I had a rather stupid security incident recently, and want you to check if your company network is vulnerable to the same issue.
The network in question had a bastion host, with an externally accessible SSH server and a (more recently added) VPN. Because of someone's laziness (i.e. routing not configured properly), the bastion host was performing network address translation, so, no matter who connected, over VPN or just double SSH, all internal hosts would see incoming SSH connections from the internal IP address of the bastion host.
The real issue that triggered a short SSH outage was that internal hosts ran fail2ban, and I had a slightly broken keyboard (already replaced as of now) that would sometimes double-register one of the keys, and the character on that key was in my password. So, I managed to mistype my password twice when connecting to one of the internal servers.
Result: that internal server banned me. Or rather, it banned the IP address of the bastion host. And that's not only me, all other people who were working there also got banned for some time - which is clearly unwanted.
Here are some recommendations how to avoid repeating this incident.
The network in question had a bastion host, with an externally accessible SSH server and a (more recently added) VPN. Because of someone's laziness (i.e. routing not configured properly), the bastion host was performing network address translation, so, no matter who connected, over VPN or just double SSH, all internal hosts would see incoming SSH connections from the internal IP address of the bastion host.
The real issue that triggered a short SSH outage was that internal hosts ran fail2ban, and I had a slightly broken keyboard (already replaced as of now) that would sometimes double-register one of the keys, and the character on that key was in my password. So, I managed to mistype my password twice when connecting to one of the internal servers.
Result: that internal server banned me. Or rather, it banned the IP address of the bastion host. And that's not only me, all other people who were working there also got banned for some time - which is clearly unwanted.
Here are some recommendations how to avoid repeating this incident.
- If you can, make sure that internal servers can see the VPN IP address of the admin, not just the internal IP of the bastion host. I.e., remove NAT.
- If you can't remove NAT, or have to support double-SSH through the bastion host, make sure that it is whitelisted. OTOH, fail2ban does protect against the situation when there is some password-guessing malware on the admin's PC. So, my recommendation would be, once NAT is removed, to deprecate the double-SSH workflow.
- Use SSH keys.
- When it is acceptable (i.e. would not be counted as an incident), actually perform the test, just to see if the ban is applied correctly.
