You spin up a brand-new VPS, check the logs an hour later, and find hundreds of failed SSH logins from IPs all over the world. Nobody knows your server exists yet. So who is attacking you, and why?
The reassuring answer, up front: almost certainly nobody is targeting you specifically — you're seeing internet background noise. The moment a server has a public IP and an open SSH port, automated bots that scan the whole internet find it and start guessing credentials. It never stops, and it isn't a sign you've been singled out. What matters is not making the noise go away but making sure it stays noise — that no amount of guessing gets a bot in. That comes down to a short, layered checklist to secure a Linux server: log in with keys instead of passwords, close every door you don't use, slow the guessers down, and patch automatically.
Why a fresh server gets attacked within minutes
There's no mystery and no malice aimed at you. The public internet is constantly scanned by automated tools hunting for machines with exposed services — SSH, databases, admin panels, anything that answers. When a bot finds an open port 22, it starts a brute-force or dictionary attack: root with 123456, admin with password, and thousands of other common pairs, hoping one machine left a weak credential in place. Because this runs at internet scale, your server doesn't need to be findable, famous, or valuable — a routable IP is enough. That's why the failed logins appear before you've published anything and never stop.
This reframes the problem: you can't end the attempts, so the goal is to make every one fail by design. A server that accepts only cryptographic keys, exposes nothing it doesn't need, and stays patched absorbs that traffic indefinitely without being at risk.
The one change that ends password guessing: SSH keys
The single highest-value security decision on any Linux server is to disable password authentication entirely and log in only with SSH keys. A brute-force attack works by guessing passwords; remove passwords and the entire category of attack you're watching in your logs can't succeed — there's nothing left to guess.
An SSH key pair is a private key you keep on your machine and a public key you place on the server. Authentication proves you hold the private key without ever sending a guessable secret, and keys are far too long to brute-force in practice — which is why they close the door passwords leave ajar.
ssh-keygen -t ed25519 -C "[email protected]"
Place the public key on the server (many providers let you paste it in at creation time), confirm you can log in with it, then turn passwords off in /etc/ssh/sshd_config:
PermitRootLogin no
PasswordAuthentication no
Reload SSH afterward, and always test a new login in a second terminal before closing your current session — that way a typo locks out a spare window, not you. Disabling root login also forces an attacker to guess a valid non-root username and its key, instead of attacking the one username (root) present on every box. The full first-boot version of this lives in our VPS setup guide; this article is the ongoing posture once those basics are in place.
Close every door you don't use: a default-deny firewall
Securing SSH only covers one service. The broader principle is to expose the smallest possible surface — every open port is a door a bot can rattle, so deny everything by default and allow only what you serve. On most setups that's SSH plus your web ports (80 and 443).
ufw default deny incoming
ufw default allow outgoing
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw enable
The reasoning behind "default deny" matters more than the commands. People add allow rules for what they want reachable but never set the baseline to deny, leaving everything else quietly open — a database, a cache, or a forgotten dev service where a scanner will find it. Bind internal services to localhost or a private network and let the firewall refuse the rest.
Slow the guessers down: rate-limiting and fail2ban
Key-only SSH already makes brute-force pointless, but you can also stop attempts from filling your logs and burning resources. Rate-limiting SSH with ufw throttles any IP making too many connections in a short window:
ufw limit OpenSSH
fail2ban goes further, watching your logs and banning IPs that rack up failed logins so they can't keep hammering you:
sudo apt install fail2ban
# /etc/fail2ban/jail.local
[sshd]
enabled = true
maxretry = 5
bantime = 1h
Be honest about what this buys you, though: both are defense-in-depth, not primary protection — source IPs are disposable and a determined operator just rotates them. Your real safety is that passwords are off and keys can't be guessed; the bans only keep logs readable and load down.
Stay patched: the vulnerability you don't know about yet
Brute-force gets the attention because it's visible in your logs, but the quieter risk is an unpatched vulnerability in software you run. Bots scan for known flaws in specific service versions just as eagerly as they guess passwords, and an outdated package can hand over a server without anyone guessing anything. So keep the system patched automatically — low effort, high impact.
On Debian and Ubuntu, enable unattended security upgrades so patches land on their own:
sudo apt install unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades
The point isn't this tool; it's that an unpatched server degrades every day it sits untouched. Pair automatic updates with removing packages you no longer use — less software means fewer things to patch or exploit.
None of these four controls is sufficient alone, and that's the point: they overlap so a gap in one is covered by another. Key-only SSH removes the password-guessing attack class, a default-deny firewall shrinks the surface to the ports you serve, fail2ban cuts the noise, and automatic patching closes known holes. Work down that list and the attacks in your logs become exactly that — background. They keep coming, they keep failing, and your server keeps running.
FAQ
Is my server actually being hacked, or is this normal?
Almost always normal. Failed logins from random IPs are the internet's background noise — automated bots scanning every public address and guessing credentials. Seeing them doesn't mean you've been targeted or breached; it means your server is online. The real question is whether any attempt can succeed, which is what disabling password login and keeping software patched prevents.
How do I stop SSH brute-force attacks for good?
Disable password authentication entirely and use SSH keys, then disable root login. Brute-force guesses passwords, so once there are none to guess, it can't work no matter how many attempts a bot makes. A default-deny firewall and fail2ban reduce the noise, but key-only authentication is the change that actually ends the threat.
Do I need fail2ban if I already use SSH keys?
Not strictly — with password login disabled, brute-force can't succeed, so fail2ban is defense-in-depth rather than your main protection. It's still worth running: it keeps logs readable, cuts wasted load from persistent bots, and bans IPs that probe other services too. Treat it as a useful extra layer, not the thing keeping you secure.
Should I change the SSH port to stop attacks?
Moving SSH off port 22 reduces automated noise, since most bots target the default port, but it's obscurity, not security — a real scan finds the new port quickly. It's a fine noise-reduction tactic on top of proper hardening, never a substitute for key-only authentication and a firewall. Don't let a moved port lull you into skipping the controls that matter.
What's the single most important thing to secure first?
SSH authentication. Switch to key-based login and turn off passwords before anything else, because that one change neutralizes the most common attack you'll see in your logs. After that, set the firewall to default-deny and enable automatic security updates. Those three steps cover most of the practical risk to a public Linux server.
Next step
The attacks won't stop, so build a server that doesn't care. Start with the change that ends password guessing — key-only SSH, root login disabled — then a default-deny firewall that exposes only the ports you serve, fail2ban and rate-limiting to quiet the noise, and automatic security updates so known holes close themselves. None of it is exotic, and together it turns a relentlessly probed box into one that absorbs the traffic and keeps running. Get reliable, security-minded hosting and server solutions at just-server.net.
