What Is SSH? A Beginner's Guide to Secure Remote Login with SSH Keys

If you manage a Linux server, a Raspberry Pi, or a VPS, you’ll connect to it over SSH sooner or later. Typing a password every time is slow, and passwords are also the weakest part of the setup. SSH keys fix both problems.

This guide explains what SSH is and how it works, then walks through generating an SSH key pair, installing it on a server, and turning off password logins.

What Is SSH?

SSH (Secure Shell) is a network protocol that lets you securely log in to and control a remote computer over an untrusted network such as the internet. It runs on TCP port 22 by default and encrypts everything that passes through it: your login, your commands, and their output.

SSH replaced older tools like Telnet and rlogin, which sent everything in clear text. Besides remote terminal sessions, SSH is also used for:

• Secure file transfer with SFTP and scp
• Port forwarding and tunneling to reach services behind a firewall
• Running remote commands and automation scripts
• Git operations over [email protected]: style URLs

How SSH Works

An SSH connection involves two sides: the client (your computer) and the server (the remote machine running an SSH daemon, sshd).

1. Connection and encryption. The client and server agree on encryption algorithms and create a shared session key, so everything that follows is encrypted.
2. Server verification. The server proves its identity with its host key. The first time you connect, your client asks you to confirm the server’s fingerprint and saves it in ~/.ssh/known_hosts. If it ever changes, you get a warning, which protects you from impersonation.
3. User authentication. You prove who you are, either with a password or with an SSH key.

Password vs. SSH Key Authentication

How SSH Keys Work

An SSH key is a pair of files:

• A private key stays on your computer and must never be shared.
• A public key is copied to every server you want to access. It’s safe to share.

When you connect, the server uses your public key to issue a challenge that only the matching private key can answer. Your private key never leaves your computer, and nothing secret travels over the network.

Step 1: Generate an SSH Key Pair

On your client machine (Linux, macOS, or Windows 10/11 with the built-in OpenSSH client), open a terminal and run:

ssh-keygen -t ed25519 -C "[email protected]"

Ed25519 is a modern algorithm that is fast, secure, and produces short keys. If you need to support very old systems, use RSA instead:

ssh-keygen -t rsa -b 4096 -C "[email protected]"

You’ll be asked two questions:

1. File location. Press Enter to accept the default (~/.ssh/id_ed25519).
2. Passphrase. Set one. It encrypts your private key on disk, so a stolen key file is useless on its own.

This creates two files:

~/.ssh/id_ed25519       # private key (keep secret)
~/.ssh/id_ed25519.pub   # public key (share this)

Step 2: Copy the Public Key to the Server

The easiest way on Linux and macOS is ssh-copy-id:

ssh-copy-id username@server-ip

Enter your password one last time. The command appends your public key to ~/.ssh/authorized_keys on the server and sets the right permissions.

On Windows (PowerShell)

ssh-copy-id isn’t available there, so pipe the key over SSH instead:

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh username@server-ip "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

Manual method

If neither works, log in with your password and add the key by hand:

mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/authorized_keys     # paste the contents of your .pub file on a new line
chmod 600 ~/.ssh/authorized_keys

Step 3: Test the Key Login

Connect again:

ssh username@server-ip

If everything is set up correctly, you’re logged in without a server password. If you set a passphrase, you’ll be asked for that instead, which only unlocks your local key.

To use a key stored somewhere else, point to it with -i:

ssh -i ~/.ssh/my_other_key username@server-ip

Step 4: Avoid Retyping Your Passphrase with ssh-agent

A passphrase keeps your key safe, but typing it every time defeats the convenience. The SSH agent holds your unlocked key in memory for the session:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

On macOS, you can store the passphrase in the keychain with ssh-add --apple-use-keychain ~/.ssh/id_ed25519. On Windows, the ssh-agent service can be set to start automatically.

Step 5: Add a Config Shortcut (Optional)

Instead of remembering usernames, IPs, and key paths, add an entry to ~/.ssh/config on your client:

Host myserver
    HostName 203.0.113.10
    User username
    IdentityFile ~/.ssh/id_ed25519

Now connecting is just:

ssh myserver

Step 6: Disable Password Logins (Recommended)

Once key login works, turn off passwords so attackers can’t guess their way in. Test key login in a separate terminal first, and keep your current session open until you’ve confirmed it works.

On the server, edit the SSH daemon configuration:

sudo nano /etc/ssh/sshd_config

Set or change these lines:

PubkeyAuthentication yes
PasswordAuthentication no
PermitRootLogin no

Some distributions also include extra files in /etc/ssh/sshd_config.d/ that can override these settings, so check there if a change doesn’t take effect.

Validate the file, then restart the service (ssh on Ubuntu/Debian, sshd on RHEL-based systems):

sudo sshd -t
sudo systemctl restart ssh

Troubleshooting Common Problems

• Still asked for a password. The usual cause is permissions. On the server, ~/.ssh must be 700 and authorized_keys must be 600, and your home directory must not be writable by others.
• “Permission denied (publickey)”. The public key isn’t in authorized_keys, or you’re using the wrong key or username. Run ssh -v username@server-ip for detailed output.
• “WARNING: UNPROTECTED PRIVATE KEY FILE!” Your private key is too open. Fix it with chmod 600 ~/.ssh/id_ed25519.
• “REMOTE HOST IDENTIFICATION HAS CHANGED!” The server’s host key differs from the one on record. If you rebuilt the server, remove the old entry with ssh-keygen -R server-ip. If you didn’t, treat it as a possible attack.
• Locked out after disabling passwords. Use your hosting provider’s web console or recovery mode to re-enable PasswordAuthentication temporarily.

SSH Key Best Practices

• Always use a passphrase on keys that live on laptops or shared machines.
• Use a separate key per device, so you can revoke one without affecting the others.
• Never share or upload your private key, and don’t commit .ssh files to Git.
• Remove old keys from authorized_keys when a device is retired or lost.
• Add fail2ban or change firewall rules if your server is exposed to constant login attempts.

Conclusion

SSH gives you an encrypted channel for managing remote machines, and SSH keys make that channel both safer and faster to use. Generate an Ed25519 key pair, copy the public key to your server with ssh-copy-id, confirm the key login works, and then disable password authentication to shut the door on brute-force attacks.

Once you’re comfortable with keys, the same setup works for SFTP transfers, Git hosting, and automation, all without typing a password. If you want to build on this, see How to Set Up an SFTP Server on Linux for secure file transfers over the same SSH connection.