This is based on a talk I gave for the UMBC LUG on SSH.  It might be of some use to other people.  I make no guarantees.

§ Concepts

# Encryption

Broadly speaking, encryption is a process of scrambling data that depends on a secret value of some sort to unscramble the data.  To anyone not in possession of the secret value, the scrambled data should be unintelligible.

There are two sorts of approaches to that secret value: shared secrets and public/private keypairs.

Encryption that uses shared secrets uses the same value to encrypt and decrypt the data.  This can be as simple as you typing a passphrase to encrypt your computer’s hard drive and then typing the same password later to unlock and boot the computer.  Or two people can share a password and use it to send encrypted data back and forth to each other.

In public key cryptography, there are a pair of values; one value is used to encrypt data and the other value must be used to decrypt it.  The two values are mathematically related to each other, but if you only have one of them, you shouldn’t be able to figure out the other one.

One of those two values is usually shared publicly and is called the public key.  The other remains secret and is called the private key.  Often the encryption key is the public one, so that people can encrypt things with an assurance that only the person who has the corresponding private key will be able to read it.  The reverse is not uncommon, though.  If the person with the private key encrypts something using their key, anyone with the public key can decrypt it (so the contents won’t be private), but the ability to decrypt the data serves as a sort of proof of ownership.  Only the owner of the corresponding private key could have encrypted the data, so you know it must have actually come from them.

Public keys are often very long chunks of binary data.  When checking the identity of a key, it’s common to use the key’s fingerprint rather than looking at every single binary byte.  A fingerprint is a string of characters calculated from the key data that’s long enough to be effectively unique, but short enough that manually comparing it to other strings isn’t too difficult.

# Accessing machines remotely

For many computers, you use them directly.  You either sit directly in front of it or hold it in your hand and then you type on its keyboard or you touch its screen.  That’s local access.

Remote access is everything else: making use of a computer that’s physically distant from you via use of a network.

For the purposes of this tutorial, we’re primarily concerned with shell access, where you type on a keyboard into a command-line user interface (usually abbreviated CLI).

SSH is the principal mechanism for using a CLI on a remote computer.  You run an SSH client on your local computer, it establishes a connection with the remote computer, and then anything you type is sent to the remote computer, while any output from your programs is sent back to your computer.  All communications over the network are encrypted to keep your data private.

There are and have been other mechanisms for remote access, like telnet, rlogin, and HTTP-based shell interfaces.  They’re either very niche or explicitly discouraged nowadays, so we’ll focus exclusively on SSH here.

SSH has also gone through some version changes and differing implementations over time.  But nowadays, pretty much everyone uses the OpenSSH implementation of version 2 of the SSH protocol or is compatible with OpenSSH’s implementation.

§ Basic SSH

# Standard case

The simplest and one of the most common ways to run SSH is to type ssh in your local shell followed by the name of a remote host to connect to.

$ ssh somehost
user@somehost's password: [not shown]
somehost$

Note that the only things that were typed were the ssh command and the account’s password.  The remote account name was never given explicitly.  By default, SSH assumes the username on the remote computer is the same as that on the local one.

# Host keys

The above is probably not what you’ll see the first time you use SSH to connect to a remote system.

SSH uses host keys to identify each system you can connect to.  Much as one use of public key cryptography serves to verify someone’s identity, an SSH host key verifies a server’s identity.  If your SSH client knows a server’s host key but gets a different key when it tries to connect, it knows something’s wrong.

The system of host keys is designed to prevent man-in-the-middle attacks, often abbreviated MITM.  In a MITM attack, someone intercepts your network traffic and pretends to be the server you want to communicate with.  This can allow the attacker to steal your passwords and other sensitive information.  But since SSH host keys are designed to not be replicable, an attacker can’t forge one.

The net result is that when you connect to a remote system for the first time, your SSH client will inform you that it doesn’t know whether it should trust the system’s host key.  (What if there was a MITM attack the very first time you tried to connect?)  It will show you the server’s key fingerprint and ask what you want to do.  That will look something like this:

$ ssh somehost
The authenticity of host 'somehost (192.168.1.1)' can't be established. 
RSA key fingerprint is MD5:90:9c:46:ab:03:1d:30:2c:5c:87:c5:c7:d9:13:5d:75.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'somehost' (RSA) to the list of known hosts.
user@somehost's password: [not shown]
somehost$

At the first prompt, about continuing to connect, you can take time to verify the host key fingerprint, if you’re able to.  Many systems publish their host key fingerprints on HTTPS websites.  (Using HTTPS ensures that the web page can’t be intercepted and altered in a MITM attack.)  Perhaps you can contact the person who runs the server via a communication medium you trust to verify the fingerprint.

If you have a source for the fingerprint you can copy and paste from (like a website), you can paste the fingerprint at the prompt instead of typing yes or no.  That lets your SSH client do the checking; it will proceed only if what you pasted matches what it got from the server.

Most of the time, it’s pretty safe to just accept host keys at this prompt.  (This is called trust-on-first-use security, or TOFU.)  If you’re in a controlled environment like a university campus or business network, the odds of someone intercepting your SSH sessions are pretty low.  When connecting to systems over the Internet, you might want to be a little more cautious, but TOFU is at least a little less risky than not having any checks on remote servers’ identities.

Just for reference, here’s what it looks like when you try to connect to a system but its host key doesn’t match the one your SSH client has on record for the system:

$ ssh somehost
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! 
Someone could be eavesdropping on you right now (man-in-the-middle attack)! 
It is also possible that the RSA host key has just been changed. 
The fingerprint for the RSA key sent by the remote host is
MD5:90:9c:46:ab:03:1d:30:2c:5c:87:c5:c7:d9:13:5d:75. 
Please contact your system administrator. 
Add correct host key in /home/user/.ssh/known_hosts to get rid of this message. 
Offending key in /home/user/.ssh/known_hosts:2
Password authentication is disabled to avoid man-in-the-middle attacks. 
Agent forwarding is disabled to avoid man-in-the-middle attacks. 
X11 forwarding is disabled to avoid man-in-the-middle attacks. 
Permission denied (publickey,password,keyboard-interactive).
$ 

If you see that message, you should check with whomever runs the remote system to see if there’s a problem.  Sometimes systems’ host keys change; if that’s the case, you can delete the stored key.  (In the above example, the stored key is on line two of /home/user/.ssh/known_hosts.)  If nothing’s change with the remote system, you have probably been the subject of a MITM attack.  You will need to address that problem before continuing (and doing so is beyond the scope of this tutorial).

# User names

SSH assumes that your username on the remote computer is the same as it is locally.  If the remote username is different, you can specify the remote username on the command line with an at sign separating it from the remote hostname:

$ ssh othername@somehost
othername@somehost's password: [not shown]
somehost$

# scp

SSH also has a program for copying files across the network.  It encrypts everything, of course, so neither your password nor the data is visible to prying eyes.

You use scp much like cp:  You give the existing file or files first and the destination for it (or them) last.  The difference from cp is that either the existing file or the destination can be in a remote location.  To indicate a remote location, put the remote hostname before the path to the file, with a colon separating them.  As with ssh, you can specify a remote username if it’s different from the local one.

$ scp localfile somehost:public_html/
user@somehost's password: [not shown]
localfile            100% |*****************************|  2048 KB 00:03
$ scp othername@somehost:public_html/index.html ~/public_html/
othername@somehost's password: [not shown]
index.html           100% |*****************************| 10198 KB 00:05
$ 

Note that you cannot copy from a remote location to a remote location.  Either the source or destination must be on your local computer.

§ Intermediate SSH

# Remote commands

You don’t have to connect to a shell on the remote computer.  You can just tell ssh to run a command remotely and give you its output.  To do this, just add the command to run remotely on your command line after the remote hostname.

$ hostname
thishost
$ ssh somehost hostname
user@somehost's password: [not shown]
somehost
$ 

Some programs manipulate the terminal as they run.  Usually these are interactive programs, like pine or emacs.  Normally, ssh will run the remote command non-interactively and just spit its output back at you. For terminal-using programs, you must add the -t parameter to your command line to tell ssh to allocate and use a pseudoterminal on the remote system.

$ ssh -t somehost pine
user@somehost's password: [not shown]
[pine opens; user reads mail; user exits]
$ 

# X11 forwarding

Graphical programs use the X Window protocol to display on your screen. With X, you can run a program on a remote computer have have it display locally.  Normally, X runs unencrypted over the network.  With ssh’s -X option, you can tell it to use its encrypted connection for X programs.

$ ssh -X -f somehost xclock
user@somehost's password: [not shown]
[xclock appears on screen]
$ 

Note that the example also used the -f on the command line.  The -f parameter puts ssh in the background after it asks for your password.  The above command is the remote equivalent of xclock&.

Without the -f parameter, you would not get back to a shell prompt on your local host until the xclock program exited.

# Config files

SSH’s configuration file is ~/.ssh/config and can contain lots of stuff.  (Note that only the local options file is important—your SSH client reads it before connecting to the remote host.)

Host lines

The file is broken up into chunks by Host directives.  Everything after a Host line applies to that host.  (Until the next Host line, of course.) The argument after Host is the name that you’d type on the command line.

There is a special hostname, * (an asterisk), which represents all hosts.  You can use this hostname to set default values for settings.  Note that OpenSSH gives priority to things that are earlier in the file, so Host * must appear last if you want more specific Host settings to override the generic ones.

Alternate host names

You can use the HostName directive to give the network name of the remote host.  For example, if there’s a really long name you don’t want to type all the time, you can effectively make a shorter alias for it.

This ~/.ssh/config file:

Host supes
  HostName supercalifragilisticexpialidocious

Would lead to this:

$ host supes
supes does not exist
$ host supercalifragilisticexpialidocious
supercalifragilisticexpialidocious is 192.168.1.2
$ ssh supes
user@supes's password: [not shown]
supercalifragilisticexpialidocious$

Alternate usernames

Instead of typing the remote username on the command line (à la ssh othername@somehost), you can use a User directive in the config file:

Host somehost
  User othername
$ ssh somehost
othername@somehost's password: [not shown]
somehost$

Compression

If you set Compression yes in the config file, SSH will try to compress all data before sending it over the network.  This can help with some slow connections.

If you put the setting in your Host * section, it will apply to all hosts.

Other things

Check the man page (man ssh_config) for all of the options available.

§ Advanced SSH

# Public key authentication

SSH can use a number of methods to authenticate you to the remote host.  Passwords are the default, since they’re the most common.  A more flexible approach is to use public keys.  These work in much the same way that an SSH client uses the server’s host key to verify the remote computer’s identity.

There are a number of advantages to using public key authentication.

First, your authentication credentials can’t be compromised remotely.  When you use a password to log in, your SSH client sends the password to the remote system.  Although the transmission is encrypted, so no one on the network can see the password, the remote system has to be able to see the password to verify it.  But if someone has broken into the remote system and is running malicious software there, they can steal the passwords as they’re sent.  In contrast, public key authentication works by the client proving it has the right private key without ever revealing it to the server.  So even a compromised server cannot steal your private key.

Second, you can use a locally-running “agent” to hold your private key.  This can make remote logins quite easy and convenient, while still maintaining reasonable security.  I’ll discuss agents more in a bit.

Finally, you can restrict specific keys to only run specific programs when they’re used.  For instance, if you have a backup script, you can designate one key that just runs that script when it’s used.  Then you can save that key somewhere untrusted, if necessary, because the key can’t be used to do anything other than run the backup script.  Check the “authorized_keys file format” section of the sshd man page (man sshd) for more details.

Creating a public key

You use the ssh-keygen program to create a keypair.  I recommend the -t ed25519 parameter to make an ED25519 key, but some older servers don’t yet support those.  Use -t rsa -b 4096 if you have problems with an ED25519 key.

$ ssh-keygen -t ed25519
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/user/.ssh/id_ed25519): [press enter for default]
Enter passphrase (empty for no passphrase): [not shown]
Enter same passphrase again: [not shown]
Your identification has been saved in /home/user/.ssh/id_ed25519. 
Your public key has been saved in /home/user/.ssh/id_ed25519.pub. 
The key fingerprint is: 
SHA256:4tjzwCeqT9M41yi4mTQE+gkJlMctZNojKaWpHoc/HbU user@localhost
$ 

Using the key for logins

You need to take the contents of the public key (/home/user/.ssh/id_ed25519.pub in the above example) and add them on a single line to ~/.ssh/authorized_keys on the remote host.

That could look like this:

$ scp ~/.ssh/id_ed25519.pub somehost:.
user@somehost's password: [not shown]
id_rsa.pub           100% |*****************************|   225 00:00
$ ssh somehost
user@somehost's password: [not shown]
somehost$ cat id_ed25519.pub >> ~/.ssh/authorized_keys
somehost$ rm id_ed25519.pub
somehost$ exit
$ ssh somehost
Enter passphrase for key '/home/user/.ssh/id_ed25519': [not shown]
somehost$ 

There is a program named ssh-copy-id that exists to simplify that process.  Simply give it the name of the remote system and it will do most of the work for you.

$ ssh-copy-id remotehost
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Host key fingerprint is SHA256:83zW/9RGLCs087xEhsrGLdrv7xmifZtUpcMxirM6/PA
user@somehost's password: [not shown]

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'somehost'"
and check to make sure that only the key(s) you wanted were added. 

$ 

You should definitely follow its recommendation to check the remote host’s ~/.ssh/authorized_keys file afterwards, just to make sure no extraneous keys were added.

# SSH agent

An SSH agent is a program that keeps a copy of your key in memory for ssh programs to use.  (Normally, you have to type the passphrase for the key every time its needed.)  It makes SSH as easy as this:

$ ssh somehost
somehost$

Agent security

The agent is pretty good about security.  It never even divulges your key—ssh programs hand data to it and it does the needed work with the key itself.  However, there are some possible security problems.

For one thing, if you walk away from your computer, someone can sit down at it and ssh places as you.  Solution: remove the key from the agent when you’re going to be gone from your computer.  (And/or just lock the screen.)

If someone else on the computer can act as you (root can do this), they can give data to the agent and have it processed as if you asked.  Solution: don’t run an agent on a computer where you don’t trust the admin.  As an extra security measure, use the -c parameter to ssh-add when adding a key (described more below).

Finally, someone with access to the computer’s memory (usually, this is only root) can search through the agent’s data and get an unencrypted copy of the key.  Solution: again, don’t run an agent on a computer where you don’t trust the admin.

Starting the agent

The procedure for running the agent differes slightly depending on whether you’re running a Bourne Shell derivative such as bash, ksh, or zsh (this is probably the case) or a C-Shell derivative such as csh or tcsh (this is less likely, and you probably already know if this applies to you).

For a Bourne Shell derivative, use:

$ eval `ssh-agent -s`
$ 

For a C-Shell derivative, use:

> eval `ssh-agent -c`
> 

Adding and removing keys

Key management with an agent is done via the ssh-add program, after the agent has been started.

To add a key, use:

$ ssh-add -c ~/.ssh/id_rsa
Enter passphrase for /home/user/.ssh/id_rsa: [not shown]
Identity added: /home/user/.ssh/id_rsa (/home/user/.ssh/id_rsa)
$ 

Note that the -c parameter is not technically necessary.  It requires the agent to ask you, usually by popping up a window in your GUI, every time a program wants to use a key.  You can just hit enter to close the window and accept the use of your key.  But this can act as a safeguard against a partial compromise of your computer.  Even if someone can send requests to your agent, they won’t be able to use your identity without your approval.

To show the keys that have been loaded into the agent, use the -l parameter:

$ ssh-add -l
1024 75:a4:2c:9b:b1:58:8f:9c:96:d8:99:77:fc:01:0d:8a /home/user/.ssh/id_rsa (RSA)
$ 

To remove a key, use the -d parameter and the key filename:

$ ssh-add -d ~/.ssh/id_rsa
Identity removed: /home/user/.ssh/id_rsa (/home/user/.ssh/id_rsa.pub)
$ 

Using the agent

Once the keys are loaded, the agent is used automatically.

$ ssh somehost
somehost$ 

Note on multiple shells

The agent’s setup is only designed to work with a single shell.  All you really need to do to have multiple shells (e.g. multiple xterms, etc.) use the same agent is to copy the value of the environment variable $SSH_AUTH_SOCK from one to the other.

$ echo $SSH_AUTH_SOCK
/tmp/ssh-XXLN11Tv/agent.953
[In another shell on the came computer:]
$ export SSH_AUTH_SOCK=/tmp/ssh-XXLN11Tv/agent.953
$ 

(Note that in C-Shell derivatives, the command is setenv SSH_AUTH_SOCK path).

I use a program called keychain to manage multiple shells with the same SSH agent.  You can install it and read the documentation for more info, but basically, all you need to do is run keychain --confirm ~/.ssh/id_ed25519 in your shell’s startup files, and it will check for an existing agent, connect to it if it exists, and create a new one (loading the supplied key) if it doesn’t.

Exiting the agent

From the shell where you started it, run eval $(ssh-agent -k) and the running agent will be shut down.  If you copied $SSH_AUTH_SOCK to other shells, those variables will have to be unset by hand.  If you’re using Keychain, run keychain -k all.

# Port forwarding

The ssh program can also be told to listen on arbitrary ports either on the local or remote computer, forward any data through the encrypted connection, and send it to some other destination from the other end.

Let’s say I have a web proxy on a remote computer.  I want my local browser to use an encrypted connection to that computer to use the proxy.  So SSH will listen for connections locally, forward the data across the encrypted connection and send it to the proxy on the remote computer.  My web browser will be told to use a port on the local browser as its proxy.

Let’s say the remote computer is “somehost”, the local computer is “localhost”, the proxy is listening on port 5865 on the remote host, and I want SSH to use port 5001 on the local host.  I would run:

$ ssh -L 5001:somehost:5865 somehost
user@somehost's password: [not shown]
somehost$ 

The -L option tells SSH to listen on the local host and forward to the remote host.  (Use the -R option for the reverse situation.) The first number is the port to listen on at the local computer.  In this case, it’s 5001.

The second value is the computer to which the data should go after it comes out of the encrypted connection.  (Since this example has the data going to the same computer as the connection goes to, I could have used the special name “localhost”, but avoided that, as it would confuse things.)  The computer specified can be any computer on the network.  The last number is the port that the data will be sent to on the destination computer.  In this example, that port is 5865, where the web proxy is listening.

I would then set my browser to use “localhost:5001” as its HTTP proxy.

The tunnel will remain open for as long as you’re connected via ssh.

Normally, SSH will only accept connection from the computer on which it’s listening.  If you want other computers on the network to use the encrypted tunnel, you must also use the -g option to ssh.  (This can be a security risk!)

SSH has a -N option that can be used to just create an encrypted tunnel without opening a shell on the remote computer.  The tunnel remains open until the client ssh is killed.

§ Other Resources

The man pages should be the authoritative reference for OpenSSH.

Mozilla has OpenSSH guidelines for secure settings on the server, the client, and with key generation.  The guidelines are periodically updated to take new developments in cryptography into account.