Jul 24

Managing Multiple SSH Keys


SSH keys are a very efficient way to make secure connections over the internet, for a large variety of things.

Many things use SSH keys regularly, for example using Git on GitHub requires you to register a key to use when accessing repositories.
Most of the time, however, their help guides only explain how to create a single key and use it with their service. If you use multiple services that use SSH, you could use the same key for all of them but that would be less secure and would mean having to change your key on each one if you have to change it at any point.

The obvious thing to do is to create multiple keys. This is a quick little tutorial on how to create and manage multiple SSH keys using a Linux terminal. If anyone knows the Windows equivalent, please let me know and I’ll add it to this post.

Creating Keys

Let’s start with the basics, the most important thing to be able to do here is to create a key.

Simply open up a new terminal and do the following:
cd ~/.ssh
If you get a response saying the directory does not exist run mkdir ~/.ssh and then repeat.
You will be prompted for an output file, just leave it blank and press enter.
You will now be prompted for a passphrase, and then to repeat the passphrase. Do not forget this passphrase, it’s the “lock” for the key.
Once the key has been created you need to add it to the key manager, using ssh-add id_rsa and enter the passphrase you just used.

It should go something like this:

Script started on Tue 24 Apr 2012 14:15:45 BST
[email protected]:~$ cd ~/.ssh
[email protected]:~/.ssh$ ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/home/joe/.ssh/id_rsa): [Press Enter] 
Enter passphrase (empty for no passphrase): [Enter A Passphrase] 
Enter same passphrase again: [Repeat Passphrase] 
Your identification has been saved in /home/joe/.ssh/id_rsa.
Your public key has been saved in /home/joe/.ssh/id_rsa.pub.
The key fingerprint is:
c0:73:11:d7:6e:bc:09:0e:89:62:30:38:e8:e3:fc:17 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|..      o...     |
|+ o  .   o  .    |
|.. o  +... o     |
| o  o .+o . +    |
|o .. .  So o o   |
| o   E    . o    |
|  .   .          |
|   . .           |
|    .            |
Script done on Tue 24 Apr 2012 14:16:13 BST
[email protected]:~/.ssh$ ssh-add id_rsa
Enter passphrase for id_rsa: 
Identity added: id_rsa (id_rsa)

Moving the Key

To organise keys, I simply move my keys into a directory named for their purpose, for example “git” for the git keys.

This leaves me with something like the following layout:

[[email protected] ~]$ cd .ssh
[[email protected] .ssh]$ ls -l
total 16
-rw-r--r--. 1 joe joe  407 May  4 22:07 config
drwx------. 2 joe joe 4096 Apr 24 13:27 fedoraproject
drwx------. 2 joe joe 4096 Apr 24 13:20 git
-rw-r--r--. 1 joe joe 2061 May  4 22:06 known_hosts
[[email protected] .ssh]$ cd git
[[email protected] git]$ ls -l
total 8
-rw-------. 1 joe joe 1766 Jan  5  2012 id_rsa
-rw-r--r--. 1 joe joe  403 Jan  5  2012 id_rsa.pub
[[email protected] git]$

So in each of the subfolders you will have id_rsa and id_rsa.pub.

Pointing to the Key

The only problem now, is that when ssh searches for a key, it won’t be able to find the one it’s looking for. You need to correctly edit your config file that should be located in your ~/.ssh/ folder. If it isn’t already there, just create a new file.

Mine looks like the following:

Host github.com
	User git
	Hostname github.com
	PreferredAuthentications publickey
	IdentityFile ~/.ssh/git/id_rsa
Host fedoraproject.org
	Hostname fedoraproject.org
	PreferredAuthentications publickey
	IdentityFile ~/.ssh/fedoraproject/id_rsa
Host fedorapeople.org
	Hostname fedorapeople.org
	PreferredAuthentications publickey
	IdentityFile ~/.ssh/fedoraproject/id_rsa

It’s fairly self-explanatory what each part of the file is, you could even just use mine as a template.


Enjoy managing your multiple SSH keys easily!



1 ping

Skip to comment form

  1. Jim

    You need to

    chmod 700 .ssh

    Before it will work

  2. David Weinraub

    Thanks, this is exactly what I have been looking for for managing my keys.

  3. Damon

    Would there be a disadvantage or reason not to naming the key for the service instead of creating subfolders?

    So instead of `~/.ssh/github/id_rsa & .pub,` just name the key github, giving you `~/.ssh/github` & `~/.ssh/github.pub`?

    1. Joe

      No, that’s a perfectly acceptable way to do this too.

  4. ChuckCottrill

    Add a simple shell script to the mix, to generate keys for specific systems:

    COMMENT=${2:-“Key for $SYSTEM work”}
    ssh-keygen -t dsa -f ~/.ssh/id_dsa.$SYSTEM -C “$COMMENT”
    chmod 600 ~/.ssh/id_dsa.$SYSTEM
    chmod 600 ~/.ssh/id_dsa.$SYSTEM.pub

  5. Marc-A. Berube


    Any way to have multiple ssh per host? My very own problem is how to use different ssh on a single host for handling different accounts (github or heroku per example)

    1. Joe


      Whatever you write in the “Host” field can effectively be used as an alias for that host.
      For example:
      Host git_user1
      User git
      Hostname github.com
      PreferredAuthentications publickey
      IdentityFile ~/.ssh/git/id_rsa

      Host git_user2
      User git
      Hostname github.com
      PreferredAuthentications publickey
      IdentityFile ~/.ssh/git2/id_rsa

      Notice the different Host and IdentityFile settings.
      Then when connecting, just use the alias, for example “ssh git_user1” or “git clone [email protected]_user1”

      [Edit] I noticed the formatting of this comment isn’t very helpful. If you have any problems getting this working just reply to the comment, and I’ll add it as an update to the post with proper formatting.

  6. Juri Sinitson


    Thanks for the instructions.

    I also have multiple ssh’s I want to connect to and I use KDE4. So beside of instructions above I used also this ones:

    To work for multiple hosts, the script ~/.kde4/Autostart/ssh-add.sh has to look like this:
    export SSH_ASKPASS=/usr/lib/ssh/ksshaskpass
    /usr/bin/ssh-add ~/.ssh/git/id_rsa ~/.ssh/fedoraproject/id_rsa

    If you have a large List of the key-files of your several hosts you may also use Variables of the shell.

  7. Mike

    This is a very good article on SSH login without password. Here is another one that worked for me when I first started doing this. It’s very simple, concise and easy to understand. http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/ [LINK EDITED BY ADMIN. Linking via advertisements is not allowed]

  8. kernc

    I don’t think one needs to use ssh-add if they specify IdentityFile in their ssh_config. Correct?

    1. Joe

      That’s true – unless you want to avoid entering your password each time you use the key.

      If you use ssh-add ~/path/to/key it will prompt you for your password and then add the key to the ssh-agent. Then, when you use this key you won’t have to enter the password.

      If you specify an IdentityFile setting for a key in the config you will still have to enter the password when the key is used.

      As far as I know, that’s the only difference between these two techniques.

  9. Adam

    Surely the passphrase would be the key for the lock, not the lock for the key… 😛 Hehe sorry, couldn’t help myself.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>