Having the ssh key protected by a passphrase is great, in case of theft the attacker will have to overcome a second obstacle, but it is also very cumbersome to have to enter the passphrase every time we want to connect to a server. For this we have ssh-agent that will ask us for the password once and cache it. This way we will have the key protected and our work as sysadmin will remain just as agile.

To start ssh-agent at X startup, it is as simple as entering an eval ssh-agent and an ssh-add before starting the window manager. Depending on the Unix system we use, we will have to do it in one way or another.

FreeBSD:

setxkbmap es  
xrdb .Xresources  
xbindkeys &  
shutter --min_at_startup &  
export SSH_ASKPASS=/usr/local/bin/x11-ssh-askpass ;export SSH_ASKPASS  
eval `ssh-agent -s`  
ssh-add &  
exec ck-launch-session dbus-launch --sh-syntax --exit-with-session awesome

Linux:

setxkbmap es  
xrdb /home/kr0m/.Xresources  
eval `/usr/bin/ssh-agent`  
/usr/bin/ssh-add < /dev/null  
exec dbus-launch --sh-syntax --exit-with-session awesome

If for some reason we kill the ssh-agent and do not want to restart the X, we can start it manually:

But we must add the following configuration in the .bashrc to check if there is a previous agent:

#
# ssh-agent configuration
#
if [ ! -z "$(pgrep ssh-agent)" ]; then
    export SSH_AGENT_PID=$(pgrep ssh-agent)
    export SSH_AUTH_SOCK=$(find /tmp/ssh-* -name agent.*)
fi