Cloud providers have made a killing from neatly-packaged managed services for years. Whether it be databases or message brokers, developers like ourselves don't seem to have a problem paying a bit extra to have things taken care of. But wait, aren't we typically the last people to opt for less optimization and less control? Why is this the time we decide otherwise? If I had to make a guess, I'd wager it's because server-side devops kind of sucks.

As a developer, configuring or debugging a VPS is usually work which is unaccounted for, and it isn't particularly rewarding. At best, your application will probably end up running the same as your local environment. How could we make this inevitable part of our jobs better? Well, we could automate  it.

paramiko and scp are two Python libraries we can use together to automate tasks we'd want to run on a remote host such as restarting services, making updates, or grabbing log files. We're going to take a look at what scripting with these libraries looks like.

Setting up SSH Keys

To authenticate an SSH connection, we need to set up a private RSA SSH key (not to be confused with OpenSSH). We can generate a key using the following command:

$ ssh-keygen -t rsa

This will prompt us to provide a name for our key. Name it whatever you like:

Generating a public/private rsa key pair.
Enter the file in which you wish to save they key (i.e., /home/username/.ssh/id_rsa):

Next, you'll be prompted to provide a password (feel free to leave this blank).

Now that we have our key, we need to copy this to our remote host. The easiest way to do this is by using ssh-copy-id:

$ ssh-copy-id -i ~/.ssh/mykey username@my_remote_host.org

Verifying our SSH Key

If you'd like to check which keys you already have, these can be found in your system's .ssh directory:

$ cd ~/.ssh

We're looking for keys which begin with the following header:

-----BEGIN RSA PRIVATE KEY-----
...

Feel free to do the same on your FPS.

Starting our Script

Let's install our libraries. Fire up whichever virtual environment you prefer and let em rip:

$ pip3 install paramiko scp

Just one more thing before we write some meaningful Python code! Create a config file to hold the variables we'll need to connect to our host. We'll need to have three values handy:

  • Host: The IP address or URL of the remote host we're trying to access.
  • Username: This is the username you use to SSH into your server.
  • Passphrase: If you specified a passphrase when you created your ssh key, specify that here. Remember that your SSH key passphrase is not the same as your user's password.
  • SSH Key: The file path of the key we created earlier. On OSX, these live in your system's ~/.ssh folder.

Feel free to use the following to grab these variables from environment variables:

from os import environ


class Config:
    remote_url = environ.get('REMOTE_HOST')
    remote_username = environ.get('REMOTE_USERNAME')
    remote_passphrase = environ.get('REMOTE_PASSPHRASE')
    remote_ssh_key = environ.get('REMOTE_SSH_KEY')
config.py

Creating an SSH Client

We're going to create a class called Client to handle the interactions we'll be having with our remote host. We'll assume the Config class is getting passed in to instantiate our client:

class Client:

    def __init__(self, config):
        self.remote_url = config.remote_url
        self.remote_user = config.remote_user
        self.remote_ssh_key = config.remote_ssh_key
        self.client = None
        self.pkey = self.__get_ssh_key()
client.py

remote_url, remote_username, remote_ssh_key are self-explanatory: these are the variables we'll utilize to connect to our host in a bit.

We're going to store the object representing our client as self.client. By storing this as a high-level variable in our class, we can create class methods that reuse our client.

self.pkey is what we'll be using to grab the SSH key we need to connect to our host. This will be the first method we add to our client: __get_ssh_key().

Connecting to our Client

We'll add a method to our client called connect() to handle connecting to our host:

from paramiko import SSHClient, AutoAddPolicy, RSAKey
from paramiko.auth_handler import AuthenticationException


class Client:
    ...

    def __connect(self):
        """Connect to remote."""
        if self.client is None:
            try:
                client = SSHClient()
                client.load_system_host_keys()
                client.set_missing_host_key_policy(AutoAddPolicy())
                client.connect(self.remote_url,
                               username=self.remote_user)
            except AuthenticationException:
                raise AuthenticationException('Authentication failed: did you remember to create an SSH key?')
            finally:
                return client
        return self.client
client.py

Let's break this down:

  • client = SSHClient(): This sets the stage for creating an object representing our SSH client. The following lines will configure this object to make it more useful.
  • load_system_host_keys(): "Loading system keys" instructs our client to look for all the hosts we've connected to in the past by looking at our system's known_hosts file and finding the SSH keys our host is expecting. If we've never connected to our host in the past, we're going to have to specify our SSH key explicitly.
  • set_missing_host_key_policy(): We can tell Paramiko what to do in the event of an unknown key pair. This is expecting a "policy" built-in to Paramiko, to which we're going to specific AutoAddPolicy(). Setting our policy to "auto-add" means that if we attempt to connect to an unrecognized host, Paramiko will automatically add the missing key locally.

Now we can connect to our client! If Paramiko was able to resolve our SSH key, all we need to pass here is our host URL and username. We could also pass variables for things like port and password, if we had elected to connect to our host this way. If there's trouble finding our key, we'll need to set this explicitly.

Getting Our SSH Key

The process of grabbing our SSH key isn't fun or interesting, but we can make it quick. We'll handle grabbing our SSH key in a new method called __get_ssh_key():

from io import StringIO
from paramiko import RSAKey


class Client:
    ...
        
    def __get_ssh_key(self):
    	"""Get our SSH key."""
        f = open(self.remote_ssh_key, 'r')
        s = f.read()
        keyfile = StringIO(s)
        pkey = RSAKey.from_private_key(keyfile)
        return pkey
client.py

There are three things happening:

  1. First, we grab the contents of our SSH key by opening our key file using open(self.remote_ssh_key, 'r'), and read the contents with f.read().
  2. Next we stream the contents of this file using StringIO: keyfile = StringIO(s).
  3. Finally, we parse the contents of the file as private RSA key using RSAKey.from_private_key(keyfile).

Feel free not to dwell on this aspect of our script. The result of this is we can now modify our connect logic to explicitly pass the correct private key:

client.connect(self.remote_url,
               username=self.remote_user,
               pkey=self.pkey)

Disconnect From Our Client

Let's definitely not forget to add a disconnect() method next:

class Client:
    ...

    def disconnect(self):
        """Close ssh connection."""
        self.client.close()
client.py

Executing Unix Commands

Let's actually do some stuff! Now that we can connect, we can start interacting with our remote host using Python. To accomplish this, we'll add an execute() method:

class Client:
    ...

    def execute(self, cmd):
        """Executes a single unix command."""
        self.client = self.__connect()
        stdin, stdout, stderr = self.client.exec_command(cmd)
        return stdout.readlines()
client.py

The first thing we want to do is check if we have an open connection to our client, and if not, go ahead and open one. This gives us the benefit of being able to execute commands without explicitly connecting each time.

cmd will be a string containing whatever unix command you'd like to execute. This can be anything you'd like! Feel free to mess around with restarting services, checking logs, or whatever it is you'd like to automate. The output returned by this command will be stored as a variable called stout, which we'll return. Since we're going to mess with uploading files via SCP next, a good start might be creating a directory with mkdir and setting user permissions on the resulting directory using chown or chmod.

Depending on what you're looking to achieve, you may actually already have enough to take the ball and run with it.  exec_command() in itself gives us a lot of power!

Uploading & Downloading Files via SCP

SCP refers to the protocol for copying files to hosts (secure copy protocol) as well as the Python library which utilizes this. We've already installed the SCP library, so we can go ahead and import this. Let's see how we'd upload files to our host this way:

from paramiko import SSHClient, AutoAddPolicy, RSAKey
from paramiko.auth_handler import AuthenticationException
from scp import SCPClient, SCPException
from io import StringIO


class Client:
    ...

    def upload(self, file, remote_directory):
        """Upload a single file to a remote directory"""
        self.client = self.__connect()
        scp = SCPClient(self.client.get_transport())
        try:
            scp.put(file,
                    recursive=True,
                    remote_path=remote_directory)
        except SCPException:
            raise SCPException.message
        finally:
            scp.close()
client.py

Our method is expecting to receive two strings: the first being the local path to our file, and the second being the path of the remote directory we'd like to upload to.

The SCP and Paramiko were meant to complement one another, which makes uploading via SCP super easy. SCPClient() creates an object which expects  "transport" from Paramiko which we provide with self.conn.get_transport().

SCP's put() method will upload a local file to our remote host. This will replace existing files with the same name if they happen to exist at the destination we specify. That's all it takes!

Downloading Files

The counterpart to SCP's put() is the get() method:

class Client:

    ...

    def download(self, file):
        """Download file from remote host."""
        if self.conn is None:
            self.conn = self.connect()
        scp = SCPClient(self.conn.get_transport())
        scp.get(file)
client.py

Tracking Upload/Download Progress

SCP lets us see the progress of our file transfers via command-line easily. First, we need to create a new __progress() method:

class Client:
    ...

    def __progress(filename, size, sent):
    	"""Display SCP progess."""
    	sys.stdout.write("%s\'s progress: %.2f%%   \r" % (filename, float(sent)/float(size)*100) )
client.py

To utilize this method, all we need to do is go back to where we set our SCPClient() and pass an extra argument:

scp = SCPClient(self.conn.get_transport(),
                progress=self.__progress)

Putting Our Script to the Test

We now have a pretty sick class to handle SSH and SCP with a remote host. Let's put it to work! The following snippet is a quick way to test what we've built so far. In short, this script looks for a local folder filled with files (in my case, I filled the folder with fox gifs 🦊).

We'll be accomplishing this with the final addition to our script, main.py:

import os
from config import Config
from client import Client


def main():
    client = Client(Config)
    local_files = os.walk(os.path.abspath(Config.local_files))
    for root, dirs, files in local_files:
        for file in files:
            local_file = root + '/' + file
            client.upload(local_file, '/uploads')
            filename = file.split('/')[-1]
            print(f'Uploaded {filename} to {Config.remote_url}:/uploads.')
    client.disconnect()
main.py

Here's the output:

Uploaded fox1.gif to 174.138.46.30:/uploads.
Uploaded fox2.gif to 174.138.46.30:/uploads.
Uploaded fox3.gif to 174.138.46.30:/uploads.

Take It And Run With It

For your convenience, I've uploaded the source for this tutorial to Github. Feel free to take this and run with it! To close things out, I'll leave you with the meat and potatoes of the Client class we put together:

from paramiko import SSHClient, AutoAddPolicy, RSAKey
from paramiko.auth_handler import AuthenticationException
from scp import SCPClient, SCPException
from io import StringIO


class Client:

    def __init__(self, config):
        self.remote_url = config.remote_url
        self.remote_user = config.remote_user
        self.remote_ssh_key = config.remote_ssh_key
        self.pkey = self.__get_ssh_key()
        self.client = None

    def __get_ssh_key(self):
        """Get our SSh key."""
        f = open(self.remote_ssh_key, 'r')
        s = f.read()
        keyfile = StringIO(s)
        pkey = RSAKey.from_private_key(keyfile)
        return pkey

    def __connect(self):
        """Connect to remote."""
        if self.client is None:
            try:
                client = SSHClient()
                client.load_system_host_keys()
                client.set_missing_host_key_policy(AutoAddPolicy())
                client.connect(self.remote_url,
                               username=self.remote_user,
                               pkey=self.pkey)
            except AuthenticationException:
                raise AuthenticationException('Authentication failed: did you remember to create an SSH key?')
            finally:
                return client
        return self.client

    def upload(self, file, remote_directory):
        """Upload a single file to a remote directory"""
        self.client = self.__connect()
        scp = SCPClient(self.client.get_transport())
        try:
            scp.put(file,
                    recursive=True,
                    remote_path=remote_directory)
        except SCPException:
            raise SCPException.message
        finally:
            scp.close()

    def execute(self, cmd):
        """Executes a single unix command."""
        self.client = self.__connect()
        stdin, stdout, stderr = self.client.exec_command(cmd)
        return stdout.readlines()

    def disconnect(self):
        """Close ssh connection."""
        self.client.close()