Fabric tutorial 2 – File transfer & error handling

On the first turorial we learned to run commands on remote hosts with Fabric. Now we move on to
transfering files. Transfering new configuration files is usually quite important part of system administration.
Also retrieving log files from the remote machines might be useful.

Sending files

Let’s assume we’ve made a new ssh_config file with important changes and we want to send it to our
remote hosts. Here’s a task for sending files.

def file_send(localpath,remotepath):
    put(localpath,remotepath,use_sudo=True)

Run it with:
fab file_send:path/to/edited/ssh_config,/etc/ssh/ssh_config
or if the modified ssh_config is in the directory where you’re running Fabric:
fab file_send:ssh_config,/etc/ssh/ssh_config

If we’re sending the file to a location that doesn’t need sudo eg. /tmp/, we don’t need the use_sudo=True.

Another example:

def ssh_config():
    put("path/to/edited/ssh_config","/etc/ssh/ssh_config", use_sudo=True)

Run:
fab ssh_config

You can also set file modes in your file transfer tasks like this:

def file_send_mod(localpath,remotepath):
    put(localpath,remotepath,mode=0755)

or use this to keep the local mode:

def file_send_oldmod(localpath,remotepath):
    put(localpath,remotepath,mirror_local_mode=True)

or by setting the mode as parameter:

def file_send_mod(localpath,remotepath,mod):
    put(localpath,remotepath,mode=int(mod, 8))

The number 8 is there so that we can use octal numbers with file mode (chmod) such as 755.

You can run it with:
fab file_send_mod:testfile,/tmp/testfile,777

Getting files

Retrieving files from the remote machines is as simple as sending them. But we might want to add a small
thing that will tell us from which machine did the file come from.

def file_get(remotepath, localpath):
    get(remotepath,localpath+"."+env.host)

Now if we run it with:
fab get_file:/var/log/auth.log,/tmp/auth.log

We will get files that have the hostname in the filename eg.
auth.log.10.10.10.10
auth.log.webserver.local

If we didn’t use this .env.host in the path we’d get just one file which would be rewritten by each host that the task was ran on.

Now you’r able to send files to remote hosts and download files from them. Next some error handling.

Error handling

Sometimes one of your remote hosts is not reachable. Fabric normally aborts the execution if it can’t
reach one of the hosts. Since Fabric 1.4.1 it’s now possible to skip the offline hosts. Just add this
to your fabfile:

env.skip_bad_hosts=True

And you should also add a timeout for the connection check. Default is 10 seconds but even 1 or 2 seconds has
worked well for me. Add 2 second timeout like this:

env.timeout=2

For some reason this host skipping seemed to give me some error messages with parallel execution but it still worked fine.

Fabric will also abort execution if it encounters an error during task execution. We can set Fabric to only give a warning instead of aborting when something fails.

env.warn_only=True

or only set it for certain commands:

def cmd(cmd):
    with settings(warn_only=True):
        run(cmd)

Now that the execution won’t abort during failures. We can add some reactions to fails.

def cmd(cmd):
    with settings(warn_only=True):
        if run(cmd).failed:
            sudo(cmd)

Now if the defined command fails, it will be run again with sudo.

With these first two basic guides you should be able to control your remote devices with Fabric. There’s still
many features in Fabric but these were the most important ones. The examples were very simple but you
can modify and combine them to make very complicated tasks that fit your needs.

The complete fabfile of this tutorial:
fabfile.py

from fabric.api import *

env.hosts=["simo@10.10.10.10", "webserver.local"]
env.user="hng"
env.skip_bad_hosts=True
env.timeout=2
#env.warn_only=True
env.parallel=True

def file_send(localpath,remotepath):
    put(localpath,remotepath,use_sudo=True)

def file_send_mod(localpath,remotepath,mod):
    put(localpath,remotepath,mode=int(mod, 8))

def file_get(remotepath, localpath):
    get(remotepath,localpath+"."+env.host)

@serial
def cmd(cmd):
    with settings(warn_only=True):
        if run(cmd).failed:
            sudo(cmd)

Leave a comment if you have questions. See fabfile.org for official Fabric documentation.

Next Fabric tutorial: Fabric Tutorial 3 – Settings and roles

Sources:
http://fabfile.org

3 responses to “Fabric tutorial 2 – File transfer & error handling

  1. Pingback: Fabric – Wrappers and more error handling « awaseroot

  2. noob September 29, 2014 at 6:43 pm

    Thanks for sharing this with us!

  3. George December 23, 2014 at 1:21 am

    What a well written article. Thank you so much.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: