Fabric tutorial 2 – File transfer & error handling
April 25, 2012Posted by on
On the first turorial we learned to run commands on remote hosts with Fabric. Now we move on to
transfering files. Transfering new configuration files is usually quite important part of system administration.
Also retrieving log files from the remote machines might be useful.
Let’s assume we’ve made a new ssh_config file with important changes and we want to send it to our
remote hosts. Here’s a task for sending files.
def file_send(localpath,remotepath): put(localpath,remotepath,use_sudo=True)
Run it with:
or if the modified ssh_config is in the directory where you’re running Fabric:
If we’re sending the file to a location that doesn’t need sudo eg. /tmp/, we don’t need the use_sudo=True.
def ssh_config(): put("path/to/edited/ssh_config","/etc/ssh/ssh_config", use_sudo=True)
You can also set file modes in your file transfer tasks like this:
def file_send_mod(localpath,remotepath): put(localpath,remotepath,mode=0755)
or use this to keep the local mode:
def file_send_oldmod(localpath,remotepath): put(localpath,remotepath,mirror_local_mode=True)
or by setting the mode as parameter:
def file_send_mod(localpath,remotepath,mod): put(localpath,remotepath,mode=int(mod, 8))
The number 8 is there so that we can use octal numbers with file mode (chmod) such as 755.
You can run it with:
Retrieving files from the remote machines is as simple as sending them. But we might want to add a small
thing that will tell us from which machine did the file come from.
def file_get(remotepath, localpath): get(remotepath,localpath+"."+env.host)
Now if we run it with:
We will get files that have the hostname in the filename eg.
If we didn’t use this .env.host in the path we’d get just one file which would be rewritten by each host that the task was ran on.
Now you’r able to send files to remote hosts and download files from them. Next some error handling.
Sometimes one of your remote hosts is not reachable. Fabric normally aborts the execution if it can’t
reach one of the hosts. Since Fabric 1.4.1 it’s now possible to skip the offline hosts. Just add this
to your fabfile:
And you should also add a timeout for the connection check. Default is 10 seconds but even 1 or 2 seconds has
worked well for me. Add 2 second timeout like this:
For some reason this host skipping seemed to give me some error messages with parallel execution but it still worked fine.
Fabric will also abort execution if it encounters an error during task execution. We can set Fabric to only give a warning instead of aborting when something fails.
or only set it for certain commands:
def cmd(cmd): with settings(warn_only=True): run(cmd)
Now that the execution won’t abort during failures. We can add some reactions to fails.
def cmd(cmd): with settings(warn_only=True): if run(cmd).failed: sudo(cmd)
Now if the defined command fails, it will be run again with sudo.
With these first two basic guides you should be able to control your remote devices with Fabric. There’s still
many features in Fabric but these were the most important ones. The examples were very simple but you
can modify and combine them to make very complicated tasks that fit your needs.
The complete fabfile of this tutorial:
from fabric.api import * env.hosts=["email@example.com", "webserver.local"] env.user="hng" env.skip_bad_hosts=True env.timeout=2 #env.warn_only=True env.parallel=True def file_send(localpath,remotepath): put(localpath,remotepath,use_sudo=True) def file_send_mod(localpath,remotepath,mod): put(localpath,remotepath,mode=int(mod, 8)) def file_get(remotepath, localpath): get(remotepath,localpath+"."+env.host) @serial def cmd(cmd): with settings(warn_only=True): if run(cmd).failed: sudo(cmd)
Leave a comment if you have questions. See fabfile.org for official Fabric documentation.
Next Fabric tutorial: Fabric Tutorial 3 – Settings and roles