Distributing and Deploying a Linux Python Application

After finishing a large application for a couple clients and having it thoroughly tested on development machines, the hardest part was yet to come. We had no control over their environment. We tested with Python 2.7 but they could have 2.4, which some of our dependencies couldn’t handle. It isn’t right to force clients to upgrade or overwrite their current Python installation, so we had to have the least intrusive installation as possible. In the end, we found one. Below is how we achieved it.

Custom Built Version of Python

This may sound extreme, but distributing Python alongside your application is the only way to ensure that the client has a compatible version. For libraries or small Python modules, this is inappropriate but when your clients are few and high paying, you learn to work on their turf.

In our situation, we had to not only distribute the Python code but we actually had to modify it to include SSL support. The fix was simple to add SSL, uncomment the edit the SSL variable in Modules/Setup. This did require openssl and openssl-devel packages to be installed. I’ve talked about SSL in Python before, hopefully I won’t have to in the future as it will be so simple.

Include All Dependencies in Distribution

Assuming that the client has an internet connection would be too much. Our application dealt somewhat with sensitive material as well, so we couldn’t pip install our dependencies like your average Joe. Instead, the solution is to give pip file paths to install. Our dependencies were stored in a directory called “deps” so pip install deps/* did the trick.

The next task is to ensure that these dependencies don’t mess with what is installed on the system so a virtual environment is needed. Unfortunately, since hardly anything can be assumed, the source to virtualenv must be distributed as well.

Installation

LOC="path/to/app"

# Extract Python onto client machine and set Python to current directory

# Configure Python to be installed in an alternate location and for it to not
# override the current installation. I recommend running make -j {number of
# cores} so that compiling is sped up.
./configure --prefix="${LOC}/python"
make
make altinstall

# With Python installed, we now how to install our virtual environment with our
# version of Python because there is a possibility that the current installation
# is too old or not compatible. Then set --python to point to our Python
# installation so that inside the virtual environment, we use the right version.
# I'm assuming that current directory is pointing to the virtual environment
# source directory
${LOC}/python/bin/python2.7 virtualenv.py "${LOC}/pythonEnv/" --python="${LOC}/python/bin/python2.7"

cd ${LOC}

# Activate the virtual environment so everything here on will be effect the
# local version of Python
source pythonEnv/bin/activate
pip install deps/*

I left it purposely ambiguous whether or not the custom Python version and the virtual environment are recreated for each version of our app’s release. Our current decisions is that, given that updates won’t be often, to always distribute Python and virtual environment and have them installed into a new subdirectory. The obvious downside is the amount of disk used, but it gives us ultimate flexibility. If a new version of Python or virtual environment is released with either needed features or bug fixes, we can deploy these without effecting previous installations. Thus if the new versions don’t pan out, it is trivial to reset to a previous version.

I would like to conclude by reiterating the installation technique showed here should be the exception. The code we were deploying would be sent to drastically varying environments – even compromised environments, and to ensure complete isolation sometimes it takes a bit of extremism in installation.