Build artifacts with pex
For Continuous Integration in my Python application (as opposed to library) projects, I generally want to run my tests as well as build a Docker image, making use of the same artifacts and environment for both: I want to test what I'm actually deploying, rather than something merely similar. Previously this meant doing pip install twice; once into a virtualenv to run the tests, then again in the Docker image build. Sharing the pip cache between these steps speeds things up a lot, but this still ends up using quite a bit of time on network roundtrips etc.
Now that pex works with pypy, I have developed a slightly better workflow for this. Briefly speaking, Pex is a tool for assembling a Python application into a single runnable file that embeds the dependencies of the application; at runtime, the dependencies are ziploaded or extracted to a temporary location as necessary to run the application.
The workflow:
- Build a pex file.
- Run the tests against the pex file.
- Copy the pex into the Docker image.
This is similar to what I would do with a language like Go or Haskell that would produce a single executable as part of the build process.