Build artifacts with pex

  |   Source

For Continuous Integration in my Python application (as opposed to library) projects, I generally want to run my tests as well as build a Docker image, making use of the same artifacts and environment for both: I want to test what I'm actually deploying, rather than something merely similar. Previously this meant doing pip install twice; once into a virtualenv to run the tests, then again in the Docker image build. Sharing the pip cache between these steps speeds things up a lot, but this still ends up using quite a bit of time on network roundtrips etc.

Now that pex works with pypy, I have developed a slightly better workflow for this. Briefly speaking, Pex is a tool for assembling a Python application into a single runnable file that embeds the dependencies of the application; at runtime, the dependencies are ziploaded or extracted to a temporary location as necessary to run the application.

The workflow:

  1. Build a pex file.
  2. Run the tests against the pex file.
  3. Copy the pex into the Docker image.

This is similar to what I would do with a language like Go or Haskell that would produce a single executable as part of the build process.

Comments powered by Disqus