Distributing Python Apps

What does it mean to Distribute a Python App?

Distributing a pure Python app to other Python users usually means sending out a source archive and some unpacking instructions, or using distutils to do this for you.

At the other end of the spectrum is sending out native executables, with all Python origins hidden. In between come ideas like archives of compiled Python, and minimal private Python installations. Complexity depends on both how many dependcies your app has, and how completely you want to shield your users from those dependencies.

The following pages describe my solutions (borrowing heavily from others) to these problems.

Archives

I took Fredrik's Squeeze idea, and Greg Stein's Small distribution and combined them. The idea was to use modulefinder from Freeze, and package the results into an archive file. Then import hooks (courtesy of Greg Stein's imputil) were installed (one line of code) which let code be imported directly from the archive. (At this point, I have written my own replacements for modulefinder and imputil.) The result is cross-platform, small and fast (much less I/O for an import). More details on archives here.

Standalones

The next step was to create small "embedding" apps (based on ideas from Christian Tismer, with help from Thomas Heller) that understand archives, and can almost completely isolate the Python environment of your app from anything the user might have installed.

For Windows, depending on #DEFINES, you get either a console app or a GUI app. Binaries are included, so Windows users don't need a C compiler. For Linux and Unix you build the app once, based on your local Python configuration. More details on the executables here.

Packaging

Finally, I created a system for assembling archives and standalones in just about any configuration you can imagine. You can:

• Pack everything into one self-extracting (and self-cleaning) executable (including all the dlls).
• Pack the pure Python together in an archive, attach this to the "embedding" app, and distribute this and it's binary dependencies separately, (this is the default).
• Do either of the above with the archive and executable separate (for non-ELF platforms).
• Package up COM servers, both as exe's and as dll's (Windows only, of course).

The package can be downloaded here.

Installer

• Release 5
• Download

Other Solutions

Freeze

The source distribution comes with a tool called "Freeze". It analyzes the script you want to distribute and (to the best of its ability) finds all the module dependencies of your script. These are then compiled, and the byte-code frozen into enormous C byte arrays. These arrays, and a module called "frozenmain.c" are compiled and linked to the python library, and the result is the executable you distribute.

If the Python library it links to is completely static, the result is completely standalone, undoubtedly large, and unable to load dynamic extension modules.

If the Python it links against was built with some modules declared *SHARED* then it should be able to load dynamic extension modules, but your job will be harder, because you now have to distribute the required shared libraries, and make sure that your executable finds the ones you distributed, not the ones from some other Python installation.

Of course, on Windows, all of Python is dynamically linked, and Python's rules about finding things are a bit different.

Squeeze

Fredrik Lundh of Pythonware wrote Squeeze a number of years ago, which takes a different approach. Squeeze packages up compiled byte code into an archive, and uses a special script to open an run the stuff in the archive. The user must have Python installed, but once they do, distribution and installation of a Squeezed script (and supporting modules) is a no-brainer. Especially since the result is cross-platform. Of course, if you rely on C extensions, you have some more work to do.

Python2C

Bill Tutt and Greg Stein developed something called Python2C, now officially abandonned. This translated Python to C code. Mostly, the C code ended up manipulating PyObjects in much the same way as if you had recreated the Python code in C using the high level Python / C API. That is, you got a speed up from the fact that there was no byte-code interpreter loop, but that's less than most people expect (maybe 15%).

py2exe

When I was slow in coming out with a Python 2.0 version of Installer, Thomas Heller created py2exe. It uses distutils to create a directory with an exe and the required dlls and pyds, (like my --onedir configuration). Windows only, and you'll need a separate build for each Python version.

Pyco

Pyco is a recent Windows-only entry that uses a much simpler scheme to package and ship interpreter + scripts. One neat capability is adding modules to the executable dynamically. Dependency tracking is up to you.