When Things Go Wrong
Finding out What Went Wrong
When an Analysis step runs, it produces a warnings file (named
warnproject.txt) in the spec file's directory. Generally, most of these warnings are harmless. For example,
os.py (which is cross-platform) works by figuring out what platform it is on, then importing (and rebinding names from) the appropriate platform-specific module. So analyzing
os.py will produce a set of warnings like:
W: no module named dos (conditional import by os)
W: no module named ce (conditional import by os)
W: no module named os2 (conditional import by os)
Note that the analysis has detected that the import is within a conditional block (an
if statement). The analysis also detects if an import within a function or class, (delayed) or at the top level. A top-level, non-conditional import failure is really a hard error. There's at least a reasonable chance that conditional and / or delayed import will be handled gracefully at runtime.
Ignorable warnings may also be produced when a class or function is declared in a package (an
__init__.py module), and the import specifies package.name. In this case, the analysis can't tell if name is supposed to refer to a submodule of package.
Warnings are also produced when an
eval statement is encountered. The
__import__ warnings should almost certainly be investigated. Both
eval can be used to implement import hacks, but usually their use is more benign.
Any problem detected here can be handled by hooking the analysis of the module. See Listing Hidden Imports below for how to do it.
Getting Debug Messages
debug=1 on an
EXE will cause the executable to put out progress messages (for console apps, these go to
stdout; for Windows apps, these show as
MessageBoxes). This can be useful if you are doing complex packaging, or your app doesn't seem to be starting, or just to learn how the runtime works.
Getting Python's Verbose Imports
You can also pass a -v (verbose imports) flag to the embedded Python. This can be extremely useful. I usually try it even on apparently working apps, just to make sure that I'm always getting my copies of the modules and no import has leaked out to the installed Python.
You set this (like the other runtime options) by feeding a phone TOC entry to the EXE. The easiest way to do this is to change the EXE from:
EXE(..., anal.scripts, ....)
EXE(..., anal.scripts + [('v', '', 'OPTION')], ...)
These messages will always go to stdout, so you won't see them on Windows if
Helping Installer Find Modules
Extending the Path
When the analysis phase cannot find needed modules, it may be that the code is manipulating
sys.path. The easiest thing to do in this case is tell Analysis about the new directory through the second arg to the constructor.
In this case, the Analysis will have a search path:
anal = Analysis(['somedir/myscript.py'],
['somedir', 'path/to/thisdir', 'path/to/thatdir'] + sys.path
You can do the same when running
Makespec.py --paths=path/to/thisdir;path/to/thatdir ...
(on *nix, use
: as the path separator).
Listing Hidden Imports
Hidden imports are fairly common. These can occur when the code is using
__import__ (or, perhaps
eval), in which case you will see a warning in the
warnproject.txt file. They can also occur when an extension module uses the Python/C API to do an import, in which case Analysis can't detect anything. You can verify that hidden import is the problem by using Python's verbose imports flag. If the import messages say "module not found", but the
warnproject.txt file has no "no module named..." message for the same module, then the problem is a hidden import.
Hidden imports are handled by hooking the module (the one doing the hidden imports) at Analysis time. Do this by creating a file named
hook-module.py (where module is the fully-qualified Python name, eg,
hook-xml.dom.py), and placing it in the
hooks package under Installer's root directory, (alternatively, you can save it elsewhere, and then use the
hookspath arg to
Analysis so your private hooks directory will be searched). Normally, it will have only one line:
When the Analysis finds this file, it will proceed exactly as though the module explicitly imported
hiddenimports = ['module1', 'module2']
module2. (Full details on the analysis-time hook mechanism is here).
If you successfully hook a publicly distributed module in this way, please send me the hook so I can make it available to others.
Extending a Package's __path__
Python allows a package to extend the search path used to find modules and sub-packages through the
__path__ mechanism. Normally, a package's
__path__ has only one entry - the directory in which the
__init__.py was found. But
__init__.py is free to extend its
__path__ to include other directories. For example, the
win32com.shell.shell module actually resolves to
win32com/win32comext/shell/shell.pyd. This is because
../win32comext to its
__init__.py is not actually run during an analysis, we use the same hook mechanism we use for hiddenimports. A static list of names won't do, however, because the new entry on
__path__ may well require computation. So
hook-module.py should define a method
mod argument is an instance of
mf.Module which has (more or less) the same attributes as a real
module object. The
hook function should return a
mf.Module instance - perhaps a brand new one, but more likely the same one used as an arg, but mutated. See mf for details, and
hook/hook-win32com.py for an example.
Note that manipulations of
__path__ hooked in this way apply to the analysis, and only the analysis. That is, at runtime
win32com.shell is resolved the same way as
win32com.__path__ knows nothing of
Once in awhile, that's not enough.
Changing Runtime Behavior
More bizarre situations can be accomodated with runtime hooks. These are small scripts that manipulate the environment before your main script runs, effectively providing additional top-level code to your script.
At the tail end of an analysis, the module list is examined for matches in
rthooks.dat, which is the string representation of a Python dictionary. The key is the module name, and the value is a list of hook-script pathnames.
So putting an entry:
rthooks.dat is almost the same thing as
except that in using the hook,
anal = Analysis(['path/to/somescript.py', 'main.py'], ...
path/to/somescript.py will not be analyzed, (that's not a feature - I just haven't found a sane way fit the recursion into my persistence scheme).
Hooks done in this way, while they need to be careful of what they import, are free to do almost anything. One provided hook sets things up so that
win32com can generate modules at runtime (to disk), and the generated modules can be found in the
Adapting to being "frozen"
In most sophisticated apps, it becomes necessary to figure out (at runtime) whether you're running "live" or "frozen". For example, you might have a configuration file that (running "live") you locate based on a module's
__file__ attribute. That won't work once the code is packaged up. You'll probably want to look for it based on
run executables set
sys.frozen=1 (and, for in-process COM servers, the embedding DLL sets
For really advanced users, you can access the
sys.importManager. See iu for how you might make use of this fact.
Accessing Data Files
--onedir distribution, this is easy: pass a list of your data files (in
TOC format) to the
COLLECT, and they will show up in the distribution
directory tree. The
name in the
(name, path, 'DATA') tuple can be a relative
path name. Then, at runtime, you can use code like this to find the file:
--onefile, it's a bit trickier. You can cheat, and add the files to the
BINARY. They will then be extracted at runtime into the work directory by the C code (which does not create directories, so the
name must be a plain name), and cleaned up on exit. The work directory is best found by
os.environ['_MEIPASS2']. Be awawre, though, that if you use
--upx, strange things may happen to your data -
BINARY is really for shared libs / dlls.
If you add them as
'DATA' to the
EXE, then it's up to you to extract them.
Use code like this:
to get the contents as a binary string. See
import sys, carchive
this = carchive.CArchive(sys.executable)
data = this.extract('mystuff')
support/unpackTK.py for an advanced example (the TCL and TK lib files are in a
PKG which is opened in place, and
then extracted to the filesystem).
Report bugs (or feature requests) here. Please make sure you set Product to
Installer. (If you choose not to become a registered user, please include some contact information in the report itself.) Subscribe to the Installer Mailing List to discuss Installer related issues. And don't forget to show your appreciation here.
Pmw comes with a script named
bundlepmw in the
bin directory. If you follow the instructions in that script, you'll end up with a module named
Pmw.py. Ensure that Builder finds that module and not the development package.
If you're using
popen on Windows and want the code to work on Win9x, you'll need to distribute
win9xpopen.exe with your app. On older Pythons with Win32all, this would apply to
win32popenWin9x.exe. (On yet older Pythons, no form of popen
worked on Win9x).