Skip to content.

plope

Personal tools
You are here: Home » Members » chrism's Home » Import Time Side Effects and You
 
 

Import Time Side Effects and You

Let's learn about import time side effects.

I have blogged before about this topic, but what the hell, I'll do it again.

Lots of people want to "use Python" for everything. They'd like to use Python for configuration files, for populating external registries as the result of some import statement, and other things. They inevitably define "use Python" as "put lots of code at module scope in a Python module". I don't really mind this so much, but please, if you do, don't make me use the result. In particular, don't make me use a framework or library in which this behavior is relied upon.

There are reasons that every credible book about Python, in the first chapter, when describing an executable Python script, places the following kind of clause in the script, which allows execution when a module is a script, but prevents it when it is imported:

  if __name__ == '__main__':
      .. do stuff ..

Since the script may (at some point) be imported as a module by another module, and the script author does not want the code inside the "if" block to be executed at import time (in the case where the script is imported by another, __name__ will not be __main__), it makes sense to code "defensively" like this.

Many people seem to forget this "my first Python" lesson when developing larger systems. It's usually insane for the import of a module to have side effects other than the following:

  • An import of another module or global.
  • Assignment of a variable name in the module to some constant value.
  • The addition of a function via a def statement.
  • The addition of a class via a class statement.
  • Control flow which may handles conditionals for platform-specific handling or failure handling of the above.

Any other sort of logic inside the top level execution path of a Python module (any code that would be executed during "import") should be regarded with great suspicion and perhaps even loathing.

The codepath represented by top-level statements in modules is owned by Python, not by you. Python is in charge of its execution; you are not. You can never predict when it will be executed.

Here's an example of such an antipattern (code is at module scope):

   import atexit
   import logging

   def _exit_function():
       logging.info('process shutting down')

   atexit.register(_exit_function)

This is an example I've taken (in spirit) from a module in the Python standard library.

First of all, any library code that thinks its smart enough to register an atexit function is mistaken. Only an application (and not a library) can possibly be smart enough to register an atexit function, because only an application is responsible for controlling the horizontal and vertical of application startup and shutdown, not any of the libraries employed by the application.

That's a bit besides the point, though. The real problem here is that the code which performs the atexit.register registration executes at module scope. This means that when this module is imported, it has the side effect of registering an atexit function. But what if this module just happened to be imported by another "innocently", but its functionality never used?

It's totally absurd to try to build a system that configures or starts itself based on import time side effects. When you build a system that relies on import time side-effects, you have no control over when the code gets executed: it might get imported much earlier, or much later than you expect it to be. You may not even have any control over why the code is executed: some code scanner, such as a testing tool, may import it, and may fail as a result, possibly at shutdown.

If you do this sort of thing, great. Just keep it to yourself and wash your hands afterwards. And maybe don't do it. OK. Just don't do it; that's the best solution.

Created by chrism
Last modified 2009-10-19 01:43 AM