Skip to content.

plope

Personal tools
You are here: Home » Members » chrism's Home » Imperative Configuration in BFG 1.2
 
 

Imperative Configuration in BFG 1.2

A short description of the "imperative configuration" mode present in BFG 1.2+.

Yesterday I blogged about the view predicates feature of repoze.bfg 1.1 . BFG 1.1 was released several months ago, and since that time, we've had a good number of alpha releases of the subsequent major release, 1.2. The most recent version in the 1.2 line as of this writing is 1.2a9. 1.2 final is due out within the next few weeks.

The most important new feature in the BFG 1.2 line is "imperative configuration". "Imperative configuration" is a funny, loaded term so let's take it one word at a time.

Imperative: this term contrasts a requirement in previous releases that configuration be at least partially declarative. In BFG releases prior to 1.2, it was necessary to have at least two files for any given application: a Python file representing the code and a ZCML file.

Configuration: "configuration" in terms of BFG means the stuff that wires up specific URLs to specific views, or that wires up up event subscribers to event emissions, etc. The stuff that turns the BFG framework into a particular application deployment.

The declarative configuration "story" provided by BFG is stolen almost entirely from Zope. The zope.configuration package is used to process declarations made in ZCML files. These declarations mutate an "application registry", adding various registrations as the ZCML files are processed. At the end of processing, the application registry represents all the stuff needed to run a particular application.

In BFG 1.0 and 1.1, you were required to have at least one ZCML file per application. This ZCML file, in turn, needed to contain at least one statement in it: one which kicked off a "scan", which is a process that scans a package or module for view configuration decorators. Alternately, you were permitted to disuse view decorators entirely and configure view mappings via <view> declarations.

The requirement that a ZCML file exist in 1.0 and 1.1 was a holdover from BFG's Zope heritage. In BFG 1.2+, it is possible to create an application entirely in Python without any ZCML (or even any decorators) in sight. For example:

   from webob import Response
   from paste.httpserver import serve
   from repoze.bfg.configuration import Configurator

   def hello_world(request):
       return Response('Hello world!')

   def goodbye_world(request):
       return Response('Goodbye world!')

   if __name__ == '__main__':
       config = Configurator()
       config.begin()
       config.add_view(hello_world)
       config.add_view(goodbye_world, name='goodbye')
       config.end()
       app = config.make_wsgi_app()
       serve(app)

If you put this stuff in a Python file, and invoke it with an interpreter that has the repoze.bfg software installed, you'll get a running helloword application. The add_view method of the configurator does effectively the same thing as a ZCML <view> declaration; in fact the ZCML view directive code calls into this method when a ZCML <view> declaration is found.

While it's sort of neat to be able to run an application as a single file, it is sort of a "gee whiz" feature that isn't very useful in practice. Every sizeable application will eventually need to split itself across multiple files. But the act of making this possible really helped me clean up the code a lot; rather than coding for configurability via ZCML, and then adding weird sort of stubs for testability, the Configurator can be used to configure both the application and test setup for an application in exactly the same way. Likewise, lots and lots of code got centralized into the configuration module (a lot removed from code that drove various ZCML directives), and we were able to document the resulting API much more effectively and cleanly.

The end result? Well, now, truly, if you don't want to use ZCML, you needn't. Ever. But even so we didn't wind up with a system that leads you towards making a false choice between "convention over configuration" and ZCML: both the declarative and the imperative configuration modes are still extremely explicit (they are really mirror copies of each other, as the declarative code uses the imperative API). The API used by imperative configurators is not "bare" ZCA registrations: it's just more domain specific, like helper ZCML directives in Zope. And hopefully we've brought down to earth the kinds of configuration tasks you can potentially perform, because they're now all enumerated in the Configurator API documentation and via various narrative documentation chapters. All of the older declarative/ZCML configuration still works as it did in 1.0/1.1, so there's no backwards compatibility concerns either. ZCML will always be a reasonable way to configure a BFG app.

I think 1.2 is the best of the bunch as far as BFG releases go, because, although BFG still has an obvious Zope heritage, 1.2 unglosses over some of the things that Zope has been glossing over for a long time, such as the relationship from the code in a ZCML directive to code that needs to be invoked imperatively. BFG also eschews some dogma-driven assumptions Zope has been making for a long time. When I say this I am reminded of this passage from the ZCML chapter of Stephan Richter's Zope 3 Developer's Handbook :

  While the developer is certainly the one that writes the initial cut of the 
  configuration, this user is not the real target audience. Once the product
  is written, you would expect a system administrator to interact much more 
  frequently with the configuration, adding and removing functionality or 
  adjust the configuration of the server setup. System administrators are 
  often not developers, so that it would be unfortunate to write the 
  configuration in the programming language, here Python. But an administrator 
  is familiar with configuration scripts, shell code and XML to some extend. 
  Therefore an easy to read syntax that is similar to other configuration 
  files is of advantage.

I think it's safe to say that this assumption has turned out to be just false. Sysadmins never change ZCML files. I have never, ever, not once in the last seven years, had a sysadmin with enough context that allowed him to change a ZCML file in any meaningful way. Only developers change ZCML files. So let's just un-dogma-ify this assumption, and put programmers back in charge to whatever extent they'd like to be. Demystification of this stuff is the only reasonable way forward, even if it means admitting we made really big design mistakes in the past.

Created by chrism
Last modified 2010-01-01 12:01 PM

Thanks

I'm really enjoying this series of BFG internal design explanations. :)

Personally, I think there's still a place for ZCML, even if some of the justifications used in old (and, sadly, largely unmaintained and so woefully out of date and not necessarily representative of the current views of the Zope community) seem hollow. I don't think the assumption that ZCML for per-package configuration would be changed by sysadmins would ever have stood up to scrutiny. However, there are some things that are done in ZCML, which I think are useful to keep out of Python code. These may also have a place for integrators (not sysadmins) to be able to configure or override without having to modify or even deeply understand others' Python code.

Personally, these days I prefer working with the factored-out-of-grok convention-over-configuration style of component writing. Things like grokcore.component for ZCA configuration and grokcore.view for view configuration. But I also eschew some of the things that Grok encourages you to do in Python, such as registering resource directories or permissions. Those things are pure configuration, and belong outside Python. Compare that with, say, an adapter registration, which is imperative Python code that requires some declarations to signal its use by the framework. I'm not very comfortable creating pseudo-classes in Python that are never instantiated, but are simply a place to hang directives.

One of the things that people may do in a "pure" Zope 3 application is to configure things like permissions and principals in ZCML via a central site.zcml. For that style of application, that's probably a good use of ZCML and may even be configurable by a sysadmin. I think that we've moved towards such things being configured through the web or inferred from the packages that make up a system, though, which means it may be less common. But I do also think it's quite useful to have a "canonical" configuration syntax and a sensible framework (in zope.configuration) for managing that configuration. I just don't think it's ideal to split application-specific wiring of components into the framework into two places: a Python file with the logic and an XML file with the wiring itself.

Martin

cant afford to have an opinion

Glad you're enjoying it!

I guess as a rule, I often can't really afford to have an opinion
about whether some configuration style is "better" in any specific
circumstance. The answer is always "it depends".

In particular, I don't think imperative vs. declarative configuration
is an either-or choice. Using ZCML is good for creating frameworks
and other extensible applications. Grok-style declarative
configuration is good for providing better locality of reference to
configuration statements. But both under the hood eventually call
into imperative code to get their jobs done. Exposing this imperative
code via a nice API I think is useful, because it centralizes and
canonizes the definition of what the framework actually does.

The framework shouldn't make an arbitrary decision about which sort of
configuration goes where based on opinion; I'd rather just make all of
it available in both an imperative and a declarative form and let
folks make up their own minds about which kind they prefer. I don't
believe it's my place (nor am I really smart enough) to impose any
limitations on how a developer chooses to work with the framework. If
he wants to use all-imperative config, fine. If he chooses to use
all-declarative config, same. If he chooses a hybrid, also fine.
It's easy enough to offer him the option of using one or the other, or
both. The BFG docs do recommend using ZCML, but this is only because
most examples already in the wild use ZCML; recommending ZCML isn't a
statement of opinion.

I also think a lot of the talk about servicing "integrators" just sort
of muddies the waters. Integrators are really just inexperienced
programmers; the fact that they don't really *want* to be programmers
doesn't really change the fact that they are. Limiting the set of
configuration options available to all programmers based on an subset
in their ranks is sometimes useful, but not *always* useful.
Sometimes its nice to be able to just bash to fit without taking an
integrator into account, because taking the integrator into account
often leads to premature generalization. On many projects, there is
exactly no one playing the integrator role at all; on these sorts of
projects, I think it makes sense to ignore the integrator until a
genuine person playing this role appears. It's not like you can't
later change the code to be more general, and make use of ZCML, and
other bits that make parts of the application overrideable.

All that said, to me, it's obvious in hindsight that a "canonical"
configuration API should be an imperative API, and the declarative
helper stuff should just use it.