The Zope community currently asserts that there is a benefit to using the current set of conventions and APIs that forms the Zope Component Architecture as a mechanism: use of its mechanisms and concepts can produce a system of higher quality and one which is more understandable than a system built without these mechanisms and concepts.

The promised benefits of obeying the conventions and constraints of the ZCA are:

• A system based on use of the component architecture machinery is "pluggable", meaning that implementations can be swapped out for one another as necessary without changing consumer code.
• If you've paid enough attention to detail over time that the contract spelled by an interface still matches the concrete implementations of the interface, the interface forms a natural set of obviously-correct documentation for that component. For a large system where there is an existing body of interface documentation, it is on average easier for someone new to the system to quickly get productive.
• The concept of adaptation, where one object can be converted to another "type" through introspection of its interfaces, allows the average developer to understand the interaction between discrete components in a large system more easily than in a comparable system which doesn't use adaptation.

I hope to explain via this blog entry that the first promise is completely true, the second is at least half-true, and the third one is just plain false.

The constraints and conventions required by the ZCA are these:

• "Interfaces" are type markers that have metadata.
• An object may declare that it "implements" one or more interfaces.
• An interface may optionally specify a contract: objects which implement the interface are presumed to offer this contract.
• "Adaptation" is the act of converting an object that implements one interface to an object that implements another interface.

Let's compare the promises of following these constraints and conventions to their actual benefits.

Components as Plugpoints

Use of a ZCA "component" as a plugpoint works extremely well. These uses include: testing dependency injection, view overrides, and other extremely high-level policy plugpoints. The ZCA really shines here, as long as you use it in small doses and take care to document the resulting system narratively in terms that don't necessarily require an understanding of component architecture concepts.

I am obviously biased, but I believe repoze.bfg is such a system. However, it actually does not use any documented API defined within in the zope.component package itself. It just uses a ZCA registry as machinery and disuses much of the worldview surrounding the traditional definition of adaptation as documented in various tutorials about the ZCA. It doesn't try to use interfaces as documentation, either. The API docs are generated from the implementation classes rather than from interfaces. The presence of interfaces at all is largely just an implementation detail.

Interfaces As Documentation and Component Creation

Interfaces can act as great documentation. If you're conscientious about maintaining the interface definition for some implementation (or set of implementations), there is incontrovertible benefit to interface documentation.

However, designing a good interface is hard. Can anyone really be expected to sit down and design a formal interface that represents the contract for some component before he even codes up the first implementation of that component and some tests for it? No, of course not. Just can't happen, sorry, unless you've done this particular thing ten times before. API design is a highly iterative process: you throw ten revisions of an API away before you get one that is mostly right.

Therefore, you almost never want to promise that a particular component even has any interface until you've got that interface right in the context of some larger system full of consumers. Until you've got the interface right, the interaction between code and components that might implement some notional interface implied by a component is just an implementation detail. It might be an important implementation detail, but it's often OK if it stays one forever. Some components don't need their interfaces spelled out because they are purely an implementation detail. If there's only one or two consumers of a particular "component", it's probably OK for that component to not have a formal interface. A system that has fewer, more meaningful and highly formalized contracts is usually easier to understand than a system that has many small contracts that all have a high probability of changing radically over time, because the overhead of maintaining the interface definitions for smaller contracts makes them less likely to be true. Wrong definitions are worse than none.

So here we have this system, the ZCA, that uses interfaces as a primary mechanism to perform higher-level operations such as adaptation. You are required to declare an interface for some component to use those higher-level operations. This is required by the worldview that states that adaptation is the conversion of an object implementing one interface to another, period, full stop. Worse, once you define an interface, you are expected to maintain its definition over the lifetime of the project; ideally it should never fall out of sync with the true interface expected by the consumers of an object which implements it. This is usually unreasonable in systems with dozens or hundreds of interface definitions. On projects with many contributors over time, and many interfaces, it's not easy to ensure that interface definitions are both correct and comprehensive. You can define an interface as a plain "marker" without any attribute or method definitions. This essentially means that you promise no contract of an adaptation to this interface. But this is frowned upon in a worldview where the point of adaptation is to convert an object implementing one interface to an object implementing another. It's considered bad form, an "abuse", a cast to a void pointer.

Meanwhile, the adaptation machinery provided by the ZCA is useful in many contexts: particularly for dependency injection in unit tests, so it's extremely tempting to want to make use of it for this purpose alone. Almost all Zope-related projects use the ZCA for unit testing dependency injection. Defining "interfaces" for all your "components" is required there because it's the only way to get what you want for testing dependency injection purposes.

There should actually be very few true plugpoints in any system. Far, far, far fewer, by maybe even an order of magnitude, than those implied by use of adaptation in the typical Zope-related project. Probably nine out of ten uses of adaptation in ZCA-using application source code is there only to provide a place to hang some testing dependency injection point. These are extremely hard to tell apart from the "real" plugpoints in any given system, because they quack exactly like the plugpoint duck. Another questionable use for an interface include using one to look up a "utility" (especially an "unnamed" utility) where the utility is something that has an "obvious" API such as a dictionary. Documenting the API of this obvious thing in the interface definition is just silly. Interfaces tend to be added to any system of consequence for similarly questionable reasons.

So now at this point, if you've been following along, we've built a system that probably has incomplete documentation for its "components", because, really, there aren't any yet. Or maybe very few "true" components. We're not really smart enough to define any "components" yet. We may never be smart enough to define them properly. Our project just uses adaptation for several discrete purposes, none of them actually related to increasing comprehensibility or providing plugpoints: we've just hacked in some dependency injection points for unit tests that use the adapter machinery to good purpose. But we didn't do this to increase comprehensibility; we did it because it makes unit testing easier. We've thus effectively misdirected the casual code reader into believing that there are lots and lots of "components" with well-thought-out "interfaces" when really all we've been trying to do is to test the system more easily. The system is still in major flux, and its "interfaces" only represent some half-baked transitional state of affairs, probably not even correct. The contracts aren't truly formalized yet at all. But it looks pretty impressive.

A developer new to the project needs to be able to detect and discount the adaptations in the code done purely for unit testing dependency injection and convenience and the adaptations that imply true component plugpoints. That task is understandably difficult because we're using the same machinery for entirely different purposes.

As a result, we've actively subverted the original goal: to make the system more understandable.

Use of Adaptation Makes A System More Understandable

The idea that mere use of adaptation as a mechanism to improve large system comprehensibility for a new team member is utterly ludicrous. Mere use of the adaptation pattern adds no additional comprehensibility whatsoever to any system, small or large. Seeing "getMultiAdapter((foo, bar), IFrobnozz)" in code lends no particular insights as to the intent of the developer who wrote it. I'm not talking about syntax here either; the adaptation could be spelled "config['frobnozz](foo, bar)" and it would have the same level of comprehensibility in isolation.

Believing that mere use of any particular adaptation syntax in the code itself helps a new developer understand the intent of the adaptation is a complete fantasy. The only way to explain the intent of any particularly important adaptation is via hand-crafted narrative documentation explaining why some particular adaption takes place in a particular spot. The component architecture doesn't help at all there; you're on your own.

Writing software is hard. Writing software documentation is even harder. But it needs to be done. No tool is going to remove this requirement. Mere use of adaptation in code is no substitute whatsoever for writing high-level design documentation that explains how the system components work together. Using adaptation without explaining the "why" just hurts comprehensibility. Folks who pretend that they've made their system comprehensible simply by using adaptation are fooling themselves in the same way that folks who write doctests but don't test all their code and don't write narrative docs have fooled themselves into thinking they've written both good tests and good docs.

Worse, when we do use adaptation without explanation, a new developer must first go understand the machinery that implements adaptation to get a sense of the intent. That machinery is not particularly well-documented. There is no explanation of the lookup ordering that takes place when components are adapted at all, in particular.

As a result, we've actively subverted the original stated goal: to make the system more understandable.

Conclusions

I believe that following the current trajectory of "the ZCA" is unwise. I am happy to continue using the "zope.component" package, because its machinery works really, really well and it is useful in many scenarios, particularly for unit testing dependency injection and for building systems that can be considered "pluggable" after the fact (systems with additional documentation that explains the how and why of each true plugpoint). However, I no longer buy in to the idea that extremely casual use of the concept of adaptation defined formally as converting one interface to another laid on top of this machinery is of any benefit whatsoever when producing a large system in isolation.

I think we can improve this state of affairs via the following:

• Create a package with only machinery which can be used to implement an adaptation pattern in the package. Document the shit out of it, like it was for your grandma. Don't impose any particular "worldview" on consumers of this machinery: in particular, do not require the use of Zope interfaces as inputs. Maybe provide extra behavior (such as inheritance) when the inputs are interfaces, but allow for registration and lookup markers that are not interfaces.
• Base the API of what is currently zope.component on the machinery in the package I've suggested above. zope.component would be a consumer of this package, and would be free to impose the "an adaptation is the conversion of one interface to another" worldview on its consumers.
• Provide some tools that make it easier to do unit testing dependency injection without needing to declare formal interfaces (this might be the package in the first bullet point). Or at least think up a convention where it's possible for a casual source reader to immediately know that some adaptation is present only for test dependency purposes, as opposed to a true plugpoint.

I'd be willing to work on projects that go in these directions. I'm not willing to work on projects that take the current zope.component and add APIs to it without factoring out the underlying machinery bits, though.