Skip to content.

plope

Personal tools
You are here: Home » Members » chrism's Home » New Web Framework: repoze.bfg
 
 

New Web Framework: repoze.bfg

Another web framework never hurt anybody.

I've been working with Zope now for about nine years. While things have changed for the better by a lot with the introduction of interfaces and Zope 3 views, Zope's various object publishers have continued to make me sad. I like object publishing via graph traversal; it fits my obviously-warped-by-Zope brain. But the complexity of the Zope2 publisher is mindblowing (I recently reimplemented it ... it took about five months to get mostly-right). Zope 3's object publisher is a bit better but I still just don't like it very much. It just seems overcomplicated and slightly too close to Z2's publisher for my tastes.

In the meantime, as part of Repoze , I've been working a good bit with other Python web frameworks. All of them have things I really like. Django has great docs, and a nice view system. Pylons is steadfastly agnostic about how and what you should plug in to it. TurboGears is embracing third-party WSGI components in a big way.

But none of them have object publishing in the same vein as Zope does. Even TurboGears' object publishing story just isn't quite the same as Zope's, as you walk a code graph rather than a model graph. I could probably learn to live with URL-based dispatch ala Routes or Django's URL dispatcher, but I'm not quite sure I'm ready to give up my object-publishing blanket just yet; we also seem to do a lot of work where arbitrary-depth model hierarchies are important. At the same time, I just don't feel like continuing to use Z2 or Z3 is a viable option for me in the long term, mostly due to those frameworks' propensity for feature-creep and complexity. Z2 has been "berry, berry good to me", and Z3 technologies have made things bearable for the last few years, but I can't help feeling like in some way it's just time to move on.

So, for the last few weeks (at the expense of much else), with a lot of help from Paul Everitt and Tres Seaver, I've been working on a new web framework. This framework is named (provisionally) repoze.bfg ("Big Fine Gun"). It uses the Zope CA, and Zope interfaces, and a bunch of other Zope libraries, but doesn't use the Z2 or Z3 publishers; it has its own publisher. It also does not use any form of Zope security; it implements its own security model. It has the same concept of views that Zope and Django have. Its development model is closer to Pylons' or Django's than it is to Zope's. It uses the z3c.pt templating system, which is a reimplementation of ZPT, or you can also use XSL, or plug in whatever templating system you like really. It doesn't assume any particular persistence mechanism; you provide your own (ZODB, relational, filesystem, etc); in a tip of the hat to Pylons, that's considered an application decision. It depends heavily on a variety of Ian Bicking creations, including WebOb and Paste. It's very small, currently clocking in at a little over 2000 lines of code, minus dependencies. That said, it does about what I want out of a web framework.

Yup, the Python web framework field is a crowded one. But then again, I gotta admit don't care very much. I'm not really writing it for you. I'm writing it for me. If its existence confuses you, so be it, sorry. But I am sensitive to starting out right with documentation . I'm trying to be dogged about keeping those docs in sync with the code.

It's still in a pretty early stage, but it has definitely moved its way out of "toy" phase. We've managed to create several applications using the framework so far. The very first one was repoze.virginia , which is the application which serves up Repoze.org these days. It's a simple file hierarchy publisher with slight dynamicism. The second one Paul wrote, and it's named repoze.lxmlgraph , where he demonstrates how one might publish a website that was represented entirely by a single XML document. The third one is named repoze.cluegun , which is a port of Rocky Burt's ClueBin application to repoze.bfg. That code runs the ClueGun website . Of the three, ClueGun is probably the most real-world app (albeit small).

I'm sort of excited about it. It's sort of like getting out of jail or something to be able to have your own web framework to write as you see fit. We'll likely continue to develop applications using repoze.bfg, pushing features into the framework as we find them useful.

Created by chrism
Last modified 2008-07-21 01:05 AM

well done

What a coincidence. It was only last weekend when I decided to try leveraging the Zope interfaces and components within Django. I stopped short because I ended up needing to reimplement the Zope hierarchy which now dominates my way of thinking about webapps. So, this project is completely spot on. I wouldn't call it a new framework, rather I would call it a new python web configuration. This is good, because it'll hopefully allow better skills transfer.

My particular need for a non-zope environment is smaller projects which are hosted on shared servers, like dreamhosts, behinds fast cgi. While I've not built much of an app yet the bare-bones startup is quick, and easy to host behind fast cgi using flup.

Cheers guys

#!virtualenv/python
def make_app(global_config, **kw):
# paster app config callback
from repoze.bfg import make_app
from test.models import get_root
import test
app = make_app(get_root, test)
return app

app = make_app(None)
WSGIServer(app).run()

Congrats!

This is exciting...

You forgot to mention that it's FAST

I believe one of the top reasons for building "Da Gun" was performance: in his very first cut at it, Chris was getting 550 requests per second, on his laptop, when rendering a view against live data.

Since then, "freaking fast" is a mantra. Not to the extent that it ruins the development experience, which is a higher priority. Hence the choice of z3c.pt over regular ZPT, or at least, one of the factors in the decision.

Other tidbits: Chris wrote a Paster template to make getting started really easy. I wrote a tutorial where I document the writing of my sample app (the XML document publisher thingy). Tres added "pushpage" support.

Chris and I chatted yesterday, and the most important thing at this point is the docs, moreso than the code. Doing a killer job on docs would make the code a lot more valuable.

Model graph?

I'd like to hear more about model graphs versus object graphs. I have a feeling you could've written a CherryPy 3 dispatcher for that in a couple hours and been done.

turn that around?

Would you be perturbed if I turned that around and asked you to take a look at http://static.repoze.org/bfgdocs/narr/traversal.html#a-traversal-example and tell me how I'd go about doing that sort of thing in CherryPy 3? In bfg, no user code runs during traversal except the __getitem__ of model objects. If each of your models inherits from Python "dict", the framework walks a graph of model objects without consulting any custom app code. The app code is only found as a result of finding a model and a "view name" and then translating that pair to a "view" (a "PageHandler" in CP terms, if I've got things right). The view is called at the end of traversal, not during traversal. I *think* the main difference here is that bfg does not assume that there is a graph of page handlers, instead it assumes there is a graph of model objects and finds a single page handler at the end (aka a view).

repoze and GAE

Hi Chris

I have just spent the last few weeks getting a core of zope3 running in GAE, (ie using GAE as the persistent store) getting an object model that supports traversal etc....
Most of it is working, and in fact a chunk of what I have done mirrors some of what you are doing. My base zope components pretty much match what you have,
though I have had to gut zope.proxy to get zope.deferredimport working, and also gut zope.security (no 'c' modules in GAE ;-)

I have also ignored zcml at the moment and doing explicit component registrations in code instead (for now)

The last bit I am trying to get working is z3c.form but having trouble tracking down all the required zcml directives and trying to remove as many zope.app dependancies as well.
So you approach with formencode is a cheaper starting point, so I will have a look at that. (Widgets at the moment don't get 'view' in their context for some reason during getMultiAdaptor)

I suppose the question I was going to ask is I wonder if we could work to make repoze the basis for a pure python starting point so that the basic zope3 component ideas etc... could be used
and runnable, (the full blown versions of zope.proxy and zope.security with there c bits could be optional extras) so that repoze (core) for instance could be a starting point for
GAE based apps.

Just a thought ;-)

Rgds

Tim


problems installing repoze

Oh one last thing, I am having trouble getting repoze to install correctly under 2.5 (bits missing) and paster keeps complaing about being unable to expand the template (bfg) if I do an
install under 2.4 (I am using virtualenv). Will keep trying and let you know why its failing in case other people have the same problem. (it's quite probably something I am doing ;-)

GAE...

Hi Tim,

I've just fixed the index so the right stuff should work under Py 2.5. I also tested that the sample paster template application rendered. Thanks for the heads up.

Kapil Thangavelu worked on making most Z3 libs work on GAE (you've probably seen it..).. http://blog.kapilt.com/2008/05/28/17/ .. i'd be interested in helping out in this effort so repoze.bfg could also share the love. Maybe we could talk about it on the Repoze-dev maillist?

C code...

Here are the compiled files under an install of repoze.bfg:

/lxml-2.1beta3-py2.5-macosx-10.3-i386.egg/lxml/etree.so
./lxml-2.1beta3-py2.5-macosx-10.3-i386.egg/lxml/objectify.so
./zope.hookable-3.4.0-py2.5-macosx-10.3-i386.egg/zope/hookable/_zope_hookable.so
./zope.i18nmessageid-3.4.3-py2.5-macosx-10.3-i386.egg/zope/i18nmessageid/_zope_i18nmessageid_message.so
./zope.interface-3.4.1-py2.5-macosx-10.3-i386.egg/zope/interface/_zope_interface_coptimizations.so
./zope.proxy-3.4.1-py2.5-macosx-10.3-i386.egg/zope/proxy/_zope_proxy_proxy.so
./zope.security-3.5.1-py2.5-macosx-10.3-i386.egg/zope/security/_proxy.so
./zope.security-3.5.1-py2.5-macosx-10.3-i386.egg/zope/security/_zope_security_checker.so

I'm not sure which of these you've gotten under control, and which Kapil has. lxml is a bit of a special case; it's required by zc3.pt. Perhaps that means it should not be part of bfg but instead be an addon.