Mon, 24 Mar 2008

PyCon 2008

I was in Chicago last week for PyCon 2008. It was my first time in the windy city, and I must say that I was thoroughly impressed. As expected in any city, we got a chance to see a lady get her purse snattched, and a mentally unstable gentleman on the train yelling profanities at god. Anyway, the conference itself was extremely well done, and tons of awesome innovation happened at the sprints afterwords.

Day 1: Tutorials
8+ hours of TurboGears/Pylons/WSGI tutorials. Awesome. I'm really excited with what is in the works for TurboGears2. By wielding Pylons, the TG2 team was able to completely re-write their framework with minimal amounts of code, while at the same time, gaining a *ton* of new features and some amazing middleware. Mark Ramm and Ben Bangert took turns walking us through the deep internals of their frameworks, while also giving some examples how to use them.

During the 3-day conference portion of PyCon, there was a vast plethora of incredibly interesting sessions and conversations. You can find a schedule of the talks and some slides here. Everything was video taped as well, so the sessions should be making their way on to YouTube hopefully at some point soon.

Here are some things that caught my attention while I was there.

Defined by Phillip J. Eby in PEP-333, the Web Server Gateway Interface is a simple interface between web servers, applications, and frameworks. Or, as explained by Ian Bicking: WSGI is a series of Tubes. The basic idea is that it lets you connect a bunch of different applications together into a functioning whole. Since TurboGears2 is based on Pylons, it will be a full blown WSGI application out the box, loaded with lots of useful middleware (WebError, Routes, Sessions, Caching, etc), and will allow you to use any WSGI server that you wish (Paste, CherryPy, orbited, mod_wsgi, etc). An example of a basic Hello World WSGI application:

def wsgi_app(environ, start_response):
    start_response('200 OK', [('content-type', 'text/html')])
    return ['Hello world!']

So, what is WSGI middleware? Well, it's essentially the WSGI equivalent of a python decorator, but instead of wrapping one function in another, you're wrapping one web-app in another. You can see a list of some existing WSGI middleware here.

With so many new shiny python programs to play with, I really tried to resist the urge to easy_install everything into my global Python site-packages so I could tinker with things. This is generally a Bad Thing in a distribution, as easy_install not only installs things behind your package managers back, but it also lacks the ability to uninstall anything with it, unless you want to take Zed's easy_fucking_uninstall approach ;) During the TurboGears tutorial, I was introduced to a tool call virtualenv, which will setup a virtual python environment in which you can easy_install as many eggs as you want without worrying about butchering your site-packages.

$ easy_install virtualenv
$ virtualenv --no-site-packages foo
$ cd foo; source bin/activate
$ easy_install <shiny python programs>

I've been in love with nose since day one, but realized that I haven't been utilizing it to it's fullest abilities. I blogged in the past about nose's profiler plugin. Come to find out, nose offers a lot more plugins that can seriously help make your life easier:

$ nosetests --pdb --pdb-failures
.............................................................> /home/lmacken/tg1.1/turbogears/turbogears/identity/tests/
-> assert abs(should_expire - expires) < 3
(Pdb) locals()
{'morsel': <Morsel: tg-visit='452c94de3900fc2adff2cd6b0b0f04c4533e3e9e'>, 'self': <turbogears.identity.tests.test_visit.TestVisit testMethod=test_cookie_permanent>, 'expires': 1206228604.0, 'should_expire': 1206232205.0, 'permanent': False}

You can also measure code coverage during your unit test execution using the '--with-coverage' option, which utilizes

Also known as "the greatest object-relational-mapper created for any language. ever.", 0.4 has seen vast improvements since 0.3. Among them, a new declarative API is now available that essentially lets you define your class, Table and mapper constructs "at once" under a single class declaration (giving you a similar ActiveMapper feel like SQLObject or Elixir).

from sqlalchemy.ext.declarative import declarative_base

engine = create_engine('sqlite://')
Base = declarative_base(engine)

class SomeClass(Base):
    __tablename__ = 'some_table'
    id = Column('id', Integer, primary_key=True)
    name = Column('name', String(50))

Unicode, demystified.
By far, the most frustrating problems I've ever encountered in Python have been unicode related. I was fortunate enough to catch Kumar McMillan's presentation, "Unicode in Python, Completely Demystified". This presentation helped enlighten many on the concept of unicode, clear up many misconceptions, and explain how to handle it properly in Python. Check out his slides for more details, but the general idea here is to follow these three rules:

His solution to decoding to unicode turns out to be quite elegant compared to some nasty try/except UnicodeDecodeError blocks that I have written in the past ;)
def to_unicode_or_bust(obj, encoding='utf-8'):
    if isinstance(obj, basestring):
        if not isinstance(obj, unicode):
            obj = unicode(obj, encoding)
    return obj

Later that night I went and shined some light on some dark corners of certain projects that I've been working on to try and handle unicode the Right Way.

After the code sprints, I got a chance to see these guys show off their hard work. grassyknoll is a search engine written in Python. With the ability to handle multiple backends, frontends, and wire formats, grassyknoll has a ton of potential to revolutionize the open source search engine. There has been recent talk in Fedora land about what kind of search engine to use, and I think grassyknoll is definitley a viable option.

Packaging BOF
Toshio, Spot, and I attended a Packaging BOF where we discussed our experiences with distutils and setuptools with a bunch of people from various companies and distros. This then sparked discussions on python-dev and the distutils-sig mailing lists. You can also find the details of the BOF session on the Python wiki. There is definitely a lot of energy behind this, so hopefully we'll see some good changes in setuptools in the near future that will make our lives as distro packagers much easier :)

Orbited is an HTTP daemon that is optimized for long-lasting comet connections. This allows you to write real-time web applications with ease. For example, embeding an irc channel anywhere:

You can also use orbited as a WSGI server! Toshio did some brief benchmarking of of CherryPy{2,3}, Paste, and Orbited WSGI servers, and orbited seemed to be the clear winner in all scenerios. There is a good chance that we will be using orbited to handle our comet widgets within MyFedora :)

Code Sprints
I stayed the entire time for the code sprints, and mainly focused on TurboGears hacking. This is what I ended up working on:

Want to read more blog posts about PyCon 2008? You can find links to lots of PyCon related posts here and on Planet Python.

posted at: 22:05 | link | Tags: , | 2 comments

Posted by ignacio at Mon Mar 24 19:55:37 2008

daMaestro started a Fedora package for virtualenv, but it doesn't work properly on x86_64 and I've been working on fixing it.

Posted by NursingEsaysUK at Wed Apr 22 11:16:58 2020

well the best thing apart all of this code hectic and frenzy it is the only thing which can best be applicable to all other programming platforms and sometimes it becomes quite imperative to add different threads to maintain the performance sustainability from different languages like ASP PHP and most notably Java in this scenario