Mon, 24 Mar 2008
PyCon 2008
I was in Chicago last week for PyCon 2008. It was my first time in the windy city, and I must say that I was thoroughly impressed. As expected in any city, we got a chance to see a lady get her purse snattched, and a mentally unstable gentleman on the train yelling profanities at god. Anyway, the conference itself was extremely well done, and tons of awesome innovation happened at the sprints afterwords.
Day 1: Tutorials
8+ hours of TurboGears/Pylons/WSGI tutorials. Awesome. I'm really
excited with what is in the works for TurboGears2. By wielding Pylons, the
TG2 team was able to completely re-write their framework with minimal amounts
of code, while at the same time, gaining a *ton* of new features
and some amazing middleware. Mark Ramm and Ben Bangert took turns walking us through the
deep internals of their frameworks, while also giving some examples how to use
them.
Sessions
During the 3-day conference portion of PyCon, there was a vast plethora of
incredibly interesting sessions and conversations. You can find a schedule of
the talks and some slides here. Everything was
video taped as well, so the sessions should be making their way on to YouTube
hopefully at some point soon.
Here are some things that caught my attention while I was there.
WSGI
Defined by Phillip J. Eby in PEP-333, the Web Server Gateway Interface is a simple interface between web servers, applications, and frameworks. Or, as explained by Ian Bicking: WSGI is a series of Tubes. The basic idea is that it lets you connect a bunch of different applications together into a functioning whole.
Since TurboGears2 is based on Pylons, it will be a full blown WSGI application out the box, loaded with lots of useful middleware (WebError, Routes, Sessions, Caching, etc), and will allow you to use any WSGI server that you wish (Paste, CherryPy, orbited, mod_wsgi, etc).
An example of a basic Hello World WSGI application:
def wsgi_app(environ, start_response):
start_response('200 OK', [('content-type', 'text/html')])
return ['Hello world!']
So, what is WSGI middleware? Well, it's essentially the WSGI equivalent of a python decorator, but instead of wrapping one function in another, you're wrapping one web-app in another. You can see a list of some existing WSGI middleware here.
virtualenv
With so many new shiny python programs to play with, I really tried to resist
the urge to easy_install everything into my global Python site-packages so I
could tinker with things. This is generally a Bad Thing in a distribution, as
easy_install not only installs things behind your package managers back,
but it also lacks the ability to uninstall anything with it, unless you want to take Zed's easy_fucking_uninstall
approach ;) During the TurboGears tutorial, I was introduced to a tool call
virtualenv, which will setup a virtual python environment in which you can
easy_install as many eggs as you want without worrying about butchering
your site-packages.
$ easy_install virtualenv
$ virtualenv --no-site-packages foo
$ cd foo; source bin/activate
$ easy_install <shiny python programs>
nose
I've been in love with nose since day
one, but realized that I haven't been utilizing it to it's fullest abilities.
I blogged in the past about nose's
profiler plugin. Come to find out, nose offers a lot more plugins that can
seriously help make your life easier:
$ nosetests --pdb --pdb-failures
.............................................................> /home/lmacken/tg1.1/turbogears/turbogears/identity/tests/test_visit.py(92)test_cookie_permanent()
-> assert abs(should_expire - expires) < 3
(Pdb) locals()
{'morsel': <Morsel: tg-visit='452c94de3900fc2adff2cd6b0b0f04c4533e3e9e'>, 'self': <turbogears.identity.tests.test_visit.TestVisit testMethod=test_cookie_permanent>, 'expires': 1206228604.0, 'should_expire': 1206232205.0, 'permanent': False}
(Pdb)
You can also measure code coverage during your unit test execution using the '--with-coverage' option, which utilizes coverage.py.
SQLAlchemy
Also known as "the greatest object-relational-mapper created for any language. ever.", 0.4 has seen vast improvements since 0.3. Among them, a new declarative
API is now available that essentially lets you define your class, Table and
mapper constructs "at once" under a single class declaration (giving you a
similar ActiveMapper feel like SQLObject or Elixir).
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('sqlite://')
Base = declarative_base(engine)
class SomeClass(Base):
__tablename__ = 'some_table'
id = Column('id', Integer, primary_key=True)
name = Column('name', String(50))
Unicode, demystified.
By far, the most frustrating problems I've ever encountered in Python have been
unicode related. I was fortunate enough to catch Kumar McMillan's
presentation, "Unicode in Python, Completely Demystified". This presentation
helped enlighten many on the concept of unicode, clear up many misconceptions,
and explain how to handle it properly in Python. Check out his slides for more details, but
the general idea here is to follow these three rules:
- decode early
- unicode everywhere
- encode late
def to_unicode_or_bust(obj, encoding='utf-8'):
if isinstance(obj, basestring):
if not isinstance(obj, unicode):
obj = unicode(obj, encoding)
return obj
Later that night I went and shined some light on some dark corners of certain projects that I've been working on to try and handle unicode the Right Way.
Grassyknoll
After the code sprints, I got a chance to see these guys show off their hard
work. grassyknoll is a
search engine written in Python. With the ability to handle multiple backends,
frontends, and wire formats, grassyknoll has a ton of potential to
revolutionize the open source search engine. There has been recent talk in
Fedora land about what kind of search engine to use, and I think grassyknoll is
definitley a viable option.
Packaging BOF
Toshio, Spot, and I attended a Packaging BOF where we discussed our
experiences with distutils and setuptools with a bunch of people from various
companies and distros. This then sparked discussions on python-dev and the
distutils-sig mailing lists. You can also find the details of the BOF session
on the Python wiki. There
is definitely a lot of energy behind this, so hopefully we'll see some good changes
in setuptools in the near future that will make our lives as distro packagers much easier :)
Orbited
Orbited is an HTTP daemon that is optimized for long-lasting comet
connections. This allows you to write real-time web applications with
ease. For example, embeding an irc channel anywhere:
You can also use orbited as a WSGI server! Toshio did some brief benchmarking of of CherryPy{2,3}, Paste, and Orbited WSGI servers, and orbited seemed to be the clear winner in all scenerios. There is a good chance that we will be using orbited to handle our comet widgets within MyFedora :)
Code SprintsI stayed the entire time for the code sprints, and mainly focused on TurboGears hacking. This is what I ended up working on:
- Added SQLAlchemy support to turbogears.testutil.DBTest (Ticket #1764). When you inherit from this class, it will automatically set up and tear down your SQLObject or SQLAlchemy database before and after each of your unit tests.
- Added a FlotWidget using ToscaWidgets to twTools This widget allows you to create attractive graphs with ease.
- Made the TurboGears2 templating engine configurable (Ticket #1680). Things were hardcoded to use genshi; this is no longer the case.
- WebTest integration for unit test (Ticket #1762). I wrote a some high level unit testing classes that wrap a WebTest object around your WSGI app. This gives you an extremely powerful API to write "framework independent" unit tests. The WebTest.get/post methods simply return WebOb objects, which allow for drastic simplification of your unittests. This also helped decouple the TG testutils from using CherryPy internals (one step closer to CherryPy3 support in TurboGears). As I mentioned on the TurboGears-trunk list, these changes will make writing unit tests a breeze:
class TestPages(testutil.DBWebTest):
def test_forbidden(self):
self.app.get('/hot_action', status=403)
def test_webob_response(self):
user = User(user_name=u"test", password=u"test")
self.login_user(user)
res = self.app.get('/hot_action')
assert "Hot WSGI action" in res
assert res.namespace['tg_flash'] == u'Hot WSGI action'
Want to read more blog posts about PyCon 2008? You can find links to lots of PyCon related posts here and on Planet Python.
posted at: 17:05 | link | | 1 comments
Posted by ignacio at Mon Mar 24 14:55:37 2008
daMaestro started a Fedora package for virtualenv, but it doesn't work properly on x86_64 and I've been working on fixing it.
