Mon, 18 Jan 2010

nose 0.11

I know nose 0.11 is old news, but I've only recently discovered it's new multiprocess module. 

lmacken@tomservo ~/bodhi $ nosetests
................................................................................................
----------------------------------------------------------------------
Ran 96 tests in 725.111s

OK

lmacken@tomservo ~/bodhi $ nosetests --processes=50
................................................................................................
----------------------------------------------------------------------
Ran 96 tests in 10.915s

OK

Nose 0.11 is already in rawhide, and will soon be in updates-testing.

Note to self (and others): Buy the nose developers beer at PyCon next month



posted at: 16:58 | link | Tags: , , , , | 2 comments

Wed, 30 Dec 2009

My dream machine

In an effort to optimize my home office, I recently donated my server rack to a local Boston record label, to hold their Red Hat servers. I'm also in the process of donating all of my computer hardware for re-use/recycling (~10 or so frankenstein boxen).

So, once I clear everything out, I'm going to replace it with a new machine. I usually sit in front of 1-3 laptops (thindpads and XOs) on the daily, and I absolutley love them, but I need something beefier. I do most of my work on remote machines, but I have found that I still spend too much time waiting on computers.

I'm not much of a gamer, so I probably don't need too high-end of a graphics chip, let alone SLI/Crossfire. The extent of my gaming these days consists of the occassional wesnoth, open arena, nethack, and my current favorite Cube 2: Sauerbraten. I just want a card that will work well in Linux, ideally without having to install proprietary drivers.

Anyway, I haven't built a desktop machine from the ground up in 12 years, and I've been out of the hardware game for a long time, so let me know what's good! Here is what I've been looking at so far...

Intel DX58SO Extreme Series X58 ATX Tri-Channel DDR3 16GB SLI or CrossFireX LGA1366 Overclocking Utility Desktop Board
or
ASUS Rampage II Extreme LGA1366 Intel X58 DDR3-1600 ATX Motherboard

Intel Core i7 975 Extreme Edition 3.33GHz 8M L3 Cache LGA1366 Desktop Processor
Cooler Master V8 Nickel Plated Copper Base Aluminum Fins 8 Heatpipes Core i7 1366 CPU Cooler - (RR-UV8-XBU1-GP)

Corsair 6 GB Dominator GT PC3-16000 2000Mhz 240-pin Triple Channel DDR3 Memory Kit
Corsair CMPSU-1000HX 1000-Watt HX Professional Series 80 Plus Certified Power Supply

Intel X25-E Extreme SATA Solid-State Drive
or
Corsair 256 GB Internal Solid State Drive (SSD) CMFSSD-256GBG2D

DELL ULTRASHARP 3008WFP - 30-inch Widescreen Flat Panel Monitor
EVGA 01G-P3-1180-AR GeForce GTX285 1024 MB DDR3 PCI-Express 2.0 Graphics Card

Endurapro
Cooler Master HAF 932 High Air Flow ATX Full Tower Case

Also, if you appreciate my software and want me to write it faster... donations are accepted ;)



posted at: 19:05 | link | Tags: | 14 comments

Mon, 28 Dec 2009

RIP Fedora 10

Fedora 10 (Cambridge) (2008-11-25 -- 2009-12-17)

Updates

Source: bodhi
Fedora 10 Updates
Most updates per developer in Fedora 10
Most Updated Packages in Fedora 10
Packages with best karma
Top Fedora 10 testers
Most tested Fedora 10 packages

Torrent

Source: fedoracommunity (upcoming release)

Torrent NameNumber of completed downloads
Fedora-10-i386-DVD 112,807
Fedora-10-x86_64-DVD 65,965
Fedora-10-i386-CDs 10,621
Fedora-10-ppc-DVD 6,851
Fedora-10-source-DVD 3,740
Fedora-10-x86_64-CDs 3,141
Fedora-10-ppc-CDs 1,336
Fedora-10-i686-AOS 666
Fedora-10-source-CDs 662
Fedora-10-i686-Live 599
Fedora-10-x86_64-Live 336
Fedora-10-x86_64-AOS 274
Fedora-10-i686-Live-KDE 201
Fedora-10-x86_64-Live-KDE 78
Fedora-10-i686-Live-XFCE 37
Fedora-10-i686-Live-Developer 13
Fedora-10-i686-Live-FEL 12
Fedora-10-x86_64-Live-XFCE 5
Fedora-10-i686-Live-broffice 3
Fedora-10-x86_64-Live-Developer 3
Fedora-10-x86_64-Live-FEL 2
Fedora-10-x86_64-Live-edu-math 1
Fedora-10-i686-Live-edu-math 1
Fedora-10-x86_64-Live-broffice 0
Total 207,354

Yum Data

Source: wiki/Legacy_statistics
Connections to yum
Week Dates New Unique IPsTotal Unique IPsTotal compared to F9
1 2008-11-25 -- 2008-12-0167,421 67,421 73%
2 2008-12-02 -- 2008-12-0881,674 149,095 97%
3 2008-12-09 -- 2008-12-1560,759 209,854 97%
4 2008-12-16 -- 2008-12-2262,527 272,381 93%
5 2008-12-23 -- 2008-12-2968,375 340,756 97%
6 2008-12-30 -- 2009-01-0573,585 414,341 97%
7 2009-01-06 -- 2009-01-1294,166 508,507 103%
8 2009-01-13 -- 2009-01-1985,557 594,064 106%
9 2009-01-20 -- 2009-01-2687,678 681,742 107%
10 2009-01-27 -- 2009-02-0291,014 772,756 110%
11 2009-02-03 -- 2009-02-0995,238 867,994 113%
12 2009-02-10 -- 2009-02-1695,967 963,961 115%
13 2009-02-17 -- 2009-02-23109,800 1,073,761 115%
14 2009-02-24 -- 2009-03-0285,246 1,159,007 --
15 2009-03-03 -- 2009-03-09100,610 1,259,617 --
16 2009-03-10 -- 2009-03-16100,323 1,359,940 --
17 2009-03-17 -- 2009-03-23100,819 1,460,759 --
18 2009-03-24 -- 2009-03-30102,843 1,563,602 --
19 2009-03-31 -- 2009-04-06101,978 1,665,580 136%
20 2009-04-07 -- 2009-04-1399,586 1,765,166 --
21 2009-04-14 -- 2009-04-20101,808 1,866,974 --
22 2009-04-21 -- 2009-04-27100,230 1,967,177 --
23 2009-04-28 -- 2009-05-0497,584 2,064,761 --
24 2009-05-05 -- 2009-05-1195,923 2,160,684 137%
25 2009-05-12 -- 2009-05-1895,632 2,256,316 --
26 2009-05-19 -- 2009-05-2592,377 2,348,693 --
27 2009-05-26 -- 2009-06-0191,747 2,440,440 --
28 2009-06-02 -- 2009-06-0891,513 2,531,953 --

Direct downloads

Source: wiki/Legacy_statistics
The following table shows the number of direct downloads of Fedora 10 media from unique IP addresses, as shown in the web proxy logs. The actual number of raw downloads tends to be much higher.
Week Dates Downloads this week Total downloads
1 2008-11-25 -- 2008-12-01 236,886 236,886
2 2008-12-02 -- 2008-12-08 105,994 342,880
3 2008-12-09 -- 2008-12-15 83,740 426,620
4 2008-12-16 -- 2008-12-22 76,982 503,602
5 2008-12-23 -- 2008-12-29 66,351 569,953
6 2008-12-30 -- 2009-01-05 65,102 635,055
7 2009-01-06 -- 2009-01-12 72,729 707,784
8 2009-01-13 -- 2009-01-19 73,301 781,085
9 2009-01-20 -- 2009-01-26 72,082 853,167
10 2009-01-27 -- 2009-02-02 71,788 924,955
11 2009-02-03 -- 2009-02-09 72,529 997,484
12 2009-02-10 -- 2009-02-16 69,071 1,066,555
13 2009-02-17 -- 2009-02-23 69,216 1,135,771
14 2009-02-24 -- 2009-03-02 67,669 1,203,440
15 2009-03-03 -- 2009-03-09 66,666 1,270,106
16 2009-03-10 -- 2009-03-16 65,524 1,335,630
17 2009-03-17 -- 2009-03-23 63,218 1,398,848
18 2009-03-24 -- 2009-03-30 62,930 1,461,778
19 2009-03-31 -- 2009-04-06 59,813 1,521,591
20 2009-04-07 -- 2009-04-13 57,102 1,578,693
21 2009-04-14 -- 2009-04-20 55,871 1,634,564
22 2009-04-21 -- 2009-04-27 55,117 1,689,681
23 2009-04-28 -- 2009-05-04 50,815 1,740,496
24 2009-05-05 -- 2009-05-11 48,139 1,788,635
25 2009-05-12 -- 2009-05-18 47,813 1,836,448
26 2009-05-19 -- 2009-05-25 46,077 1,882,525
27 2009-05-26 -- 2009-06-01 44,969 1,927,494
28 2009-06-02 -- 2009-06-08 44,835 1,972,329



posted at: 18:05 | link | Tags: , , , | 0 comments

Thu, 10 Dec 2009

FUDCon Toronto 2009

Another FUDCon is in the books, this time in Toronto. It was great to catch up with many people, put faces to some names, and meet a bunch of new contributors. I gave a session on Moksha, which I'll talk about below, and was also on the Fedora Infrastructure panel discussion.

My goal this FUDCon wasn't to crank out a ton of code, but to focus on gathering and prioritizing requirements and to help others be productive. Here are some of the projects I focused on.

Moksha

Moksha is a project I created a little over a year ago, which is the base of a couple of other applications I've been working on as well: Fedora Community and CIVX. I'll be blogging about these in more detail later.

One of the main themes of FUDCon this year was Messaging (AMQP), and Moksha is a large part of this puzzle, as it allows you to wield AMQP within web applications. During my session the demo involved busting open a terminal, creating a consumer that reacts to all messages, creating a message producer, and then creating a live chat widget -- all of which hooked up to Fedora's AMQP broker.

I'll be turning my slides into an article, so expect a full blog post explaining the basics soon. In the mean time, I found Adam Miller's description to be extremely amusing:

"I walked into a session called "Moksha and Fedora Community -- Real-time web apps with Python and AMQP" which blew my mind. This is Web3.0 (not by definition, but that's what I'm calling it), Luke Macken and J5 completely just stepped over web2.0 and said "pffft, childs play" (well not really but in my mind I assume it went something like that). This session showed off technology that allows real time message passing in a web browser as well as "native" support for standard protocols. The project page is https://fedorahosted.org/moksha/ and I think everyone on the planet should take some time to go there and enjoy the demo, prepare to have your mind blown. Oh, and I also irc transcribed that one as well http://meetbot.fedoraproject.org/fudcon-room-3/2009-12-05/fudcon-room-3.2009-12-05-22.07.log.html ... presentation slides found: http://lmacken.fedorapeople.org/moksha-FUDConToronto-2009.odp"

Fedora Community

So after we released v1.0 of Fedora Community for F12, all of us went off in seperate directions to hack on various things. J5 wrote AMQP javascript bindings, which I then integrated into Moksha. Máirín Duffy built a portable usability lab and has been doing great research on the usability of the project. And I dove back into Moksha to solidify the platform.

After we deploy our AMQP broker for Fedora, and once we have start adding shims into our existing infrastructure, we'll then be able to start creating live widgets and message consumers that can react to events, allowing us to wield Fedora in real-time. This will let us to keep our fingers on the pulse of Fedora, automate and facilitate tedious tasks, and gather metrics as things happen.

During the hackfests I also did some work on our current Fedora Community deployment. Over the past few weeks some of our widgets randomly died, and we haven't been receiving proper error messages. So, I successfully hooked up WebError and the team is now getting traceback emails, which will help us fix problems much faster (or at least nag the hell out of us about them).

I also worked with Ian Weller on the new Statistics section of the dashboard, which has yet to hit production. Ian and I wrote Wiki metrics, Seth Vidal wrote BitTorrent metrics, and I wrote Bodhi metrics. We've also got many more to come. My main concern was a blocker issue that we were hitting with our flot graphs when you quickly bounce between tabs. I ended up "fixing" the bug, so I'll be pushing what we have of the stats branch into production in the near future.

TurboGears2

TurboGears has definitely been our favorite web framework within Fedora's Infrastructure for many years now. TurboGears2, a complete re-invention of itself, has been released recently, and is catching on *very* quickly in the community. Tons of people are working on awesome new apps, and loving every minute of it. I was also able to convert a rails hacker over to it, after he was able to quickly dive into one of the tutorials with ease. See my previous blog post about getting up and running with TG2 in Fedora/EPEL.

python-fedora

One of my main tasks during the hackfests was to pull the authentication layer in Fedora Community that authenticates against the Fedora Account System, and port it over to python-fedora, so we can use it in any TurboGears2 application. I committed the initial port to python-fedora-devel, and have started working on integrating it into a default TG2 quickstart and document the process. There are still a couple of minor things I want to fix/clean up before releasing it, so expect a blog about it soon.

Bodhi

It seems like yesterday that I was an intern at Red Hat working on an internal updates system for Fedora Core. Coming up on 5 years later, and I am now working on my 3rd implementation of an updates system, Bodhi v2.0. What's wrong with the current Bodhi you ask? Well, if you talk to any user of it, you'll probably get a pretty long list. Bodhi is the first TurboGears application written & deployed in Fedora Infrastructure, and uses the vanilla components (SQLObject, kid, CherryPy2). The TG1 stack has been holding up quite nicely over the years, and is still supported upstream, but bodhi's current implemention and design does not make it easy to grow.

Bodhi v2.0 will be implemented in TurboGears2, using SQLAlchemy for an ORM, Mako for templates, and ToscaWidgets2 for re-usable widgets. It will be hook-based and plugin-driven, and will be completely distribution agnostic. Another important goal will be AMQP message-bus integration, which will allow other services or users to react to various events inside of the system as they happen.

So far I've ported the old DB model from SQLObject to SQLAlchemy, and have begun porting the old unit tests, and writing new ones. Come the new year, I'll be giving this much more of my focus.

During the hackfests I got a chance to talk to Dennis Gilmore about various improvements that we need to make with regard to the update push process. It was also great to talk to many different users of bodhi, who expressed various concerns, some of which I've already fixed. I also got a chance to talk to Xavier Lamien about deploying Bodhi for rpmfusion. On the bus ride home I helped explain to Mel how Bodhi & Koji fit into the big picture of things.

During the BarCamp sessions I also attended a session about the Update Experience, where we discussed many important issues surrounding updates.

liveusb-creator

So I got a chance to finally meet Sebastian Dziallas, of Sugar on a Stick fame, and was able to fix a few liveusb-creator issues on his laptop. I ended up pushing out a new release a couple of days ago that contains some of those fixes, along with a new version of Sugar on a Stick.

The liveusb-creator has been catching a lot of press recently (see the front page for a list). Not only did it have a 2 page spread in Linux Format, but it was also featured in this weeks Wired.com article New Sugar on a Stick Brings Much Needed Improvements. Rock.

Python

There was lot of brainstorming done by Dave Malcolm, Colin Walters, Toshio Kuratomi, Bernie Innocenti, I, and many others about various improvements that we could make to the Python interpreter. From speeding up startup time by doing some clever caching to potentially creating a new optimized compiled binary format. We also looked into how WebError/abrt gather tracebacks, and discussed ways of enabling interactive traceback debugging for vanilla processes, without requiring a layer of WSGI middleware.

There was also work done on adding SystemTap probes to Python, which is very exciting. There are many ideas for various probe points, including one that I blogged about previously.

Intel iMac8,1 support

My iMac sucks at Linux. This has been something that has been nagging me for a long time, and I've been slowly trying to chip away at the problems. First, I've been doing work on a Mac port of the liveusb-creator. I also started to work on a kernel patch for getting the EFI framebuffer working, and discussed how to do it with ajax and pjones. The screen doesn't display anything after grub, and since we don't know the base address of the framebuffer, it involves writing code to iterate over memory trying to find some common pixel patterns. I'm still trying to wrap my head around all of it, but I'll probably end up just buying them beer to fix it for me.

Thincrust

Thincrust is a project that I've been excited about for a while, and I actually have some appliances deployed in a production cloud. I was able to run some ideas for various virtual appliances by one of the authors over some beers. Some pre-baked virtual appliances that you can easily throw into a cloud that I would like to see:

dogtail

I'm glad to see that dogtail is still exciting people in the community. It still has a lot of potential to improve not only the way we test graphical software, but we also discussed ways of using it to teach people and automate various desktop tasks. What if you logged in after a fresh install and got the following popup bubble:

Hi, welcome to Fedora, what can I help you do today?

Each task would then allow Fedora to take the wheel and walk the user through various steps. I had this idea a while ago, when dogtail first came out, and I still think it would be totally awesome. Anyway, this was not a focus of the hackfests, but merely a conversation that I had while walking to lunch :)



posted at: 11:49 | link | Tags: , , , , , , , | 0 comments

Thu, 19 Nov 2009

TurboGears2 in Fedora & EPEL

I'm excited to announce that the TurboGears2 web application stack is now available in Fedora 12, 11 and EPEL-5.

What is TurboGears2?

TurboGears 2 is the built on top of the experience of several next generation web frameworks including TurboGears 1 (of course), Django, and Rails. All of these frameworks had limitations which were frustrating in various ways, and TG2 is an answer to that frustration. We wanted something that had:
  • Real multi-database support
  • Horizontal data partitioning (sharding)
  • Support for a variety of JavaScript toolkits, and new widget system to make building ajax heavy apps easier
  • Support for multiple data-exchange formats.
  • Built in extensibility via standard WSGI components

Installing the TurboGears2 stack & development tools

Fedora 12
yum install TurboGears2 python-tg-devtools
Fedora 11
yum --enablerepo=updates-testing install TurboGears2 python-tg-devtools
Red Hat Enterprise Linux 5 (with EPEL)
yum --enablerepo=epel-testing install TurboGears2 python-tg-devtools

Creating your first TG2 app

paster quickstart

Run your test suite

nosetests

Run your application

paster serve development.ini

Read the documentation

http://www.turbogears.org/2.0/docs

Contribute

If you're interested in helping maintain and improve the TG2/Pylons stack within Fedora/EPEL, please let me know. We're always looking for new Python hackers to join the team. There are still a few more components that need to be packaged and reviewed (eg: chameleon.genshi), so please take a look at the TurboGears2 page on the Fedora wiki for more details..



posted at: 00:00 | link | Tags: , , | 1 comments

Tue, 17 Nov 2009

Fedora 12 is here!



Install it with the liveusb-creator!



posted at: 00:00 | link | Tags: , | 0 comments

Sun, 08 Nov 2009

New liveusb-creator release!

So I've gotten some pretty inspiring feedback from various users of the liveusb-creator recently, so I decided to put some cycles into it this weekend and crank out another release.

"As a non-Linux person, Live-USB Creator has improved the quality of my life measurably!" --Dr. Arthur B. Hunkins
Yesterday I released version 3.8.6 of the liveusb-creator. Changes in this release include:
Windows
https://fedorahosted.org/releases/l/i/liveusb-creator/liveusb-creator-3.8.6.zip

Fedora
https://admin.fedoraproject.org/updates/liveusb-creator-3.8.6-1.fc11
https://admin.fedoraproject.org/updates/liveusb-creator-3.8.6-1.fc12

Source
https://fedorahosted.org/releases/l/i/liveusb-creator/liveusb-creator-3.8.6.tar.bz2

Trac
http://liveusb-creator.fedorahosted.org



posted at: 20:39 | link | Tags: , , , | 1 comments

Tue, 13 Oct 2009

Good Python Habits: vim + pyflakes

Here is a neat little hack for running pyflakes on Python files after you save them. I like using pyflakes for quickly catching dumb errors, but you could easily replace it with a more comprehensive tool like pychecker, or pylint for more strict PEP8 compliance.

All you have to do is throw this in your ~/.vimrc

au BufWritePost *.py !pyflakes %

This has saved me *tons* of time and frustration over the past few weeks, and I have no idea I lived without it.



posted at: 08:32 | link | Tags: , , | 3 comments

Sun, 11 Oct 2009

Fedora 12 filesystem showdown



posted at: 18:02 | link | Tags: , , , | 4 comments

Tue, 14 Jul 2009

Fedora 9 Updates Metrics

Fedora 9 Updates


Most updated packages


Packages with the best karma


Most updates per developer


Most tested packages


Top testers



posted at: 12:00 | link | Tags: , , | 8 comments