Mon, 26 Jan 2009
bodhi updates push process
Bodhi's push process is something that is usually quite opaque to Fedora package maintainers. Once an update request goes into bodhi, the developer sits back and waits for the update to go to where it needs to go. The ball is then in releng's court, as they must sign the packages, and tell bodhi to begin the push. From there, bodhi does it's thing for a while, and then updates magically end up on our users machines. Yay!
Pushing updates used to take the better part of a day, mostly due to dumb code and lots of filesystem churn over NFS. Thankfully, a lot of the code is now much smarter, and people like jkeating and mmcgrath have been helping to address the NFS & infrastructure bottlenecks.
Hopefully I can help shed some light on one of the dark corners of bodhi known as The Masher. Here are some statistics of the last updates push that happened earlier today.
| Initial push request from releng | |
| Check koji tag / bodhi status consistency | 38s |
| Move all of the build tags in Koji | 9m32s |
| Update the comps CVS module | 11s |
| Mash f9-updates-testing | 4m16s |
| Mash f9-updates | 1h3m8s |
| Mash f10-updates-testing | 12m43s |
| Mash f10-updates | 37m51s |
| Set update ids, state modifications, updates-testing digest generation | 1m57s |
| Generate updateinfo.xml | 5m55s |
| Repo sanity checks & symlinking to go live | 1m4s |
| Cache latest repodata, and remove old | 1m14s |
| Wait for updates to hit the master mirror | 1h1s |
| Send update notices, update/close bugs, notify developers/commenters | 11m11s |
| Total | 3h49m42s |
So we've obviously made some great improvements here, and once the signing server is deployed, you can probably expect a much more frequent/consistent flow of updates. However, I definitely think there is still a lot of low-hanging fruit in this process, and many steps can probably be done in parallel. We're going to be adding DeltaRPM generation into the mix in the near future, so I'll give an update a bit later with some details as to how that effects the process.
Anyway... if you know Python, and enjoy optimizing code -- come talk to me :)
posted at: 23:15 | link | | 0 comments
