lewk.org

The blog of Luke Macken

Sun, 14 Dec 2008

>>> from fedora.client import Wiki

I created a simple Python API for interacting with Fedora's MediaWiki a while back, in an attempt to gather various metrics. I just went ahead and committed it to the python-fedora modules. Here is how to use it:

>>> from fedora.client import Wiki
>>> wiki = Wiki()
>>> wiki.print_recent_changes()
From 2008-12-07 20:59:01.187363 to 2008-12-14 20:59:01.187363
500 wiki changes in the past week

== Most active wiki users ==
 Bbbush............................................ 230
 Konradm........................................... 25
 Duffy............................................. 22
 Jreznik........................................... 21
 Ianweller......................................... 14
 Jjmcd............................................. 14
 Geroldka.......................................... 10
 Gdk............................................... 9
 Anouar............................................ 7
 Gomix............................................. 6

== Most edited pages ==
 Features/KDE42.................................... 21
 SIGs/SciTech/SAGE................................. 15
 FUDCon/FUDConF11.................................. 14
 Special:Log/upload................................ 13
 How to be a release notes beat writer............. 12
 Special:Log/move.................................. 11
 Design/SETroubleshootUsabilityImprovements........ 10
 PackageMaintainers/FEver.......................... 9
 User:Gomix........................................ 6
 Zh/主要配置文件..................................... 5

>>> for event in wiki.send_request('api.php', req_params={
...         'action': 'query',
...         'list': 'logevents',
...         'format': 'json',
...         })['query']['logevents']:
...     print '%-10s %-15s %s' % (event['action'], event['user'], event['title'])
...
patrol     Ianweller       User:Ianweller/How to create a contributor business card
move       Nippur          REvanderLuit
patrol     Ianweller       Project Leader
move       Ianweller       FPL
upload     Anouar          Image:AnouarAbtoy.JPG
move       Liangsuilong    ZH/Docs/FetionOnFedora
move       Liangsuilong    FetionOnFedora
patrol     Ianweller       User:Ianweller


It uses the fedora.client.BaseClient, which is a class that simplifies interacting with arbitrary web services. Toshio and I created it a while back as a the core client for talking with our various TurboGears-based Fedora Services (bodhi, pkgdb, fas, etc.), but it has now seemed to morph into a much more flexible client for talking JSON with web applications.

from datetime import datetime, timedelta
from collections import defaultdict
from fedora.client import BaseClient

class Wiki(BaseClient):

    def __init__(self, base_url='http://fedoraproject.org/w/', *args, **kwargs):
        super(Wiki, self).__init__(base_url, *args, **kwargs)

    def get_recent_changes(self, now, then, limit=500):
        """ Get recent wiki changes from `now` until `then` """
        data = self.send_request('api.php', req_params={
                'list'    : 'recentchanges',
                'action'  : 'query',
                'format'  : 'json',
                'rcprop'  : 'user|title',
                'rcend'   : then.isoformat().split('.')[0] + 'Z',
                'rclimit' : limit,
                })
        if 'error' in data:
            raise Exception(data['error']['info'])
        return data['query']['recentchanges']

    def print_recent_changes(self, days=7, show=10):
        now = datetime.utcnow()
        then = now - timedelta(days=days)
        print "From %s to %s" % (then, now)
        changes = self.get_recent_changes(now=now, then=then)
        num_changes = len(changes)
        print "%d wiki changes in the past week" % num_changes

        users = defaultdict(list) # {username: [change,]}
        pages = defaultdict(int)  # {pagename: # of edits}

        for change in changes:
            users[change['user']].append(change['title'])
            pages[change['title']] += 1

        print '\n== Most active wiki users =='
        for user, changes in sorted(users.items(),
                                    cmp=lambda x, y: cmp(len(x[1]), len(y[1])),
                                    reverse=True)[:show]:
            print ' %-50s %d' % (('%s' % user).ljust(50, '.'), len(changes))

        print '\n== Most edited pages =='
        for page, num in sorted(pages.items(),
                                cmp=lambda x, y: cmp(x[1], y[1]),
                                reverse=True)[:show]:
            print ' %-50s %d' % (('%s' % page).ljust(50, '.'), num)



I added a Wiki.login method to the latest version, but it isn't quite working yet. This is due to some minor limitations in the ProxyClient, so we currently cannot handle authenticated requests. However, this shouldn't be very difficult to implement. The reason for this is that we need to be able to run authenticated queries as a 'bot' account in order to mitigate the 500 entry API return limit.

This module makes it easy to talk to MediaWiki's API, so if you do anything cool with it feel free to send patches here. It's currently not being shipped in a python-fedora release, so you'll have to grab the code from Bazaar:

bzr branch bzr://bzr.fedorahosted.org/bzr/python-fedora/python-fedora-devel

posted at: 17:12 | link | Tags: Fedora, Python, code, TurboGears | 4 comments

Linkage

• Twitter || identi.ca
• Fedora Profile
• Resume
• Photos
• Bookmarks
• libre.fm || last.fm
• Code
• Blog
  • RSS
  • RSS 2.0
  • Atom

Projects

Tip Jar

Contact me | Back to top
© 2008-2009 Luke Macken