Python RSS Code

Articles:

The PythonWeb services developer: RSS for Python
http://codeboje.de/blog/archives/Feeds-and-python-II.html - Short articles about using Universal Feed Parser

Libraries:

the Universal Feed Parser - Reads 9 RSS versions and Atom
RSS.py - reads most RSS versions, produces RSS 1.0
SpycyRoll - I don't know much about - this is an aggregator, and perhaps other stuff as well
PyRSS2Gen - produces RSS 2.0

Feed Parser

Feed Parser is an awesome RSS reader. It is now hosted on Google Code & Sourceforge - Universal Feed Parser on Google Code (Project Page on SourceForge).

Universal Feed Parser documentation.

Download it, and then start a Python prompt in the same directory.

   1 import feedparser
   2 
   3 python_wiki_rss_url = "http://www.python.org/cgi-bin/moinmoin/" \
   4                       "RecentChanges?action=rss_rc"
   5 
   6 feed = feedparser.parse( python_wiki_rss_url )

You now have the RSS feed data for the PythonInfo wiki!

Take a look at it; There's a lot of data there.

Of particular interest:

`feed[ "bozo" ]`	`1` if the feed data isn't well-formed XML.
`feed[ "url" ]`	URL of the feed's RSS feed
`feed[ "version" ]`	version of the RSS feed
`feed[ "channel" ][ "title" ]`	`"PythonInfo Wiki"` - Title of the Feed.
`feed[ "channel" ][ "description" ]`	`"RecentChanges at PythonInfo Wiki."` - Description of the Feed
`feed[ "channel" ][ "link" ]`	Link to RecentChanges - Web page associated with the feed.
`feed[ "channel" ][ "wiki_interwiki" ]`	"Python``Info" - For wiki, the wiki's preferred InterWiki moniker.
`feed[ "items" ]`	A gigantic list of all of the RecentChanges items.

For each item in feed["items"], we have:

`item[ "date" ]`	`"2004-02-13T22:28:23+08:00"` - ISO 8601 date
`item[ "date_parsed" ]`	`(2004,02,13,14,28,23,4,44,0)`
`item[ "title" ]`	title for item
`item[ "summary" ]`	change summary
`item[ "link" ]`	URL to the page
`item[ "wiki_diff" ]`	for wiki, a link to the diff for the page
`item[ "wiki_history" ]`	for wiki, a link to the page history

Aggregating Feeds with Feed Parser

If you're pulling down a lot of feeds, and aggregating them:

First, you may want to use Future threads to pull down your feeds. That way, you can send out 5 requests immediately, and wait for them all to come back at once, rather than sending out a request, waiting for it to come in, send out another request, wait for it to come back in, etc., etc.,.

   1 from future import Future
   2 
   3 hit_list = [ "http://...", "...", "..." ] # list of feeds to pull down
   4 
   5 # pull down all feeds