This post originated from an RSS feed registered with Python Buzz by Aaron Brady. Original Post: Check a Google Sitemap for bad URLs Feed Title: insom.me.uk Feed URL: http://feeds2.feedburner.com/insommeuk Feed Description: Posts related to using Python. Some tricks and tips, observations, hacks, and the Brand New Things. Latest Python Buzz Posts Latest Python Buzz Posts by Aaron Brady Latest Posts From insom.me.uk

Create cagsmfbu.py:

import sys
import httplib2
import xml.dom.minidom as md

H = httplib2.Http()
X = md.parse(open(sys.argv[1]))
locs = X.getElementsByTagName("loc")
for loc in locs:
    url = loc.childNodes[0].nodeValue.encode('u8')
    try:
        res, content = H.request(url)
        print "%s\t%d" % (url, res.status)
    except:
        print "%s\tTOOMANY" % url
    sys.stdout.flush()

And then

% python cagsmfbu.py sitemap.xml | tee output.tdf