Podcast Metadata Scraping

Turns out scraping metadata from iTunes compatible podcast RSS feeds is extremely easy. I couldn't remember which episode Lebanese Politics Podcast had the hosts talking about the Beirut trash protests, so I went to iTunes to try and find out. However, the search in iTunes is shockingly bad. Plus I wanted to run a grep through the thing for my own purposes anyways, so I decided to scrape this information down.

The code is at https://github.com/peixian/podcast-description-search and distributed under an MIT license.

It takes in the iTunes RSS feed and can either do a simple query on the podcast description, or throw back a csv:

$ ./scrape.py -h
usage: search.py [-h] [-q QUERY] [-o OUT] itunes_url

Takes in an itunes podcast id, searches for a specific string, also dumps all the shows to a csv.

positional arguments:
itunes_url            URL from itunes for podcast

optional arguments:
-h, --help            show this help message and exit
-q QUERY, --query QUERY
Specific string to search for. This is quite dumb so the csv with a more complex engine might be better
-o OUT, --out OUT     Path to dump the results to a csv

Since iTunes compatible RSS feeds are standardized, turns out the public API left up at https://itunes.apple.com/lookup takes a single ID url param, which is a iTunes global unique ID. You can do a GET on this endpoint without throwing any auth, and it'll return back where the original RSS feed is hosted at. In my case, it was hosted at Soundcloud, which then provides the open endpoint to grabbing the iTunes standarized RSS format.