A talk by @benjamincoe
Birdeater is a command-line tool for backing up a user's public Tweets in JSON format.
First, make sure you have node package manager installed:
curl | sh
Then run the following commands to install and run Birdeater:
npm install birdeater -gbirdeater --user=shitmydadsays
Be mindful when running it, as Twitter limits the number of requests that a single client can make per hour.
Birdeater does not use Twitter's API. It was built as a demonstration of an approach I like to use for parsing structured information from unstructured HTML. Here's how it works:
An http connection is made to a user's public timeline using the request library:
This returns an HTML representation of the tweets. JQuery is used to extract structured information from this:
I find that Node.js, coupled with JQuery, works great for building web crawlers:
This approach has become my hammer when web scraping tasks come up.