A simple continuous harvester for twitter
This application is able to capture tweets which happen around the world. Currently it works only with the Twitter stream API 1.1.
cfg/cfg.jsonand create at least one capture
mail_alertflag in main configuration)
true(default), the captured tweets are written to the file system with the following convention:
true(should be false by default), a kind of queue is created (directory 'data/TODO') where filenames to consume by an external process. This allow to write the tweets to any db
$ npm install --save twitter-harvest
$ npm install -g forever$ forever start twitter-harvest.js
With forever it is possible to run the task 'forever'. And leave your session.
put all the agent definition files to the agent directory (one file per agent).
$ cat cfg/agents/*.json
to capture all the tweets where there is a mention of geneva word for several languages.
to capture all the tweets which are posted around Geneva area (Switzerland).
truethis agent is launched
more API twitter doc https://dev.twitter.com/streaming/overview/request-parameters
One mail is also sent when the system is started, you should received this mail on your mail box if all well configured.
note : supported mail system is given by nodemailer node module (here is the supported service https://github.com/andris9/nodemailer-wellknown#supported-services), but only gmail was tested for gmail, it is possible you have to decrease the security level of your mail account (so don't use a personal account) and to authorize specifically the application by using this url: https://g.co/allowaccess
Note that currently, we have 3 errors messages when twitter-harvest is launched. This is not important. Here are theses Error messages
MIT © Arnaud Gaudinat