Tweet Sentiment for AWS CDK
This is an AWS CDK library that allows you to search for terms in tweets and detect the sentiment in the search results.
It works by periodically polling the freely available public Twitter Standard Search API for the given search terms and feeding the tweets in a stream for analysis and aggregation.
The result is exposed as a DynamoDB table. Essentially, the table contains one item per term with the breakdown of tweets per sentiment.
$ aws dynamodb scan --table-name CdkSampleAppStack-TweetSentimentB1251CEC-LNLOYTJ6H5JB | jq '.Items'[ , , ]
Usage
Create an AWS CDK project and install the npm
package.
npm install cdk-tweet-sentiment
Add the CdkTweetSentiment
construct to your stack.
; ...new CdkTweetSentimentthis, 'TweetSentiment', ;...
Note that CdkTweetSentiment
is essentially a @aws-cdk/aws-dynamodb.Table
object and can be used as such.
Here you can find a self-contained CDK application that uses this construct.
Architecture
- A CloudWatch Event Rule triggers the producer lambda function periodically.
- The last received tweet ID for each term is stored as a checkpoint in a DynamoDB table.
- The producer lambda function queries the Twitter Standard Search API for each of the given terms. It updates each checkpoint and feeds the tweets in a Kinesis Firehose delivery stream.
- The processor lambda function transforms each tweet. First, it translates the text of the tweet, if it is not in English, using the Translate service. Then, it detects the sentiment of the English text using Comprehend.
- The tweets are loaded from the stream to an S3 bucket and kept for further processing by the user.
- S3 monitors the bucket and invokes the aggregation Lambda function when an object in inserted. The aggregator retrieves the tweets in the object and updates a DynamoDB table with the sentiments.
Twitter API
The tweets are retrieved via the Twitter Standard Search API. In order to use the Twitter API, your need to
- Apply for a Twitter developer account.
- Create a Twitter app.
- Generate the consumer keys and bearer token for the application-only authentication.
License
This repository is under the Apache License 2.0.