Volos SQS connector
The Volos SQS connector is a Node.js module that lets you use Amazon Simple Queue Service (SQS) through a RESTful API. The module is designed to work on Apigee Edge but can be run anywhere Node.js applications can run. You can use this module without any dependency on Apigee.
Quick example
This module allows maps SQS operations to RESTful API resources and query parameters. You can perform CRUD operations on objects like messages and queues.
For example, you might ask for a list of your SQS subscriptions like this:
curl http://localhost:9099/queues
and get back a JSON response like this:
{
"action": "GET",
"params": {
"qp": {}
},
"path": "/queues",
"url": "/queues",
"data": {
"ResponseMetadata": {
"RequestId": "799655555-55555-55555-55555-cf14b4bc55555"
},
"QueueUrls": [
"https://sqs.us-east-1.amazonaws.com/650324455555/another2",
"https://sqs.us-east-1.amazonaws.com/650324455555/another3",
"https://sqs.us-east-1.amazonaws.com/650324455555/myqueue"
]
},
"timestamp": 1405525899498,
"duration": 1545,
"applicationName": "volos-sqs"
}
...
To get a larger set of attributes, use the query parameter expand=true. This option gives you the flexibility to have a small message payload for a subset of fields if those are all that are required. For example:
curl http://localhost:9058/subscriptions?expand=true
Installation
The volos-sqs module is designed for Node.js and is available through npm:
$ npm install volos-sqs
Usage
There are two examples below, one basic example and one that uses the avault (Apigee Vault) Node.js module, which is a secure local storage module. Apigee Vault is used to encrypt sensitive login credentials sent to the backend database.
Simple example without Apigee Vault
The example below shows a simple usage of the volos-sqs connector using the http module to proxy requests to the connector.
Note: In this example, the SQS credentials are specified in plaintext. This is not a best practice.
var sqsConnector = require('volos-sqs');
var http = require('http');
var configuration = require('./configuration.js');
var profile = {
region: 'myregion'
accessKeyId: 'myaccesskeyid',
secretAccessKey: 'mysecretkey'
};
vvar sqsConnectorObject = new sqsConnector.SqsConnector({"profile": profile, "configuration": configuration});
var svr = http.createServer(function (req, resp) {
sqsConnectorObject.dispatchRequest(req, resp);
});
svr.listen(9089, function () {
sqsConnectorObject.initializePaths(configuration);
console.log(sqsConnectorObject.applicationName + ' node server is listening');
});
Simple example using the Apigee Vault for local secure storage
This example shows the usage of the avault module to provide a secure local storage option for credentials and endpoint configuration.
This example assumes you have configured a vault and loaded a configuration profile with a key 'my_profile_key'. See the section "SQS connection profile" below for a quick example. For a complete description of the avault module see the Apigee Vault page on GitHub.
var sqsConnector = require('volos-sqs');
var http = require('http');
var vault = require('avault').createVault(__dirname);
var sqs;
vault.get('my-profile-key', function(profileString) {
if (!profileString) {
console.log('Error: required vault not found.');
} else {
var profile = JSON.parse(profileString);
var svr = http.createServer(function(req, resp) {
sqs.dispatchRequest(req, resp);
});
svr.listen(9100, function() {
sqs = new snsConnector.SqsConnector({"profile": profile, configuration: configuration, receiveMessageCallback: receiveMessageCallback});
sqs.initializePaths(configuration.restMap);
console.log(sqs.applicationName + ' server is listening');
});
}
function receiveMessageCallback(body, message) {
var dfd = Q.defer();
console.log('body: ' + body + ', message: ' + message);
dfd.resolve('');
return(dfd.promise);
}
});
Getting started with your app
To use this connector you need a correctly configured SQS connection for your AWS SQS account.
SQS connection profile
The SQS configuration profile is used by the connector to establish a connection to the backend SQS account. The profile includes the following fields:
- region - The AWS region.
- acessKeyId - The access key ID for your Amazon Web Services SQS account.
- secretAccessKey - The secret access key for your Amazon Web Services SQS account.
Tip: Log in to your SQS account to find your security credentials, including AWS access keys.
Example:
var profile = {
region: 'myregion',
accessKeyId: 'myaccesskeyid',
secretAccessKey: 'mysecretkey'
};
Optional: Encrypting the connection profile with Apigee Vault
The avault module provides local, double-key encrypted storage of sensetive information such as credentials and system endpoints. This provides an option to store these kinds of data in a format other than text/plain.
In order to insert a value into the vault a command-line tool is provided called vaultcli. This tool comes with the avault module. Here's an example:
$ vaultcli --verbose --value='{"region"; "myregion", "accessKeyId":"myaccesskeyid", "secretAccessKey": "mysecretaccesskey"}' my-vault-name
Note: These are the same keys that are required in the plaintext version of the profile. If this command completes successfully you will find two new files:
store.jsandkeys.js. Place them in the root directory of thevolos-sqsmodule.
For more detailed usage of the avault module refer to the Apigee Vault page on GitHub.
Using the SQS connector
If you run the Node.js server and call the API like this:
curl http://localhost:9100/
The server will return the following API usage information:
{
"usage": {
"Commands": [
"GET /messages Query Parameters: queueUrl, maxNumberOfMessages, visibilityTimeout, waitTimeSeconds",
"POST /messages FormVars: queueUrl, messageBody Optioanl FormVars: delaySeconds",
"DELETE /messages Query Parameters: queueUrl, receiptHandle",
"GET /queues/:id",
"GET /queues Query Parameters: prefix",
"POST /queues FormVars: queueName",
"PUT /queues FormVars: queueUrl",
"DELETE /queues Query Parameters: queueUrl"
],
"Common Optional Parameters": [
"limit=<maxResults>",
"expand=true"
]
}
}
For example, to get a list of all of your queues:
curl http://localhost:9099/queues
To get a list of all the messages in a specific queue:
curl http://localhost:9100/messages?queueUrl=https://sqs.us-east-1.amazonaws.com/555555555555/myqueue
You might get a response like this:
{
"action": "GET",
"params": {
"qp": {
"queueUrl": "https://sqs.us-east-1.amazonaws.com/555555555555/myqueue"
}
},
"path": "/messages",
"url": "/messages?queueUrl=https://sqs.us-east-1.amazonaws.com/555555555555/myqueue",
"data": {
"receiveResult": {
"ResponseMetadata": {
"RequestId": "5555-5555-5555-ad1e-ec975555364c"
},
"Messages": [
{
"MessageId": "437b5555-5555-5555-5555-6c44015555",
"ReceiptHandle": "+eXJYhj5rDo/1645555a7fW5555uezfvnYi9PE6gx4m+43C5555/obWbtYqGi5TA5555ndGsE2UpeIh5555s1J1mjyHEFHCzxxP0G9XxPq9T95555Yxc/BQj5555FKwlXqR5555b2ZZnY++JyD/IGQIJgduD2D5555/Ulc7qD9WKkSoCGQXT5555rycoG1NlJuW9wz/mG75555/YS5555I7NMm2msK2DlArdYXnzD535Jw8G51pR55551EGJ6vEMtgNNXQQ=",
"MD5OfBody": "9b55552f99d25c52f175b55553c",
"Body": "Hello world"
}
]
},
"dequeueResult": {
"ResponseMetadata": {
"RequestId": "a61ca5555-5555-5555-5555-6c118555525"
}
}
},
"timestamp": 1405527082391,
"duration": 4018,
"applicationName": "volos-sqs"
}