api-core
The generic component for creating an API with Blueprint support.
This repo is meant to be required as a module on other domain-specific API repos.
- Design decisions
- How do I set this up?
- How do I configure this?
- How do I make requests? (example run)
- How do I add a new endpoint?
- How do I test this?
- How do I make a release?
Design decisions
Authenticaton
Redis is used the state for OAuth2 (tokens, etc...).
The OAuth2 scopes are defined as follows:
-
if none is specified, the service is used for internal OAuth2 authentication purposes and, thus, is not exposed through HTTP
-
if
public
the service is available publicly through HTTP (i.e. it does not require any authentication) -
for any other value, the service is available through HTTP, and this property is used for OAuth2 scope authorization purposes
You can overview the decisions behind authentication processes here.
HTTP
The content type of all responses is application/json
. For security reasons, only application/json
requests are accepted.
-
When a resource does not exist:
-
GET a single resource
- return an HTTP 404 response
-
GET a list of resources
- return an empty array
-
UPDATE a resource
- return an HTTP 404 response
-
Data persistence
PostgreSQL is currently the only supported database.
Services
There was the need to decouple data access operations from business logics for maintainability reasons.
For this reason, there are two kinds of services: DataAccess
and Business
.
Business
services deal with implementing business logics and/or interacting with system components that are external to an API. They may communicate with multiple DataAccessService
to achieve this goal (0..*).
DataAccess
services aim to fetch data from a database (e.g. PostgreSQL). They may be shared between Business
services (1..*).
How do I set this up?
-
Install Node.js and the project's dependencies
-
get NVM
-
cd api-core
-
nvm install
-
npm install
-
macOS
-
Install Redis
brew install redis
-
install PostgreSQL
brew install postgres
Ubuntu 14.04
-
Get ready to use PPAs
-
we're trusting Chris Leas's PPA. He's legit because of Chris Lea Joins Forces With NodeSource
sudo apt-get -y install python-software-properties sudo add-apt-repository -y ppa:chris-lea/redis-server sudo apt-get -y update
-
-
Install Redis
- we need >= 3.0.x to be safe
apt-get install redis-server
-
Install PostgreSQL
apt-get install postgresql-9.4
How do I configure this?
This module's configuration is environment variable-based.
The following env vars are supported:
-
TOKEN_SIZE
-
is the size of both access and refresh tokens in bytes
-
e.g.
16
days
-
-
CLIENTS
-
is the comma-separated set of colon-separated client properties to setup on Redis on API load. These can be used for the
client_credentials
OAuth2 grant type. -
note that all client scopes must be included on the
SCOPES
env var -
client_type:client_id:client_secret:scope -> e.g.
confidential:test:test:test
days
-
-
SCOPES
-
is the comma-separated list of colon-separated pairs of scopes and their expiration relative to the default
ACCESS_TOKEN_DURATION
-
e.g.
SCOPES=user:5,retailer:1,scraper:Infinity
-
-
ACCESS_TOKEN_DURATION
-
is the amount of days an access token lasts
-
e.g.
0.04
days ~1
hour
-
-
REFRESH_TOKEN_DURATION
-
is the amount of days an refresh token lasts
-
e.g.
7
days
-
-
DBS
-
is the comma-separated list of allowed DBs
-
e.g.
DBS=operational,analytics,billing
-
-
ANALYTICS_DB
,OPERATIONAL_DB
and vars of the formX
_DB-
they are postgres URIs
-
e.g. ANALYTICS_DB=postgres://postgres:@localhost/analytics
-
e.g. OPERATIONAL_DB=postgres://postgres:@localhost/operational
-
-
OPERATIONAL_DB_HOST
-
is the hostname where the operational db is located
-
e.g.
localhost
-
-
NODE_DEBUG
-
may be used to expose debug logs for different components
-
e.g.
NODE_DEBUG=runner,config,services,postgres,store,api,oauth,web,auth node index.js api.json
-
How do I run this?
-
Pick your poison
-
to start it as a regular Node.js application pick either option, where
CONFIG
is the path to theapib
file:-
node index.js $CONFIG
-
CONFIG=$CONFIG node index.js
-
-
to run it with PM2
- ensure it's installed
* `npm install -g pm2`
-
pm2 start ecosystem.json --env env_name
-
where
env_name
is the environment to run the API under (e.g. production) -
--env env_name
should be ommited if running on development mode
-
-
How do I make requests? (example run)
Note: this repo needs a supermodule to include it as a submodule. This section is meant as an example of steps that should be executed from the said project.
-
Run the API
-
Get an
access_token
-
with curl
-
DATA="{ \"username\": \"$EMAIL\", \"password\": \"$PASS\", \"grant_type\": \"password\", \"scope\": \"retailer\", \"client_id\": \"test\", \"client_secret\": \"test\" }"
-
export ACCESS_TOKEN="$(curl -v http://localhost:3000/oauth/token -H "Content-Type: application/json" -X POST -d "$DATA" | JSONStream 'access_token' | tr -d '[]"')"
-
-
with the api-client
-
-
check if you got an
access_token
-
with curl
echo $token
-
with the api-client
-
-
Make a request using the
access_token
you just got-
with curl
-
curl -iv http://localhost:3000/dummy -X GET -H "Authorization: Bearer $token"
- if everything's fine, you should have gotten a HTTP 200 OK response and some JSON payload
-
-
with the api-client
-
How do I add a new endpoint?
-
Declare your new endpoint's behaviour. To do so , you may use:
-
an *.apib API Blueprint file
Note: Using an
.apib
file is preferrable, as Dredd can be used to validate its structure.-
Use JSON SCHEMA to define the various request-reply flows for the endpoint. More info on JSON Schema draft v4.
-
the OAuth
scope
can be defined on theTransaction
section before the service name-
e.g.
retailer:updateRetailerSettings [POST]
-
e.g.
public:getDomainProductPrice [GET]
-
Note: to have a service that is used only for internal authenticaton purposes, only the service name should be provided, i.e
email [/oauth/token]
:
-
-
the
db
to use can be defined on theRequest
section after the stringto
-
the respective env var is fetched from this value.
-
e.g.
Request to analytics (application/json; charset=utf-8)
-
-
the
data
services that will be used by the associated business service can be defined on theRequest
section after the stringwith data
- e.g.
Request to analytics with data [service], [service] ... (application/json; charset=utf-8)
- e.g.
-
-
a configuration file
*.json
-
Expose the services through an array of service objects whose options are as follows:
-
name
[mandatory]: the name of the service for which to spawn data and business components -
data
[optional]: an array of data access services the business service is able to communicate with- default: the value for
name
- default: the value for
-
route
[optional]: the HTTP route to reach the service-
default: the value for
name
-
look into express for more information
-
-
method
[optional]: the HTTP method to reach the service- default:
GET
- default:
-
scope
[optional]: a string or boolean to specify the service visibility- default: the service is used for internal authentication purposes and is not exposed
-
db
: the address for the service's PostgreSQL database or the name of the DB to use-
e.g.
postgres://postgres:@localhost/db_name
-
e.g.
operational
-
-
-
-
-
Setup the data access service
-
Note: your service should inherit from the base service class and override the
getSQL
method- alternatively, if there's a helper class under
data/*.js
, you might only have to overridegetSQL
- alternatively, if there's a helper class under
-
create a stub under
data/serviceName.js
const DataAccessService = require('../api-core/lib/data/index'); module.exports = class Service extends DataAccessService { getSQL(params) { // return the SQL the service is going to execute // it should be parameterized with ${parameterName} return this.SQL`the sql`; } }
- don't forget to remove the comments
-
-
(Optionally) setup the business service
-
Note: if there's no need to extend the default BusinessService functionality, you shouldn't create any file. A default service will be loaded instead
-
create a stub under
business/serviceName.js
const BusinessService = require('../api-core/lib/business/index'); module.exports = class Service extends BusinessService { run(params) { // Coordinates how different requests to data access services should be orchestrated. // If not overridden, all services described under the `data` config // will be called simultaneously and their results aggregated // It should return a Promise. } }
-
-
Implement your services
-
Fill in the code to make them useful
-
Before committing any external dependency be sure to check the project with NSP and be sure no new vulnerabilities are found.
-
How do I test this?
npm test
How do I make a release?
-
Install git-flow
-
npm version $SEMVER
-
git flow release start $VERSION
to start a new release -
git flow release finish $VERSION
to finish the release-
develop
gets merged tomaster
-