A MEAN.js app for all Cliques advertiser, publisher & administrator management and reporting. Currently live in two environments:
The Console is a single-page-app (SPA) with full client-side MVC functionality provided by Angular.js(v1.x) interacting w/ a RESTful API service powered by Node.js & Express.js. All CSS files are compiled from LESS. For staging & production servers, PM2 is used to run & manage Node.js server processes.
Node.js server-side packages (public & private) are (obviously) managed by NPM. Bower handles management of all client-side packages. Grunt handles all building & linting steps, and can also be used to run the app in development.
This README only provides setup & deployment instructions as well as a high-level overview of the system's components. To learn more, please check out the following resources:
Use the following instructions to install system dependencies & deploy the Console.
The Console is only designed to be deployed on UNIX-based operating systems. It's possible to run it on Windows, but it's more complicated.
Also, in order to successfully deploy the console in any environment, you MUST have read/pull access the cliques-config repository. All network credentials, API keys, etc. are stored in this repository. If you don't have access to this repository, there is likely a reason for that.
The repository depends on cliques-node-utils, which is installed as a private NPM dependency under the NPM @cliques organization. You must have an NPM account and be added as a member of the @cliques organization in order to install this package, or else the setup process will fail.
Once you've cloned the repository locally, run
../cliques-configand add symlink
cliques-configto repository root, if not already present.
NOTE: In order for
npm install to run successfully, you will need to have
gcc installed. On MacOSX,
gcc comes packaged with XCode command line tools. If you run into any NPM errors (especially installing
node-gyp) during install, make sure you've installed XCode command line tools by running
The setup process only needs to be run when the Node.js environment is updated via an update to
console_environment.cfg or if you're cloning the repository for the first time.
To run the server using grunt in local-test mode, run
$ grunt # the default grunt task starts the server w/ Nodemon in `local-test` mode
Once you see the following output, your server should be running on localhost:5000
MEAN.JS application started on port 5000Mongoose connection open to mongodb://xxx.xxx.xx.xx:27017/exchange_devConnected to exchange connection as default mongo DB connection
For client-side development, running the server with grunt is useful because of the various file-watchers & live-reloading functionality. However, I'd recommend setting up a remote debugger like the one provided by WebStorm to run the server.
This script will perform the same steps as
setup-mac.sh, except it will also install or update a couple of system dependencies (namely
libfontconfig1, a Phantom.js dependency) using
apt-get as well. For this step, you'll need to have
sudo privileges on your machine.
The deployment process for Debian has been packaged into a single script. Simply run:
$ ./deploy-console.sh -e dev # use the -e flag to pass the appropriate NODE_ENV environment variable
This script will:
source activate_env.sh -e [NODE_ENV]to activate the Node.js environment
npm installto update NPM & Bower packages
grunt buildstep, again depending on the environment you pass in. This will build & minify all application CSS & JS files.
cliques-console-[dev]PM2 process(es). NOTE: pm2 processes are run with
-i 0, which will spawn concurrent cliques-console processes on all available server cores. So if you are running a server w/ 4 CPU's, you will see 4 processes spin up.
https-redirectprocess (see https-redirect.js), which just listens for non-secure connections over http and redirects them to https.
There are really two distinct "environment" scopes that I'll address below:
production) which specifies database/API credentials and build steps.
The Node.js environment is configured independently from the deployment environment. See below for details.
I use Node Version Manager (NVM) to manage the specific version of Node.js interpreter being used to run the Console. This gets around any potential incompatibilities between different versions of Node.js installed on local systems.
This setup works almost identically to Python's powerful virtualenv package, in that it creates an isolated Node.js environment that depends on explicitly-declared global package versions (see Environment Config below).
NVM installs a "local" Node.js to the
.nvm directory, and all "global" packages are installed to this directory rather than in the system node modules directory.
All versioning information for NVM and "global" (not actually "system" global but installed to the NVM directory) Node packages are stored in the console_environment.cfg config file found in the cliques-config. You may or may not have access to this repository depending on your team status, please refer to Config Permissions for details.
There are three primary deployment environments for the Console. The
NODE_ENV environment variable is used to indicate which environment should be loaded:
application.jsfile, but not minified to still allow for some debugging.
You can use the
activate_env.sh shell script to "activate" the local NVM Node.js environment AND a specific deployment environment (all this does is just set the
NODE_ENV environment variable). This script is used in the deployment step for
production, but it is also useful on its own when you are working on the command line and need to access to the Console's specific environment.
# use the `-e` flag to pass in a deployment environment# default is 'production'$ source activate_env.sh -e dev
Mongoose schemas are found in this repository under
However, only models that are specific to the Console reside here (ex: Users, Organizations, Payments, etc.). Models that need to be accessed by multiple repositories are found in the shared cliques-node-utils repo in order to centralize shared model specs.
Below is an EER diagram that should help visualize how all Cliques models relate to one another (Last updated 2016-12-16, release 0.6.55):
There are two MongoDB "databases" (which are like "schemas" in SQL parlance), one for production data & one for development:
The production database. Contains both schemas specified in this repository, as well as those specified in cliques-node-utils.
This is the development database. All collections are copied from
exchange every 2 hours via a Python ETL.
Each time the collections are synced, all changes to
exchange_dev collections are wiped clean and replaced with the the most recent data from production.
For aggregate collections like
hourlyadstats, in the interest of preserving storage space, only the last 72 hours of data will be kept. So don't be surprised when you only see the last 3 days' worth of data in charts when running the console in dev!