node package manager

meetup-web-platform

A collection of Node web application utilities developed for the Meetup web platform

npm version Build Status Coverage Status

Web platform

This is the base platform for serving Meetup web apps including the public website and admin. It provides a Hapi webserver and a set of conventions for composing applications with React + Redux.

In general, application-specific code will live outside of this package.

Docs

Usage

Environment

Platform configuration is read from environment variables, which must be declared in .mupweb.config in your home directory (i.e. $HOME/.mupweb.config) in the following format:

API_HOST=api.meetup.com
API_PROTOCOL=https
DEV_SERVER_PORT=8000
ASSET_SERVER_HOST=0.0.0.0
ASSET_SERVER_PORT=8001
OAUTH_AUTH_URL=https://secure.meetup.com/oauth2/authorize
OAUTH_ACCESS_URL=https://secure.meetup.com/oauth2/access
MUPWEB_OAUTH_KEY=<check with an admin>
MUPWEB_OAUTH_SECRET=<check with an admin>
PHOTO_SCALER_SALT='<check with admin>'  # single quotes are required
CSRF_SECRET='<any random string over 32 characters long>'
COOKIE_ENCRYPT_SECRET='<any random string over 32 characters long>'

Note: you can use dev.meetup.com URLs for API_HOST, OAUTH_AUTH_URL, and OAUTH_ACCESS_URL, but you will need to ensure that your devbox is up and running with a recent build of Meetup classic.

To automatically add these env variables into your terminal session, source the config file in your .bashrc or .zshrc:

set -a  # auto-export all subequent env variable assignments
source $HOME/.mupweb.config
set +a  # turn off auto-export of env variables

Releases

This package uses semver versioning to tag releases, although the patch version is determined exclusively by the Travis build number for pushes to master. Major and minor versions are hard-coded into the Makefile.

Manual pushes to master and PR merges to master will be built by Travis, and will kick off the yarn publish routine. The currently-published version of the package is shown on the repo homepage on GitHub in a badge at the top of the README.

Development/Beta releases

When developing a consumer application that requires changes to the platform code, you can release a beta version of the platform on npm by opening a PR in the meetup-web-platform repo. When it builds successfully, a new beta version will be added to the list of available npm versions. The generated version number is in the Travis build logs, which you can navigate to by clicking on 'Show all checks' in the box that says 'All checks have passed', and then getting the 'Details' of the Travis build.

screen shot 2016-10-29 at 10 25 20 am screen shot 2016-10-29 at 10 25 29 am

At the bottom of the build log, there is a line that echos the GIT_TAG. If you click the disclosure arrow, the version number will be displayed, e.g. 0.5.177-beta.

screen shot 2016-10-29 at 10 25 59 am screen shot 2016-10-29 at 10 26 06 am

You can then install this beta version into your consumer application with

> yarn install meetup-web-platform@<version tag>

Each time you push a change to your meetup-web-platform PR, you'll need to re-install it with the new tag in your consumer application code.

The overall workflow is:

  1. Open a PR for your meetup-web-platform branch
  2. Wait for Travis to successfully build your branch (this can take 5+ minutes)
  3. Get the version string from the build logs under GIT_TAG
  4. (if needed) Push changes to your meetup-web-platform branch
  5. Repeat steps 2-3

Introductory Resources

Basic knowledge of reactive programming using RxJS 5 is a pre-requisite for being able to work in this repository. https://www.learnrxjs.io/ manages a good list of starting resources, specifically:

Suggestions:

  • Reference the api docs regularly while watching videos (http://reactivex.io/rxjs/).
  • Play around with the JSBin in the egghead.io videos (console.log to each transformation step, etc).

Modules

Server

The server module exports a startServer function that consumes a mapping of locale codes to app-rendering Observables, plus any app-specific server routes and plugins. See the code comments for usage details.

Middleware/Epics

The built-in middleware provides core functionality for interacting with API data - managing authenticated user sessions, syncing with the current URL location, caching data, and POSTing data to the API.

Additional middleware can be passed to the makeRenderer function for each specific application's client and server entry points.

Epic middleware

Based on redux-observable, this middleware provides the following functionality through "Epics":

Auth epics/auth.js

in development - API will provide most auth needs

Auth/login requires side-effect interaction with an authentication server, which has an endpoint distinct from the regular data API - the current branch implements it as part of the API server, responding to requests to /api/login.

on LOGIN_REQUEST, which provides credentials

  1. Format credentials for API using a query for login
  2. Make API call
  3. Trigger LOGIN_SUCCESS containing API response (could format before triggering action if needed)
  4. Write auth cookie to maintain login across sessions

on LOGOUT (no attached data)

  • delete auth cookie (not currently implemented)
  • clearing local state is not a side effect, so it's handled directly by the auth reducer in coreReducers

Sync epics/sync.js

This epic is currently only responsible for fetching the data from the API server on initial render or client-side user navigation.

on server/RENDER or LOCATION_CHANGE, which provide a location (URL):

  1. Match location to defined routes and extract the renderProps like URL path and querystring params
  2. Check routes for query functions that return data needs, and process them into an array
  3. Trigger API_REQUEST containing the queries

on API_REQUEST, which provides queries:

  1. Send the queries to the application server, which will make the corresponding external API calls.
  2. When the application server returns data, trigger API_SUCCESS action containing API response array and query array
  3. If the application server responds with an error, trigger API_ERROR

Interesting feature: navRenderSub is a Rx.SerialDisposable, which means that when a user navigates to a new page, any "pending" API requests will not be processed. This is a Very Good Thing because it means that we won't be calling API_COMPLETE for a page load that is no longer applicable. A similar tool is used for the AuthEpic so that only one login request can be processed, but it's less likely to be an issue there since it's rare that users would be trying to log in repeatedly without waiting for previous login requests to be processed.

Cache epics/cache.js

The cache epic provides optimistic state updates for any data specified by active route queries, similar to the SyncEpic, but using locally cached data rather than an API. It is client-side only, and it does not suppress data fetched from the server - it simply pre-empts it before the API response returns and overwrites everything with the latest server data.

on API_REQUEST, which provides queries for the requested route

  1. For each query, use the JSON-encoded query as a key into the cache
  2. Format responses from the cache to look like API responses
  3. Trigger CACHE_SUCCESS containing the cache hits

on API_SUCCESS, which provides fresh info from the server

  • make an entry in the cache with key as the JSON-encoded query and the value as the corresponding response

The cache is stored locally in the user's browser using IndexedDB so that it will survive multiple sessions. All API queries are stored, and the data is refreshed each time the query is re-requested.

In dev, be aware that the cache may be masking failed API requests - keep your network dev tools open and watch the server logs to make sure you haven't broken anything. We will add tooling to make such cases more obvious in the future.

Disable cache

By design, the cache masks slow responses from the API and can create a 'flash' of stale content before the API responds with the latest data. In development, this behavior is not always desirable so you can disable the cache by adding a __nocache param to the query string. The cache will remain disabled until the the page is refreshed/reloaded without the param in the querystring.

http://localhost:8000/ny-tech/?__nocache

POST

POST API requests are handled by PostMiddleware, which provides a generalized interface for sending data to the API and handling return values. The middleware only responds to actions that have a POST_ prefix or a _POST suffix in the action type. Furthermore, the action.payload must have a Query object (with a type, ref, and params) as well as onSuccess and onError actionCreators to receive the data response from the API - they must be actionCreators because their return values will be dispatched by the middleware in the API success/error case.

Use reducers to parse the response and update application state.

Client

Rendering 'empty' state with <NotFound>

To correctly render a 'not found' state for a feature, you should render a <NotFound> component, which the server will use to set the response status to 404.

Example:

import NotFound from 'meetup-web-platform/lib/components/NotFound';
 
class GroupContainer extends React.Component {
    render() {
        if (!this.props.group) {
            return (
                <NotFound>
                    <h1>Sorry, no matching group was found</h1>
                </NotFound>
            );
        }
 
        return <GroupDetail group={this.props.group} />;
    }
}

Tracking

When starting the server, applications provide a platform_agent identifier, e.g. 'mup-web' that is used to tag all of the automatically-generated tracking data produced by platform-related activity, including data requests, browser sessions and login/logout actions. Over time, this system will expand to include click tracking and other types of tracking defined by the Data team and implemented through platform-provided unique IDs.

More info in Confluence here

Dev patterns

Async

Use Promises or Observables to handle async processing - the latter tends to provide more powerful async tools than the former, particularly for long processing chains or anything involving sequences of values, but Promises are good for single async operations. Do not write functions that fire callbacks.

When using Observables, you can always throw an Error and expect the subscriber to provide an onError handler. When using Promises, call Promise.reject(new Error(<message>)) and expect that the caller will provide a .catch() or onRejected handler.

Error Handling

Guidelines:

  1. Use Error objects liberally - they are totally safe until they are paired with a throw, and even then they can be usefully processed without crashing the application with a try/catch.
  2. Use throw when there is no clear way for the application to recover from the error. Uncaught errors are allowed to crash the application and are valuable both in dev and in integration tests.
  3. Populate state with the actual Error object rather than just a Boolean or error message String. Error objects provide better introspection data. For example, a validation process might return null (for no validation errors) or new Error('Value is required') rather than true (for "is valid") or false.
  4. Many errors will have an associated Redux action, such as LOGIN_ERROR - keep the corresponding state updates as narrow as possible. For example, LOGIN_ERROR should only affect state.app.login - all affected UI components should read from that property rather than independently responding to LOGIN_ERROR in a reducer. Do not create a high-level error properties state
  5. When using Promises or Observables, always provide an error handling function (catch for Promises, error for Observables)