@lonelyplanet/lp-analytics

lp-analytics

👉 A JS library providing a standard interface for event tracking for analytics services across our frontend codebases, and a set of tools for implementing said interface across a variety of JS environments.

FAQ

Where is the implementation spec?

Here

How do I use it?

Install from npm:

npm install --save @lonelyplanet/lp-analytics

...aaand from that point it depends on what sort of JS environment you're working in.

See the Very Broad Overview for details, under subheading "that's still pretty abstract", where three representative setups are examined.

How do I run tests?

npm run test

How do I publish to NPM?

Make a PR into develop without worrying about versioning/etc. Get it reviewed & merge it.

Then use standard-version to update the version/changelog & push to develop.

(per https://www.npmjs.com/package/standard-version, with some tweaks):

1. when you land commits on your master branch, select the Squash and Merge option.

• JQ note: it's optional to merge develop into master prior to publishing.

2. add a title and body that follows the Conventional Commits Specification.
3. when you're ready to release to npm:
4. git checkout develop; git pull origin develop
5. run npm run preversion (this tests, then builds), then npm run version:patch, npm run version:minor, or npm run version:major.

• 👉 semvar cheatsheet

6. ensure that the version has been changed as desired in package.json and package-lock.json, and the changelog is as you expect.
7. run npm run push. Now the NPM package should be updated.
8. PR the changes generated by the release from develop to master

standard-version does the following:

1. bumps the version in package.json/bower.json (based on your commit history)
2. uses conventional-changelog to update CHANGELOG.md
3. commits package.json (et al.) and CHANGELOG.md
4. tags a new release

How do I update the static scripts?

npm run update_static

For that matter, what are the static scripts?

You'll notice the presence of two scripts in /src/scripts: initializeDataLayer/index.ts and initializeTrackingContainers/index.ts.

These handle setting up the tracking framework and including gtm.js on the page; really they aren't doing too much, just making sure that window.lp.analytics.dataLayer and window.lp.analytics.track are where we expect them to be, running GTM's snippet, etc.

If you use any of React/ES6 tooling, you won't have to worry about including these since you'll be making use of the same methods it invokes directly, which will handle all of the same setup.

But if you're in a legacy codebase and need to include your tracking code as script tags directly in a html/haml/etc. file, you won't be able to import those methods and thus you'll need to include these already-webpacked-babelified-etc. script tags as static assets.

The actual links you'll use are:

• https://assets.staticlp.com/lp-analytics/initialize-data-layer.js
• https://assets.staticlp.com/lp-analytics/initialize-tracking-containers.js

(See the Very Broad Overview's examination of destinations-next for details)

When updates are made that would affect either of these scripts, you'll want to update the static assets as well.

Do so via npm script:

npm run update_static

This will build /dist and then upload the scripts, so it's all you'll need to do.

📝 Note 📝 that if multi-factor authentication is enabled for your AWS account (it probably is), you'll need to authenticate from the CLI for the script to work.

One avenue for doing so is to install aws-mfa as detailed here: https://github.com/broamski/aws-mfa.

A Very Broad Overview Of What (And Why) This Is

data layer? 🧐

We need to be able to draw inferences about how users interact with our sites in order to guide our digital strategy. Why devote time & resources to a page noone visits? etc.

We could keep track of all that data ourselves, but there are many third-party analytics services that can do a lot of the heavy lifting for us.

A standard frontend interface for integrating with these services is the dataLayer, a JavaScript array that exists somewhere within the user's web browser's JS environment.

When things happen on the page that we're interested in tracking (a page loaded, a button clicked, etc.), we push a JavaScript object into the dataLayer array. That object is a data payload. Who clicked what? etc.

And the third party services take it from there. Not that our work necessarily ends there, but anything else would be done via the third-party service, not our JS.

that sounds simple why did you make this library 🧐

Well, we have a lot of discrete codebases that have evolved independently over a long time. Our codebases

1. Use a variety of JS setups
2. Implement(-ed) a variety of third-party analytics services
3. Do not necessarily use consistent payload shapes for the same event occurring in different codebases
4. Do not necessarily use consistent keys for the key/value pairs that those payloads contain

At some point shortly before your author's hiring at LP, our organization decided to devote resources to improving the integrity of our analytics data. Evidently our decision-makers have not been comfortable making decisions based on our existing analytics implementations.

This led to a partnership with Analytics Pros.

The first collaboration between AP & LP yielded an analytics implementation for landing & marketing page events. More on that in a sec**

The second collaboration called for a complete overhaul of our existing analytics implementations, everywhere. Your author has been working on this.

AP favors a setup that pushes data to Google Tag Manager (GTM), and then uses GTM to push events to Google Analytics (GA) based on that data.

** It called for event payloads that were unknown to the author during the majority of the work that went on during the second collaboration. Late into the process, the specifications for the first collaboration were added into the spec for the second (labeled as "moved from old spec"), which is why their implementations may only be present in select codebases. They may also not have their interfaces/types/enums/etc. present in lp-analytics at all yet-- please add them as needed (you might be asked to track an event defined in that first collaboration in another codebase, for example).

in conclusion 🤓

• lp-analytics (this library) sets up a standard interface for event tracking (to address points 3 & 4 above), and provides a set of tools for implementing said interface across a variety of JS environments (to address point 1).

• The specific impetus for its creation has been to implement the GTM/GA-based tracking setup that AP has prescribed for us (as the second LP/AP collaboration).

• As such, it currently only provides for that implementation, but is extensible should the need arise to add other tracking providers.

Further Into The Weeds 🌱

Broadly speaking, here's how the setup works:

• We have a dataLayer array
• Analytics event payloads get pushed there
• GTM receives that data
• GTM pushes events to GA (per its configuration, which we need to set up; that setup is not currently explored here)
• GA makes reports from data from the events
• LP makes informed decisions

Let's get a bit more concrete. How do we implement all that?

• First, note that where dataLayer is relative to the window object is mostly straightforward, but please don't interact with/push to it directly, lest you run afoul of a workaround that was needed for codebases that implement rizzo's head JS. Long story short, there might not be a .push method where you expect one to be.

• Instead, we'll 🚨 (STEP 1) run some setup that'll make for a standardized, abstracted interface.

• Next, take a look at AP's implementation spec, page 10 ("Core Data Layer").

• The spec describes a payload (henceforth called dataLayer-initialized) that we'll want to have placed in the data layer on each page as the page loads.

• So 🚨 (STEP 2) track (ie, append to dataLayer) the dataLayer-initialized payload next.

• Finally, 🚨 (STEP 3), we need to initialize GTM. This just involves running their snippet-- that snippet will append a script tag for gtm.js to the page, which does a bunch of stuff that we won't worry about here. Just know that it has to be done in the <head> tag, and after the dataLayer-initialized payload has been appended to the dataLayer.

• That's the end of the complicated stuff.

• Step 4 is to set up tracking in reaction to client-side user activity (button clicks, etc.). We'll go over this separately (todo).

that's still pretty abstract 🤓

You're right, it's hard to get too specific without looking at a particular implementation since it really depends on the JS setup of the codebase we're looking at.

So let's do that.

🌎 destinations-next 🌎

application_detail.html.hbs (annotated)

{{!-- From Destinations/app/templates/layouts/partials/doctype --}}
{{> "layouts/partials/doctype" }}
<head>
  <script src="https://assets.staticlp.com/lp-analytics/initialize-data-layer.js"></script> 🚨 (**STEP 1**)
  {{> "layouts/partials/data_layer_detail"}}
  {{> "layouts/partials/data_layer_init_track_detail" }} 🚨 (**STEP 2**)
  {{> "layouts/partials/lp_js_details" }}
  <script src="https://assets.staticlp.com/lp-analytics/initialize-tracking-containers.js"></script> 🚨 (**STEP 3**)
  {{> "layouts/partials/meta_detail" }}
<!-- ...etc -->

Step 1: There is some setup required to get our dataLayer situation situated. If you're in a React or ES6 JS environment, you won't actually need to do this as a discrete step, because the methods you import will handle it all for you upon being invoked.

We don't use ES modules here in destinations-next, though, so we'll instead run a script that'll do what we need and has been webpacked and minified for us ahead of time.

See the /src/scripts directory? Everything in there winds up being processed & made available in this manner. Should the need arise, the scripts can be updated to reflect code changes with the command npm run update_static.

Anyway, end result of initialize-data-layer.js is that we're ready to push events to the dataLayer by means of the window.lp.analytics.track method (note that this does not directly interface with the dataLayer or use a .push method).

Step 2: Let's look at the handlebars partial that is being included by this line:

🌎 destinations-next 🌎

data_layer_init_track_detail.hbs

<script>
  (function() {
    window.lp.analytics.track({
      adblock: undefined,
      applicationName: "{{application_name}}",
      articleName: "{{article_name}}",
      articleType: "{{article_type}}",
      atlasId: "{{atlas_id}}",
      campaignPageName: undefined,
      contentCountry: "{{content_country}}" || undefined,
      contentContinent: "{{content_continent}}" || undefined,
      contentCity: "{{content_city}}" || undefined,
      contentNeighborhood: undefined,
      contentRegion: "{{content_region}}" || undefined,
      contentType: undefined,
      destinationDirectory: undefined,
      destinationSubNav: undefined,
      dispatchVariant: undefined,
      event: "{{event_name}}",
      forumCategory: undefined,
      forumContinent: undefined,
      forumCountry: undefined,
      forumPostTitle: undefined,
      hasPhoto: {{has_photo}},
      hasVideo: {{has_video}},
      loggedIn: undefined,
      poiAttributes: undefined,
      poiName: undefined,
      poiType: undefined,
      poiVenueType: undefined,
      siteSection: "{{site_section}}",
      userId: undefined,
    });
  })();
</script>

Note that all possible key/value pairs for dataLayer-initialized are present-- If anything is unknown or irrelevant to the page being tracked, undefined is used to fill in the value. This is by design; it's part of AP's spec.

🎵 Note 🎵 : leave userId, dispatchVariant and loggedIn undefined. We have some machinations in place that'll handle those for us.

If you define any variables prior to calling window.lp.analytics.track, be sure to wrap everything up in an IIFE so as to avoid polluting global scope (this code doesn't define any, but probably used to, hence the IIFE).

By the way, you probably noticed {{> "layouts/partials/data_layer_detail"}} right above {{> "layouts/partials/data_layer_init_track_detail" }} in application_detail.html.hbs. We haven't addressed that yet, but it sure smells like part of our analytics stew, right?

Yep, that's the legacy pageload payload. I've left that code alone for the time being, since

🗣 removing the legacy pageload payload (if present), or changing its index within the dataLayer may break existing gtm-based ad code.

That ad code may be hardcoded to look for specific (legacy) key/value pairs on the object at index zero of the dataLayer, which it expects to be the legacy payload.

This is why the new dataLayer-initialized payloads are being tracked after any existing pageload payloads, if present.

We'll need to continue efforts to audit & update ad code prior to cleaning the old payloads out.

Step 3: This, like step 2, loads a webpacked & minified script for us since we don't have access to ES modules.

initialize-tracking-containers.js does three things:

• Make a remote call to dotcom-connect and await the user's auth details (if logged in). Upon hearing back, it will

• Add data we got from dotcom-connect to the dataLayer-initialized payload that we tracked in (step 2). Don't worry about what index it occupies within the dataLayer-- our machinery here will find the appropriate payload.

• Run GTM's snippet code that kicks everything into motion.

That wraps it up.

Obviously this might look pretty different depending on your codebase, but for a non-React/ES6 setup, these are the notes you'll want to hit.

Philosophical Note 🗿

I shied away from adding any of the steps above to rizzo, rizzo-next, etc.

The data being tracked by dataLayer-initialized comes from the codebase itself, of course, and its timing relative to the other steps described above is pretty rigidly prescribed.

As such, I figured it made sense to keep it all together in the codebase where the data lives.

And given how many codebases require this setup (a lot), I figure that verbosity that is predictable and easy to find is better than brevity that may not be.

Anyway, let's move on to a React/ES6 setup.

🎯 dotcom-pois 🎯

list.jsx (annotated)

// ...etc
import {
  analytics,
  ApplicationNames,
  createDataLayerScript,
  DataLayerInitializer,
  DestinationDirectories,
  DestinationSubNavs,
  SiteSections,
} from "@lonelyplanet/lp-analytics";
// ...etc

export default class ListComponent extends React.Component {
  // ...etc
  render() {
    // ...etc
    return (
      <div className="PageContainer">
        <DataLayerInitializer
          script={createDataLayerScript({
            [analytics.applicationName]: ApplicationNames.dotcomPois,
            // ...etc
            [analytics.destinationDirectory]: DestinationDirectories.poiList,
            // ...etc
            [analytics.siteSection]: SiteSections.destinations,
          }, 1, asyncUserStatusWrapper)}
          helmet
        /> 🚨 (**STEPS 1, 2, 3**)
        // ...etc
    );
  }
}

Step 1: As noted earlier, when using ES6 modules, (step 2) doesn't require any action on your part-- you're going to import and directly use the methods from which initialize-data-layer.js was generated in the first place.

Step 2: Pretty good amount to unpack here.

<DataLayerInitializer> is essentially just a wrapper around <script dangerouslySetInnerHTML={{ __html: ...[script content] }} />, except that it also allows you to pass in the boolean prop helmet (as is present here).

helmet does what you'd think-- react-helmet will move the resultant script up into the <head> tag of the document if present. The script will stay wherever it is if not. react-helmet is not a peer dependency; lp-analytics has its own version.

We generate our argument to the script prop via createDataLayerScript (imported from lp-analytics).

The first argument it accepts is the payload to be tracked.

Note that createDataLayerScript will accept whatever you have and add in every other key with undefined for its value. You don't need to manually include every single key/value pair as you would for the non-ES6 setup. You'll still wind up with a complete dataLayer-initialized payload.

📓 Note 📓: leave userId, dispatchVariant and loggedIn out. Just don't worry about them at all. We have some machinations in place that'll handle those for us.

The second argument to createDataLayerScript is the index at which the payload should be inserted.

You may recall from above that we have some GTM-based ad code that could break (depends on what ad code is on the page in question) if we remove or change the index of the legacy pageload payload. This is how we're getting around the problem in a React/ES6-based setup: tell the script we're generating to put the new payload at a specific index.

The argument defaults to 0 if not included.

Finally, note the variables analytics, ApplicationNames, DestinationDirectories and SiteSections here.

analytics is an object full of constants for all the key names that'll be present on payloads we track via lp-analytics.

ApplicationNames, DestinationDirectories and SiteSections are TS enums. In practice, they work like analytics above (as a collection of constants), but have a more specific use within TypeScript in that they define acceptable values that may be provided for a certain key. i.e., every possible value that could be used for the analytics.applicationName key should be contained within ApplicationNames. Naturally, you'll only use these for value sets that are finite and countable (e.g. user registrations sources-- facebook, twitter, etc.), not infinite (e.g. user IDs).

This may seem like a chore-- you'll probably need to add to these enums as you develop. But I think it's worth doing in that it promotes consistency across codebases and also allows lp-analytics to serve as a set of living documentation for what values, payload shapes, etc. we're actually using in practice.

Even if you aren't coding in an ES6-based JS environment and can't use the enums, for example, you can peruse lp-analytics's enums.ts file and see what values are being used for a particular key, or by checking interfaces.ts you can glean what key/value pairs are present for a certain payload type.

Step 3: The third argument to createDataLayerScript allows you to specify a wrapper into which the produced script will be placed. You will, in all cases, want to import & pass in asyncUserStatusWrapper**. It wasn't originally meant to be used in all cases like this, but, well, that changed a day or two ago, and now it is.

** Update: makeAsyncUserStatusWrapper will be the preferred means of implementing this functionality going forward, instead of using asyncUserStatusWrapper directly. It allows you to pass in an argument for the url you want it to use for dotcom-connect; this will enable use of QA/staging/production urls as the case may be. Leaving the argument blank (or using asyncUserStatusWrapper directly) will result in the default value, which is production. It returns a function that can be used in place of asyncUserStatusWrapper, e.g.

createDataLayerScript({ ...payload }, 1, makeAsyncUserStatusWrapper("http://connect.qa.lonelyplanet.com"));

Anywhoo, it will implement the same functionality vis-a-vis dotcom-connect as is built into initialize-tracking-containers.js-- you'll get userId, loggedIn and dispatchVariant, these will be added to your dataLayer-initialized payload for you, and then the tracking container(s) will be initialized (i.e., the GTM snippet will be executed).

Now I must confess that dotcom-pois is one of the simpler React setups you'll see. More commonly, you'll find <DataLayerInitializationData> components littered throughout a React codebase. For example,

🎬 dotcom-video 🎬

app/assets/app.jsx

// ...etc
<DataLayerInitializationData
  data={{ [analytics.applicationName]: ApplicationNames.dotcomVideo }}
  indexForEntirePayload={1}
  scriptWrapperForEntirePayload={asyncUserStatusWrapper}
/>
// ...etc

That's one of several. The idea behind DataLayerInitializationData is that you add an instance wherever you happen to have some data that should be added to the dataLayer-initialized payload.

Then, by means of react-side-effect, they are composed into a single payload that gets fed to createDataLayerScript, just like we did in the previous example, when we provided a single payload to createDataLayerScript from the outset.

You then feed that script to DataLayerInitializer (if still in React-land), or otherwise toss it into a <script> tag through whatever means are at your disposal (the example here uses handlebars):

🎬 dotcom-video 🎬

pov_controller.js

// ...etc
locals.dataLayer = DataLayerInitializationData.rewind();
// ...etc

🎬 dotcom-video 🎬

meta.hbs

<!-- ...etc -->
<script>{{{dataLayer}}}</script>
<!-- ...etc -->

This way you don't have to know everything about the payload in just one spot in the code.

Note that since there is no single place to specify the index or script wrapper, this setup allows you to set the index or script wrapper for the entire payload on any single instance. I've been adding those props to every instance so as to be totally unambiguous about how the payload will wind up upon being composed.

My thoughts: this setup has merits but is quite finicky. Some instances don't get picked up by react-side-effect and it can be time-consuming to figure out where/why, and you've got to be very diligent about calling rewind after each renderToString, or else incorrect data could be appended to your page (copilot used to have a problem with this when publishing all pages at once).

My recommendation is to favor the simpler setup found in dotcom-pois where possible.