@outsmartly/bigquery-collector

BigQuery Collector for use with Outsmartly intercepts

This allows you to post events directly into BigQuery using Outsmartly's edge as a collector.

Getting Started

Before using you will need to create a BigQuery project, dataset, and table(s). You will also need to create a Service Account and download the JSON with permissions to write to your BigQuery project.

Install

npm install --save @outsmartly/bigquery-collector

Adding to your outsmartly.config.js

import { bigQueryCollector } from "@outsmartly/bigquery-collector";
import { serviceAccount } from "./service-account";

const collectorConfig = {
  client_email: serviceAccount.client_email,
  private_key_id: serviceAccount.private_key_id,
  private_key: serviceAccount.private_key,
  project_id: serviceAccount.project_id,
  dataset: "Testing_001",
  collectorPathname: "/__analytics__",
  debug: true, // if debug is true then you will see the response returned from BigQuery
};

const [bigQueryEndpoint] = bigQueryCollector(collectorConfig);

export default {
  // ...
  routes: [
    bigQueryEndpoint,
    // ...
  ],
};

Sending events from client-side:

fetch(collectorPathname, {
  method: "POST",
  body: JSON.stringify({
    // this is your table name
    table: "myTableName",
    // this is the row you would like to write
    row: {
      hello: "hi",
      another: "what up",
    },
  }),
  headers: {
    "Content-Type": "application/json",
  },
});

Optional server (edge) side tracking

You can optionally destructure an additional async function that will allow you to write directly to BigQuery.

import { bigQueryCollector } from "@outsmartly/bigquery-collector";
import { serviceAccount } from "./service-account";

const collectorConfig = {
  client_email: serviceAccount.client_email,
  private_key_id: serviceAccount.private_key_id,
  private_key: serviceAccount.private_key,
  project_id: serviceAccount.project_id,
  dataset: "Testing_001",
  collectorPathname: "/__analytics__",
  debug: true, // if debug is true then you will see the response returned from BigQuery
};

const [bigQueryEndpoint, writeToBigQuery] = bigQueryCollector(collectorConfig);

export default {
  // ...
  routes: [
    {
      path: "*",
      async intercept(event) {
        const request = event.request;
        const url = new URL(request.url);
        const { pathname } = url;

        const table = "edgeLogs";
        const rows = [
          {
            json: {
              pathname,
            },
          },
        ];
        event.waitUntil(writeToBigQuery(table, rows));

        // continue to other intercepts / overrides
        return;
      },
    },
    bigQueryEndpoint,
    // ...
  ],
};