Vytal

An accurate gaze-tracking API with real time predictions and inferencing.

Installation

Using NPM:

$ npm install vytal

Using yarn:

$ yarn add vytal

Go to our API portal{:target="_blank"} to get credits to attain accurate gaze coordinates before you get started.

This package requires TailwindCSS{:target="_blank"}. Please install and setup Tailwind before using the package.

Real Time Predictions Example

import React, { useState, useRef, useEffect } from 'react';
import realTimePreds from 'vytal/dist/main';
import RealTimeCalibration from "vytal/dist/main";

const RealTimePredictions = () => {
  const apiKey = 'sk-...';
  const videoRef = useRef(null);
  const [predictions, setPredictions] = useState([]);
  const [calibMatrix, setCalibMatrix] = useState(null);
  const [windowInformation, setWindowInformation] = useState(null)

  const handleMatrix = (matrix) => {
    setCalibMatrix(matrix);
  };

  useEffect(() => {
    setWindowInformation(window);
  }, []);

  useEffect(() => {
    const startVideoStream = async () => {
      try {
        const stream = await navigator.mediaDevices.getUserMedia({ video: true });
        if (videoRef.current) {
          videoRef.current.srcObject = stream;
          await videoRef.current.play();
        }
      } catch (err) {
        console.error("Error accessing camera:", err);
      }
    };

    startVideoStream();

    return () => {
      if (videoRef.current && videoRef.current.srcObject) {
        const tracks = videoRef.current.srcObject.getTracks();
        tracks.forEach(track => track.stop());
      }
    };
  }, []);

  useEffect(() => {
    if (videoRef.current && calibrationMatrix) {
      const { predictions } = realTimePreds(apiKey, calibrationMatrix, videoRef);

      const predictionWatcher = () => {
        setPredictions(predictions);
      };

      predictionWatcher();
      const watcherInterval = setInterval(predictionWatcher, 100);

      return () => {
        clearInterval(watcherInterval);
      };
    }
  }, [apiKey, calibrationMatrix]);

  return (
    <div>
      {calibMat ? (
        <>
          <video ref={videoRef} style={{ display: 'none' }} />
          <ul>
            {predictions.map((prediction, index) => (
              <li key={index}>Prediction: {JSON.stringify(prediction)}</li>
            ))}
          </ul>
          <button onClick={startStream}>Start Predictions</button>
        </>
      ) : (
        windowInformation && (
          <RealTimeCalibration
            window={windowInformation}
            apiKey="sk-..."
            setCalibrationMatrix={handleMatrix}
          />
        )
      )}
    </div>
  );
};

export default RealTimePredictions;

Inferencing Example

import Calibration from "vytal/dist/main";
import { useEffect, useState } from "react";

export default function VideoInferencing() {
  const [windowInformation, setWindowInformation] = useState(null);
  const [calibMatrix, setCalibMatrix] = useState(null);

  useEffect(() => {
    setWindowInformation(window);
  }, []);

  const handleMatrix = (matrix) => {
    setCalibMatrix(matrix);
  };

  const handleInference = async () => {
    // Access user video stream (implementation omitted for brevity)
    const videoBlob = await getVideoBlobFromStream(); // Replace with your video capture logic

    try {
      const preds = await inference(videoBlob, apiKey, calibMatrix);
      setPredictions(preds);
    } catch (error) {
      console.error('Error during inference:', error);
    }
  };

  return (
    <div>
      {windowInformation && (
        <Calibration
          window={windowInformation}
          apiKey="sk-..."
          calibrationMatrix={handleMatrix}
        />
      )}
      {calibMatrix && (
          <button onClick={handleInference}>Get Predictions</button>
      )}
      {predictions && <pre>{JSON.stringify(predictions, null, 2)}</pre>}
    </div>
  );
}

Parameters

Calibration

• window (object), required: information about the window of the user's currrent screen
• apiKey (str), required: API key for calibration
• calibrationMatrix (func), required: retrieve matrix from calibration to use for inferencing
• redirect (str), optional: Link with where to redirect after calibration is complete. Default is "#"

Inference

• videoBlob (Blob), required: information about the video that was recorded
• apiKey (str), required: API key for inferencing
• calibrationMat (str), required: calibration matrix string from Calibration to retrieve accurate inferences

Real Time Calibration

• window (object), required: information about the window of the user's currrent screen
• apiKey (str), required: API key for calibration
• setCalibrationMatrix (func), required: retrieve matrix from calibration to use for inferencing

Real Time Predictions

• apiKey (str), required: API key for real time predictions
• calibMat (str), required: calibration matrix string from Calibration to retrieve accurate predications
• feed (Video), required: information about the current stream

Error Handling

Any errors (e.g., API Key validation, model errors, and file input issues) will be in the console.