RequestSRC

Overview

RequestSRC is a lightweight, server-side middleware for real-time monitoring of HTTP requests in Express applications. It acts as a middleware that attaches location data to requests, logs them into a local SQL database, and provides a dashboard displaying a sortable table and a filterable graph.

• Minimal latency (~5 seconds) between request logging and dashboard update.
• All server-side logic, including database operations and dashboard rendering, is handled on the server for optimal performance.
• Built-in support for GeoIP MaxMind City Database (bundled with the Feb 2025 version) for geolocation enrichment.

Features

✔️ Automatic Request Logging: Captures IP address, geolocation, user agent, timestamp, and custom request type.
✔️ Built-in Admin Dashboard: Sortable table and filterable graph to analyze logs.
✔️ Database Support: Stores logs in PostgreSQL or MySQL.
✔️ Privacy-Friendly: IP anonymization available for GDPR/CCPA compliance.
✔️ Customizable:

• Anonymization options,
• Dashboard route,
• Log retention period,
• Data refresh rate (WIP),
• Data expiration (WIP).

✔️ Graph Filtering:

• Filter graph data by IP, city, region, country, or request type.
• Visual differentiation of request types via color-coded lines matching the log entries.

✔️ Advanced User-Agent Detection:

• Identifies whether requests are local, public, or through port forwarding.
• Highlights anomalies or suspicious access patterns.

✔️ Log Customization:

• Anonymize logs,
• Color-coded request types for easy identification.

✔️ Plug-and-Play Setup:

• After PostgreSQL installation and setup, the integration is seamless.

---

Installation

Install RequestSRC via npm:

npm install request-src

---

Usage

Basic Setup

Integrate RequestSRC into your Express app:

const express = require("express");
const RequestSRC = require("request-src");

const app = express();

// ✅ Update RequestSRC configuration dynamically
RequestSRC.updateConfig({
    anonymize: false, // Enable anonymization
    dashboardRoute: "/custom", // Custom dashboard route
    retentionPeriod: 30, // Keep logs for 30 days
});

// Use the middleware
app.use(RequestSRC.router);

app.get("/", (req, res) => {
    RequestSRC.add(req, "test-request");
    res.send("Hello World with RequestSRC!");
});

app.listen(3000, () => console.log("Server running on port 3000"));

Logging Requests

Add a log to your initialized database:

app.post("/login", (req, res) => {
    RequestSRC.add(req, "user_login"); // Logs as 'user_login'
    res.send("User logged in");
});

Log without saving to the database:

app.get("/log", async (req, res) => {
    const log = await RequestSRC.log(req, "temp_log");
    res.json(log); // Returns log data without saving
});

---

API Methods

requestSRC.add(req, reqType)

📌 Logs request metadata into the database.

• Extracts IP, user-agent, timestamp, and geolocation.
• Stores the request in the SQL database.
• Example Usage:

app.post("/register", (req, res) => {
    requestSRC.add(req, "user_creation"); // Logs as 'user_creation'
    res.send("User registration processed");
});

---

requestSRC.log(req, reqType)

📌 Extracts request metadata without storing it.

• Returns an object with request details for debugging.
• Example Usage:

app.get("/debug", (req, res) => {
    try {
        const log = await RequestSRC.log(req, 'debug log');
        res.json(log); // Returns log data as JSON without saving to DB
    } catch (error) {
        console.error("Error logging request:", error);
        res.status(500).json({ error: "Failed to log request" });
    }
});

• Example Output:

{
    "ip": "192.168.1.100",
    "anonymized_ip": "192.168.1.0",
    "user_agent": "Mozilla/5.0",
    "timestamp": "2025-01-27T12:34:56Z",
    "geo": { "country": "US", "city": "San Francisco", "region": "California" },
    "reqType": "debug log"
}

---

Configuration Options

When initializing RequestSRC, you can configure:

Option	Type	Description
anonymize	Boolean	Masks last octet of IP (default: false).
dashboardRoute	String	URL path for admin dashboard (default: /requestSRC).
retentionPeriod	Number	Auto-delete logs older than X days (0 = disable).

---

Accessing the Dashboard

Once installed, the traffic monitoring dashboard is available at:

http://your-domain/requestSRC

It provides:

• A sortable table view for traffic logs.
• A graph view to visualize request patterns over time, with color-coded request types.

---

---

Using RequestSRC Without a Database

If you want to log request data without saving it to a database, you can use the .log() method. This method extracts request metadata (IP, user-agent, geolocation, etc.) and returns it as a JSON object without storing it.

Example Usage:

app.get('/log', async (req, res) => {
    try {
        const log = await RequestSRC.log(req, 'temporary_log');
        res.json(log); // Returns log data as JSON without saving to DB
    } catch (error) {
        console.error("Error logging request:", error);
        res.status(500).json({ error: "Failed to log request" });
    }
});

Example Output:

{
    "ip": "192.168.1.100",
    "anonymized_ip": "192.168.1.0",
    "user_agent": "Mozilla/5.0",
    "timestamp": "2025-01-27T12:34:56Z",
    "geo": { "country": "US", "city": "San Francisco", "region": "California" },
    "reqType": "temporary_log"
}

This is useful for debugging or temporary monitoring without writing to the database.

---

Setting Up PostgreSQL

To fully utilize RequestSRC with persistent log storage, you'll need to install and configure PostgreSQL.

1. Install PostgreSQL:
   Follow the official guide to install PostgreSQL based on your operating system:
   👉 PostgreSQL Download

2. Create a Database:
   After installation, create a new database for logging:

   psql -U postgres
   CREATE DATABASE requestsrc;
   \c requestsrc

3. Initialize the Database Structure:
   Create a logs table to store request data:

   CREATE TABLE logs (
       id SERIAL PRIMARY KEY,
       timestamp TIMESTAMP NOT NULL,
       ip VARCHAR(45) NOT NULL,
       city VARCHAR(100),
       region VARCHAR(100),
       country VARCHAR(100),
       user_agent TEXT,
       req_type VARCHAR(100)
   );

---

Configuring Environment Variables

To securely manage your database credentials, use a .env file. This should be added to .gitignore to prevent sensitive data from being exposed.

1. Create a .env File:

   touch .env

2. Add Database Credentials to .env:

   DB_USER=postgres
   DB_HOST=localhost
   DB_NAME=requestsrc
   DB_PASSWORD=your_password
   DB_PORT=5432

3. Add .env to .gitignore:

   echo ".env" >> .gitignore

---

Basic Setup

Now that the database is ready, integrate RequestSRC into your Express app:

const express = require('express');
const RequestSRC = require('request-src');
require('dotenv').config();

const app = express();

// ✅ Update RequestSRC configuration dynamically
RequestSRC.updateConfig({
    anonymize: false, // Enable anonymization
    dashboardRoute: "/requestSRC", // Dashboard route
    retentionPeriod: 30, // Retain logs for 30 days
});

// Use the middleware
app.use(RequestSRC.router);

// Example route to log requests
app.get("/", (req, res) => {
    RequestSRC.add(req, "home_page");
    res.send("Hello World with RequestSRC!");
});

app.listen(3000, () => console.log("Server running on port 3000"));

With this setup, RequestSRC will now log request data into your PostgreSQL database and provide a dashboard at:

http://localhost:3000/requestSRC

License

MIT License © 2025 Xavier Pimentel

---

Contributing

Feel free to submit issues, feature requests, or pull requests on GitHub.

🔗 GitHub Repository: https://github.com/XavierPim/requestSRC