NextCollect is a Next.js framework for reliable collection of user behavioral data

Overview

How Next Collect Works

Overview

NextCollect.js is a framework for server-side user event collection for Next.Js. It is designed from the ground up to work in Serverless environment.

NextCollect is destination agnostic. It could send data to multiple destinations at once. So far we support Jitsu, Segment and PostgREST (compatible with Supabase)

Server-Side vs Client-Side event collection

Client-Side user tracking means that once user loads the page (or makes another action), piece of JavaScript code sends the data to collector server. This is how most of the analytics trackers works: GoogleAnalytics, Segment, Amplitude, etc. Server-Side tracking happens when server (backend) renders the page or answers API call. No JavaScript code is touched.

Why go Server-Side

  • Reliability. 100% events will be recorded. Unlike client-side trackers, server side trackers are insusceptible to AdBlockers and Safari's Tracking Prevention.
  • Better user identification. Server-side cookies are more reliable, especially when cookies are set under the same domain name as the main app
  • Better user experience. Less javascript request means faster website
  • No middleman. It's possible to distribute data to end destinations directly, bypassing Segment and other similar probles

Best of the both worlds

nextjs-collect allows collect data client-side collection too. It exposes a first-parti api route /api/event-sink, so the events can be from client-side code. A good example of such event is a user actions which do not call any server API, such button click on a button.

The data will be sent to /api/collect, and sensitive params such userId, ip address and so on will be resolved server-side. Since the api call is first-party (goes to the same host), it won't be blocked by AdBlockers or tracking prevention.

See a full instruction on how to use client-side tracking below

Getting Started

npm install --save next-collect. Make sure that Next.Js >= 12.0

Usage

Create page/_middleware.[js|ts] file within you Next App:

import {collectEvents} from 'next-collect/server'

export default collectEvents({destination: ["jitsu", "segment"]});

or wrap an existing middleware:

import {collectEvents} from 'next-collect/server'

const middleware = (req, res) => {
}

export default collectEvents({
    middleware: middleware,
    destinations: ["jitsu", "segment"]
});

Destinations

NextCollect is destination agnostic. It could send data to multiple destinations at once. We support Jitsu, Segment and PostgREST (Supabase). See destination reference below

Most reads config from env variables, or config can be passed to destination directly. Example:

import {withEventSink} from 'next-collect/server'

const middleware = (req, res) => {
}

export default collectEvents({
    middleware: middleware,
    destinations: [{type: "jitsu", opts: {key: "{API KEY}", host: "{JITSU HOST}"}}]
});

See a full list of destinations with param references below

Event type mapping

By default, NextCollect sends a page_view event. You can define your own types with pattern matching:

export default collectEvents({
    middleware,
    drivers: [...],
    eventTypes: [
        {"/api*": "api_call"},
        {"/img*": null},
        ["/favicon*", null],
        {"/*": "page_view"}],
})

In this example all requests to /img* and /favicon* won't be logged, request to /api* will be logged as api_call and the rest will be tagged as page_view. Order of mapping matters! First rule takes precedence./api/test matches both first and last rule, but it will be tagged as api_view event

Custom properties

NextCollect allows adding custom properties event. You probably want to do so if you authorize users, hence you want to see user id / email attached to the event. Here's an example:

function getUser(req) {
    return {id: ..., email:...}
}


export default collectEvents({
    middleware,
    drivers: [...]
    eventTypes: [...],
    extend: (req: NextRequest) => {
        return {
            user: getUser(req),
            projectId: req.page?.params?.projectId,
        }
    },
})

The easiest way to get user id and email is to save it to cookies, and get it from req.cookies in getUser() function.

Furthermore, this example adds projectId to event. projectId is taken from page property. You can add as many properties as you want

Client-Side Data Collection - useCollect() hook

Not all events can be tracked on server-side. Some events happen when user interacts with UI, and no server code is touch.

In this case, you should use useCollect() hook. Event goes to a /api/collect (handled by Page Middleware), and the "hydrated" on server

const collect = useCollect()

return <button onClick={() => collect.event("button_click", {buttonId: "Sign Up"})}>Click Me!</button>

Advanced: Custom API Route

/api/collect is handled by Page Middleware. If by any reason, you can't use Pages Middleware, you could define an API Route explicitly.

./pages/api/collect-data.ts file:

export default nextEventsCollectApi({
    ...nextCollectBasicSettings,
    extend: (req: NextApiRequest) => {...},
})

The syntax is the same as for collectEvents() except that NextApiRequest is passed to configuration functions, not NextRequest.

Next, you should redefine an API route for useCollect() hook in pages/_app.ts with EventCollectionProvider:

<EventCollectionProvider options={{apiPath: "/api/collect-events"}}>
   ...
</EventCollectionProvider>

Destination Reference

At the moment, NextCollect supports Jitsu (jitsu), Segment(segment) and PostgREST (postgres).

Jitsu

Config

Parameter Documentation
opts.key or process.env.JITSU_KEY
(required *)
Jitsu Server API key
opts.key or process.env.JITSU_KEY
(required *)
Jitsu host. Must start with https:// or http://. Example: `t.jitsu.com

Example

Segment

Config

Parameter Documentation
opts.key or process.env.SEGMENT_KEY
(required *)
Segment write API key

PostgREST

Jitsu supports PostgREST (including Supabase which is based on PostgREST).

A table with all fields should be created prior to using this destination.

Config

Parameter Documentation
opts.url or process.env.POSTGREST_URL
(required *)
Url of PostgREST server
opts.url or process.env.POSTGREST_URL
(required *)
Url of PostgREST server

Contributing

Please see CONTRIBUTING.md

You might also like...

This is a demo of updating a map to show air quality data for the user’s current location using Next.js Advanced Middleware, powered by Netlify Edge Functions.

Show Local Air Quality Based on the User's Location Use AQI data to show the air quality near the current user. This is built using Next.js Advanced M

Nov 4, 2022

✏️ A small jQuery extension to turn a static HTML table into an editable one. For quickly populating a small table with JSON data, letting the user modify it with validation, and then getting JSON data back out.

jquery-editable-table A small jQuery extension to turn an HTML table editable for fast data entry and validation Demo 👉 https://jsfiddle.net/torrobin

Jul 31, 2022

A collection of preloaded and essential files that makes the website more attractive, smooth and user friendly

Web-Docs A collection of preloaded and essential files that makes the website more attractive, smooth and user friendly How to use: git clone https://

Dec 17, 2022

An online library for adding and removing a different number of books from a user collection, keeping track of the books you've read and the one's you are yet to read

An online library for adding and removing a different number of books from a user collection, keeping track of the books you've read and the one's you are yet to read

Awesmoe Books A Website demo for our project of book store, The website has ability of adding and removing you books from yor library, Thats reflects

Jul 8, 2022

Awesome-book is an online library website where a user can store a collection of books. Different book can be added and removed. Built with JavaScript using Dom

Awesome-book Description Awesome-book is an online library website where a user can store a collection of books. Differents book can be added and remo

Sep 9, 2022

⚡️ Next-generation data transformation framework for TypeScript that puts developer experience first

⚡️ Next-generation data transformation framework for TypeScript that puts developer experience first

TypeStream Next-generation data transformation framework for TypeScript that puts developer experience first Nowadays, almost every developer is worki

Nov 26, 2022

A curated collection of common interview questions to help you prepare for your next interview.

A curated collection of common interview questions to help you prepare for your next interview.

30 Seconds of Interviews A curated collection of common interview questions to help you prepare for your next interview. This README is built using ma

Jan 7, 2023

tooldb is a (soon) massive collection of frameworks and tools. It's build on Flowbite, Next.js, Tailwind CSS and uses Supabase.

tooldb is a (soon) massive collection of frameworks and tools. It's build on Flowbite, Next.js, Tailwind CSS and uses Supabase.

tooldb is a (soon) massive collection of frameworks and tools. It's build on Flowbite, Next.js, Tailwind CSS and uses Supabase.

Jul 14, 2022

This project entails a To-do-List whereby a user can input the tasks they want to do, check the tasks done and also clear all tasks when all of them are completed. It is efficient for a user who want to manage their time and keep track of their day.

This project entails a To-do-List whereby a user can input the tasks they want to do, check the tasks done and also clear all tasks when all of them are completed. It is efficient for a user who want to manage their time and keep track of their day.

Screenshot Here is a screenshot for the project. To-Do-List Project This is a Microverse project that entails a to-do-list which one is able to add an

Jun 16, 2022
Comments
  •  Failed to get nextJsPageMiddleware.props: The request.page has been deprecated in favour of `URLPattern`.

    Failed to get nextJsPageMiddleware.props: The request.page has been deprecated in favour of `URLPattern`.

    On Next.js 12.0.x we're starting to see these logs:

     [WebServer] 2022-08-17 01:54:47.069 UTC [ERROR] (next-collect/server) - Failed to get nextJsPageMiddleware.props: The request.page has been deprecated in favour of `URLPattern`.
       Read more: https://nextjs.org/docs/messages/middleware-request-page
        [Error: The request.page has been deprecated in favour of `URLPattern`.
       Read more: https://nextjs.org/docs/messages/middleware-request-page
       ]
    
    opened by zomars 1
Owner
Jitsu
Jitsu (YC S20) is an open-source data integration and event collection platform.
Jitsu
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
AppRun is a JavaScript library for developing high-performance and reliable web applications using the elm inspired architecture, events and components.

AppRun AppRun is a JavaScript library for building reliable, high-performance web applications using the Elm-inspired architecture, events, and compon

Yiyi Sun 1.1k Dec 20, 2022
A fast & reliable transaction API for web3 Games, Bridges and other projects

Gelato Relay SDK SDK to integrate into Gelato Multichain Relay Table of Contents Installation Introduction Quick Start Payment Types Request Types Sen

Gelato 17 Dec 31, 2022
chain-syncer is a module which allows you to synchronize your app with any ethereum-compatible blockchain/contract state. Fast. Realtime. Reliable.

Chain Syncer Chain Syncer is a JS module which allows you to synchronize your app with any ethereum-compatible blockchain/contract state. Fast. Realti

Miroslaw Shpak 10 Dec 15, 2022
Open source infrastructure for scalable, reliable native integrations in B2B SaaS products

Open-source infrastructure for native integrations Native, customer-facing integrations for your B2B SaaS made simple, reliable and extensible. Explor

Nango 225 Jan 2, 2023
This project is built with JavaScript, Webpack, HTML & CSS, Leaderboard api. When user clicks on Refresh button it hits the api and responds with the data, The user can also post data to the api

leaderboad Description the project. this project is about the leaderboad i did during Microverse to build a website for adding Data to the API and fet

Emmanuel Moombe 4 May 30, 2022
FormGear is a framework engine for dynamic form creation and complex form processing and validation for data collection.

FormGear is a framework engine for dynamic form creation and complex form processing and validation for data collection. It is designed to work across

Ignatius Aditya Setyadi 91 Dec 27, 2022
Obsidian-Snippet-collection - A collection of snippet to customize obsidian

This repo is a collection of CSS snippets for Obsidian.md. To install them on PC

Mara 110 Dec 22, 2022