A library for prototyping realtime hand detection (bounding box), directly in the browser.

Overview

Handtrack.js

npm version npm

View a live demo in your browser here.

Handtrack.js is a library for prototyping realtime hand detection (bounding box), directly in the browser. Underneath, it uses a trained convolutional neural network that provides bounding box predictions for the location of hands in an image. The convolutional neural network (ssdlite, mobilenetv2) is trained using the tensorflow object detection api (see here).

FPS Image Size Device Browser Comments
21 450 * 380 Macbook Pro (i7, 2.2GHz, 2018) Chrome Version 72.0.3626 --
14 450 * 380 Macbook Pro (i7, 2.2GHz, mid 2014) Chrome Version 72.0.3626 --

This work is based on the the coco-ssd tensorflowjs sample. Definitely check it out if you are interested in detecting/tracking any of the 90 classes in the coco dataset.

The library is provided as a useful wrapper to allow you prototype hand/gesture based interactions in your web applications. without the need to understand. It takes in a html image element (img, video, canvas elements, for example) and returns an array of bounding boxes, class names and confidence scores.

The library also provides some useful functions (e.g getFPS to get FPS, renderPredictions to draw bounding boxes on a canvas element), and customizable model parameters.

Tests on a Macbook Pro 2.2 GHz Intel Core i7, achieve 21 FPS.

How does this work?

  • Trained using egohands dataset. You will notice the model works better when the hands in an image is viewed from a top (egocentic) view.
  • Trained model is converted to the Tensorflowjs format
  • Model is wrapped into an npm package, and can be accessed using jsdelivr, a free open source cdn that lets you include any npm package in your web application. You may notice the model is loaded slowly the first time the page is opened but gets faster on subsequent loads (caching).

When Should I Use Handtrack.js

If you are interested in prototyping gesture based (body as input) interactive experiences, Handtrack.js can be useful. The usser does not need to attach any additional sensors or hardware but can immediately take advantage of engagement benefits that result from gesture based and body-as-input interactions.

Code examples for the pong game control shown above is in the demo folder.

Some (not all) relevant scenarios are listed below: 

  • When mouse motion can be mapped to hand motion for control purposes.
  • When an overlap of hand and other objects can represent meaningful interaction signals (e.g a touch or selection event for an object).
  • Scenarios where the human hand motion can be a proxy for activity recognition (e.g. automatically tracking movement activity from a video or images of individuals playing chess). Or simply counting how many humans are present in an image or video frame.
  • You want an accessible demonstration that anyone can easily run or tryout with minimal setup.

How Do I Use Handtrack.js in my Web App?

via Script Tag

You can use the library by including it in a javacript script tag.

<!-- Load the handtrackjs model. -->
<script src="https://cdn.jsdelivr.net/npm/handtrackjs/dist/handtrack.min.js"> </script>

<!-- Replace this with your image. Make sure CORS settings allow reading the image! -->
<img id="img" src="hand.jpg"/> 
<canvas id="canvas" class="border"></canvas>

<!-- Place your code in the script tag below. You can also use an external .js file -->
<script>
  // Notice there is no 'import' statement. 'handTrack' and 'tf' is
  // available on the index-page because of the script tag above.

  const img = document.getElementById('img'); 
  const canvas = document.getElementById('canvas');
  const context = canvas.getContext('2d');
  
  // Load the model.
  handTrack.load().then(model => {
    // detect objects in the image.
    model.detect(img).then(predictions => {
      console.log('Predictions: ', predictions); 
    });
  });
</script>

via NPM

npm install --save handtrackjs
import * as handTrack from 'handtrackjs';

const img = document.getElementById('img');

// Load the model.
handTrack.load().then(model => {
  // detect objects in the image.
  console.log("model loaded")
  model.detect(img).then(predictions => {
    console.log('Predictions: ', predictions); 
  });
});

API

Loading the model: handTrack.load()

Once you include the js module, it is available as handTrack. You can then load a model with optional parameters.

const modelParams = {
  flipHorizontal: true,   // flip e.g for video 
  imageScaleFactor: 0.7,  // reduce input image size for gains in speed.
  maxNumBoxes: 20,        // maximum number of boxes to detect
  iouThreshold: 0.5,      // ioU threshold for non-max suppression
  scoreThreshold: 0.79,    // confidence threshold for predictions.
}

handTrack.load(modelParams).then(model => {

});

Returns a model object.

Detecting hands: model.detect()

model.detect takes an input image element (can be an img, video, canvas tag) and returns an array of bounding boxes with class name and confidence level.

model.detect(img).then(predictions => { 
        
});

Returns an array of classes and confidence scores that looks like:

[{
  bbox: [x, y, width, height],
  class: "hand",
  score: 0.8380282521247864
}, {
  bbox: [x, y, width, height],
  class: "hand",
  score: 0.74644153267145157
}]

Other Helper Methods

  • model.getFPS() : get FPS calculated as number of detections per second.
  • model.renderPredictions(predictions, canvas, context, mediasource): draw bounding box (and the input mediasource image) on the specified canvas. predictions are an array of results from the detect() method. canvas is a reference to a html canvas object where the predictions should be rendered, context is the canvas 2D context object, mediasource a reference to the input frame (img, video, canvas etc) used in the prediction (it is first rendered, and the bounding boxes drawn on top of it).
  • model.getModelParameters(): returns model parameters.
  • model.setModelParameters(modelParams): updates model parameters with modelParams
  • dispose() : delete model instance
  • startVideo(video) : start webcam video stream on given video element. Returns a promise that can be used to validate if user provided video permission.
  • stopVideo(video) : stop video stream.

Run Demo

Commands below runs the demo example in the demo folder.

npm install npm run start

The start script launches a simple python3 webserver from the demo folder using http.server. You should be able to view it in your browser at http://localhost:3005/. You can also view the pong game control demo on same link http://localhost:3005/pong.html

How was this built?

The object detection model used in this project was trained using annotated images of the human hand (see here) and converted to the tensorflow.js format. This wrapper library was created using guidelines and some code adapted from the coco-ssd tensorflowjs.

Citing this Work (see abstract)

Paper abstract of the paper is here. (a full paper will be added when complete).

If you use this code in your project/paper/research and would like to cite this work, use the below.

Victor Dibia, HandTrack: A Library For Prototyping Real-time Hand TrackingInterfaces using Convolutional Neural Networks, https://github.com/victordibia/handtracking

@article{Dibia2017,
  author = {Victor, Dibia},
  title = {HandTrack: A Library For Prototyping Real-time Hand TrackingInterfaces using Convolutional Neural Networks},
  year = {2017},
  publisher = {GitHub},
  journal = {GitHub repository},
  url = {https://github.com/victordibia/handtracking/tree/master/docs/handtrack.pdf}, 
}

TODO (ideas welcome)

  • This thing is still compute heavy (your fans may spin after while). This is mainly because of the neural net operations needed to predict bounding boxes. Perhaps there might be ways to improve/optimize this.
  • Tracking id's across frames. Perhaps some nifty methods that assigns ids to each had as they enter a frame and tracks them (e.g based on naive euclidean distance).
  • Add some discrete poses (e.g. instead of just hand, detect open hand, fist).
Comments
  • Update @tensorflow/tfjs to the latest version

    Update @tensorflow/tfjs to the latest version

    https://github.com/tensorflow/tfjs/releases tfjs recently deprecated methods loadFronzenModel (as well as using .pb models) and fromPixels. The model should be converted to .json

    opened by johnSamilin 10
  • Demo breaking with the JSDelivr load

    Demo breaking with the JSDelivr load

    I tried cloning the repo just to check out the demo - both index and pong break with seemingly the same issue:

    Uncaught ReferenceError: require is not defined                                                                      handtrack.min.js:2464:294
        <anonymous> https://cdn.jsdelivr.net/npm/handtrackjs@latest/dist/handtrack.min.js:2464
        <anonymous> https://cdn.jsdelivr.net/npm/handtrackjs@latest/dist/handtrack.min.js:1
        <anonymous> https://cdn.jsdelivr.net/npm/handtrackjs@latest/dist/handtrack.min.js:1
    

    and

    Uncaught TypeError: handTrack.load is not a function
        <anonymous> http://localhost:3005/js/index.js:86
    

    If I switch to loading from <script src="lib/handtrack.js"> </script> instead of JSDelivr everything runs fine. But loading from <script src="handtrack.min.js"> </script> produces the same error. (seems like the libs folder still has the older version from 2018).

    The same thing happens when trying to run this example https://github.com/victordibia/handtrack.js#import-via-script-tag :

    Uncaught ReferenceError: require is not defined                                                                        handtrack.min.js:2464:294
        <anonymous> https://cdn.jsdelivr.net/npm/handtrackjs@latest/dist/handtrack.min.js:2464
        <anonymous> https://cdn.jsdelivr.net/npm/handtrackjs@latest/dist/handtrack.min.js:1
        <anonymous> https://cdn.jsdelivr.net/npm/handtrackjs@latest/dist/handtrack.min.js:1
    

    and Uncaught SyntaxError: await is only valid in async functions and async generators index.html:21:22

    Line 22 in index.html is const predictions = await model.detect(img); from the example

    opened by ConnectedCat 6
  • code error, realtime handtrack is not working

    code error, realtime handtrack is not working

    Hi

    https://victordibia.github.io/handtrack.js/#/doodle on chrome, i've got the js error below when clicking on Start Video doodle

    react-dom.production.min.js:232 Uncaught TypeError: Cannot read property 'then' of undefined
        at t.value (Doodle.jsx:164)
        at Object.<anonymous> (react-dom.production.min.js:49)
        at p (react-dom.production.min.js:69)
        at react-dom.production.min.js:73
        at S (react-dom.production.min.js:140)
        at T (react-dom.production.min.js:169)
        at N (react-dom.production.min.js:158)
        at D (react-dom.production.min.js:232)
        at En (react-dom.production.min.js:1718)
        at Is (react-dom.production.min.js:5990)
    

    Same thing for https://victordibia.github.io/handtrack.js/#/ when clicking on button Start Video

    opened by franck34 5
  • Webcam faces wrong direction

    Webcam faces wrong direction

    When I tried to use this demo on my Microsoft Surface computer, the webcam detection was working with the front-facing webcam instead of the rear-facing webcam, so it was facing away from my keyboard. Could this demo be modified to allow switching webcams, so that it will work correctly on computers with more than one webcam?

    enhancement 
    opened by jarble 3
  • Using my own model

    Using my own model

    I tried to train a smaller size ssdmobilenetv1 model and convert it into tfjs_graph_model format,but how can I use it in the project? Is it necessary to modify the handtrack.min.js code because I found basepath and defaultParams variables in it . Any help would be greatly appreciated!!

    opened by Carreraa 3
  • Cannot load model

    Cannot load model

    Dear author, I have face with this problem when try to do example const video = document.getElementById('largeVideo'); // const canvas = document.getElementById("canvas"); // const context = canvas.getContext("2d");

        // Load the model.
        handTrack.load().then(model => {
            // detect objects in the image.
            console.log("model loaded")
            model.detect(video).then(predictions => {
                console.log('Predictions: ', predictions);
                // model.renderPredictions(predictions, canvas, context, video);
                // requestAnimationFrame(runDetection);
            });
        });
    

    --> "UnhandledError: Request to https://cdn.jsdelivr.net/npm/handtrackjs@latest/models/webmodel/centernet512fpn/base/model.json failed with status code 404. Please verify this URL points to the model JSON of the model to load" Could you help me. Thanks

    opened by ngochai27595 2
  • Fix rollupjs config

    Fix rollupjs config

    Fix issues in #44:

    • Resolve crypto issues by setting browser flag
    • Include required commonjs modules
    • Switch rollupjs config from reactdemo output to handtrack.min.js
    opened by matthiasmiller 1
  • Bump dot-prop from 4.2.0 to 4.2.1

    Bump dot-prop from 4.2.0 to 4.2.1

    Bumps dot-prop from 4.2.0 to 4.2.1.

    Release notes

    Sourced from dot-prop's releases.

    v4.2.1

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • (docs) update README.md

    (docs) update README.md

    Impressive work! I am looking forward to playing around with this.

    However, reading the README from the CLI was difficult due to multiple issues. Proposed are fixes.

    • Fix spelling/grammar
    • Fix spacing
    • Use proper markdown formatting
    opened by vladdoster 1
  • Bump lodash from 4.17.14 to 4.17.19

    Bump lodash from 4.17.14 to 4.17.19

    Bumps lodash from 4.17.14 to 4.17.19.

    Release notes

    Sourced from lodash's releases.

    4.17.16

    Commits
    Maintainer changes

    This version was pushed to npm by mathias, a new releaser for lodash since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Camera freezing on iOS Safari

    Camera freezing on iOS Safari

    When trying to use the basic example on iOS Safari (iPhone 11 Pro), and toggling video, it either shows all black or just what shows on the camera for the first frame - then freezes.

    I've stopped runDetection(); running within the startVideo() function when camera is enabled to rule this out, and the same behaviour occurred then.

    opened by jamesdowen 1
  • handtrack not work in vue3+vite3

    handtrack not work in vue3+vite3

    engine.ts:586 Uncaught (in promise) TypeError: t3 is not a function at engine.ts:586 at engine.ts:424 at t2.scopedRun (engine.ts:435) at t2.tidy (engine.ts:422) at f2 (engine.ts:586) at engine.ts:599 at t2.scopedRun (engine.ts:435) at t2.runKernelFunc (engine.ts:596) at t2.runKernel (engine.ts:493) at clone_2 (clone.ts:47)

    opened by Jackie-Tang 0
  • ERROR Error: Uncaught (in promise): Error: Failed to compile fragment shader.

    ERROR Error: Uncaught (in promise): Error: Failed to compile fragment shader.

    Below is the Code in Angular 14. Getting ERROR Error: Uncaught (in promise): Error: Failed to compile fragment shader.

    async handposes() { const img = document.getElementById('videoid');

    handTrack.startVideo(img);
    /* start camera input stream on the provided video tag. returns a promise with message format
    { status: false, msg: 'please provide a valid video element' } 
    */
    //const img = this.canvas;
    const model = await handTrack.load();
    const predictions = await model.detect(img);
    console.log('getpredictions');
    

    }

    ngAfterViewInit(): void { /this._recognizer.initialize( this.canvas.nativeElement, this.video.nativeElement );/ this.handposes(); }

    opened by cloudyape 0
  • Bump terser from 5.6.1 to 5.14.2

    Bump terser from 5.6.1 to 5.14.2

    Bumps terser from 5.6.1 to 5.14.2.

    Changelog

    Sourced from terser's changelog.

    v5.14.2

    • Security fix for RegExps that should not be evaluated (regexp DDOS)
    • Source maps improvements (#1211)
    • Performance improvements in long property access evaluation (#1213)

    v5.14.1

    • keep_numbers option added to TypeScript defs (#1208)
    • Fixed parsing of nested template strings (#1204)

    v5.14.0

    • Switched to @​jridgewell/source-map for sourcemap generation (#1190, #1181)
    • Fixed source maps with non-terminated segments (#1106)
    • Enabled typescript types to be imported from the package (#1194)
    • Extra DOM props have been added (#1191)
    • Delete the AST while generating code, as a means to save RAM

    v5.13.1

    • Removed self-assignments (varname=varname) (closes #1081)
    • Separated inlining code (for inlining things into references, or removing IIFEs)
    • Allow multiple identifiers with the same name in var destructuring (eg var { a, a } = x) (#1176)

    v5.13.0

    • All calls to eval() were removed (#1171, #1184)
    • source-map was updated to 0.8.0-beta.0 (#1164)
    • NavigatorUAData was added to domprops to avoid property mangling (#1166)

    v5.12.1

    • Fixed an issue with function definitions inside blocks (#1155)
    • Fixed parens of new in some situations (closes #1159)

    v5.12.0

    • TERSER_DEBUG_DIR environment variable
    • @​copyright comments are now preserved with the comments="some" option (#1153)

    v5.11.0

    • Unicode code point escapes (\u{abcde}) are not emitted inside RegExp literals anymore (#1147)
    • acorn is now a regular dependency

    v5.10.0

    • Massive optimization to max_line_len (#1109)
    • Basic support for import assertions
    • Marked ES2022 Object.hasOwn as a pure function
    • Fix delete optional?.property
    • New CI/CD pipeline with github actions (#1057)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Background loading?

    Background loading?

    handTrack.load(modelParams) will freeze the UI for 5..10s during loading time. This is unacceptable for a background task where responsiveness is more important than quick loading times. If it's just a network call it needs to be able to async, if it's a computationally expensive operation a web worker might be more useful. Both of these are hard to extend from the outside.

    opened by ModischFabrications 0
  • Bump node-fetch from 2.6.1 to 2.6.7

    Bump node-fetch from 2.6.1 to 2.6.7

    Bumps node-fetch from 2.6.1 to 2.6.7.

    Release notes

    Sourced from node-fetch's releases.

    v2.6.7

    Security patch release

    Recommended to upgrade, to not leak sensitive cookie and authentication header information to 3th party host while a redirect occurred

    What's Changed

    Full Changelog: https://github.com/node-fetch/node-fetch/compare/v2.6.6...v2.6.7

    v2.6.6

    What's Changed

    Full Changelog: https://github.com/node-fetch/node-fetch/compare/v2.6.5...v2.6.6

    v2.6.2

    fixed main path in package.json

    Commits
    • 1ef4b56 backport of #1449 (#1453)
    • 8fe5c4e 2.x: Specify encoding as an optional peer dependency in package.json (#1310)
    • f56b0c6 fix(URL): prefer built in URL version when available and fallback to whatwg (...
    • b5417ae fix: import whatwg-url in a way compatible with ESM Node (#1303)
    • 18193c5 fix v2.6.3 that did not sending query params (#1301)
    • ace7536 fix: properly encode url with unicode characters (#1291)
    • 152214c Fix(package.json): Corrected main file path in package.json (#1274)
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by endless, a new releaser for node-fetch since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Add Typescript definitions

    Add Typescript definitions

    Seems like type definitions are missing, which makes integration with a typescript (svelte) project cumbersome:

    Hint: Could not find a declaration file for module 'handtrackjs'. '/handtrackjs/src/index.js' implicitly has an 'any' type.
      Try `npm i --save-dev @types/handtrackjs` if it exists or add a new declaration (.d.ts) file containing `declare module 'handtrackjs';` (ts)
    
    opened by ModischFabrications 0
Releases(0.0.6)
  • 0.0.6(Feb 11, 2019)

    First release. Support for methods

    • load(params) : load model
    • detect(input) : get predictions given input
    • startVideo(video) : start video stream on given video element
    • getFPS() : get FPS calculated as
    • renderPredictions(predictions, canvas, context, mediasource): draw bounding box on canvas.
    • dispose() : delete model instance
    Source code(tar.gz)
    Source code(zip)
Owner
Victor Dibia
Research Engineer at Cloudera Fast Forward Labs, developer, designer! Interested in the intersection of Applied AI and HCI.
Victor Dibia
architecture-free neural network library for node.js and the browser

Synaptic Important: Synaptic 2.x is in stage of discussion now! Feel free to participate Synaptic is a javascript neural network library for node.js a

Juan Cazala 6.9k Dec 27, 2022
A speech recognition library running in the browser thanks to a WebAssembly build of Vosk

A speech recognition library running in the browser thanks to a WebAssembly build of Vosk

Ciaran O'Reilly 207 Jan 3, 2023
This is a JS/TS library for accelerated tensor computation intended to be run in the browser.

TensorJS TensorJS How to use Tensors Tensor operations Reading values Data types Converting between backends Onnx model support Optimizations Running

Frithjof Winkelmann 32 Jun 26, 2022
Deep Learning in Javascript. Train Convolutional Neural Networks (or ordinary ones) in your browser.

ConvNetJS ConvNetJS is a Javascript implementation of Neural networks, together with nice browser-based demos. It currently supports: Common Neural Ne

Andrej 10.4k Dec 31, 2022
Run Keras models in the browser, with GPU support using WebGL

**This project is no longer active. Please check out TensorFlow.js.** The Keras.js demos still work but is no longer updated. Run Keras models in the

Leon Chen 4.9k Dec 29, 2022
The Fastest DNN Running Framework on Web Browser

WebDNN: Fastest DNN Execution Framework on Web Browser WebDNN is an open source software framework for executing deep neural network (DNN) pre-trained

Machine Intelligence Laboratory (The University of Tokyo) 1.9k Jan 1, 2023
Train and test machine learning models for your Arduino Nano 33 BLE Sense in the browser.

Tiny Motion Trainer Train and test IMU based TFLite models on the Web Overview Since 2009, coders have created thousands of experiments using Chrome,

Google Creative Lab 59 Nov 21, 2022
Bayesian bandit implementation for Node and the browser.

#bayesian-bandit.js This is an adaptation of the Bayesian Bandit code from Probabilistic Programming and Bayesian Methods for Hackers, specifically d3

null 44 Aug 19, 2022
Simple Javascript implementation of the k-means algorithm, for node.js and the browser

#kMeans.js Simple Javascript implementation of the k-means algorithm, for node.js and the browser ##Installation npm install kmeans-js ##Example (JS)

Emil Bay 44 Aug 19, 2022
Clustering algorithms implemented in Javascript for Node.js and the browser

Clustering.js ####Clustering algorithms implemented in Javascript for Node.js and the browser Examples License Copyright (c) 2013 Emil Bay github@tixz

Emil Bay 29 Aug 19, 2022
A neural network library built in JavaScript

A flexible neural network library for Node.js and the browser. Check out a live demo of a movie recommendation engine built with Mind. Features Vector

Steven Miller 1.5k Dec 31, 2022
A lightweight library for neural networks that runs anywhere

Synapses A lightweight library for neural networks that runs anywhere! Getting Started Why Sypapses? It's easy Add one dependency to your project. Wri

Dimos Michailidis 65 Nov 9, 2022
A JavaScript deep learning and reinforcement learning library.

neurojs is a JavaScript framework for deep learning in the browser. It mainly focuses on reinforcement learning, but can be used for any neural networ

Jan 4.4k Jan 4, 2023
A WebGL accelerated JavaScript library for training and deploying ML models.

TensorFlow.js TensorFlow.js is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models. ⚠️ We recent

null 16.9k Jan 4, 2023
Linear Regression library in pure Javascript

Lyric Linear Regression library in pure Javascript Lyric can help you analyze any set of x,y series data by building a model that can be used to: Crea

Flurry, Inc. 43 Dec 22, 2020
Machine Learning library for node.js

shaman Machine Learning library for node.js Linear Regression shaman supports both simple linear regression and multiple linear regression. It support

Luc Castera 108 Feb 26, 2021
Support Vector Machine (SVM) library for nodejs

node-svm Support Vector Machine (SVM) library for nodejs. Support Vector Machines Wikipedia : Support vector machines are supervised learning models t

Nicolas Panel 296 Nov 6, 2022
machinelearn.js is a Machine Learning library written in Typescript

machinelearn.js is a Machine Learning library written in Typescript. It solves Machine Learning problems and teaches users how Machine Learning algorithms work.

machinelearn.js 522 Jan 2, 2023
FANN (Fast Artificial Neural Network Library) bindings for Node.js

node-fann node-fann is a FANN bindings for Node.js. FANN (Fast Artificial Neural Network Library) is a free open source neural network library, which

Alex Kocharin 186 Oct 31, 2022