[UNMAINTAINED] Simple feed-forward neural network in JavaScript

Overview

This project has reached the end of its development as a simple neural network library. Feel free to browse the code, but please use other JavaScript neural network libraries in development like brain.js and convnetjs.

brain

brain is a JavaScript neural network library. Here's an example of using it to approximate the XOR function:

var net = new brain.NeuralNetwork();

net.train([{input: [0, 0], output: [0]},
           {input: [0, 1], output: [1]},
           {input: [1, 0], output: [1]},
           {input: [1, 1], output: [0]}]);

var output = net.run([1, 0]);  // [0.987]

There's no reason to use a neural network to figure out XOR however (-: so here's a more involved, realistic example: Demo: training a neural network to recognize color contrast

Using in node

If you have node you can install with npm:

npm install brain

Using in the browser

Download the latest brain.js. Training is computationally expensive, so you should try to train the network offline (or on a Worker) and use the toFunction() or toJSON() options to plug the pre-trained network in to your website.

Training

Use train() to train the network with an array of training data. The network has to be trained with all the data in bulk in one call to train(). The more training patterns, the longer it will probably take to train, but the better the network will be at classifiying new patterns.

Data format

Each training pattern should have an input and an output, both of which can be either an array of numbers from 0 to 1 or a hash of numbers from 0 to 1. For the color constrast demo it looks something like this:

var net = new brain.NeuralNetwork();

net.train([{input: { r: 0.03, g: 0.7, b: 0.5 }, output: { black: 1 }},
           {input: { r: 0.16, g: 0.09, b: 0.2 }, output: { white: 1 }},
           {input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 }}]);

var output = net.run({ r: 1, g: 0.4, b: 0 });  // { white: 0.99, black: 0.002 }

Options

train() takes a hash of options as its second argument:

net.train(data, {
  errorThresh: 0.005,  // error threshold to reach
  iterations: 20000,   // maximum training iterations
  log: true,           // console.log() progress periodically
  logPeriod: 10,       // number of iterations between logging
  learningRate: 0.3    // learning rate
})

The network will train until the training error has gone below the threshold (default 0.005) or the max number of iterations (default 20000) has been reached, whichever comes first.

By default training won't let you know how its doing until the end, but set log to true to get periodic updates on the current training error of the network. The training error should decrease every time. The updates will be printed to console. If you set log to a function, this function will be called with the updates instead of printing to the console.

The learning rate is a parameter that influences how quickly the network trains. It's a number from 0 to 1. If the learning rate is close to 0 it will take longer to train. If the learning rate is closer to 1 it will train faster but it's in danger of training to a local minimum and performing badly on new data. The default learning rate is 0.3.

Output

The output of train() is a hash of information about how the training went:

{
  error: 0.0039139985510105032,  // training error
  iterations: 406                // training iterations
}

Failing

If the network failed to train, the error will be above the error threshold. This could happen because the training data is too noisy (most likely), the network doesn't have enough hidden layers or nodes to handle the complexity of the data, or it hasn't trained for enough iterations.

If the training error is still something huge like 0.4 after 20000 iterations, it's a good sign that the network can't make sense of the data you're giving it.

JSON

Serialize or load in the state of a trained network with JSON:

var json = net.toJSON();

net.fromJSON(json);

You can also get a custom standalone function from a trained network that acts just like run():

var run = net.toFunction();

var output = run({ r: 1, g: 0.4, b: 0 });

console.log(run.toString()); // copy and paste! no need to import brain.js

Options

NeuralNetwork() takes a hash of options:

var net = new brain.NeuralNetwork({
  hiddenLayers: [4],
  learningRate: 0.6 // global learning rate, useful when training using streams
});

hiddenLayers

Specify the number of hidden layers in the network and the size of each layer. For example, if you want two hidden layers - the first with 3 nodes and the second with 4 nodes, you'd give:

hiddenLayers: [3, 4]

By default brain uses one hidden layer with size proportionate to the size of the input array.

Streams

The network now has a WriteStream. You can train the network by using pipe() to send the training data to the network.

Example

Refer to stream-example.js for an example on how to train the network with a stream.

Initialization

To train the network using a stream you must first create the stream by calling net.createTrainStream() which takes the following options:

  • floodCallback() - the callback function to re-populate the stream. This gets called on every training iteration.
  • doneTrainingCallback(info) - the callback function to execute when the network is done training. The info param will contain a hash of information about how the training went:
{
  error: 0.0039139985510105032,  // training error
  iterations: 406                // training iterations
}

Transform

Use a Transform to coerce the data into the correct format. You might also use a Transform stream to normalize your data on the fly.

Comments
  • Stream Training Ability

    Stream Training Ability

    This pull request will give the network the ability to train using streams. Currently the only way to train is to supply an array of all the data. If your dataset is large enough you wont be able to load the entire thing into memory.

    This pull request shouldn't break any of the current features or functionality. It only adds in the ability to train using streams.

    The network is now a Write Stream that can be written to for each training iteration. It helps if you think of the network when training, as something that you are writing data to. ie. like pushing data into a brain.

    opened by nickpoorman 17
  • Train Multiple Times

    Train Multiple Times

    This is an enhancement request: I think we should be able to train one neural network multiple times.

    Without this feature, we need to keep a record of everything we've used for previous trainings, append our new data, and train again, creating a new network. But with this feature, we can pass around a neural network export, and train again when needed.

    opened by tlhunter 17
  • .gitignore for node_modules?

    .gitignore for node_modules?

    When I fork this repo, and do "npm install", the folder is filled with node_modules, and since there is no .gitignore, git tries to commit them...

    From what I read, there are two options for handling node_modules in git projects:

    A. Keep all the node_modules in the git repository, together with my project, or:

    B. Don't keep any node_modules in the git repository, and have a ".gitignore" file that contains "node_modules".

    However, here, there are no node_modules, but also no .gitignore file...

    I am not an expert, so maybe it's my mistake.

    opened by erelsgl 12
  • How to use this for self learning AI?

    How to use this for self learning AI?

    One way to have self learning AI is to evolve neural networks, which is what I want to use this for.

    What I need to be able to do this:

    1. Create a random neural network with totally random weights.
    2. Take a few sensory inputs and pass them into the neural network each frame.
    3. The network should then quickly return some outputs based on the inputs. For example, all I might need is 1 output, which is how fast to turn & what direction, 1 being hard right, -1 being hard left.
    4. Ability to tweak some of the weights slightly (mutation).

    Is that possible with this library, or is that not what it's designed for and I should look elsewhere? Thanks!

    opened by farzher 5
  • Redis backend for Brain?

    Redis backend for Brain?

    Is anyone interested-in or working-on a Redis backend for this project? Is there a reason there isn't one already (something I may be overlooking), or any fundamental reasons this wouldn't work?

    opened by christopherdebeer 5
  • What type of neural network is this

    What type of neural network is this

    I'm trying to use this library for an image recognition assignment. We're getting pretty poor results, around 40% accuracy, compared to the high 90% range for all the other techniques we've tested. That's with all our preprocessing and training over 100s of iterations with a very low error threshold.

    I'm wondering what type of neural network this is, and whether there's anything fundamental in its implementation that makes it poorly suited to image recognition? Convolutional NNs are generally great at it, but I was thinking that this library might be a different type.

    Does it generally perform well on high dimensional data? We're using all features with non-zero variance from our image set, which is in the order of 100 or so pixels/features. The colour contrast example given only has 3 dimensions, so I was thinking we might be able to get better performance with some aggressive feature extraction/dimensionality reduction.

    opened by lavelle 4
  • Make brain bower installable

    Make brain bower installable

    If you don't know, Bower is a package manager for javascript. I've already created a bower.json file for you, so all you need to do is publish it by running the following.

    sudo npm install -g bower
    bower register brain git://github.com/harthur/brain.git
    

    If you have any questions, please let me know.

    opened by levicc00123 4
  • Write null breaks streaming conventions

    Write null breaks streaming conventions

    Would be better to have something like write ahead log similar to databases. Where there is a max amount of data events before training is started. If max is not reached in some time frame, then go ahead with training and begin queuing up more data while training. however lock the training while currently training.

    At the end of each training session, trained values should be copied, and the run function should work on these copies.

    By removing the null we can

    socket.on('data', function (buffer) { brainstream.write(buffer); } 
    

    And all will work. I may try to create a pull request.

    opened by miketheprogrammer 4
  • fromJSON does not like output:

    fromJSON does not like output: "0" ... converts results to Array

    Nice Neural Net! I have two suggestions for improvement:

    1. Suggest adding brain.fromJSON shortcut:
       brain.fromJSON = function (json) { 
          return new brain.NeuralNetwork().fromJSON(
               typeof json === "string" ? JSON.parse(json) : json
          ); 
       }
    

    Let me know if you want a pull request for this.

    1. Found a small "gotcha" when using fromJSON and training for output:"0".

    When training, if you have a string output of zero "0" it converts the results of run to an Array if created using fromJSON. This could happen when creating a portable NeuralNet to convert images (for example, on a license plate) of letters and numbers to a string in Javascript.

    Example:

    net.train([
      {input:inputX,output:{"X":1}},
      {input:input1,output:{"1":1}},
      {input:input0,output:{"0":1}}
    ]);
    
    net.run(inputX).constructor === Object
    
    // net.run(inputX) will return an Object,
    // but if cloned it will return an Array:
    
    var net_clone = brain.fromJSON(net.toJSON());
    net_clone.run(inputX).constructor === Array;
    

    toFunction is unaffected by this bug.

    Fuller example at this gist: https://gist.github.com/josher19/5218613

    Let me know if you would like me to add a unit test to brain/test/unit/json.js or any other way I can help.

    Best Regards, ->> Josh <<-

    opened by josher19 4
  • Missing build instructions or v0.6.1 download

    Missing build instructions or v0.6.1 download

    I'm having trouble getting brain to build properly. I was trying to get v0.6.1 working for a browser-based game I'm working on, but didn't see it listed on the downloads page. So, I cloned it, checked out version v0.6.1 and tried a build:

    git clone git://github.com/harthur/brain.git
    git checkout 8ba611b771d6186d2ff63405978b951c7b49cbb1 # 0.6.1
    npm install
    npm install browserify
    jake build
    npm install [email protected]
    jake minify
    

    Which worked (no errors), but wouldn't run in the browser.

    So, I tried

    git checkout 20114ae3844bbdbe4b1e5abb2fc38f50b940bfa4 # 0.6.0
    jake build
    jake minify
    wget -O brain-0.6.0-official.js https://github.com/downloads/harthur/brain/brain-0.6.0.js
    diff brain-0.6.0.js brain-0.6.0-official.js > brain.diff
    

    and compared my generated brain-0.6.0.js to the official brain-0.6.0.js and it was way different and also wouldn't run in the browser.

    How do you build brain?

    opened by benmanns 3
  • demo: input/data[0] not defined

    demo: input/data[0] not defined

    The demo works in none of the browsers I've tested (firefox, chrome, opera). Gives me

    Cannot read property 'input' of undefined
    

    and

    data[0] is undefined
    
    opened by pvdz 3
  • Training error : NaN with numeric values

    Training error : NaN with numeric values

    Hi.

    I try to learn to detect the gender of names. The TrainingData looks like this

    { input: [ 2, 26, 11, 8, 13, 4 ], output: { h: 0, f: 1 } }, { input: [ 2, 4, 17, 8, 18, 4 ], output: { h: 0, f: 1 } }, { input: [ 2, 4, 18, 0, 8, 17, 4 ], output: { h: 1, f: 0 } }, { input: [ 2, 26, 18, 0, 17 ], output: { h: 1, f: 0 } }, { input: [ 2, 7, 0, 13, 19, 0, 11 ], output: { h: 0, f: 1 } }, { input: [ 2, 7, 0, 13, 19, 26 ], output: { h: 0, f: 1 } }, { input: [ 2, 7, 0, 17, 11, 8, 13, 4 ], output: { h: 0, f: 1 } }, { input: [ 2, 7, 0, 17, 11, 14, 19 ], output: { h: 1, f: 0 } },

    My code : https://gist.github.com/lucaspojo/1a4c7c848f18074ccd195eb8d7828b6a

    Do you know what I'm doing wrong?

    Thx!

    opened by lucydjo 1
  • Node-Red example flow doesn't work.

    Node-Red example flow doesn't work.

    You have two flows: Train and Run. I import it to Node-Red and press the inject node to Train it. I get this in the debug window. "TypeError: May not write null values to stream"

    If the demo doesn't work, I am a bit worried.

    opened by Just-another-pleb 1
  • Training RSSI->distance to location in brain

    Training RSSI->distance to location in brain

    Hi, I am trying to use brain for training RSSI and sensors coordinates as inputs and position as output in x,y,z normalized in single number. can you suggest on data modeling this into brain.

    inputs : [{x1,y1,z1,distance1},{{x2,y2,z2,distance2},{x3,y3,z3,distance3}} output: {x100+y10+z} as position.

    i get constant error while training even up to 20000 iterations.

    Regards

    opened by rpant1979 2
  • How to run this project

    How to run this project

    So I try to run this project, here is the steps I take, not sure if it is correct

    yarn install
    

    When it asks

    Couldn't find any versions for "esprima-six" that matches "0.0.3"
    ? Please choose a version from this list: (Use arrow keys)
    

    Choose 1.0.1

    Then, create a file like basic.js

    var assert = require("assert"),
        brain = require("./lib/brain");
    
    var net = new brain.NeuralNetwork();
    
    net.train([{input: { r: 0.03, g: 0.7, b: 0.5 }, output: { black: 1 }},
               {input: { r: 0.16, g: 0.09, b: 0.2 }, output: { white: 1 }},
               {input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 }}]);
    
    var output = net.run({ r: 1, g: 
    

    Run node basic.js to see the result. Hope it help

    opened by onmyway133 1
  • Cannot read property 'bias' of undefined neuralnetwork.js:366:58 when runing fn fromJSON with server Hapi

    Cannot read property 'bias' of undefined neuralnetwork.js:366:58 when runing fn fromJSON with server Hapi

    If I running single my script to recognition a charecter then Neral Network skip layer input, but if require it from server Hapi error occur at line: 366, 367. And I resolved it: this.biases[i][j] = layer[node] ? layer[node].bias : 0; this.weights[i][j] = layer[node] ? _(layer[node].weights).toArray() : 0;

    opened by TatChu 0
Owner
Heather
Heather
Deep Neural Network Sandbox for JavaScript.

Deep Neural Network Sandbox for Javascript Train a neural network with your data & save it's trained state! Demo • Installation • Getting started • Do

Matias Vazquez-Levi 417 Nov 8, 2022
architecture-free neural network library for node.js and the browser

Synaptic Important: Synaptic 2.x is in stage of discussion now! Feel free to participate Synaptic is a javascript neural network library for node.js a

Juan Cazala 6.8k Nov 22, 2022
Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX, TensorFlow Lite, Caffe, Keras, Darknet, Paddle

Lutz Roeder 20.6k Nov 22, 2022
Powerful Neural Network for Node.js

NeuralN Powerful Neural Network for Node.js NeuralN is a C++ Neural Network library for Node.js with multiple advantages compared to existing solution

TOTEMS::Tech 274 Nov 5, 2022
FANN (Fast Artificial Neural Network Library) bindings for Node.js

node-fann node-fann is a FANN bindings for Node.js. FANN (Fast Artificial Neural Network Library) is a free open source neural network library, which

Alex Kocharin 186 Oct 31, 2022
Deep Learning in Javascript. Train Convolutional Neural Networks (or ordinary ones) in your browser.

ConvNetJS ConvNetJS is a Javascript implementation of Neural networks, together with nice browser-based demos. It currently supports: Common Neural Ne

Andrej 10.4k Nov 25, 2022
DN2A - Digital Neural Networks Architecture in JavaScript

DN2A (JavaScript) Digital Neural Networks Architecture About DN2A is a set of highly decoupled JavaScript modules for Neural Networks and Artificial I

Antonio De Luca 464 Oct 27, 2022
DN2A - Digital Neural Networks Architecture

DN2A (JavaScript) Digital Neural Networks Architecture About DN2A is a set of highly decoupled JavaScript modules for Neural Networks and Artificial I

Antonio De Luca 464 Oct 27, 2022
A lightweight library for neural networks that runs anywhere

Synapses A lightweight library for neural networks that runs anywhere! Getting Started Why Sypapses? It's easy Add one dependency to your project. Wri

Dimos Michailidis 65 Nov 9, 2022
DN2A - Digital Neural Networks Architecture

DN2A (JavaScript) Digital Neural Networks Architecture About DN2A is a set of highly decoupled JavaScript modules for Neural Networks and Artificial I

Antonio De Luca 464 Oct 27, 2022
Simple Javascript implementation of the k-means algorithm, for node.js and the browser

#kMeans.js Simple Javascript implementation of the k-means algorithm, for node.js and the browser ##Installation npm install kmeans-js ##Example (JS)

Emil Bay 44 Aug 19, 2022
Pure Javascript OCR for more than 100 Languages 📖🎉🖥

Version 2 is now available and under development in the master branch, read a story about v2: Why I refactor tesseract.js v2? Check the support/1.x br

Project Naptha 28.8k Nov 29, 2022
WebGL-accelerated ML // linear algebra // automatic differentiation for JavaScript.

This repository has been archived in favor of tensorflow/tfjs. This repo will remain around for some time to keep history but all future PRs should be

null 8.5k Nov 25, 2022
A JavaScript deep learning and reinforcement learning library.

neurojs is a JavaScript framework for deep learning in the browser. It mainly focuses on reinforcement learning, but can be used for any neural networ

Jan 4.3k Nov 28, 2022
Differential Programming in JavaScript.

April 19, 2018 TensorFlow.js was recently released. It is well engineered, provides an autograd-style interface to backprop, and has committed to supp

Propel 2.7k Nov 21, 2022
Machine learning tools in JavaScript

ml.js - Machine learning tools in JavaScript Introduction This library is a compilation of the tools developed in the mljs organization. It is mainly

ml.js 2.3k Nov 22, 2022
A WebGL accelerated JavaScript library for training and deploying ML models.

TensorFlow.js TensorFlow.js is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models. ⚠️ We recent

null 16.8k Nov 23, 2022
JavaScript API for face detection and face recognition in the browser and nodejs with tensorflow.js

face-api.js JavaScript face recognition API for the browser and nodejs implemented on top of tensorflow.js core (tensorflow/tfjs-core) Click me for Li

Vincent Mühler 14.5k Nov 30, 2022
Call Python packages in JavaScript.

Introduction to Boa Boa is the Python Bridge Layer in Pipcook, it lets you call Python functions seamlessly in Node.js, it delivers any Python module

imgcook 62 Nov 15, 2022