Interactive digital art with head-coupled perspective effect using Three.js and TensorFlow.js

Overview

Interactive digital frame with head-tracking using Three.js & TensorFlow.js

Using TensorFlow.js and Three.js, this project is a prototype of an interactive digital art frame using head-tracking to create an effect of head-coupled perspective.

A famous use of this effect can be seen in Ghost Protocol.

I wanted to experiment with making interactive art pieces so this project is a PWA that can be run fullscreen on an iPad.

⚠️ This is a prototype made as a side project so the code is not production-ready. Its performance is not optimised so the first load can take a while. I developed and tested it on a Macbook Pro and iPad Pro so if you try it on another device and it doesn't work or doesn't look super responsive, I don't intend to fix it. With that in mind, here's the live demo that works with both mouse movements and head-tracking. ⚠️

Demo

If you'd like to read more about it, check out the blog post!

Local development

  • Clone this repo
  • Run npm install to install the dependencies
  • Run npm start to start the local server
  • Your browser should automatically open a new tab to http://localhost:3000
You might also like...

Train and test machine learning models for your Arduino Nano 33 BLE Sense in the browser.

Train and test machine learning models for your Arduino Nano 33 BLE Sense in the browser.

Tiny Motion Trainer Train and test IMU based TFLite models on the Web Overview Since 2009, coders have created thousands of experiments using Chrome,

Nov 21, 2022

A react native cached, animated, and auto chooses the best-resolution image for the current screen.

A react native cached, animated, and auto chooses the best-resolution image for the current screen.

A react native cached, animated and auto chooses the best-resolution image for the current screen. This is a simple react native image component that

Mar 30, 2022

Visualizer for neural network, deep learning, and machine learning models

Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX, TensorFlow Lite, Caffe, Keras, Darknet, Paddle

Jan 5, 2023

Run XGBoost model and make predictions in Node.js

XGBoost-Node eXtreme Gradient Boosting Package in Node.js XGBoost-Node is a Node.js interface of XGBoost. XGBoost is a library from DMLC. It is design

Nov 15, 2022

Bayesian bandit implementation for Node and the browser.

#bayesian-bandit.js This is an adaptation of the Bayesian Bandit code from Probabilistic Programming and Bayesian Methods for Hackers, specifically d3

Aug 19, 2022

Simple Javascript implementation of the k-means algorithm, for node.js and the browser

#kMeans.js Simple Javascript implementation of the k-means algorithm, for node.js and the browser ##Installation npm install kmeans-js ##Example (JS)

Aug 19, 2022

Unsupervised machine learning with multivariate Gaussian mixture model which supports both offline data and real-time data stream.

Gaussian Mixture Model Unsupervised machine learning with multivariate Gaussian mixture model which supports both offline data and real-time data stre

Oct 7, 2022

Clustering algorithms implemented in Javascript for Node.js and the browser

Clustering.js ####Clustering algorithms implemented in Javascript for Node.js and the browser Examples License Copyright (c) 2013 Emil Bay github@tixz

Aug 19, 2022

An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more

An NLP library for building bots, with entity extraction, sentiment analysis, automatic language identify, and so more

NLP.js If you're looking for the version 3 docs, you can find them here Version 3 "NLP.js" is a general natural language utility for nodejs. Currently

Dec 29, 2022
Owner
Charlie
Senior Front-end Developer @netlify , former Software Dev @atlassian & @thoughtworks & Creative Developer @nytimes. Always tinkering with new technologies
Charlie
A state of the art* image classification app that tells users if the photo they uploaded is a pizza or is not a pizza

Not Pizza Have you ever seen something and wondered if it was a pizza or not? No? Doesn't matter. What matters is that we have a state of the art imag

Kei the Pizza 21 Oct 24, 2022
DN2A - Digital Neural Networks Architecture

DN2A (JavaScript) Digital Neural Networks Architecture About DN2A is a set of highly decoupled JavaScript modules for Neural Networks and Artificial I

Antonio De Luca 464 Jan 1, 2023
DN2A - Digital Neural Networks Architecture

DN2A (JavaScript) Digital Neural Networks Architecture About DN2A is a set of highly decoupled JavaScript modules for Neural Networks and Artificial I

Antonio De Luca 464 Jan 1, 2023
DN2A - Digital Neural Networks Architecture in JavaScript

DN2A (JavaScript) Digital Neural Networks Architecture About DN2A is a set of highly decoupled JavaScript modules for Neural Networks and Artificial I

Antonio De Luca 464 Jan 1, 2023
Run Keras models in the browser, with GPU support using WebGL

**This project is no longer active. Please check out TensorFlow.js.** The Keras.js demos still work but is no longer updated. Run Keras models in the

Leon Chen 4.9k Dec 29, 2022
NEW⚡ PancakeSwap Prediction Bot using AI live recomendations. ~70% Win rate 🔮

?? PancakeSwap Prediction SmartBot PancakeSwap Prediction Bot using live TradingView AI recomendations. ~70% Win rate. ⭐ Please consider giving a star

Eashita Sora 19 Dec 15, 2022
architecture-free neural network library for node.js and the browser

Synaptic Important: Synaptic 2.x is in stage of discussion now! Feel free to participate Synaptic is a javascript neural network library for node.js a

Juan Cazala 6.9k Dec 27, 2022
A JavaScript deep learning and reinforcement learning library.

neurojs is a JavaScript framework for deep learning in the browser. It mainly focuses on reinforcement learning, but can be used for any neural networ

Jan 4.4k Jan 4, 2023
A WebGL accelerated JavaScript library for training and deploying ML models.

TensorFlow.js TensorFlow.js is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models. ⚠️ We recent

null 16.9k Jan 4, 2023