About Introduction to Machine Learning for the Arts, IMA, Tisch School of the Arts, New York University, Fall 2022

Overview

Introduction to Machine Learning for the Arts

About Introduction to Machine Learning for the Arts, IMA, Tisch School of the Arts, New York University, Fall 2022

Info

  • Friday, 5:20 PM - 8:30 PM, 370 Jay Street, Room 409 Loc: Brooklyn Campus
  • Office Hours

OUTLINE

0: Intro!

Friday 9/2

  • Introduction to Machine Learning

1: ml5.js

Friday 9/9

  • Session 1: ml5.js
  • Session 2: Image Classification

2: Transfer Learning

Friday 9/16

  • Session 1: Transfer Learning with Teachable Machine
  • Session 2: Transfer learning with ml5.js

3: Machine Learning Models

Friday 9/23

  • Session 1: Pre-trained models 1
  • Session 2: Pre-trained models 2

4: Training a Model

Friday 9/30

  • Session 1: Real-Time Data Collection
  • Session 2: Real-Time Model Training

5: Convolutional Neural Networks

Friday 10/7

  • Session 1: Image Data
  • Session 2: Convolutional Neural Networks

6: What are data?

Friday 10/14

  • Session 1: Working with Data 1
  • Session 2: Working with Data 2

7: Recurrent Neural Networks

Friday 10/21

  • Session 1: SketchRNN (Drawing)
  • Session 2: charRNN (Text)

8: Pre-Trained Models with RunwayML

Friday 10/28

  • Session 1 (Tu 10/26): Introduction to RunwayML
  • Session 2 (Th 10/28): RunwayML Model Workshop

9: Generative Models with RunwayML

Friday 11/4

  • Session 1: Hosted Models and Networking, Model Training
  • Session 2: Generative Adversarial Networks, Interactive Image Synthesis

10: AI Enhanced video with Runway Sequel

Friday 11/11

  • Session 1: AI Rotoscoping, Optical Flow, Depth Estimation
  • Session 2: Sequel Workshop

11: Final Project Proposals

Friday 11/18

  • Session 1: Group 1
  • Session 2: Group 1

Thanksgiving break

12: Final Project Development and Testing

Friday 12/2

  • Session 1: Group 1
  • Session 2: Group 1

13: Final Project Presentations

Friday 12/9

  • Session 1: Group 1
  • Session 2: Group 2

CODE OF CONDUCT

Please read and review the ITP/IMA Code of Conduct. The Code of Conduct will be reviewed and discussed as part of the course introduction.

The ITP/IMA Code of Conduct is an evolving work-in-progress document that establishes and communicates the commitment of the ITP/IMA community to uphold a key set of standards and obligations that aim to make ITP/IMA an inclusive and welcoming environment.

COURSE DESCRIPTION

An introductory course designed to provide students with hands-on experience developing creative coding projects with machine learning. The history, theory, and application of machine learning algorithms and related datasets are explored in a laboratory context of experimentation and discussion. Examples and exercises will be demonstrated in JavaScript using the p5.js, ml5.js, and TensorFlow.js libraries. In addition, students will learn to work with open source pre-trained models in the cloud using Runway. Principles of data collection and ethics are introduced. Weekly assignments, team and independent projects, and project reports are required.

COURSE OBJECTIVES

At the completion of this course, the student will:

  • Develop an intuition for and high level understanding of core machine learning concepts and algorithms, including supervised learning, unsupervised learning, reinforcement learning, transfer learning, classification, and regression.
  • Be able to apply machine learning algorithms to real-time interaction in media art projects using pre-trained models and “transfer learning” in JavaScript and related tools.
  • Learn how to collect a custom dataset to train a machine learning model and
  • Develop a vocabulary for critical discussions around the social impact and ethics of data collection and application of machine learning algorithms.
  • Become familiar with the current landscape of new media art generated from machine learning algorithms. Understand how to use a machine learning model to generate media: words, sound, and images.

EQUIPMENT

You will need a modern laptop (4 years old or younger is a good rule of thumb). Most required software is freely available. The department has all required commercial software installed on laptops available for checkout.

COURSE TEXTS

There is no textbook for the class. Readings and videos will be assigned on the individual session notes pages.

GRADING AND ATTENDANCE

Grades for the course will follow the standard A through F letter grading system and will be determined by the following breakdown:

  • 25% Participation
  • 50% Assignments (including reading responses and other written work)
  • 25% Final project

At most two (2) unexcused absences will be tolerated without effect to your grade. Any more than two (2) unexcused absences will result a lowering of your final grade by one whole grade for each unexcused absence. For example, three (3) unexcused absences will result in your highest possible grade being a B instead of an A. Four (4) unexcused absences will result in your highest possible grade being a C and so on. Six (6) unexcused absences would result in an automatic F for the course. Two (2) late arrivals will count for one (1) absence.

PARTICIPATION:

This class will be highly participatory. You are expected to contribute to discussions, engage in group work, give feedback to your peers, and otherwise fully participate in class.

PHONE, TABLET, LAPTOP, AND OTHER ELECTRONICS USE

Recreational use of phones, music players, video game systems and other devices are forbidden. Laptops and tablets are allowed for note taking, in class work, as well as relevant research only. Activities not related to the class, such as recreational use of the internet, including all social media websites, email and instant messaging, game playing, and work for other classes, will not be permitted. Such activities are disrespectful to the instructor and distracting to others. Your laptop should always be closed whenever a fellow student is presenting.

TEACHING STYLE

Classes will be a mixture of lecture, discussion, hands-on tutorials, homework review, presentations, and group work. You will need to come to class prepared with a laptop and any other supplies specified for that class.

COURSE SCHEDULE

The course will be one (1) times per week for three (3) hours and thirty minutes for a total of 14 weeks.

ASSIGNMENTS

There will be regular assignments that are relevant the class material. These assignments must be documented (written description, photos, screenshots, screen recording, code, and video all qualify based on the assignment) on a web platform such as a blog or google doc. You are required to link to your assignment from the course repo (you may choose to use a privately shared google doc or password protected website if you prefer.) The due dates are specified on the assignment page.

It is expected that you will spend 6 to 8 hours a week on the class outside of class itself. This will include reviewing material, reading, watching video, completing assignments and so on. Please budget your time accordingly.

Each assignment will be marked as complete (full credit), partially complete (half credit), or incomplete (no credit). To be complete an assignment should meet the criteria specified in the syllabus including documentation. If significant portions are simply not attempted or the assignment is turned in late (up to 1 week) then it may be marked partially complete. If it is more than a week late, not turned in, or an attempt isn’t made to meet the criteria specified it will be marked incomplete.

Responses to reading and other written assignments are also due in class one week after they are assigned and must also be submitted via the class website. Written assignments are expected to be 200 to 500 words in length unless otherwise specified. Grading will follow the same guidelines as above; on time and meeting the criteria specified will be marked as complete. Late (up to 1 week) or partially completed work will be given half credit. Work that is more than a week late, not turned in, or fails to meet the criteria specified will be given no credit.

STATEMENT OF ACADEMIC INTEGRITY

Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else. More information can be found on Tisch’s page regarding Academic Integrity (http://tisch.nyu.edu/faculty/academic-integ).

USE OF FREE AND OPEN SOURCE CODE EXAMPLES

(The following is adapted from Golan Levin’s Interactivity and Computation Course (Fall 2018) at Carnegie Mellon University.)

You must cite the source of any code you use. Please note the following additional expectations and guidelines:

  1. Check the License. When using others' code, pay attention to the license under which it has been released, and be certain to fulfill the terms and requirements of those licenses. Descriptions of common licenses, and their requirements, can be found at choosealicense.com. Some licenses may require permission. If you are confused or aren’t sure how to credit code, ask one of the course instructors and make your best good faith effort. Not properly citing code sources is grounds for a 0 on an assignment.

  2. Use Libraries. The use of general, repurposable libraries is strongly encouraged. The people who developed and contributed these components to the community worked hard, often for no pay; acknowledge them by citing their name and linking to their repository.

  3. Be Careful. It sometimes happens that an artist places the entire source code for their sketch or artwork online, as a resource from which others can learn. Assignments professors give in new-media arts courses are often similar (e.g. "Clock"); you may also discover the work of a student in some other class or school, who has posted code for a project which responds to a similar assignment. You should probably avoid this code. At the very least, you should be careful about approaching such code for possible re-use. If it is necessary to do so, it is best to extract components that solve a specific technical problem, rather than those parts which operate to create a poetic experience. Your challenge, if and/or when you work with others' code, is to make it your own. It should be clear that downloading an artwork from someone's GitHub and simply changing the colors would be disgracefully lazy. And doing so without proper citation would be outright plagiarism.

ACCESSIBILITY

Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.

WELLNESS

Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYUWellness Exchange 212-443-9999.

TITLE IX

Tisch School of the Arts to dedicated to providing its students with a learning environment that is rigorous, respectful, supportive and nurturing so that they can engage in the free exchange of ideas and commit themselves fully to the study of their discipline. To that end Tisch is committed to enforcing University policies prohibiting all forms of sexual misconduct as well as discrimination on the basis of sex and gender. Detailed information regarding these policies and the resources that are available to students through the Title IX office can be found by using the following link. https://www.nyu.edu/about/policies-guidelines-compliance/equal-opportunity/title9.html

You might also like...

Solana blockchain candy machine app boilerplate on top of Metaplex Candy Machine. NextJS, Tailwind, Anchor, SolanaLabs.React, dev/mainnet automation scripts.

NFT Candy Factory NOTE: This repo will prob only work on unix-based environments. The NFT Candy Factory project is designed to let users fork, customi

Dec 30, 2022

Complete Open Source Front End Candy Machine V2 Minter dAPP Built For The Frog Nation NFT Solana Project. Built With React, Candy Machine V2, Typescript

Complete Open Source Front End Candy Machine V2 Minter dAPP Built For The Frog Nation NFT Solana Project. Built With React, Candy Machine V2, Typescript

Complete Open Source Front End Candy Machine V2 Minter dAPP Built For The Frog Nation NFT Solana Project. Built With React, Candy Machine V2, Typescript

Sep 24, 2022

JavaScript package for predictive data analysis and machine learning

scikit.js JavaScript package for predictive data analysis and machine learning. Generic math operations are powered by Tensorflowjs core layer. We are

Jan 6, 2023

Repository for machine learning research.

Machine-Learning A simple project created with django containing links and a chatbot to demonstrate how Choice Coin can be used for eLearning #how to

Oct 21, 2022

Nutrition tracking software that uses machine learning to help you reach your body composition goals.

Nutrition tracking software that uses machine learning to help you reach your body composition goals.

LogSmarter - Machine Learning Nutrition Tracker Description LogSmarter LogSmarter™ is not just another calorie counter. It is an AI nutrition coach th

Sep 4, 2022

Inferrd makes Machine Learning deployment easy and scalable.

Inferrd makes Machine Learning deployment easy and scalable.

Introduction ML Deployment made simple, scalable and collaborative The new open-source standard to deploy, monitor and secure Machine Learning service

Dec 16, 2022

LayerX-AI is a comprehensive platform to annotate and manage your machine learning data.

LayerX-AI is a comprehensive platform to annotate and manage your machine learning data.

The AI Data Platform Annotate, Manage and Deploy Training Data The end-to-end AI data management platform that helps ML teams annotate, manage and dep

Dec 18, 2022

DialogFlow Machine Learning Chatbot

 DialogFlow Machine Learning Chatbot

DialogFlow Machine Learning Chatbot Project Overview This repository contains a backend service for an intelligent chatbot that onboards clients, answ

Jul 21, 2022

An AWS Cloud Native application using CDK that defines a Serverless Event Driven application for interacting with Twitter and utilising Machine Learning / AI as a Service.

An AWS Cloud Native application using CDK that defines a Serverless Event Driven application for interacting with Twitter and utilising Machine Learning / AI as a Service.

AWS Serverless Event Driven Twitter Bot An AWS Cloud Native application using CDK (Written in TypeScript) that defines a Serverless Event Driven appli

Dec 18, 2022
Owner
ml5
ml5: friendly machine learning for the web
ml5
Tutorials for MATH 4432 Statistical Machine Learning, HKUST, Fall 2022

Tutorials for MATH 4432 Statistical Machine Learning Course information MATH 4432 Statistical Machine Learning Instructor: Prof. YANG Can Teaching ass

WANG Zhiwei 7 Dec 1, 2022
Homework Assignments for Visualization for Data Science, Fall 2022, University of Utah

Homeworks for Utah's Vis for Data Science Course In subfolders in this directory you will find the homeworks for CS 6630 / CS 5630 / DS 4630 – Visuali

Datavis Course 19 Nov 14, 2022
All five assignments and the final group project is done in class CSCI5410 (Serverless Data Processing) Fall 2021 of MACS at Dalhousie University.

Dalhousie University | Fall 2021 | CSCI5410 | SDP (Serverless Data Processing) All five assignments and the final group project is done in class CSCI5

Dhrumil Shah 1 Dec 26, 2021
From the Linux Foundation office in New York City, welcome to The Untold Stories of Open Source

From the Linux Foundation office in New York City, welcome to The Untold Stories of Open Source. Each week we explore the people who are supporting Open Source projects, how they became involved with it, and the problems they faced along the way.

The Linux Foundation 77 Jan 5, 2023
HackMIT 2022. 2nd Place in Blockchain for Society sponsored by Jump Crypto. A revolutionary web application that leverages machine learning and blockchain technology to improve the crowdsourcing experience!

?? Wikisafe ?? Wikisafe is a revolutionary new crowdsourcing web application that innovates the process of crowdsourcing information. This application

Benson Liu 5 Dec 8, 2022
Grupprojekt för kurserna 'Javascript med Ramverk' och 'Agil Utveckling'

JavaScript-med-Ramverk-Laboration-3 Grupprojektet för kurserna Javascript med Ramverk och Agil Utveckling. Utvecklingsguide För information om hur utv

Svante Jonsson IT-Högskolan 3 May 18, 2022
Hemsida för personer i Sverige som kan och vill erbjuda boende till människor på flykt

Getting Started with Create React App This project was bootstrapped with Create React App. Available Scripts In the project directory, you can run: np

null 4 May 3, 2022
Kurs-repo för kursen Webbserver och Databaser

Webbserver och databaser This repository is meant for CME students to access exercises and codealongs that happen throughout the course. I hope you wi

null 14 Jan 3, 2023
PLSC 21510/31510: Introduction to Text as Data for Social Science (Spring 2022)

PLSC 21510/31510: Introduction to Text as Data for Social Science Spring 2022 About Social scientists increasingly use large quantities of text-based

Rochelle Terman 16 Oct 17, 2022
Introduction to R for Clinical Data for the R/Medicine 2022 Virtual Conference

This is the GitHub repository for the R/Medicine 2022 pre-conference workshop R/Medicine 101: Intro to R for Clinical Data. There is a course website

Stephan Kadauke 10 Sep 23, 2022