Machine Learning with ml5.js

For this tutorial you will be introduced to Machine Learning. It will cover the basic concepts and will give you an opportunity to learn as well as create your own machine learning models. For the machine learning part we will use ml5.js which is based on TensorFlow, for the coding framework we will be using p5.js as the background to the machine learning. This is all done in the web browser so no need to download anything, subscribe to anything, it is free and more importantly easy and intuitive to use.

Before we jump into the machine learning tutorial this would be a good place to familiarise yourself with p5.js as you will be using it a lot as the basis for all your coding. You don’t have to follow the tutorial but it would be useful and helpful to you as we will be ploughing on assuming some understanding. It also tells you how to get started. Highly recommend that you go through this. Click on the link below or go to the tab:

Learn to code with p5.js

Module 1: The Perceptron

The perceptron is a single node, this is a good place to start even though it doesn’t use ml5.js. In this exercise we draw a line across the canvas and train the model to colour all the circles above the line white and all the circles below the line black. This is a simple coding exercise where you don’t need machine learning but it demonstrates what can be done.

Module 2: Classification

This tutorial takes you through the process of creating your own classification task using ml5.js.

Module 3: Regression

To demonstrate the difference between regression and classification but using a similar example we will attempt to predict the in between colours between a set of white circles and a set of black circles. This will give us a grey scale of circles depending on their closeness to either the white circles or the black circles.

Module 4: Gesture (classification)

This uses the basic ml5.js library to use data from the movement of a mouse to train it and to predict whether it is moving up, down, left or right. This is a simple illustration of Machine Learning/Artificial Intelligence in action.

Module 5: Linear (regression)

Another example that attempts to draw a straight line from a bunch of random data around a straight line. Illustrates how the model is trained and the graphical output to see how well it performs.

Module 6: sine wave (regression)

Spice it up a bit and give it a harder example, a sine wave. We will also talk about more hyperparameters.

Module 7: colour predictor (classification)

This is a nice example with only a small dataset which we provide. Here you have three sliders for red, green and blue, the model will predict what colour you create whether it is red-ish, green-ish or blue-ish.

Module 8: pixel predictor (regression)

Using the pixels from a webcam we can train the image to move a circle around the canvas.

Module 9: Load and Save (data and models)

In the real world data is downloaded, here we can save and load data and models.

Pre-trained Models

In the above examples we used ml5.js straight out of the box. Within ml5.js there a re number of very useful pre-trained models which you make use of.

Module 10: faceMesh

This uses an ml5.js built in feature, a pertained model that creates 468 data points of your face from your webcam image. it is called faceMesh. you can access any of the data points to use in any creative way you want. In this tutorial we will draw a bounding box round the mouth.

Module 11: handPose

HandPose is part of the ml5.js library and it is a pertained model that can identify the keypoints on your hands. We can use these data points to identify parts of individual fingers to control the size and position of a circle

Module 12: bodyPose

bodyPose is part of the ml5.js library and it is a pertained model that can identify the keypoints of your whole body. We can use those data points to animate

Module 13: soundClassification

This uses the pretrained soundClassification which works with 18 words. We test how good it is and then attempt to move a circle around the canvas with our voice

NeuroEvolution

NeuroEvolution is based on Genetic Algorithms that are a form of Reinforcement Learning. The key elements are: Population, Fitness, Selection, Crossover and Mutation

Module 14: Flappy Bird NeuroEvolution

We are entering the world of Genetic Algorithms and harnessing the power of the neural network to give a bird a brain. This will help it learn how to navigate the pipes in the famous Flappy Bird game with the help of ml5.js

Module 15: Smart Cars (or Bees) NeuroEvolution

We give a bunch of cars a brain each get them to chase a target using NeuroEvolution (Genetic Algorithm) approach.