Tutorial 4: Artificial Intelligence

This tutorial is designed for beginners in Artificial Intelligence, particularly those new to p5.js, ml5.js, or Machine Learning in general. It offers a simple, step-by-step approach that you can progress at your own pace. You can download the PDFs and print them out or have them displayed side by side with the p5.js web editor (recommended).

The tutorial is freely available, and the p5.js web editor is also free to download. By the end of this tutorial, you’ll have a solid understanding of Artificial Intelligence, Machine Learning, Neural Networks, and coding in p5.js. The possibilities for what you do with this knowledge are endless, and it’s up to your creativity and motivation to explore them.

The tutorial consists of three modules that guide you through a series of units that develop different concepts, ideas, and applications. While I recommend following the tutorial in a linear fashion, you’re free to skip or jump ahead if you prefer. If you’re new to coding or p5.js, I’ve included some coding snippets that cover the essential code you’ll need for this tutorial. However, I recommend not skipping these sections unless you’re already very familiar with p5.js or a seasoned coder.

Above all, enjoy the learning process and don’t be discouraged by setbacks. You can only grow in experience through doing.

Introduction to the tutorial

These three introductory sections are there to help you understanding what you are doing and why. I recommend reading through them and this will help you in the tutorial as you work through the units. I will assume that you have read through them so I don’t have to repeat myself. They are pretty brief and a quick read so don’t skimp on this despite your eagerness to get going, there are some key bits of information you will need for later on.

A brief summary of Artificial Intelligence

This quick read through gives you some background information on Artificial Intelligence, Machine Learning and in particular to Neural Networks and what they are.

An exploration of p5.js

To access this tutorial you will need a basic understanding what p5.js is and also how the web based editor and all its buttons and settings work and benefit you.

An overview of ml5.js

Although p5.js is the main coding environment it the ml5.js library that does all the heaving lifting when it comes to the Artificial Intelligence and Machine Learning units.

Module A building an AI model

Module A is where we start our journey, work through each unit and I recommend that you don’t skip a unit. The coding snippets have been kept as brief as possible so that when you are working through the AI units you have some understanding of p5.js while attempting to grasp the concepts of AI. This module uses ml5.js and explores some simple examples that will demonstrate key features.

Module A Unit #1 p5.js code snippets part 1

To create your AI models you do need to code, this is a quick introduction to the key concepts that you will need, it is ideal for those who have never coded before and a good introduction to p5.js for those who have.

Module A Unit #2 linear regression

In this first unit we plot points for a straight line with some variance (randomness). The neural network (ml5.js) will be trained on these data points and will approximate to the line. This is a regression task where you will train it on some synthetic data and then predict the outcome.

Module A Unit #3 sine wave regression

This is similar to unit #1 but a bit more challenging for the neural network to learn, here we will introduce some more hyperparameters to help in the learning process, hopefully with improved accuracy.

Module A Unit #4 mouse gesture

This is a classification task. We will give the neural network a small amount of synthetic data on the movement of the mouse, it will try to predict whether the mouse is moving up, down, left or right.

Module A Unit #5 p5.js code snippets part 2

An introduction to some key concepts used in the next section, such as using video and exploring pixels.

Module A Unit #6 cluster classification

This is a supervised learning model. We are going to click on the canvas in four areas. In each quadrant we will create a set of ten circles then we will predict the colour in the space between the circles

Module A Unit #7 cluster regression

Here we will click on the canvas to create a cluster of white circles and a cluster of black circles. We will train it on those data points and then predict what shade of grey the canvas is between those data points.

Module A Unit #8 colour predictor

Train the neural network to predict three colour groups using synthetic data and testing with sliders.

Module A Unit #9 pixel predictor

In this unit we take the pixels from an image (webcam) and train the neural network to move a circle with your head

Module A Unit #10 save and load

This unit covers saving/loading data, and saving/loading models once trained

Module B pretrained models

Module B looks at some pretrained models which means that they have done all the hard work of collecting the data and training the models. The reason for this module is that you can use these pretrained models to develop other applications and creative endeavours. It is also quite fun.

Module B Unit #1 p5.js code snippets part 3

A short unit as background coding snippets useful for the remainder of this section.

Module B Unit #2 pretrained faceMesh

This pretrained model can identify over 400 points on your face, we will use some of those points to draw a box around your mouth and locate your eyes.

Module B Unit #3 pretrained handPose

This pretrained model can identify key parts of your hand (both hands at once). In this example you will control a circle on the canvas with your fingers.

Module B Unit #4 pretrained bodyPose

This pretrained model can identify the main parts of the body, such as hands, arms, legs, eyes and so on. We can also draw a skeleton between these points.

Module B Unit #5 pretrained speech classification

This pretrained model has been trained on 18 key words, so that when you say one of those words it would recognise it. As an exercise you will move a circle round the canvas with your voice.

Module B Unit #6 pretrained body segmentation

This pretrained model can detect key body parts and follow their motion

Module C neuro evolution

Module C is a bit more challenging, it is looking at Reinforcement Learning through the use of Genetic Algorithms. The algorithm we will be using is a neuro evolution approach which means that we put a neural network in a car and a bird respectfully. We allow them to reproduce and over time we select the ones that perform best according to their fitness scores.

Module C Unit #1 p5.js coding snippets 4

These snippets will help you when you are working through the next set of units for neuroevolution smart cars and flappy bird game.

Module C Unit #2 Introduction to Genetic Algorithms

Although we will be using a neural network which means that it is a neural evolution approach, the fundamentals are based on genetic algorithms where we select the fittest and mate them to reproduce a better next generation. This is offers a bit more explanation.

Module C Unit #3 smart cars

This is the first example where cars are given their own brain and through a process of selection chase a target that is moving randomly according to perlin noise. They don’t know that they are meant to do that but only that they are rewarded for doing so.

Module C Unit #4 flappy bird

In this example we create a simple version of the flappy bird game and then give the bird a brain. Then using a neuroevolution approach allow successive generations to play the game until a bird can navigate the pipes without hitting them.