Deep Learning 101: Tensorflow Playground

hello everyone and welcome to this

lecture

in this lecture i wanted to show you

guys a tool

known as tensorflow playground

where we can visually see

how we build an artificial neural

network and how do we actually train it

as well

how to change the number of layers and

how to also change the number of neurons

activation layer and so on okay

please note that this tool is

independent of

the teachable machines but it's again

provided by google as well

okay all right so let's go ahead and get

started

so this is simply the type of artificial

neural network that we are going to

build together using tensorflow

playground

just to clarify so tensorflow is

google's framework

to build train and deploy ai and ml

models

at scale and please note

that tensorflow specifically is beyond

the scope of this course

so we are not going to learn i would say

how to code

in python or leverage you know

tensorflow and keras

here i just wanted to show you from a

very high level how can we visually

build our newer network and how we can

train it essentially live as well

without writing any

lines of code okay so please go ahead

and and go to that link which is

playground.tensorflow.org

all right and i have it open in here

okay

so as you guys can see here essentially

i have the inputs

okay here i have the first hidden layer

and what i could do is i can change the

number of neurons

and you guys would notice that here we

have what we call it a dense

or fully connected artificial neural

network

meaning that all the neurons in one

layer

are fully connected to all the neurons

in the subsequent layer

okay so if you guys can see here this

input here

is fully connected to all the neurons in

all the

subsequent hidden and

if i wanted to make or create a deep

neural network well

i can add a hidden layer so if you press

on the positive here i'm adding a new

hidden layer

and if you click on that adding neurons

you should be able to see

again additional neurons that are being

added

and the more that you add neurons the

more that you're increasing the

complexity of the network

and again you guys will notice that here

also i have a dense fully connected

artificial neural network as well

and a very important point to notice

here is that

this network hasn't been trained yet

if you guys recall when we cover the

training and testing phases of

artificial neural networks

is that we build our artificial neural

network first

we train it with the training data and

that's when we actually go

and change the values of these weights

if you guys remember

and then afterwards we go ahead and test

it and assess its performance

using the testing data set so you guys

will notice here

that all these values of weights

like minus 0.29 minus 0.19

point you know like 1 4 all these values

are randomly initialized because we

haven't trained it yet we haven't done

anything yet

here i'm just building the network i'm

randomly initializing the weights

and i haven't even trained any the

network yet

okay and what you guys can see here is

well you can see the learning rate we

are actually now familiar with the

learning rate

and this is simply how aggressive you

would like to update your network or

update the weights

so let's set it to 0.03 as is

also here you should be able to see that

we have different activation functions

if you guys remember we only covered the

basic

step activation function you guys can go

ahead and change the activation

function again we're not going to go

into the details of what's really relu

stands for rectified linear units but

don't worry about it

let's just you know let's take let's

maybe pick like relu for example for now

and we can also change the problem type

to either classification

or regression and let's stick with

classification because that's what we're

doing right now

and you can also change what we call it

the regularization

and this is simply just a term to ensure

that the network network

is able to generalize and not memorize

it's more of a penalizing term

that try to improve the generalization

ability of the network

again don't worry about it for now what

i could do right now

is i can well select the training data

that i need here

so i can select for example a very

simple two classes

and now the network is not trained yet i

haven't trained it yet

but now i can go ahead and actually

train it so if you click play

when i say play that means i'm going to

start off the training

that means now i'm going to feed in the

training data to my network

and i'm going to try to update the

values of weights

so if you click play here we go you guys

will see that the loss here

starts very high and then it decayed

afterwards

and now that boundary has been drawn

between the two classes

so now i have that green i'm sorry that

blue

area here and the orange area indicating

that now the model my ai model has been

trained

and as you guys notice here the values

of weights have actually changed

right so here for example the value of

weight becomes 1.1 for example or 1.2

and that's it that means now the model

was able to

train to learn the difference between

these two classes

and you guys will see here this is the

epochs counter if you guys remember we

also covered the definition

of epochs before and that's every time

we feed in

all the training data to my model and

update the weights

once okay so maybe if i try a little bit

harder problem like this problem for

example

and i click play again as you guys can

see the loss

start is very high or the error starts

high and then it decays afterwards

and i was able to draw that boundary

between the blue class

and the orange class okay pretty

incredible

all right and what you guys can see here

as well

is that you can increase or vary the

noise you can also

change the ratio of the training to

testing data so maybe you can change it

make it 80

to 20 you can increase the noise for

example

in the input as you guys can see as you

increase the noise

as the problem becomes much harder

because now the two

classes are are not linearly separable

and you can also change the batch size

as well by changing the number of

samples or data points that you feed

at once to the network if you guys

remember

and you can again you can click play as

well and you should be able to see that

the network also was able to trade right

now

and was able to draw the boundary to

classify the two classes

all right okay so now it's time

for a quick mini challenge actually it

might be a little bit i would say uh

hard but that's the point i want you

guys to experiment with

the um with the tool so i want you first

to choose the most challenging data set

which is the spiral data set so you guys

will see that here we had four options

before

let me go back so here we had four

options

i want you guys to start with this one

because this one the spiral one

is the most difficult of all and

i wanted to choose it first i wanted to

train the model okay

and then because it's a difficult

problem we have to do something about it

maybe we need to change the architecture

of the model for example

maybe we need to add new hidden layers

or maybe add

additional neurons or maybe change

activation functions i want you guys to

play with it

to try to improve its performance and

next step number three i wanted to tune

the model hyper parameters maybe play

with a learning rate

maybe play with the regularization and

try to achieve better model performance

and then finally i want you guys to

perform

some feature engineering to improve

model performance

so when i say feature engineering that

means instead of feeding in

the inputs as is maybe you can feed in

the input squared or maybe you can sum

up the two inputs together and feed them

as one input

or maybe add let's say x squared plus y

squared for example again there are so

many options

so that's it please go ahead pause the

video

attempt to solve the mini challenge and

i will be back with a solution

in a couple of seconds best of luck and

i will see you guys

after the challenge

all right hope you guys were able to

figure out the challenge

let's go ahead and give it a shot so

what i'm going to do

is i'm going to first select the spiral

okay and then next what i'm going to do

is i'm going to give it a shot maybe

click play without changing anything and

you guys would notice here

that the network is actually really

trying really hard

to try to come up with that boundary but

because the problem is quite difficult

you guys will see that now the loss

training loss is at 0.3

and the test loss is at 0.4 right

it's going down it's improving over time

which is good and the network here you

guys will see that now i was able to

actually

like draw the boundary which is pretty

great and the loss

settled at around 0.17

approximately okay

and again it's trying

it's trying which is not bad actually

the performance is pretty good so far

and what i could do as well is well i

can maybe

change the number of neurons maybe try

additional neurons

maybe try additional neurons too maybe

add an additional hidden layer

okay and maybe let's add an additional

hidden layer as well and give it a shot

and let's click play and here we go

you guys will see that again the network

is learning that's good sign

here we're going to ultimately of course

we would like the loss to drop to

almost zero that means the error has

each almost zero and that's a great sign

they mean the network was able to um

to generalize and here we go so now the

loss actually reduced to almost 0.06

which is again

pretty pretty great you guys will see

that here i have all the spiral here

it's pretty good and now i'm standing at

0.05

of error okay so that's again another

way of solving the problem

another way of solving the problem too

is maybe we can perform some sort of

feature engineering

so instead of only feeding x1 and x2

which is these two classes in here

maybe i can feed in let's say x1 squared

and maybe x2 squared and maybe x1 times

x2

and maybe sine x and maybe sine x1 and

sine x2

again you guys can there's a lot of

trial and error in here

and let's go ahead and run it and as you

guys notice here the network

is as well training and now

we are not only feeding in the original

data but we're also performing

again feature engineering while

increasing the number of inputs that are

being fed to the network

and you guys will see that the loss now

achieved 0.03

which is again pretty pretty good i

think it's better than the previous run

and that's it the network is pretty

stable the error is actually very very

low it's almost 0.034

and you guys will see that the the

actual blue here has been completely

separated from the other class from the

orange class which is pretty amazing

that's exactly what we're looking for

what you guys as well could do is maybe

you can increase the noise level here to

try to make it a little bit harder so

now it's much

harder problem and maybe you can give it

a shot as well and see if the network is

able to do it

and please note that you can do a lot of

stuff here maybe you can change the

activation function for example

maybe change or tweak the learning rate

for instance or maybe add regularization

to try to improve the generalization

ability of the network

and here we go the loss is standing at

0.256 and please note that this is a

much harder problem

okay and still the network was actually

able to

get it right so we are standing at .24

of loss and again it's really trying

okay again the loss dropped to 0.06 0.05

again pretty incredible and here we go

so now the model

after around 350 epochs was able to

actually draw that boundary here

and separate the two classes together

the blue from the orange class

all right okay and that's it that's all

i have for

this lecture and this is the mini

challenge solution as well if you guys

want to check it out

and that's it i hope you guys enjoy this

lecture in the next lecture we are going

to learn

how to export our model how to save and

deploy

our trained ai model so please stay

tuned and i'll see you guys

in the next lecture