I have no idea how I managed to get on my bus to DC this morning. My Citibike station I planned on, was full, then my phone is Unavailable until like 10:42 . Bus boards at 11:00am . Still got to find a citibike dock to park the bike, because the stolen citibike fee is $1,250. At 10:42 of course I open phone and have absolutely no cell service. The citibike station does not have a printed map on it....

my backprop SGD from scratch 2022-Aug 14:13 ok reviewing from last time , Yea so I had switched from relu to sigmoid on commit b88ef76daf , but yea log loss is still going up during training, so for sure got rid of the bug of how it did not make sense to map that relu output to a sigmoid since a relu only produces positive numbers and so the sigmoid therefore was only able to produce values greater than 0....

[[my back prop SGD from scratch 2022-Aug]] 13:35 yea so last time I had noticed , hey on a random initialization why was the y_prob 0.5 ? I had literally just initialized a new network and got this first example, ipdb> p x, y (array([10.31816265, -8.80044688]), 1) while running the ipdb debug mode, and inside of train_network() , ran --> 186 y_prob = feed_forward(x, model.layers, verbose=False) and got ipdb> p y_prob 0....

[[my back prop SGD from scratch 2022-Aug]] 16:38 why no learning going on hmm look at this network import network as n import dataset import plot X, Y = dataset.build_dataset_inside_outside_circle(0.5) model = n.initialize_model({"learning_rate": 0.01}) ( loss_vec, model, artifacts, X_validation, Y_validation, Y_prob ) = n.train_network(X, Y, model) 17:02 wondering if I can inspect the gradient, to see if it is pointing where it should 23:02 so for a random weight initialized network, curious at least here the response should be nonlinear right?...

[[my back prop SGD from scratch 2022-Aug]] 12:38 so what happened last time? well let me look at the 2022-08-21.html notes I created. 13:20 darn so ok spent bunch of time figuring out why I couldnt view all the images in that html but basically combination of the html references images in log seq dir and also I have to copy them to my git repo area for this repo....