Neural Networks: Weights, Activation, and Backpropagation

If you've ever looked at a neural network diagram and every wondered what is actually happening in these networks, this post is just for you. I pulled this post together after researching the properties of neural networks, and writing out what is finally there to click. No analogies that lead to no explanations, just the math behind neural networks, one piece at a time. The Simplest Possible Net——A Perceptron Before neural networks got keep, they started shallow. A perceptron is a neural net with zero hidden layers, invented back in 1943. Perceptions work when data is linearly separable, meaning you could draw a line and cleanly split the classes on either side of it. Let's take a simple example with an equation: D=y-2x-3. Everything where D>=0 is Class 1 and everything where D