For very deep neural networks, understanding their inner workings can be challenging. However, with shallow and low-dimensional networks, we can still analyze their behavior to some extent. In this blog, I will explain how neural networks perform pattern recognition from a geometric perspective. By visualizing how neural network weights interact with input data, we can gain insights into their decision-making process. I will illustrate this using various visualizations in coordinate space, primarily in 2D.

Let’s say you are in a game show and there are 3 doors. Behind 2 doors there are goats and behind the remaining one door there is a car. You do not know what’s behind which door and are asked to choose one door. Next, the host will reveal the door behind which there is a goat. Then the host asks you whether you want to switch your choice with the other closed door or not. Question is, Should you switch or not?