Artificial neural networks work like biological ones, where the inputs are summed in a weighted manner, and if enough input is received, firing occurs.
Multiple neurons can be connected to the same input (similar to a multi-class classifier). This is known as a fully connected layer.
Linear classifier can be broken down into
We can have multiple neurons connected to the same input
This corresponds to a multi-class classifier — each output node outputs the score for a class
These are often called fully connected layers or linear projection layers
The activation function is not seen as it is assumed each node in the output layer contains both the weighted sum and the sigmoid activation
Having more layers in the neural network increases its complexity, allowing it to represent more complex functions as well.
<aside> 📌 SUMMARY: Neural networks are artificial representations of biological neurons, where if sufficient input is received, the neuron will fire. Having deeper neural networks allows us to represent more complex functions as the hidden layers increases representational powers.
</aside>
<aside> 📌 SUMMARY: Since data is naturally compositional, we can use deep neural networks to represent this compositionality, allowing us to learn complex functions and build complex models. Computation graphs represent the order in which we do operations in neural networks. This allows us to know the forward and backward function, which is useful when training the model and performing gradient descent.
</aside>