What is deep learning?
Deep Learning is a subset of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks.
ML vs DL
| Deep Learning | Machine Learning |
| DL is a subset of ML. It is based on neural networks, a conceptual model of the brain | It is a subset of AI. It focuses on statistical methods to enable machine to improve with experience. |
| It learns the features by itself. Very good with processing on image data | We have to explicitly tell what features to identify. |
| Can implement hand crafted features | Can’t implement hand crafted features |
| Algorithms like CNN, RNN, GANs | Algorithms like Linear Regression, KNN, Decision Tree, ANN |
Why deep learning algorithms is called neural networks?
Computing Systems inspired by Biological Neural Networks.
So basically it works on the same idea. The most basic network of deep learning is Artificial Neural Network it is also called as vanilla network. As you can see in blow picture there are inputs to the nodes of the network, hidden layer nodes and some output nodes. All these nodes are connected with some edges and some nodes receives inputs from biases. So neural network consist of main 3 layers
- Input Layer
- Hidden Layer
- Output Layer
There are the terms called activation function and biases. So an activation function is the mathematical exponential or linear function which is to be applied on the output of a particular node and its output goes to next node in the network. There are many reasons behind the usage of the activation functions.
Let’s take an example to understand the use of activation function.
When our brain is fed with a lot of information simultaneously, it tries hard to understand and classify the information between useful and not-so-useful information. We need a similar mechanism to classify incoming information as useful or less-useful in case of Neural Networks. This is a very important in the way a network learns because not all information is equally useful. Some of it is just noise. Well, activation functions help the network do this segregation. They help the network use the useful information and suppress the irrelevant data points.
Popular Activation Functions
- Identity
- Binary Step
- Sigmoid
- Tanh
- ReLU
- Leaky ReLU
- Softmax
Bias
The bias value allows the activation function to be shifted to the left or right, to better fit the data. Hence changes to the weights alter the steepness of the sigmoid curve, whilst the bias offsets it, shifting the entire curve so it fits better. Note also how the bias only influences the output values, it doesn’t interact with the actual input data
Major Architectures
- Unsupervised Pretrained Networks
- Convolutional Neural Networks
- Recurrent Neural Networks
- Recursive Neural Networks
UPNs
- Autoencoders
- Deep Belief Networks(DBNs)
- Generative Adversarial Networks(GANs)
Use cases of UPN architectures
- Feature Extraction
- Initialization
- Synthesizing
CNNs
- Lenet-5
- AlexNet
- VGGNet
- GoogleNet(Inception)
- ResNet
- ResNext
- DenseNet
- RCNN(RegionBasedCNN)
- YOLO(YouOnlyLookOnce)
- SqueezeNet
- SegNet
Use cases of CNN architecture
- Computer Vision
- Natural Language Processing
RNNs
- Hopfield Network
- Long Short-Term Memory (LSTM)
- Gated Recurrent Unit (GRU)
Use cases of RNN architecture
- Sentiment Classification
- Image Captioning
- Language Translation
- Video Captioning
Recursive Neural Network
- Recursive Autoencoder
- Recursive Neural Tensor Network
Use cases of Recursive NN architecture
- Image scene decomposition
- NLP
- Audio-to-text transcription
Vector's Blog 