DL Notes Personal1

-

 Types of Neural Networks



Perceptron : 




x1, x2 = input

w1, w2 = weights

b = bias

z = sum of all these

now, z will be given to activation function

For example, step function, output will be 0 or 1

How to use it?

You train the model and find the value of w1, w2 and b


Perceptron is a line : 

It's a binary classifier, that divides data into 2 regions

No, matter how many features we have, it will always divide the data into 2 parts


Limitation : Perceptron will be used on linear data only

Code :

Prediction using perceptron




How do we find the correct values for weights?


Step 1 : we take the random values for each weight

Step 2 : randomly take one data point, if the point is on correct region, do nothing, else, move the line

Step 3 : repeat step 2 for 1000(n) times

How do we know that point is on correct region?

- we know that blue points should be in -ve region and green points should be in +ve regions

- so, we just have to make line transformations accordingly

How do we move towards correct values of A,B,C?

1. If you make change in C, line moves parallely


2. If you change x, line will move on the x axis, y is still same


3. same for change in y




Example :



If you want to move point in -ve region, you subtract it from line

If you want to move point in +ve region, you add it in line

but , in this way line will move very drastically, so we use learning rate.


Algorithm :

Instead of using these 2 if conditions, it can be simplified to this 1 formula



Explanation : 


3rd row = green point

4th row = red point

    

 


This method will find a line, but it is not gauranteed that it is the best fit line, because, in the empty space between 2 regions, there can be multiple lines.


Perceptron loss function



So, how do we find the values such that this Loss becomes minimum

Intuition : 
Assume that w2 and b are constant, so for w1, we have to get a point where L is minimum.



For that , we will use gradient descent.











Types of perceptron





Problem with Perceptron : 

- Works only on linear data

xor dataset, perceptron fails here


MLP - Multi layer perceptron : 






 

k=lay
er no where this is going to
i = from node
j= to node

bij => i=layer no, j=node no.





Perceptron with Sigmoid


Intuition : 






how do we overlap?










so, for this diagram, MLP will look like below : 






In multi class classification, there will be multiple nodes in output layer. dog, cat, cow

Hiden layers with correct Activation function can identify any data pattern




Trainable parameters in Neural network

what is it ? - Sum of weights + biases at each layer

Formula : Sigma (no_of_nodes_in_current_layer*no_of_nodes_in_next_layer) + bias aka no_of_nodes_in_next_layer



26 trainable params in above image



Once the training is done, make prediction












Training : 




Another Example : 


Loss Functions in DL



cost function vs loss function

loss func : applies on single row for each row

cost func : applies on batch data



loss func : 










  • mse (Mean Squared Error)

  • mae (Mean Absolute Error)

    • bce (Binary Cross-Entropy)

  • CCE (Categorical Cross-Entropy)

  • SCE (Sparse Categorical Cross-Entropy)



    • Task Type Loss Function Output Activation Function
    Regression MSE / MAE Linear (None)
    Binary Class. BCE Sigmoid
    Multi-class CCE / SCE Softmax


    Backpropagation








    y bar = 





    First 3 done







    Notice that, inside each iteration, step c will be performed 9 times as we have 9 weights

    Finally, it will look like this : 







    v17 22:44






















    Comments

    Post a Comment

    Popular posts from this blog

    Extracting Tables and Text from Images Using Python

    -
    Image
    In this blog, we'll explore a complete Python solution that detects and extracts tables and text from images using libraries like Transformers, OpenCV, PaddleOCR, and easyOCR. This step-by-step breakdown includes code to detect tables, extract content from individual table cells, and retrieve any remaining text in the image. Overview When working with scanned documents, such as invoices or forms, it is essential to accurately extract both structured information (like tables) and unstructured text. The approach we’ll explore uses Microsoft's pretrained object detection model to locate tables and OCR techniques to extract the text from both table cells and the rest of the image. Steps: 1. This code first detects table using microsoft's model. and save that image which contains detected table only  2. After that, from the detected table , we make a seperate image for each cell.  3. Then we read text from the image of each cell  4. Now, to read the extra texts except for the ...
    Read more

    Positional Encoding in Transformer

    -
    Image
    1. Why Position Matters in Transformers? Transformers rely on self‑attention, which processes tokens in parallel. This means, unlike RNNs, they don’t inherently know the order of words. So, sentences like “Ravi killed the lion” vs. “The lion killed Ravi” would look identical to a vanilla Transformer—clearly problematic! 🧪 Idea #1: The Naïve Approach A simple fix would be to add index/position of the token in an embedding vector. Issues: Unbounded values: Position IDs can become huge (e.g. 100,000+ in long texts), destabilizing training. Discrete steps: Sharp jumps between integers disrupt gradient flow. 🧪 Idea #2: Normalize the Position Numbers What if we divide the position numbers by a constant to make them small and smooth? That helps a bit—values don’t explode anymore. Issues: Now, if you observe, in both the sentences, the word at second position has got the different values. 1 for sentence1, and 0.5 for sentence2. so, the Neural network will get confused while training, what a...
    Read more

    Chain Component in LangChain

    -
    Image
    LangChain is a powerful framework for building applications with Large Language Models (LLMs). One of its core features is the Chain Component , which allows developers to link multiple operations together to create complex workflows. In this blog, we'll explore different types of chains in LangChain and demonstrate their usage with code examples. 1️⃣ LLMChain (Basic Single-Step Chain) 🔹 Executes a single LLM call based on a prompt. 🔹 The simplest chain type. ✅ Use Case: When you need a single prompt-response from an LLM. Example: 2️⃣ SequentialChain (Step-by-Step Execution) 🔹 Executes multiple chains in sequence where each step depends on the output of the previous step. ✅ Use Case: Multi-step processes where each step depends on the previous output . 📌  Example: Blog Outline → Content Expansion → Summary Example: 3️⃣ SimpleSequentialChain (Basic Sequential Execution) 🔹 Similar to SequentialChain, but only passes the last step’s output forward (less flexible). 🔹 ...
    Read more