Ensembling Techniques

-

Ensembling is a technique in machine learning where multiple models are combined to make better predictions than any single model could achieve on its own. The idea is that by using multiple models, you can reduce errors and improve accuracy.

Suppose, you are confused about choosing science or commerce for higher studies. So, you seek advice.

Possible Ways:

A: You may ask one of your friends.
Here, you are likely to make decision based on one person's belief and pov. Moreover, it might be the case that your friend has chosen science, and he wants you to choose the same, so that you both can be together.

B: Another way could be by asking 5 classmates of yours.
This should provide a better idea. This method may provide a more honest opinion as you are getting multiple perspectives. However, the problem still exists if most of your classmates have a similar background or bias.

C: How about asking 50 people?
Now, if you ask 50 people, including students, teachers, and professionals from both fields, you are more likely to get diverse and balanced opinions. Some might have chosen science, others commerce, and some might be neutral. This way, you get a comprehensive view of the pros and cons of each stream, helping you make a well-informed decision.

Analogy with Ensembling:

  • Option A is like using a single model. Your decision is based on one source, which might not be very accurate.
  • Option B is similar to using a small ensemble of models, like bagging or boosting with a few iterations. It improves accuracy but might still have some bias.
  • Option C is similar to a robust ensemble method like Random Forest or Stacking with a large number of models. It combines diverse opinions (models) to give you a well-rounded, accurate prediction.

Bagging

In bagging, we split the dataset into different parts and train multiple models on these parts. For example, suppose you have 100 data points. You might train 20 data points using k-nearest neighbors (KNN), another 20 using a decision tree, and so on. Each model makes its own prediction, and then we combine these predictions:

  • For classification: We take the majority vote. If most models say "Class A", then "Class A" is our final prediction.
  • For regression: We take the average of all predictions to get the final result.

This method reduces overfitting and makes the model more robust by averaging out the errors of individual models.

Boosting

Boosting is another powerful ensembling technique that focuses on improving the performance of weak models. Instead of training all models independently, boosting trains models sequentially. Each new model tries to correct the mistakes made by the previous ones. Here's how it works:

  1. Start with a simple model: Train a basic model on the entire dataset.
  2. Identify errors: Find where this model makes mistakes.
  3. Train next model on errors: Train the next model to correct these errors.
  4. Combine models: Combine the predictions of all models, giving more weight to models that correct errors better.

This way, each model builds on the previous one, gradually improving accuracy.










Algorithms :-





Comments

Post a Comment

Popular posts from this blog

Extracting Tables and Text from Images Using Python

-
Image
In this blog, we'll explore a complete Python solution that detects and extracts tables and text from images using libraries like Transformers, OpenCV, PaddleOCR, and easyOCR. This step-by-step breakdown includes code to detect tables, extract content from individual table cells, and retrieve any remaining text in the image. Overview When working with scanned documents, such as invoices or forms, it is essential to accurately extract both structured information (like tables) and unstructured text. The approach we’ll explore uses Microsoft's pretrained object detection model to locate tables and OCR techniques to extract the text from both table cells and the rest of the image. Steps: 1. This code first detects table using microsoft's model. and save that image which contains detected table only  2. After that, from the detected table , we make a seperate image for each cell.  3. Then we read text from the image of each cell  4. Now, to read the extra texts except for the ...
Read more

Getting Started with ML

-
Image
 Types of ML :- 1) Supervised ML :- Before going for supervised learning, you need to be aware of two terminologies: Suppose, you have the data of student's age and weight. Now, for new student, just by knowing age, you want to predict his weight. So, in this case i. Dependent Variable : Age ii. Independent Variable : Weight Regression : It is applied on Numeric Data such as temperature, weight etc. Classification : It is applied on Categorial Data. There are 2 types of classification: 1.Binary Classification - Yes/No, Spam/Not Spam, etc. 2. Multiclass Classification - Small/Medium/Large, Bad/Average/Good/Excellent etc. 2) Unsupervised ML :- i. Clustering : If we have some data where there is no dependent/independent variable, then we can use clustering. suppose, we have data for salary and age. There is no relation between salary and age. So, clustering groups the data and keep the data with same characteristics together. ii. Dimensionality Reduction : Suppose, we have 100...
Read more

Linear Regression

-
Image
 Linear Regression shows the Linear Relationship between dependent features(Y-axis) and independent features(X-axis).  There are 2 types of Linear Regression: 1. Simple Linear Regression -  It has only one independent variable. Hypothesis Function for Simple Linear Regression : where: Y is the dependent variable X is the independent variable  β0 is the intercept ( the value where the regression line crosses the y-axis) β1 is the slope/weight ex : calculate weight based on age. In above figure Red dots represent Actual Data Points and black dots represent predicted values. error is calculated by (Actual Value - Predicted Value) 2. Multiple Linear Regression -  It has more than one independent features.  Hypothesis Function for Simple Linear Regression : y= a0 + a1x1 + a2x2 + a3x3 + ..  Let's understand this by example, we all know that if the size of the house is bigger, then the price also increases. But along with this, there are many other factors ...
Read more