KMeans Clustering

-

 

What is KMeans Clustering?

KMeans clustering is an unsupervised learning algorithm used to partition a dataset into K distinct, non-overlapping subsets or clusters. The goal is to group similar data points together while ensuring that data points in different clusters are as distinct as possible.


How Does KMeans Clustering Work?






Step 1: Initialize Centroids

  • Choose K initial centroids randomly from the data points. These centroids are the initial cluster centers.

Step 2: Assign Points to Clusters

  • Assign each data point to the nearest centroid, forming K clusters.

Step 3: Update Centroids

  • Calculate the new centroids as the mean of all data points assigned to each cluster.

Step 4: Repeat

  • Repeat steps 2 and 3 until the centroids no longer change or change minimally.

Choosing the Right Number of Clusters in KMeans Clustering

Choosing the right number of clusters (K) is crucial for the effectiveness of KMeans clustering. Two popular methods to determine the optimal number of clusters are the Elbow Method and the Silhouette Score. Let's understand these methods in a simple way.

1. Elbow Method

The Elbow Method helps to determine the optimal number of clusters by plotting the within-cluster sum of squares (WCSS) against the number of clusters (K). WCSS measures the total distance between each point and the centroid of its assigned cluster.

Steps:

  1. Run KMeans for a range of cluster numbers (e.g., 1 to 10).
  2. Calculate the WCSS for each value of K.
  3. Plot the WCSS values against the number of clusters.
  4. Look for an "elbow" point where the decrease in WCSS starts to slow down.

The "elbow" point indicates the optimal number of clusters. At this point, adding more clusters doesn't significantly improve the model.




2. Silhouette Score

The Silhouette Score measures how similar an object is to its own cluster compared to other clusters. It calculates the average silhouette width for each sample, with values ranging from -1 to 1. Higher values indicate better-defined clusters.

Steps:

  1. Run KMeans for different values of K.
  2. Calculate the Silhouette Score for each K.
  3. Plot the Silhouette Scores against the number of clusters.
  4. Choose the K with the highest Silhouette Score.

A higher Silhouette Score indicates better cluster separation.


Advantages and Limitations

Advantages:

  • Simplicity: Easy to understand and implement.
  • Scalability: Efficient for large datasets.
  • Speed: Fast convergence.

Limitations:

  • Choosing K: The need to specify the number of clusters in advance.
  • Sensitivity to Initialization: Different initial centroids can lead to different results.
  • Assumes Spherical Clusters: Assumes clusters are spherical and equally sized, which may not always be the case.

Comments

Post a Comment

Popular posts from this blog

Extracting Tables and Text from Images Using Python

-
Image
In this blog, we'll explore a complete Python solution that detects and extracts tables and text from images using libraries like Transformers, OpenCV, PaddleOCR, and easyOCR. This step-by-step breakdown includes code to detect tables, extract content from individual table cells, and retrieve any remaining text in the image. Overview When working with scanned documents, such as invoices or forms, it is essential to accurately extract both structured information (like tables) and unstructured text. The approach we’ll explore uses Microsoft's pretrained object detection model to locate tables and OCR techniques to extract the text from both table cells and the rest of the image. Steps: 1. This code first detects table using microsoft's model. and save that image which contains detected table only  2. After that, from the detected table , we make a seperate image for each cell.  3. Then we read text from the image of each cell  4. Now, to read the extra texts except for the ...
Read more

Getting Started with ML

-
Image
 Types of ML :- 1) Supervised ML :- Before going for supervised learning, you need to be aware of two terminologies: Suppose, you have the data of student's age and weight. Now, for new student, just by knowing age, you want to predict his weight. So, in this case i. Dependent Variable : Age ii. Independent Variable : Weight Regression : It is applied on Numeric Data such as temperature, weight etc. Classification : It is applied on Categorial Data. There are 2 types of classification: 1.Binary Classification - Yes/No, Spam/Not Spam, etc. 2. Multiclass Classification - Small/Medium/Large, Bad/Average/Good/Excellent etc. 2) Unsupervised ML :- i. Clustering : If we have some data where there is no dependent/independent variable, then we can use clustering. suppose, we have data for salary and age. There is no relation between salary and age. So, clustering groups the data and keep the data with same characteristics together. ii. Dimensionality Reduction : Suppose, we have 100...
Read more

Linear Regression

-
Image
 Linear Regression shows the Linear Relationship between dependent features(Y-axis) and independent features(X-axis).  There are 2 types of Linear Regression: 1. Simple Linear Regression -  It has only one independent variable. Hypothesis Function for Simple Linear Regression : where: Y is the dependent variable X is the independent variable  β0 is the intercept ( the value where the regression line crosses the y-axis) β1 is the slope/weight ex : calculate weight based on age. In above figure Red dots represent Actual Data Points and black dots represent predicted values. error is calculated by (Actual Value - Predicted Value) 2. Multiple Linear Regression -  It has more than one independent features.  Hypothesis Function for Simple Linear Regression : y= a0 + a1x1 + a2x2 + a3x3 + ..  Let's understand this by example, we all know that if the size of the house is bigger, then the price also increases. But along with this, there are many other factors ...
Read more