Clustering discovers structure without labels. Classification learns a rule from labelled examples. Mixing the two questions leads to bad evaluation: a visually neat cluster is not automatically a useful customer segment, and a classifier can be accurate while failing the minority class that matters.
K-means optimizes one geometric objective
K-means alternates between assigning each point to its nearest centroid and recomputing centroids. It minimizes within-cluster squared distance. That makes feature scaling, distance choice, initialization, and k part of the model—not preprocessing trivia.
It works best for roughly spherical, similarly sized clusters. Outliers can pull centroids; categorical features and irregular shapes need other representations or algorithms. Multiple random starts reduce, but do not remove, local-optimum risk.
Decision trees explain with partitions
A decision tree chooses splits that reduce impurity, commonly entropy or Gini. Trees handle nonlinear interactions and are readable, but deep trees memorize noise. Maximum depth, minimum leaf size, pruning, and validation control that flexibility.
SVMs maximize margin
A linear SVM seeks a separating hyperplane with a wide margin. Only the support vectors determine the boundary. The penalty C trades a wider margin against training errors; kernels represent nonlinear boundaries through similarity, but introduce scale and parameter sensitivity.
| Situation | Start with |
|---|---|
| Unknown groups, numeric geometry | K-means baseline |
| Need explainable rules and mixed interactions | Decision tree |
| High-dimensional sparse features | Linear SVM |
| Nonlinear boundary, moderate dataset | Kernel SVM after scaling |
In plain terms: choose the metric before celebrating the model. For imbalance, inspect precision, recall, PR curves, and class-specific errors—not accuracy alone.
Review card
- Clustering has no target label; classification does.
- K-means assumes distance has meaning.
- Trees partition feature space and need complexity control.
- SVMs are margin models shaped by support vectors.
- Evaluation must reflect the cost of each error type.