Clustering discovers structure without labels. Classification learns a rule from labelled examples. Mixing the two questions leads to bad evaluation: a visually neat cluster is not automatically a useful customer segment, and a classifier can be accurate while failing the minority class that matters.
K-means optimizes one geometric objective
K-means alternates between assigning each point to its nearest centroid and recomputing centroids. It minimizes within-cluster squared distance. That makes feature scaling, distance choice, initialization, and k part of the model—not preprocessing trivia.
It works best for roughly spherical, similarly sized clusters. Outliers can pull centroids; categorical features and irregular shapes need other representations or algorithms. Multiple random starts reduce, but do not remove, local-optimum risk.
Decision trees explain with partitions
A decision tree chooses splits that reduce impurity, commonly entropy or Gini. Trees handle nonlinear interactions and are readable, but deep trees memorize noise. Maximum depth, minimum leaf size, pruning, and validation control that flexibility.
SVMs maximize margin
A linear SVM seeks a separating hyperplane with a wide margin. Only the support vectors determine the boundary. The penalty C trades a wider margin against training errors; kernels represent nonlinear boundaries through similarity, but introduce scale and parameter sensitivity.
| Situation | Start with |
|---|---|
| Unknown groups, numeric geometry | K-means baseline |
| Need explainable rules and mixed interactions | Decision tree |
| High-dimensional sparse features | Linear SVM |
| Nonlinear boundary, moderate dataset | Kernel SVM after scaling |
In plain terms: choose the metric before celebrating the model. For imbalance, inspect precision, recall, PR curves, and class-specific errors—not accuracy alone.
Review card
- Clustering has no target label; classification does.
- K-means assumes distance has meaning.
- Trees partition feature space and need complexity control.
- SVMs are margin models shaped by support vectors.
- Evaluation must reflect the cost of each error type.
Clustering 在沒有 label 時找 structure；classification 從 labelled examples 學 decision rule。把問題混在一起就會用錯 evaluation：視覺上整齊的 cluster 不一定是有用的客群；classifier 即使 accuracy 很高，也可能完全漏掉真正重要的 minority class。
K-means 只最佳化一個幾何 objective
K-means 反覆把 point 分給最近 centroid，再重算 centroid；它最小化 within-cluster squared distance。因此 feature scaling、distance、initialization 與 k 都是 model 的一部分，不只是 preprocessing 細節。
它最適合近似 spherical、大小相近的 cluster。Outlier 會拉動 centroid；categorical feature 與不規則形狀需要別的 representation 或 algorithm。Multiple random starts 能降低、但不能消除 local optimum。
Decision tree 用 partition 解釋規則
Decision tree 選擇能降低 impurity 的 split，常見指標是 entropy 或 Gini。Tree 能表達 nonlinear interaction，也容易閱讀，但太深會記住 noise。Maximum depth、minimum leaf size、pruning 與 validation 都是在控制這份自由度。
SVM 最大化 margin
Linear SVM 尋找 margin 夠寬的 separating hyperplane，boundary 只由 support vectors 決定。Penalty C 在寬 margin 與 training error 間交換；kernel 能表示 nonlinear boundary，但也更依賴 scaling 與 parameter。
| 情境 | 可先嘗試 |
|---|---|
| 未知群組、numeric geometry | K-means baseline |
| 需要可解釋規則與 feature interaction | Decision tree |
| High-dimensional sparse feature | Linear SVM |
| Nonlinear boundary、中型 dataset | Scaling 後的 kernel SVM |
白話來說： 選對 metric 才能談 model 好不好。Imbalanced data 要看 precision、recall、PR curve 與各 class error，不能只看 accuracy。
複習卡
- Clustering 沒有 target label；classification 有。
- K-means 假設 distance 有意義。
- Tree partition feature space，需要限制 complexity。
- SVM 是由 support vectors 塑造的 margin model。
- Evaluation 必須反映不同錯誤的實際代價。