How to handle an ML problem, end to end

1 · Observe

A request lands on your desk

Someone asks: 'Can we predict which customers will leave?' or 'Can we estimate house prices?'. The model is rarely the hard part. What decides whether the result is trustworthy is the order in which you do things and what you refuse to look at too early.

2 · Express

Each step in one line of reasoning

1. Frame it. Name the target, the decision it supports, the metric, and the cost of each kind of mistake. Write it as one sentence.
2. Look first. Check size, types, missing values, duplicates and how common each outcome is. Odd values are clues.
3. Split before you learn anything. Train to learn, validation to choose, test to confirm once. Use stratified splits when one class is rare.
4. Set a baseline. 'Always guess the most common answer' or 'use the average'. This is the bar.
5. Prepare inside a pipeline. Fill gaps, scale and encode using only training rows, so the same steps apply cleanly to new rows.
6. Start simple. A linear or logistic model you can read teaches you more than a complicated one you cannot.
7. Evaluate with the right metric. Accuracy hides a rare class; average miss in real units explains a regression.
8. Test the gain. If model B beats A by a point, ask whether that is bigger than the wobble of the score.
9. Slice the errors. Look by group, by size, by region. A good average can hide a failing group.
10. Report with limits. Say what it can and cannot do, what data it saw, and what you would monitor.

3 · Visualize

Where the effort goes

Notice how little of the week is the model itself. Beginners spend their time on algorithms; the people whose results hold up spend theirs on the first two rows.

4 · Discover

Run all of it in one short program

Here is the whole path on a real dataset built into scikit-learn: 569 patients, 30 measurements each, and a diagnosis. Each numbered comment is one of the questions above.

from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, recall_score
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True)            # 1. the data: 569 patients, 30 measurements
X_rest, X_test, y_rest, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)   # 2. lock the test set

baseline = DummyClassifier(strategy="most_frequent")  # 3. the dumbest sensible guess
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))   # 4. preparation + model travel together

print("baseline  CV accuracy:", round(cross_val_score(baseline, X_rest, y_rest, cv=5).mean(), 3))
print("pipeline  CV accuracy:", round(cross_val_score(model, X_rest, y_rest, cv=5).mean(), 3))   # 5. compare on validation folds only

model.fit(X_rest, y_rest)                              # 6. refit, then open the test set once
guess = model.predict(X_test)
acc = accuracy_score(y_test, guess)
print("test accuracy:", round(acc, 3), "| recall for class 0 (malignant):", round(recall_score(y_test, guess, pos_label=0), 3))
half = 1.96 * (acc * (1 - acc) / len(y_test)) ** 0.5   # 7. say how sure you are
print(f"about 95% interval for the true accuracy: {acc - half:.3f} to {min(1, acc + half):.3f}")

baseline  CV accuracy: 0.626
pipeline  CV accuracy: 0.969
test accuracy: 0.982 | recall for class 0 (malignant): 0.952
about 95% interval for the true accuracy: 0.958 to 1.000

The baseline already scores about 63% simply by always guessing the common answer. The pipeline reaches around 97%, and the test score is close to the validation score, which is the sign nothing leaked. The interval shows how much the test score itself could wobble with only 114 patients.

5 · Apply

Practise it on projects

Customer Churn is this path with a decision attached: you choose the cut-off from the cost of a missed leaver and check fairness. House Price is the numeric version, with a capped target that tricks the model. Titanic is the classic first competition, where a one-line rule is hard to beat.

6 · Where it goes wrong

Five common failures

7 · In Python

The skeleton to copy

The program above is the skeleton. Replace the dataset, the baseline and the pipeline, keep the order.

1. Load the data and separate the target from the features.
2. Split with train_test_split, using stratify for classification.
3. Build a DummyClassifier or DummyRegressor baseline.
4. Build make_pipeline(preparation, model).
5. Compare with cross_val_score on the training portion only.
6. Fit on the training portion, score the test set once, and report an interval.