A first model, run end to end. This post is about the workflow — what the steps are, what the vocabulary means, and the two mistakes that make a bad model look excellent. The algorithm is the least interesting part.
The vocabulary
- Features (
X) — the inputs, one row per example, one column per measurement. - Target (
y) — what you are predicting, one value per row. - Fit / train — show the model examples so it learns the mapping.
- Predict — apply it to rows it has never seen.
- Classification — predicting a category. Regression — predicting a number.
Supervised learning means every training row comes with its answer. That is the case here and it is most of what you will meet.
The data
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True, as_frame=True)
print(X.shape) # Output: (150, 4)
print(list(X.columns)[:2]) # Output: ['sepal length (cm)', 'sepal width (cm)']
print(sorted(set(y))) # Output: [0, 1, 2]
print(y.value_counts().to_dict()) # Output: {0: 50, 1: 50, 2: 50}
150 rows, 4 numeric features, 3 balanced classes. as_frame=True gives you a pandas
DataFrame so the columns keep their names — worth it, because a feature importance list of
[0, 1, 2, 3] tells you nothing.
Real data needs cleaning first: missing values filled or dropped, categories encoded as numbers, and numeric columns scaled if the algorithm cares about magnitude. That work is most of the job and none of it is glamorous.
Split before you look
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y
)
print(len(X_train), len(X_test)) # Output: 105 45
print(sorted(y_test.value_counts())) # Output: [15, 15, 15]
The test set is the only honest estimate you have of performance on new data, and it stops being honest the moment the model learns anything from it. Split first, then explore only the training half.
random_state=42 makes the split reproducible, so a change in your score is a change in
your model rather than in the shuffle. stratify=y keeps the class proportions in both
halves — without it a rare class can land almost entirely in one side.
Fit and predict
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
X, y = load_iris(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y)
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print(round(accuracy_score(y_test, predictions), 3)) # Output: 0.889
Every scikit-learn model has the same two methods, which is the library's best design decision:
swapping RandomForestClassifier for LogisticRegression changes one line and
nothing else.
88.9% — forty of forty-five correct. Start with a random forest: it handles unscaled features and mixed magnitudes, rarely needs tuning, and gives you a baseline to beat.
Accuracy is not enough
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import confusion_matrix
X, y = load_iris(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, stratify=y)
model = RandomForestClassifier(n_estimators=100, random_state=42).fit(X_train, y_train)
print(confusion_matrix(y_test, model.predict(X_test)).tolist())
# Output: [[15, 0, 0], [0, 14, 1], [0, 4, 11]]
Rows are the truth, columns the prediction, so the diagonal is correct and everything else is a mistake. Class 0 is perfect. The five errors are all confusion between classes 1 and 2, and four of them are class 2 predicted as class 1.
The single accuracy number hid all of that. On a dataset where 99% of transactions are legitimate, a
model that predicts "legitimate" every time scores 99% accuracy and catches no fraud at all — which is
why precision (of those flagged, how many were right) and recall (of
the real cases, how many were caught) exist. classification_report prints both per
class.
One split can lie
from sklearn.datasets import load_iris
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
X, y = load_iris(return_X_y=True, as_frame=True)
model = RandomForestClassifier(n_estimators=100, random_state=42)
scores = cross_val_score(model, X, y, cv=5)
print([round(float(s), 3) for s in scores])
# Output: [0.967, 0.967, 0.933, 0.967, 1.0]
print(round(float(scores.mean()), 3)) # Output: 0.967
Five different splits, five scores from 93.3% to 100%. The single 88.9% above was a pessimistic draw — and had it been a lucky one, nothing in that number would have told you.
Cross-validation trains five times, each on a different four fifths, and tests on the fifth left out. Report the mean and the spread: a model scoring 0.97 ± 0.02 is a different proposition from one scoring 0.97 ± 0.15.
Which measurements mattered
from sklearn.datasets import load_iris
from sklearn.ensemble import RandomForestClassifier
X, y = load_iris(return_X_y=True, as_frame=True)
model = RandomForestClassifier(n_estimators=100, random_state=42).fit(X, y)
ranked = sorted(zip(X.columns, model.feature_importances_), key=lambda p: -p[1])
print(ranked[0][0]) # Output: petal length (cm)
print(round(float(ranked[0][1]), 2)) # Output: 0.44
Petal length alone accounts for about 44% of the decisions, and petal width is close behind — the two sepal measurements barely matter. Worth checking for two reasons: it tells you which measurements are worth collecting, and a feature that is implausibly important is the usual symptom of leakage.
The two mistakes
Leakage. Information about the answer reaching the model through the features. Scaling before splitting, so the training data has seen the test set's statistics. Including a column derived from the target. A "days since account closed" field on a model predicting closure. The score is excellent and it is meaningless — the tell is a result that seems too good.
Overfitting. The model memorises the training data instead of learning the pattern. Near-perfect on training, mediocre on test. Score both to see it, and prefer a simpler model when the gap is large.
Both are why the test set gets touched once, at the end, and never influences a decision.
Where to go next
The workflow above is nearly all of applied machine learning: get data, split it, fit something simple, measure it honestly, improve the data. Model choice matters far less than most people expect, and feature quality far more.
Next, Interview Questions — or back to NumPy Arrays for the array layer all of this sits on.