Can We Trust the Model?

Step 1: A model that asks questions

A decision tree reaches an answer by asking a series of yes/no questions about the features: “Is the alcohol level above 13? If yes, is the color intensity above 4?…” Each question splits the data into smaller groups.

We will use another dataset that ships with scikit-learn: 178 wines, each described by 13 measurements and labeled with one of three growers.

from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

wine = load_wine()
X_train, X_test, y_train, y_test = train_test_split(
    wine.data, wine.target, test_size=0.3, random_state=0
)
print(len(wine.data), "wines,", len(wine.feature_names), "features")

178 wines, 13 features

Step 2: When a model memorizes

A tree can keep asking questions until it has a separate answer for every training example. That is overfitting. Let’s compare training accuracy with testing accuracy as we let the tree grow deeper.

for depth in [1, 2, 3, None]:
    tree = DecisionTreeClassifier(max_depth=depth, random_state=0)
    tree.fit(X_train, y_train)
    train_acc = accuracy_score(y_train, tree.predict(X_train))
    test_acc = accuracy_score(y_test, tree.predict(X_test))
    print(depth, round(train_acc, 3), round(test_acc, 3))

1 0.653 0.63
2 0.952 0.87
3 0.992 0.944
None 1.0 0.944

The columns are: depth limit, training accuracy, testing accuracy. (None means no limit.)

• At depth 1, the tree is too simple. It is wrong on both sets (underfitting).
• With no limit it scores a perfect 1.0 on training data. But on new data it does no better than depth 3.

The gap between training and testing accuracy is the warning sign. Always look at testing accuracy, never training accuracy alone.

Step 3: Whose data is it?

A model learns from the data we give it. If the data is unbalanced, the model will be too.

• A hiring model trained on past hires may learn to prefer the kinds of people who were hired before, even if they were not the best.
• A speech tool trained mostly on one accent may misunderstand others.
• A health model trained on one population may not work for another.

Nobody has to mean harm for this to happen. It comes from who is in the data and who is missing.

Step 4: Questions for a fairer model

Before you trust a model, ask:

1. Who is in the data? And who is left out?
2. How does it perform for different groups? Don’t just look at overall accuracy.
3. What happens when it’s wrong? Is a mistake annoying, or does it harm someone?
4. Who decides? For important choices (school, jobs, health) a person should review the model’s output.
5. Can people question the result? A fair system has a way to say “this is wrong”.

Think about it: Machine learning can also do a lot of good: reading X-rays in places with few doctors, forecasting floods, helping farmers spot crop disease. What problem in your community could use data?

Step 5: Your capstone

You now know the whole path: ask a question, collect data, look at it, train a model, test it fairly, and reflect on whether you can trust it. For the capstone project, choose a dataset and a depth, and prepare a one-minute talk with four parts:

1. The problem you tried to solve.
2. Your best model and its testing accuracy.
3. One thing that could go wrong.
4. One real-world use you’d be excited about, or careful with.

Good presentations are honest about both what works and what doesn’t. That honesty is what makes a data scientist trustworthy.