Looking at Data

Step 1: Meet Google Colab

Google Colab (colab.research.google.com) runs Python in your browser. Sign in with a Google account and choose New notebook. A notebook is made of cells. Click in a cell, type code and press Shift + Enter to run it.

print("Hello from Colab")

Hello from Colab

Colab already has the libraries we need, so we can start right away.

Step 2: A table in pandas

We will use a tiny made-up dataset about students: the hours each one studied for a test and the score they got.

import pandas as pd

data = pd.DataFrame({
    "hours": [1, 2, 3, 4, 5, 6, 7, 8],
    "score": [52, 55, 61, 64, 70, 74, 80, 85],
})
print(data.head())

   hours  score
0      1     52
1      2     55
2      3     61
3      4     64
4      5     70

Each row is one student. Each column is one kind of fact. The numbers on the left (0, 1, 2…) are the row numbers pandas adds for you.

Step 3: Summarizing

describe() calculates a summary for every number column.

print(data.describe().round(2))

       hours  score
count   8.00   8.00
mean    4.50  67.62
std     2.45  11.72
min     1.00  52.00
25%     2.75  59.50
50%     4.50  67.00
75%     6.25  75.50
max     8.00  85.00

• count: how many values.
• mean: the average. std tells you how spread out the values are.
• min and max: the smallest and largest.
• 50% is the median, the middle value.

You can also ask for one number at a time:

print(data["score"].mean())
print(data["hours"].corr(data["score"]).round(3))

67.625
0.998

The correlation is a number between -1 and 1. Near 1 means the two columns rise together. Near 0 means no clear link. Near -1 means one goes up while the other goes down.

Step 4: Draw it

A picture often shows what numbers hide. matplotlib draws charts.

import matplotlib.pyplot as plt

plt.scatter(data["hours"], data["score"])
plt.title("Study time and test score")
plt.xlabel("Hours studied")
plt.ylabel("Score")
plt.show()

You will see dots rising from the lower left to the upper right. That upward slant is the pattern: more study time goes with higher scores.

Careful: Correlation does not prove cause. Maybe students who study more also sleep better or pay more attention. The data shows a link, not the reason.

Step 5: Questions to ask about any dataset

Before you build anything with data, ask:

1. Where did this data come from? Who collected it and why?
2. How many examples are there? Eight rows is too few to trust. Real projects use hundreds or thousands.
3. Who or what is missing? Does it cover everyone it should?
4. Is anything strange? A score of 500 on a 100-point test is a mistake.

Our eight rows are a teaching example. In the next session we will use them to make a prediction, and you will see why more data would make us more confident.