Collecting Data

Overview

In the previous lesson, you learned how to categorize ML problems and match them to the right approach. Every ML model depends entirely on the quality and quantity of its training data — and data has to come from somewhere. In this lesson, you'll explore the main sources of data for ML projects, learn how to access public datasets in Python, and understand the properties that make data useful (or dangerous) for training a model.

Learning Objective

By the end of this lesson, you will have learned how to:

• Explore methods for data acquisition (public datasets, APIs, web scraping).
• Understand the importance of data quantity and quality for ML.

Key terms

Dataset: A structured collection of data used for analysis or model training, typically organized as rows (observations) and columns (features).

Public dataset: A dataset made freely available by governments, research institutions, or companies for public use.

API (Application Programming Interface): A standardized interface that allows programs to request data from an external service.

Web scraping: Programmatically extracting data from web pages using tools like requests and BeautifulSoup.

Data bias: A systematic skew in a dataset that causes a model trained on it to make unfair or inaccurate predictions for certain groups or scenarios.

Class imbalance: A condition where one target class has far fewer examples than another (e.g., 99% non-fraud, 1% fraud), which can cause a model to ignore the minority class.

Train/test split: Dividing a dataset into a training portion (used to fit the model) and a held-out test portion (used to evaluate generalization).

train_test_split: The scikit-learn function that randomly splits a dataset into training and test sets.

Starter Code

This lesson encourages experimentation. Use the included Colaboratory notebook to run the code as you learn, or read the lesson and then freely experiment with the notebook's code.

Introduction

The saying "garbage in, garbage out" is nowhere more true than in machine learning. A sophisticated algorithm trained on bad data will produce bad predictions. Before worrying about which model to use or how to tune it, you need data — enough of it, representative of the problem you're solving, and labeled correctly.

In practice, data scientists spend significant time sourcing, evaluating, and combining datasets. This lesson surveys the main acquisition methods and the quality considerations that determine whether a dataset is actually fit for training.

Sources of Data

1. Public Datasets

The fastest path to a working ML project is a pre-collected, clean public dataset. Thousands are freely available:

General repositories:

• Kaggle Datasets — Tens of thousands of community-contributed datasets across every domain, with associated notebooks and competitions.
• UCI Machine Learning Repository — Classic benchmark datasets used in research since the 1980s (Iris, Wine, Adult Income, etc.).
• Google Dataset Search — A search engine specifically for datasets across the web.

Government and institutional data:

• data.gov — U.S. government open data (census, health, transportation, education).
• World Bank Open Data — Global economic and development statistics.
• CDC / NIH — Public health and biomedical data.

Built into Python libraries: Several Python libraries ship with ready-to-use datasets — perfect for practice and experimentation.

# scikit-learn built-in datasets
from sklearn.datasets import (
    load_iris,           # Classic flower classification (150 rows, 3 classes)
    load_wine,           # Wine variety classification (178 rows, 3 classes)
    load_breast_cancer,  # Binary cancer classification (569 rows)
    load_diabetes,       # Diabetes regression (442 rows)
    fetch_california_housing  # Housing price regression (20,640 rows)
)

# Load the Iris dataset
iris = load_iris(as_frame=True)
df = iris.frame
print(df.shape)      # (150, 5)
print(df.head())

# seaborn also ships with several well-known datasets
import seaborn as sns

titanic  = sns.load_dataset("titanic")   # Passenger survival (891 rows)
tips     = sns.load_dataset("tips")      # Restaurant tips (244 rows)
diamonds = sns.load_dataset("diamonds")  # Diamond prices (53,940 rows)
penguins = sns.load_dataset("penguins")  # Palmer penguins (344 rows)

print(titanic.shape)
print(titanic.head())

2. APIs

An API returns structured data (usually JSON) in response to HTTP requests. Most major platforms — Twitter/X, Spotify, OpenWeather, the U.S. Census Bureau — expose data via APIs.

import requests
import pandas as pd

# Example: Open-Meteo free weather API (no API key required)
url = "https://api.open-meteo.com/v1/forecast"
params = {
    "latitude": 41.85,
    "longitude": -87.65,
    "daily": "temperature_2m_max,temperature_2m_min",
    "forecast_days": 7,
    "timezone": "America/Chicago"
}

response = requests.get(url, params=params)
data = response.json()

df_weather = pd.DataFrame({
    "date":    data["daily"]["time"],
    "temp_max": data["daily"]["temperature_2m_max"],
    "temp_min": data["daily"]["temperature_2m_min"]
})
print(df_weather)

Most commercial APIs require an API key (a token that authenticates your requests). Keep keys secret — never commit them to a public GitHub repository.

3. Web Scraping

When data is displayed on a website but not available via an API or download, you can programmatically extract it using requests (to fetch the HTML) and BeautifulSoup (to parse it):

import requests
from bs4 import BeautifulSoup
import pandas as pd

url = "https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal)"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

# Find the first table on the page
table = soup.find("table", {"class": "wikitable"})
df = pd.read_html(str(table))[0]
print(df.head())

Important: Always check a website's robots.txt and Terms of Service before scraping. Scraping is not always permitted, and excessive requests can overload servers.

4. Internal / Organizational Data

In industry, most ML training data comes from internal systems:

• Databases (SQL, as you've learned) — transaction logs, customer records, sensor readings
• Data warehouses — aggregated historical data from multiple systems
• Event logs — clickstreams, app usage, server logs
• Manually labeled data — human annotators who label images, text, or audio for training

For sensitive internal data, privacy regulations (GDPR, HIPAA) govern how data can be collected, stored, and used. Data governance is a real responsibility.

5. Synthetic Data

When real data is scarce, too sensitive, or imbalanced, synthetic data — artificially generated to mimic real distributions — can supplement training:

from sklearn.datasets import make_classification, make_regression

# Generate a synthetic classification dataset
X, y = make_classification(
    n_samples=1000,
    n_features=10,
    n_informative=5,
    n_classes=2,
    random_state=42
)
print(f"X shape: {X.shape}, class distribution: {y.mean():.2f}")

Data Quantity and Quality

How Much Data Do You Need?

There is no universal answer, but general guidance:

These are rough estimates. The required amount also depends on the number of features, the complexity of the pattern, and the acceptable error rate.

The danger of too little data: The model memorizes the training examples (overfitting) rather than learning the underlying pattern. It performs well on training data but poorly on new data.

The danger of too much noise: Even large datasets are harmful if they're mislabeled, full of errors, or irrelevant to the problem.

Data Quality Dimensions

Quantity matters, but quality matters more:

Bias and Representativeness

Data bias is one of the most serious risks in ML. A model trained on biased data learns those biases and amplifies them.

Examples:

• A hiring algorithm trained on historical hiring decisions that under-represented women will learn to discriminate against women.
• A medical diagnosis model trained only on patients from one demographic will perform poorly for patients from other demographics.
• A facial recognition system trained mostly on light-skinned faces will perform worse on darker-skinned faces.

How to reduce bias:

• Ensure your training data represents the full population the model will serve.
• Examine the data distribution across demographic groups before training.
• Audit model performance separately for different subgroups.

Class Imbalance

In classification problems, one class often has far fewer examples:

import seaborn as sns

df = sns.load_dataset("titanic")
print(df["survived"].value_counts(normalize=True))

Output:

0    0.617
1    0.383
Name: survived, dtype: float64

The Titanic dataset is moderately imbalanced (62% did not survive). A model that always predicted "did not survive" would be 61.7% accurate — without learning anything. Techniques to handle imbalance include oversampling the minority class, undersampling the majority class, or using class weights in the algorithm.

The Train/Test Split

Before any data work begins, split your labeled dataset into a training set and a test set. The test set must remain unseen until final evaluation — it is your measure of real-world performance.

import pandas as pd
from sklearn.model_selection import train_test_split

df = sns.load_dataset("titanic").dropna(subset=["survived"])

X = df[["pclass", "age", "fare", "sibsp", "parch"]].fillna(0)
y = df["survived"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,       # 20% for testing
    random_state=42,     # Reproducibility
    stratify=y           # Preserve class proportions in both splits
)

print(f"Training set: {X_train.shape[0]} rows")
print(f"Test set:     {X_test.shape[0]} rows")
print(f"Train class balance: {y_train.mean():.2f}")
print(f"Test class balance:  {y_test.mean():.2f}")

Output:

Training set: 712 rows
Test set:     179 rows
Train class balance: 0.38
Test class balance:  0.39

stratify=y ensures the class proportions in the training and test sets match the overall dataset — important for imbalanced problems.

Conclusion

In this lesson, you explored the main methods for acquiring ML training data — public datasets (including those built into scikit-learn and seaborn), APIs, web scraping, internal databases, and synthetic data generation. You also learned the data quality dimensions that matter for training — accuracy, completeness, representativeness, and class balance — and why poor-quality data produces poor models regardless of algorithmic sophistication. Finally, you performed your first train_test_split. In the next lesson, you'll learn how to prepare raw data for ML through data cleaning.

Practice

Coding Assessment

Practice the concepts from this lesson using this notebook. After completing the exercises, save your notebook to GitHub and submit the link for grading.

Knowledge Check