Big data
Big data is data so large, fast or varied that it doesn't fit on one computer, so it's stored and processed across many machines.
Select any words here to ask your Maestro about them.
The in-app survey has a million responses: 61% prefer home delivery, with a 0.1-point margin. Is that what your customers prefer?
What it's for: storing and analyzing data that arrives in volumes one computer can't handle: every click in an online store, every ride in a ride-hailing app, every reading from a factory's sensors. It's what machine learning models are trained on, and it turns reports that took days into minutes.
The hard part isn't the size: it's knowing what each row stands for.
Most teams don't have big data: they have a CSV that doesn't fit in Excel. Power Query or a database handles that.
In SQL for Data People you learn to query tables with millions of rows without opening them in a sheet.
How it's used
Is yours big data?
Count the rows before you pick a tool
- Up to a million rows: Excel.A sheet tops out at 1,048,576 rows, and it slows down well before that.
- marked: Up to tens of millions: Power Query, pandas or DuckDB on your laptop.Almost everything a company calls big data fits here.
- Beyond that, or data arriving every second: BigQuery, Snowflake or Spark.They split the work across many machines: that's big data.
When to look closer
The total that's quietly missing rows
sales_2025.csv: 1,300,000 rows
- Excel warns that the file didn't load completely. You click OK.
- marked: =SUM(D:D) → $26.2 million
You opened a year of sales from a CSV in Excel, and the sum is $26.2 million. Is that the year's total?
Ask your Maestro about Big data
It answers from the free course page closest to this word, the way it answers while you study.
You: What does "Big data" mean, in plain words?
Your Maestro: I'll explain it with a case of yours: it sticks better than a definition. How many rows does the biggest table you work with have?
Asking about: Answers from the page "Is this file on my computer or in the cloud?"
Only this browser keeps what you ask here. How it works
Learn more
Related terms
- Database
A database stores data in connected tables, with rules for each column, so many people and programs can read and change it without stepping on each other.
- Machine learning
Machine learning has a program learn rules from examples that come with answers, instead of someone writing them, so it can predict new cases.
- CSV
A CSV (comma-separated values) is a plain-text file that holds a table: one row per line, with commas between the columns.
- Power Query
Power Query is the tool in Excel and Power BI for bringing in data and cleaning it with steps that get saved: dropping columns, changing types, filtering rows. When the new file arrives, you click Refresh and every step runs again on its own.