Look at the data before you model it
Plot the raw distributions, scatterplots and simple summaries before fitting anything — a habit that catches broken data, unit errors and outliers a model would otherwise quietly absorb and hide.
The popular image of a data scientist building clever predictive models is real but incomplete. Practitioner surveys and firsthand accounts consistently describe a job where finding, cleaning and reshaping messy data eats up more hours than the modeling step most people picture.
The craft passed down inside the field is less about memorizing a specific algorithm and more about disciplined skepticism: looking at the data before trusting any model of it, checking whether a pattern holds inside subgroups, and knowing when the honest answer to a stakeholder's question is that the data cannot support one.
What the work demands
Statistics and probability85
Programming (Python, SQL, R)78
Data wrangling and pipelines70
Machine learning modeling72
Communicating findings62
Business or domain context58
Statistics and probability
Hypothesis testing, regression, and understanding what a p-value or a confidence interval actually claims — the inferential foundation everything else in the job sits on top of.
Programming (Python, SQL, R)
Writing code fluent enough to pull, clean and transform data at scale, not just prototype an analysis in a notebook that never has to run again.
Data wrangling and pipelines
Turning messy, inconsistent, real-world data — missing values, duplicate records, mismatched formats — into something a model or chart can actually use.
Machine learning modeling
Choosing, training and validating a model appropriate to the problem, and knowing when a simple regression beats an elaborate neural network.
Communicating findings
Translating a statistical result into a chart, a memo or a recommendation a non-technical executive can act on, without oversimplifying or burying the real uncertainty.
Business or domain context
Understanding what a metric actually means inside a specific business or scientific field, so an analysis answers the question that was actually asked.
A day in the life
- 8–9
Standup and dashboard check
Reviewing overnight pipeline runs and key metrics dashboards, and a short sync with the team on priorities for the day.
- 9–12
Data pulling and cleaning
Writing SQL queries against a data warehouse, joining and reshaping tables, and handling missing or inconsistent values — commonly the largest single block of a working day.
- 12–13
Lunch
A genuine break, and often an informal setting where cross-team problems get raised before they reach a scheduled meeting.
- 13–15
Modeling and analysis
Building or refining a statistical model, running an experiment's significance test, or digging into why a metric moved.
- 15–17
Stakeholder meetings and reviews
Presenting findings to a product or business team, reviewing an A/B test's results, or defending an analysis's assumptions under questioning.
- 17–8
Off the clock (mostly)
Personal time, except around a product launch or a quarterly business review, when evenings can absorb last-minute analysis requests.
The know-how
Craft knowledge practitioners actually pass on — not motivation.
Check whether the trend survives inside subgroups
An aggregate trend can reverse completely once the data is split by a relevant category — a pattern known as Simpson's paradox, named for a 1951 paper by statistician Edward H. Simpson.
Hold out a test set and do not look at it until the end
Repeatedly checking performance against the same held-out data, then adjusting the model, quietly leaks information from that data into the model — touching it only once is what keeps a final result honest.
Budget most of the project for cleaning, not modeling
Real projects consistently run closer to 80% data preparation and 20% modeling than the reverse — planning a timeline around the opposite assumption is a common way projects run late.
Ship the simplest model that could plausibly work first
A baseline — sometimes just a rule or a linear model — clarifies how much a more complex model actually adds, and is often good enough to ship on its own.
When a pattern looks too good, hunt for the confound
A surprising correlation is more often an artifact of a hidden third variable than a genuine discovery — actively search for what else could explain it before presenting a result.
Tools of the trade
Python (pandas, scikit-learn, NumPy)
The dominant language for data manipulation and classical machine learning, with pandas for tabular data and scikit-learn for standard modeling algorithms.
SQL
The query language for pulling data out of the relational and cloud data warehouses — Snowflake, BigQuery, Redshift — where most companies' data actually lives.
Jupyter Notebook
The standard interactive environment for exploratory analysis, letting a data scientist run code, view a chart and take notes in the same document.
Tableau / Power BI
Business-intelligence dashboarding tools used to turn a finished analysis into something a non-technical stakeholder can explore and monitor without writing code.
Git and cloud platforms (AWS, GCP, Azure)
Version control for code and, increasingly, for models, alongside the cloud infrastructure that stores data and runs training jobs at a scale a laptop cannot handle.
How people fail at it
Testing until something is significant
Running many statistical comparisons and reporting only the one that crosses a significance threshold produces false discoveries that will not replicate — a well-documented failure mode known as p-hacking.
Shipping a model that leaks the target
Accidentally including a feature that encodes the outcome being predicted produces unrealistically strong offline results that collapse once the model runs on genuinely unseen production data.
Building a sophisticated answer to the wrong question
A carefully tuned machine-learning model solving a problem a simple SQL query or spreadsheet formula would have answered just as well wastes effort and can miss the actual business question.
Keep exploring
More in Science & Research
⚛️
Physicist
Derives and tests the mathematical laws governing matter, energy, space and time, from a lone chalkboard to a 3,000-author particle-collider paper.
AI-resistant 65 🧬Biologist
The scientist who studies life itself, from Linnaeus naming species by hand to editing genomes with CRISPR, still testing every idea against a living organism.
AI-resistant 64 🧪Chemist
The scientist who makes and measures matter itself — from Tapputi's Babylonian perfume still to today's robot laboratories, still the one who decides what the spectrum means.
AI-resistant 70 🔭Astronomer
The scientist who measures the universe, from Babylonian clay tablets to space telescopes, still deciding which flicker in the data is a discovery.
AI-resistant 70 🧮Mathematician
From Babylonian scribes to Fields medalists and AI-assisted proof: the profession that turns hard questions into permanent certainty, one theorem at a time.
AI-resistant 70