Most data scientists spend their days in five things: Python, SQL, a notebook, Git, and a way to show results to someone who doesn’t write code. Everything else on this page is either a specialization or something you pick up on the job.
Lists of data science tools usually dump 20 logos on you and call it a day. This one is organized by what you’re doing at each stage of a project, with a verdict on each tool: what it’s for, what it costs, and when it’s worth your time.
Table of Contents
- The core stack
- Stage 1: Getting at the data
- Stage 2: Cleaning and analysis
- Stage 3: Visualization
- Stage 4: Modeling
- Stage 5: Tracking, deployment, and pipelines
- Stage 6: Collaboration
- The newest additions: AI assistants and the modern data stack
- Comparison table
- How the stack changes by role
- What to learn first
The core stack
If you need the short answer, it’s this:
- Python for analysis and modeling (R is the alternative, mostly in academia and statistics-heavy teams)
- SQL for pulling data out of databases
- Jupyter (JupyterLab) for exploratory work
- Git and GitHub for version control
- One BI or dashboard tool (Tableau, Power BI, or a Python option like Streamlit) for sharing results
Learn those five and you can do most entry-level data science work. The rest of the list below fills in the gaps by stage.
Stage 1: Getting at the data
Before any modeling happens, someone has to get data out of a warehouse, an API, or a pile of CSVs. This stage is less glamorous than it sounds and takes up more time than most beginners expect.
SQL is the one non-negotiable. Nearly every company stores its data in relational databases or warehouses that speak some dialect of it (PostgreSQL, MySQL, BigQuery, Snowflake, Redshift). You’ll write joins, aggregations, and window functions daily. Free to learn; the databases themselves range from free to expensive depending on who’s paying. Learn it if: you plan to work with data at any company that has more than a spreadsheet’s worth of it. That’s all of them.
Cloud warehouses (BigQuery, Snowflake, Redshift) are where much of that SQL runs. You rarely choose which one; your employer already did. Pricing is usage-based. Learn it if: you’re targeting a specific employer. Otherwise, the SQL transfers and the console doesn’t matter.
DuckDB is a newer option that runs SQL directly on local files like CSVs and Parquet with no server. It’s free and open source, and it’s become a favorite for analysis that’s too big for pandas but too small to justify a warehouse. Learn it if: you want to query large files on your laptop.
APIs and web scraping (Python’s requests library, Beautiful Soup) cover data that isn’t in a database at all. Learn it if: you’re building personal projects, since public datasets and APIs are how most portfolios get their data.
Stage 2: Cleaning and analysis
This is the center of the job. A commonly repeated claim says data scientists spend most of their time cleaning data, and while the exact percentages vary by survey, the direction is right: it’s the biggest slice of the work.
Python is the default language here. It’s free, and its dominance comes from its libraries rather than the language itself. The foundation is NumPy, which provides the array math that nearly everything else is built on; its design is described in a Nature paper on array programming with NumPy. On top of it sit pandas for tables, and SciPy for statistics and scientific routines, which is documented in Nature Methods. Learn it if: you want a data science job. This is the single highest-value skill on the page.
R is Python’s older rival, and it’s still strong in biostatistics, academic research, and anywhere statistical modeling matters more than software engineering. The tidyverse packages (dplyr, ggplot2) make data wrangling pleasant. Free. Learn it if: you’re in research, public health, or a team that already runs on it.
Polars is a faster, newer alternative to pandas, written in Rust, that handles larger datasets and runs operations in parallel. Free. Learn it if: you’ve outgrown pandas, not before.
Excel still shows up more than the data science community likes to admit. Stakeholders send you spreadsheets, and a quick pivot table beats writing code for a ten-minute question. Paid (Microsoft 365), though Google Sheets covers much of it for free. Learn it if: you’ll work with business teams, which is most of you.
Jupyter notebooks are where most of this happens. JupyterLab lets you mix code, output, charts, and notes in one document, so you can explore data step by step. A Nature feature on why Jupyter became the standard covers how it caught on among scientists. Free and open source. Learn it if: you do exploratory analysis. Notebooks are weak for production code, so expect to move finished work into regular scripts.
Stage 3: Visualization
You’ll make charts at two points: during exploration for yourself, and at the end for other people. They call for different tools.
Matplotlib, seaborn, and Plotly are the Python plotting libraries. Matplotlib is the foundation and gives total control at the cost of verbose code. Seaborn sits on top of it and produces good statistical charts in a line or two. Plotly adds interactive charts that work in a browser. All free. Learn it if: you use Python, which means all three, in roughly that order.
ggplot2 is R’s plotting library and the reason many people stay with R. Its grammar-of-graphics approach makes layered charts easy to build. Free.
Tableau is the long-standing dashboard tool in large companies. It’s built for drag-and-drop exploration and polished interactive dashboards. It’s paid, though Tableau Public is a free version where your work is visible to everyone. Learn it if: you’re applying to enterprise or analytics-heavy roles.
Power BI is Microsoft’s competitor and tends to win at companies already using Microsoft 365 and Azure. Power BI Desktop is free to download; sharing dashboards within an organization requires paid licenses. Learn it if: the job postings you’re targeting mention it. Many do.
Looker Studio (formerly Google Data Studio) is a free dashboard tool that connects easily to Google data sources. Learn it if: you need a quick, shareable dashboard with no budget.
Streamlit turns a Python script into a web app with a few lines of code. Free and open source. Learn it if: you want to share a model or analysis without handing off to a BI team.
Stage 4: Modeling
Here’s where the title “scientist” earns its keep, and where the tool choices depend on what kind of model you’re building.
scikit-learn is the standard library for classical machine learning: regression, classification, clustering, preprocessing, and model evaluation. Its consistent fit and predict interface is a big part of why it’s easy to learn, and it’s described in a Journal of Machine Learning Research paper. Free. Learn it if: you do any predictive modeling. Start here before deep learning.
XGBoost and LightGBM are gradient-boosted tree libraries and the workhorses for tabular data. On spreadsheet-style problems like churn prediction or fraud detection, they frequently beat neural networks. Free. Learn it if: you work with structured business data.
PyTorch is the dominant deep learning framework in research and increasingly in industry. Its code reads like normal Python, which makes debugging less painful. Free. Learn it if: you’re working with images, text, audio, or anything that needs a neural network.
TensorFlow and Keras are Google’s deep learning stack. TensorFlow was once the default; PyTorch has taken much of its share, but you’ll still meet TensorFlow in older production systems and mobile deployment. Free. Learn it if: a target employer uses it. Otherwise, pick PyTorch.
statsmodels covers classical statistics: regression with proper standard errors, hypothesis tests, and time series. Free. Learn it if: you need inference (explaining why something happens), not just prediction.
Hugging Face Transformers gives you pretrained language and vision models you can use or fine-tune without training from scratch. Free library, with paid hosting options. Learn it if: you work with text or want to use modern language models in your own pipelines.
Stage 5: Tracking, deployment, and pipelines
This is the stage most tool lists skip, and it’s where a notebook becomes something a company can depend on.
MLflow tracks experiments: which parameters you tried, what the metrics were, and which model version is in use. Without something like it, you end up with files named model_final_v3_REAL.pkl. Free and open source. Learn it if: you train more than a handful of models. Weights & Biases does similar work with a more polished interface; it’s free for personal use and paid for teams.
DVC (Data Version Control) applies Git-style versioning to large datasets and model files, which Git itself handles poorly. Free and open source. Learn it if: your results need to be reproducible months later.
Docker packages your code and its dependencies into a container so it runs the same on your laptop and a server. Free for personal use. Learn it if: you plan to deploy anything, or work with ML engineers who will.
FastAPI wraps a model in a web API so other software can call it. Free. Learn it if: you want to serve predictions yourself.
Apache Airflow schedules and monitors pipelines, such as “pull data at 2 a.m., retrain on Sundays.” Free and open source. Learn it if: your role overlaps with data engineering.
Apache Spark processes datasets too large for a single machine, typically through its Python interface, PySpark. Free and open source, though you’ll usually run it on a paid platform. Learn it if: you work at a company with truly large data. Most beginners don’t need it.
Stage 6: Collaboration
Git and GitHub record every change to your code and let teams work on the same project without overwriting each other. Git is free; GitHub and GitLab have free tiers. Also, a public GitHub profile doubles as a portfolio, which is how many hiring managers look at junior candidates. Learn it if: you write any code, which is always.
VS Code and PyCharm are code editors that handle full projects better than a notebook can. VS Code is free and also runs notebooks; PyCharm has a free community edition and a paid professional one. Learn it if: you’re moving from scripts to real projects.
Cloud notebooks (Google Colab, Kaggle Notebooks) give you a notebook in the browser with free GPU access on limited tiers. Learn it if: you’re learning and your laptop can’t train a model.
The newest additions: AI assistants and the modern data stack
Two groups of tools have moved from “interesting” to “standard” fast, and older guides leave them out.
AI coding assistants (GitHub Copilot, Cursor, and chat assistants such as Claude and ChatGPT) now sit next to most data scientists’ editors. They’re good at boilerplate: plotting code, pandas reshaping, regex, SQL drafts. They’re bad at knowing whether your analysis makes sense, and they’ll happily produce code that runs and answers the wrong question. Treat the output as a first draft you must check against your own data. Pricing ranges from free tiers to monthly subscriptions. Learn it if: you want to move faster, but only after you can read and debug the code it produces.
dbt (data build tool) lets you write SQL transformations as versioned, tested modules inside a warehouse. dbt Core is free and open source; the hosted version is paid. It’s mostly the territory of analytics engineers, but data scientists increasingly read and write dbt models. Learn it if: your company uses it, or you want to work closer to data engineering.
Databricks is a cloud platform that combines Spark, notebooks, MLflow, and data storage in one place. It’s paid and usage-based, with a free edition for learning. Learn it if: a target employer runs on it, which large enterprises often do.
Comparison table
| Tool | Stage | Cost | Priority |
|---|---|---|---|
| Python (pandas, NumPy) | Analysis | Free | Essential |
| SQL | Data access | Free (database varies) | Essential |
| JupyterLab | Exploration | Free | Essential |
| Git / GitHub | Collaboration | Free tier | Essential |
| scikit-learn | Modeling | Free | Essential for ML roles |
| Matplotlib / seaborn | Visualization | Free | Essential |
| Tableau / Power BI | Dashboards | Free tier or paid | Strongly recommended |
| R | Analysis | Free | Situational |
| PyTorch | Deep learning | Free | Situational |
| XGBoost / LightGBM | Modeling | Free | Strongly recommended |
| MLflow | Experiment tracking | Free | Nice to have |
| Docker | Deployment | Free for personal use | Nice to have |
| Spark / Databricks | Big data | Free / paid | Situational |
| dbt | Transformation | Free core / paid cloud | Situational |
| AI coding assistants | All stages | Free tier or paid | Nice to have |
How the stack changes by role
The same title means different toolkits depending on the team. The jobs a data scientist can do, and what each one pays, vary just as widely.
- Data analyst: SQL, Excel, a BI tool (Tableau or Power BI), and light Python or R. Less modeling, more dashboards and reports.
- Data scientist: Python, SQL, scikit-learn, notebooks, statistics libraries, and visualization. The generalist stack described above.
- Machine learning engineer: Python, PyTorch or TensorFlow, Docker, cloud platforms, MLflow, Airflow, and FastAPI. Much more software engineering, much less exploratory analysis.
- Research scientist or academic: R or Python, statsmodels, LaTeX, and whatever the field’s standard packages are. Reproducibility matters more than deployment.
What to learn first
Order matters more than coverage. A reasonable sequence:
- SQL and Python basics. Together they unlock most entry-level work.
- pandas and a plotting library. Clean a real, messy dataset and chart it.
- Git. Put every project on GitHub from the start.
- scikit-learn. Build, evaluate, and honestly report on a few models.
- One BI tool. Pick whichever your target jobs mention.
- Then specialize: PyTorch for deep learning, Docker and MLflow for production, Spark for big data.
Skip the urge to learn everything. Hiring managers mostly want to see that you can take a real question, get the data, clean it, model it sensibly, and explain the result. A small stack used well beats a long list of logos on a resume.
