Data Science Cheat Sheet

Data Science Cheat Sheet: Concepts, Statistics, Tools & Workflow

Data science is the practice of extracting meaningful insights from data using statistics, programming, analytical techniques, and domain knowledge.

Organizations use data science to understand customer behavior, forecast demand, detect fraud, improve marketing campaigns, optimize operations, and support business decisions.

This cheat sheet explains essential data science concepts, common terminology, statistical methods, data preparation techniques, machine learning fundamentals, and practical examples.

1. What Is Data Science?

Data science combines several disciplines to transform raw data into useful insights.

Its main components include:

• Statistics: Understanding distributions, relationships, uncertainty, and variation.

• Programming: Processing and analyzing data using tools such as Python and SQL.

• Data visualization: Presenting findings through charts and dashboards.

• Machine learning: Building models that learn patterns from data.

• Domain knowledge: Understanding the business problem and interpreting results correctly.

Common Applications

2. The Data Science Workflow

A data science project usually follows a structured process.

1. Define the problem: Identify the question or business objective.

2. Collect data: Retrieve relevant information from databases, files, APIs, or other sources.

3. Clean data: Address missing values, duplicates, errors, and inconsistent formats.

4. Explore data: Examine distributions, patterns, relationships, and unusual observations.

5. Prepare features: Select and transform variables for analysis or modeling.

6. Build a model: Apply an appropriate statistical or machine learning technique.

7. Evaluate results: Test whether the model or analysis meets the objective.

8. Communicate findings: Explain insights, limitations, and recommendations.

9. Deploy when needed: Integrate the solution into an operational workflow.

10. Monitor and maintain: Check for performance changes and data-quality issues.

The process is iterative. New findings may require additional data collection, cleaning, or modeling.

3. Types of Data

Understanding data types helps determine which analytical techniques are appropriate.

Structured Data

Data organized into a predefined structure, such as rows and columns.

Examples: Sales tables, employee records, and transaction databases.

Semi-Structured Data

Data with organizational markers but without a rigid relational table structure.

Examples: JSON documents and many XML files.

Unstructured Data

Data that does not naturally fit into a conventional table format.

Examples: Images, audio recordings, videos, and free-form documents.

Quantitative vs. Qualitative Data

Quantitative data can be discrete, such as the number of orders, or continuous, such as measured temperature.

Qualitative data can be nominal, such as city names, or ordinal, such as satisfaction ratings.

4. Essential Statistics

Statistics helps summarize data, understand variability, and draw conclusions while accounting for uncertainty.

Mean

The arithmetic average.

[
\text{Mean}=\frac{\sum x_i}{n}
]

Example: For values 10, 20, and 30:

[
\text{Mean}=\frac{10+20+30}{3}=20
]

Median

The middle value after sorting the observations. For an even number of observations, it is commonly calculated as the average of the two middle values.

The median is often less affected by extreme values than the mean.

Mode

The most frequently occurring value or category.

Range

[
\text{Range}=\text{Maximum}-\text{Minimum}
]

Variance

Variance measures the spread of values around their mean.

Population variance:

[
\sigma^2=\frac{\sum (x_i-\mu)^2}{N}
]

Sample variance:

[
s^2=\frac{\sum (x_i-\bar{x})^2}{n-1}
]

The sample formula uses (n-1) in the denominator when estimating population variance from a sample under the usual assumptions.

Standard Deviation

The square root of variance. It measures dispersion in the original units of the data.

Percentiles and Quartiles

• Percentile: A value associated with a particular position in a distribution.

• Q1: First quartile, approximately the 25th percentile.

• Q2: Median, approximately the 50th percentile.

• Q3: Third quartile, approximately the 75th percentile.

• Interquartile range: (Q3-Q1).

Exact percentile calculations can vary depending on the method used.

5. Probability Fundamentals

Probability measures how likely an event is to occur.

It ranges from 0 to 1, where 0 represents impossibility and 1 represents certainty under the stated model.

Basic Probability

For equally likely outcomes:

[
P(A)=\frac{\text{Favorable outcomes}}{\text{Total outcomes}}
]

For a fair six-sided die, the probability of rolling a 4 is (1/6).

Complement Rule

[
P(\text{not }A)=1-P(A)
]

Addition Rule

For two events:

[
P(A\cup B)=P(A)+P(B)-P(A\cap B)
]

This accounts for outcomes that belong to both events.

Conditional Probability

[
P(A\mid B)=\frac{P(A\cap B)}{P(B)}
]

This formula applies when (P(B)>0).

Independence

Two events are independent when the occurrence of one does not change the probability of the other.

For independent events:

[
P(A\cap B)=P(A)P(B)
]

Understanding probability is important when interpreting model predictions, experiments, and uncertain outcomes.

6. Data Cleaning

Data cleaning identifies and addresses data-quality problems before analysis.

Common Problems

• Missing values.

• Duplicate records.

• Incorrect data types.

• Inconsistent date formats.

• Spelling differences in categories.

• Invalid or impossible values.

• Outliers.

• Inconsistent units.

• Incorrect joins between datasets.

Example: Basic Cleaning With Pandas

import pandas as pd

df = pd.read_csv("sales.csv")

df = df.drop_duplicates()

df["sales"] = pd.to_numeric(
    df["sales"],
    errors="coerce"
)

df["sales"] = df["sales"].fillna(
    df["sales"].median()
)

print(df.head())

This example removes duplicate rows, converts the sales column to numeric values where possible, and fills missing sales values with the median.

Important: Median imputation is not appropriate for every dataset. Choose a missing-value strategy based on the meaning of the data and why values are missing. Always assess the effect of cleaning decisions on your results.

7. Exploratory Data Analysis (EDA)

Exploratory data analysis helps you understand a dataset before drawing conclusions or building models.

Common EDA Activities

• Inspect the number of rows and columns.

• Check data types.

• Summarize numeric variables.

• Examine category frequencies.

• Identify missing values.

• Detect duplicate records.

• Visualize distributions.

• Explore relationships between variables.

• Investigate unusual observations.

Pandas Examples

import pandas as pd

df = pd.read_csv("sales.csv")

print(df.head())
print(df.shape)
print(df.info())
print(df.describe())
print(df.isnull().sum())

Useful methods:

EDA helps reveal patterns, but observed relationships do not automatically establish causation.

8. Data Visualization

Data visualization communicates patterns, comparisons, distributions, and relationships.

Example With Matplotlib

import matplotlib.pyplot as plt

months = ["Jan", "Feb", "Mar", "Apr"]
sales = [100, 150, 130, 180]

plt.plot(months, sales, marker="o")
plt.title("Monthly Sales")
plt.xlabel("Month")
plt.ylabel("Sales")
plt.show()

Visualization Best Practices

• Give charts meaningful titles.

• Label axes and units.

• Use suitable scales.

• Avoid unnecessary decoration.

• Include relevant context.

• Choose chart types based on the question.

• Do not distort comparisons through misleading scales.

• Consider accessibility and color contrast.

9. Correlation vs. Causation

Correlation measures how variables change together. Causation means a change in one factor produces a change in another, under the relevant causal interpretation.

A correlation does not prove causation.

For example, advertising expenditure and sales may increase together. However, the relationship alone does not prove that advertising caused the entire sales increase.

Other factors might include seasonality, pricing, distribution, promotions, or market demand.

Correlation Coefficient

Pearson's correlation coefficient measures linear association between two numeric variables.

[
-1\leq r\leq 1
]

• (r=1): Perfect positive linear correlation.

• (r=-1): Perfect negative linear correlation.

• (r=0): No linear correlation.

A zero Pearson correlation does not necessarily mean there is no relationship; the relationship could be nonlinear.

10. Hypothesis Testing

Hypothesis testing provides a framework for evaluating claims about a population using sample data.

Key Terms

• Null hypothesis ((H_0)): The baseline claim being evaluated.

• Alternative hypothesis ((H_1)): A competing claim.

• Significance level ((\alpha)): A threshold chosen for the test.

• P-value: The probability, assuming the null hypothesis and model assumptions, of obtaining results at least as extreme as those observed.

• Type I error: Rejecting a true null hypothesis.

• Type II error: Failing to reject a false null hypothesis.

• Statistical power: Probability of detecting an effect of a specified size under the alternative.

Interpreting a P-value

A p-value below a chosen significance threshold can provide evidence against the null hypothesis.

It does not tell you the probability that the null hypothesis is true, nor does it establish that an effect is practically important.

Consider effect size, uncertainty, assumptions, study design, and multiple testing when interpreting results.

11. Machine Learning Fundamentals

Machine learning uses algorithms to identify patterns in data and make predictions, classifications, recommendations, or other outputs.

Supervised Learning

The model learns from examples containing input features and known target labels or values.

Common tasks:

• Classification.

• Regression.

Examples:

• Predicting whether a transaction is fraudulent.

• Estimating product demand.

• Predicting customer churn.

Unsupervised Learning

The model looks for structure in data without requiring a known target label for the primary task.

Examples:

• Customer clustering.

• Dimensionality reduction.

• Pattern discovery.

Reinforcement Learning

An agent learns through interactions with an environment and feedback, often represented by rewards.

Common Algorithms

Algorithm suitability depends on the dataset, target, assumptions, performance requirements, and evaluation results.

12. Training, Validation, and Test Data

Data splitting helps evaluate whether a model generalizes beyond the data used to train it.

Training set: Used to fit model parameters.

Validation set: Used to compare models, tune hyperparameters, or select a modeling approach.

Test set: Used for a final, relatively unbiased evaluation after model decisions are complete.

An illustrative split is 70% training, 15% validation, and 15% testing. The appropriate proportions depend on dataset size, task, and evaluation design.

Data Leakage

Data leakage occurs when information unavailable at prediction time improperly influences training or evaluation.

Examples include:

• Using future information to predict the past.

• Preprocessing all data before splitting when the transformation learns from the data.

• Including a feature that directly reveals the target.

• Allowing related records from the same entity to leak across splits when the evaluation requires entity-level separation.

Fit data-dependent preprocessing steps using training data only, then apply the fitted transformations to validation and test data.

For time-dependent problems, use a split strategy that respects chronology.

13. Model Evaluation Metrics

Different tasks require different metrics.

Classification

Formulas:

[
\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN}
]

[
\text{Precision}=\frac{TP}{TP+FP}
]

[
\text{Recall}=\frac{TP}{TP+FN}
]

[
F1=2\frac{\text{Precision}\times\text{Recall}}
{\text{Precision}+\text{Recall}}
]

Here, TP means true positives, TN true negatives, FP false positives, and FN false negatives.

Accuracy can be misleading for imbalanced datasets. Choose metrics based on the consequences of different errors.

Regression

For errors (e_i=y_i-\hat{y}_i):

[
MAE=\frac{1}{n}\sum_{i=1}^{n}|e_i|
]

[
RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}e_i^2}
]

MAE and RMSE are expressed in the target's units. RMSE penalizes large errors more strongly than MAE.

14. Overfitting and Underfitting

Overfitting: A model learns patterns specific to training data, including noise, and performs poorly on unseen data.

Underfitting: A model is too limited to capture important patterns in the data.

Common Solutions

More complex models are not automatically better. The goal is reliable performance on the intended real-world task.

15. Feature Engineering

Feature engineering creates, selects, or transforms variables to help an analysis or model represent useful information.

Examples:

• Extracting month or weekday from a date.

• Calculating order value from quantity and price.

• Encoding categorical variables.

• Scaling numeric features when appropriate.

• Creating customer activity summaries.

• Transforming highly skewed variables when justified.

Feature engineering should be guided by domain knowledge and validated using an appropriate evaluation strategy.

Avoid creating features that reveal future outcomes or introduce sensitive information inappropriately.

16. Essential Python Libraries for Data Science

Install only the packages needed for your project and manage dependencies using an appropriate environment.

17. Example: A Simple Data Analysis Workflow

The following example calculates descriptive statistics from a small dataset.

import pandas as pd

data = {
    "campaign": ["A", "B", "C", "D"],
    "spend": [1000, 1500, 1200, 1800],
    "conversions": [50, 75, 60, 90]
}

df = pd.DataFrame(data)

df["cost_per_conversion"] = (
    df["spend"] / df["conversions"]
)

print(df)
print("Total spend:", df["spend"].sum())
print("Total conversions:", df["conversions"].sum())
print("Average cost per conversion:",
      df["cost_per_conversion"].mean())

This example demonstrates:

• Creating a dataset.

• Organizing data into a DataFrame.

• Creating a derived metric.

• Calculating totals and averages.

The example assumes all conversion counts are positive. Real datasets should be validated before calculations, particularly when denominators can be zero or values are missing.

18. Common Data Science Mistakes

• Starting analysis without defining the problem.

• Ignoring missing values or duplicates.

• Confusing correlation with causation.

• Using inappropriate evaluation metrics.

• Allowing data leakage.

• Overfitting to training data.

• Treating a p-value as a measure of effect size.

• Ignoring class imbalance.

• Drawing conclusions from unrepresentative samples.

• Creating misleading charts.

• Failing to document assumptions.

• Deploying a model without monitoring it.

A technically correct calculation can still lead to a poor decision if the underlying data, assumptions, or business context are misunderstood.

19. Data Science Quick Reference

20. Data Science Learning Checklist

Foundations

• Understand structured, semi-structured, and unstructured data.

• Learn mean, median, variance, and standard deviation.

• Understand probability and conditional probability.

• Distinguish correlation from causation.

Data preparation and analysis

• Clean missing and duplicated data.

• Perform exploratory data analysis.

• Create suitable visualizations.

• Use Python and SQL for data manipulation.

Machine learning

• Understand supervised and unsupervised learning.

• Split data appropriately.

• Prevent data leakage.

• Select task-appropriate metrics.

• Recognize overfitting and underfitting.

Communication and deployment

• Explain findings in plain language.

• Document assumptions and limitations.

• Connect results to the original business problem.

• Understand why deployed models need monitoring.

Conclusion

Data science combines statistics, programming, domain knowledge, and analytical reasoning to turn data into useful information. A strong foundation in data cleaning, exploratory analysis, probability, visualization, and machine learning will help you approach real-world problems more systematically.

Start with small datasets and clearly defined questions. Practice explaining not only what your analysis found, but also how reliable the result is, what assumptions it depends on, and what decision it can support.