Roadmap for Building Machine Learning Models

Data Pre-processing

- This is the first step in building a machine learning model. Data pre-processing refers to the transformation of data before feeding it into the model.
- It deals with the techniques that are used to convert unusable raw data into clean reliable data.
- Since data collection is often not performed in a controlled manner, raw data often contains outliers (for example, age = 120), nonsensical data combinations (for example, model: bicycle, type: 4- wheeler), missing values, scale problems, and so on.
- Because of this, raw data cannot be fed into a machine learning model because it might compromise the quality of the results. As such, this is the most important step in the process of data science.

Model Learning

- After pre-processing the data and splitting it into train/test sets, we move on to modeling.
- Models are nothing but sets of well-defined methods called algorithms that use pre- processed data to learn patterns, which can later be used to make predictions.
- There are different types of learning algorithms, including supervised, semi-supervised, unsupervised, and reinforcement learning.

Model Evaluation

- - In this stage, the models are evaluated with the help of specific performance metrics.
  - With these metrics, we can go on to tune the hyperparameters of a model in order to improve it.
  - This process is called hyperparameter optimization.
  - We will repeat this step until we are satisfied with the performance.

Prediction

- - - Once we are happy with the results from the evaluation step, we will then move on to predictions.
    - Predictions are made by the trained model when it is exposed to a new dataset.
    - In a business setting, these predictions can be shared with decision makers to make effective business choices.

Model Deployment

- - - - The whole process of machine learning does not just stop with model building and prediction.
      - It also involves making use of the model to build an application with the new data.
      - Depending on the business requirements, the deployment may be a report, or it may be some repetitive data science steps that are to be executed.
      - After deployment, a model needs proper management and maintenance at regular intervals to keep it up and running.

Data Representation

The main objective of machine learning is to build models that understand data and find underlying patterns. In order to do so, it is very important to feed the data in a way that is interpretable by the computer.
To feed the data into a model, it must be represented as a table or a matrix of the required dimensions.
Converting your data into the correct tabular form is one of the first steps before pre-processing can properly begin.

Independent Variables

These are all the features in the DataFrame except the target variable. They are of size (m, n), where m is the number of observations and n is the number of features. These variables must be normally distributed and should NOT contain:

Missing or NULL values
Highly categorical data features or high cardinality
Outliers
Data on different scales
Human error
Multicollinearity (independent variables that are correlated)
Very large independent feature sets (too many independent variables to be manageable)
Sparse data
Special characters

Feature Matrix and Target Vector

A single piece of data is called a scalar. A group of scalars is called a vector, and a group of vectors is called a matrix.

A matrix is represented in rows and columns.

Feature matrix data is made up of independent columns, and the target vector depends on the feature matrix columns. To get a better understanding of this, let's look at the following table:

As you can see in the table, there are various columns: Car Model, Car Capacity, Car Brand, and Car Price.

All columns except Car Price are independent variables and represent the feature matrix. Car Price is the dependent variable that depends on the other columns (Car Model, Car Capacity, and Car Brand).

It is a target vector because it depends on the feature matrix data.

Open the Colab notebook

Data Cleaning

Data cleaning includes processes such as filling in missing values and handling inconsistencies. It detects corrupt data and replaces or modifies it.

Missing Values

The concept of missing values is important to understand if you want to master the skill of successful management and understanding of data.

As you can see in the table below, the data belongs to a bank;

Each row is a separate customer and each column contains their details, such as age and credit amount.

There are some cells that have either NA or are just empty.

This is missing data. Each piece of information about the customer is crucial for the bank.

If any of the information is missing, then it will be difficult for the bank to predict the risk of providing a loan to the customer.

Handling Missing Data

Intelligent handling of missing data will result in building a robust model capable of handling complex tasks. There are many ways to handle missing data.

Removing the Data

Checking missing values is the first and the most important step in data pre-processing.

A model cannot accept data with missing values.

This is a very simple and commonly used method to handle missing values: we delete a row if the missing value corresponds to the places in the row, or we delete a column if it has more than 70%-75% of missing data.

Again, the threshold value is not fixed and depends on how much you wish to fix.

The benefit of this approach is that it is quick and easy to do, and in many cases no data is better than bad data. The drawback is that you may end up losing important information, because you're deleting a whole feature based on a few missing values.

Here is an example on how we remove missing data. You will need the Banking_Marketing.csv dataset to try out this example.

Open the Colab Notebook

Mean/Median/Mode Imputation

In the case of numerical data, we can compute it's mean or median and use the result to replace missing values.

In the case of the categorical (non-numerical) data, we can compute its mode to replace the missing value. This is known as imputation.

The benefit of using imputation, rather than just removing data, is that it prevents data loss.

The drawback is that you don't know how accurate using the mean, median, or mode is going to be in a given situation.

Here is an example on how we impute missing data. You will need the Banking_Marketing.csv dataset to try out this example.

Open the Colab Notebook

Outliers

Outliers are values that are very large or very small with respect to the distribution of the other data.

We can only find outliers in numerical data. Box plots are one good way to find the outliers in a dataset, as you can see in the following figure:

Consider a sample dataset of temperatures from a place at different times:

71, 70, 90, 70, 70, 60, 70, 72, 72, 320, 71, 69

First, we'll sort the data: 60, 69, 70, 70, 70, 70, 71, 71, 72, 72, 90, 320

Next, we'll calculate the median (Q2). The median is the middle data after sorting. Here, the middle terms are 70 and 71 after sorting the list.

The median is(70+71)/2=70.5

Then we'll calculate the lower quartile (Q1). Q1 is the middle value (median) of the first half of the dataset.

First half of the data = 60, 69, 70, 70, 70, 70

Points 3 and 4 of the bottom 6 are both equal to 70.

The average is (70+70)/2 = 70

Q1=70

Then we calculate the upper quartile (Q3). Q3 is the middle value (median) of the second half of the dataset.

Second half of the data = 71, 71, 72, 72, 90, 320

Points 3 and 4 of the upper 6 are 72 and 72.

The average is (72+72)/2=72

Q3=72

Then we find the interquartile range (IQR).

IQR=Q3–Q1=72–70

IQR=2

Next, we find the upper and lower fences.

Lowerfence=Q1–1.5(IQR)=70–1.5(2)=67

Upper fence = Q3 + 1.5 (IQR) = 71.5 + 1.5(2) = 74.5

Boundaries of our fences = 67 and 74.5

Any data points lower than the lower fence and greater than the upper fence are outliers.

Thus, the outliers from this example are 60, 90 and 320.

Here is an example on how we find and remove outliers from a dataset. You will need the german_credit_data.csv dataset to try out this example.

Open the Colab Notebook

Data Integration

We combine data from different sources to get a unified structure with more meaningful and valuable information.

This is mostly used if the data is segregated into different sources.

To make it simple, let's assume we have data in CSV format in different places, all talking about the same scenario. Say we have some data about an employee in a database.

We can't expect all the data about the employee to reside in the same table. It's possible that the employee's personal data will be located in one table, the employee's project history will be in a second table, the employee's time-in and time-out details will be in another table, and so on.

So, if we want to do some analysis about the employee, we need to get all the employee data in one common place. This process of bringing data together in one place is called data integration.

To do data integration, we can merge multiple pandas DataFrames using the merge function.

Here is an example of how we merge data from multiple datasets. You would need the student.csv and mark.csv datasets to try out this example.

Open the Colab Notebook

Data Transformation

Previously, we saw how we can combine data from different sources into a unified dataframe.

Now, we have a lot of columns that have different types of data.

Our goal is to transform the data into a machine-learning-digestible format.

All machine learning algorithms are based on mathematics.

So, we need to convert all the columns into numerical format. Before that, let's see all the different types of data we have.

Taking a broader perspective, data is classified into numerical and categorical data:

Numerical: As the name suggests, this is numeric data that is quantifiable.

Categorical: The data is a string or non-numeric data that is qualitative in nature.

Numerical data is further divided into the following:

Discrete: To explain in simple terms, any numerical data that is countable is called discrete.

For example: the number of people in a family or the number of students in a class.

Discrete data can only take certain values (such as1,2,3,4,etc).

Continuous: Any numerical data that is measurable is called continuous.

For example: the height of a person or the time taken to reach a destination.

Continuous data can take virtually any value (for example, 1.25, 3.8888, and 77.1276).

Categorical data is further divided into the following:

Ordered: Any categorical data that has some order associated with it is called ordered categorical data.

For example: movie ratings (excellent, good, bad, worst) and feedback (happy, not bad, bad).

You can think of ordered data as being something you could mark on a scale.  

Nominal: Any categorical data that has no order is called nominal categorical data.

Examples include gender and country.

Handling Categorical Data

There are some algorithms that can work well with categorical data, such as decision trees. But most machine learning algorithms cannot operate directly with categorical data. These algorithms require the input and output both to be in numerical form. If the output to be predicted is categorical, then after prediction we convert them back to categorical data from numerical data.

Let's discuss some key challenges that we face while dealing with categorical data:

High cardinality: Cardinality means uniqueness in data. The data column, in this case, will have a lot of different values. A good example is User ID – in a table of 500 different users, the User ID column would have 500 unique values.
Rare occurrences: These data columns might have variables that occur very rarely and therefore would not be significant enough to have an impact on the model.
Frequent occurrences: There might be a category in the data columns that occurs many times with very low variance, which would fail to make an impact on the model.
Won’t fit: This categorical data, left unprocessed, won’t fit our model.
Encoding:  To address the problems associated with categorical data, we can use encoding. This is the process by which we convert a categorical variable into a numerical form. 
Replacing : This is a technique in which we replace the categorical data with a number. This is a simple replacement and does not involve much logical processing.

Here is an example of how we replace categorical data with a number. You would need the student.csv dataset to try out this example

Open the Colab Notebook

Label Encoding

This is a technique in which we replace each value in a categorical column with numbers from 0 to N-1.

For example, say we've got a list of employee names in a column. After performing label encoding, each employee name will be assigned a numeric label.

But this might not be suitable for all cases because the model might consider numeric values to be weights assigned to the data.

Label encoding is the best method to use for ordinal data. The scikit-learn library provides LabelEncoder(), which helps with label encoding.

Here is an example of how convert categorial data to numerical data using label encoding. You would need the Banking_Marketing.csv dataset to try out this example.

Open the Colab Notebook

One Hot Encoding

In label encoding, categorical data is converted to numerical data, and the values are assigned labels (such as 1, 2, and 3).

Predictive models that use this numerical data for analysis might sometimes mistake these labels for some kind of order (for example, a model might think that a label of 3 is "better" than a label of 1, which is incorrect).

In order to avoid this confusion, we can use one-hot encoding. Here, the label-encoded data is further divided into n number of columns.

Here, n denotes the total number of unique labels generated while performing label encoding. For example, say that three new labels are generated through label encoding.

Then, while performing one-hot encoding, the columns will be divided into three parts. So, the value of n is 3.

Open the Colab Notebook

Data in Different Scales

In real life, values in a dataset might have a variety of different magnitudes, ranges, or scales. Algorithms that use distance as a parameter may not weigh all these in the same way.

There are various data transformation techniques that are used to transform the features of our data so that they use the same scale, magnitude, or range. This ensures that each feature has an appropriate effect on a model's predictions.

Some features in our data might have high-magnitude values (for example, annual salary), while others might have relatively low values (for example, the number of years worked at a company). Just because some data has smaller values does not mean it is less significant.

So, to make sure our prediction does not vary because of different magnitudes of features in our data, we can perform feature scaling, standardization, or normalization (these are three similar ways of dealing with magnitude issues in data).

Standard Scaling

Min Max Scaling

Data Discretization

Page updated

Google Sites

Report abuse