1. remove html tags

2. remove punctuations

3. remove numbers

4. remove white spaces

normalization process

mean, median, or mode of the variable or simply assuming zero

Extract new variables from existing variables

Extract Age from Date of Birth; for example, create a new field "Total loans as a proportion of income"

Salary and Other Income——> Total Income

Filter rows for "Beijing" from the "city" column

Only select useful features

Convert the amount to USD uniformly

3 standard deviations from the mean

the 75th and the 25th percentile values ​​of the data

outside of 1.5 IQR:outliers

outside of 3 IQR:extreme values

a 5% trimmed dataset is one for which the 5% highest and the 5% lowest values ​​have been removed.

The process of replacing extreme values ​​and outliers in a dataset with the maximum (for large value outliers) and minimum (for small value outliers) values ​​of data points that are not outliers.

Sensitive to outliers and suitable for situations where the distribution is unknown

Suitable for samples that follow a normal distribution

a series that contains characters in a particular order

1 Remove html tags:

2 Remove Punctuations

3 Remove Numbers

4 Remove white spaces