Agentic AI DevOps Bootcamp · Session 6 · ML Training – Linear Regression · Summary Notes

Agentic AI DevOps Bootcamp · Session 6 · Summary Notes

Today’s topics

• The training workflow in four steps
• Seeing the pattern in the data with a graph (matplotlib)
• Deciding which features to keep
• Analysing a dataset with many features
• Encoding a categorical column (one-hot encoding)
• Why we never test on the training data, and train_test_split
• Training with many features, testing, and the r2_score
• Why 75% is not enough, and what comes next

The workflow

Raw data is never ready for training. Every ML project follows the same four steps:

1. Data analysis: how many records and columns? What type is each?
2. Feature engineering: check every feature (a feature is a column) and change or remove what is not useful.
3. Split the data into features (x) and target (y).
4. Train the model with x and y.

Today we repeat these steps on a harder dataset. In the last session the data had one feature (experience). Real salaries depend on many.

1. See the pattern: plot the data

The pattern is the relationship between x and y: when x changes, how much does y change? Reading ten numbers is easy. With hundreds of records it is not, so we plot them. The Python library for graphs is matplotlib (seaborn is another).

!python3 -m pip install matplotlib        # if it is not in requirements.txt yet
from matplotlib import pyplot as plt

plt.figure(figsize=(6, 6))
plt.scatter(x, y)                         # scatter plot: one dot per record
plt.xlabel("Experience")
plt.ylabel("Salary")
plt.title("Experience vs. salary")
plt.show()

What to look for: if every step in experience raises the salary by the same amount, the dots form a straight line. That is a linear relationship, so linear regression is a good fit.

You don’t need to memorise plotting syntax. The concepts are what matter: pandas loads the data, matplotlib visualises it, NumPy handles the numbers, and the documentation (or an AI assistant) gives you the exact code. Data scientists usually do the deep pattern analysis and choose the model, but you should know how it is done.

2. Which features should you keep?

A salary may depend on experience, projects completed, certificates, job title and more. You decide which features to include. That is domain knowledge, not code.

• If a feature has no real impact on the target (say your company ignores certificates when setting salary), leave it out. It only adds training complexity.
• Keep the features with the highest impact (the highest “weight”) on the target.
• Data scientists usually deliver a trimmed dataset, but you should still question it. If a feature looks unnecessary, tell them, and agree on the final set together.

3. Analyse the new dataset

The new dataset (salary_v2) has 100 records and 5 columns: experience, projects completed, job title, certificates completed, and salary (the target).

data.shape          # (100, 5)
data.head()
data.info()
data.nunique()      # how many unique values per column

Numerical vs. categorical. info() shows that experience, projects, certificates and salary are numbers, but job title is text (str). Text like this is categorical data. A model can only process numbers, so every categorical column must be converted. That conversion is called encoding.

nunique() and why it matters. Our counts: experience 20, job title 5, certificates 6, salary 100.
– Salary has 100 unique values, so no two employees are paid the same. That is good: the model can learn why salaries differ.
– If all 100 employees had the same salary, there would be nothing to learn, like a student whose practice questions are all identical. Send such data back to the data team.

4. Which feature matters? Plot them all, then corr()

Plot salary against each feature (a loop over the columns gives several graphs at once). What we see:

• Experience: salary clearly rises with experience, so it has a lot of influence.
• Certificates: the graph stays flat (zero or five certificates, same salary), so little influence.

With 4 features this is easy. With 20, plotting and removing weak features is part of good feature engineering.

To see the same thing as numbers, call corr() on the numerical columns:

data.corr(numeric_only=True)

For our data, experience is about 0.7, projects about 0.3 and certificates about 0.2. Experience is the strongest feature.

Good to know: corr() gives a correlation (1 = move perfectly together, 0 = unrelated). It is a good first guide to importance, not exactly the same as the weights the model learns.

5. Encode the categorical column: one-hot encoding

One-hot encoding replaces a categorical column with one 0/1 column per category. pandas does it with get_dummies:

data_encoded = pd.get_dummies(data, columns=['Job_Title'], dtype=int)
data_encoded.head()

• Job_Title becomes five columns: junior, lead, manager, mid, senior.
• In each row exactly one is 1 and the rest are 0. A mid-level employee has Job_Title_Mid = 1.
• The meaning is kept, and the data is now numerical.

From now on use data_encoded, not the old data. (You could drop one of the five columns with drop_first, but keeping all five makes the meaning obvious. There are also other encoding methods, which we will meet later.)

6. Features (x) and target (y)

X = data_encoded.drop(columns=['Salary'])    # every column except the target
y = data_encoded['Salary']                   # the target only

X holds experience, projects, certificates and the five job-title columns. y holds only the salary.

7. Never test on the data you trained on

Last session we trained on all ten records and then predicted the same ten. We got 100%. That proves nothing, like giving a student ten practice questions and then putting the same ten in the exam.

To judge a model fairly, hold back some data the model never sees during training. This is the train/test split: for example, 80 records to train and 20 to test.

from sklearn.model_selection import train_test_split

x_train, x_test, y_train, y_test = train_test_split(
    X, y, test_size=0.33, random_state=42
)

• test_size=0.33 means 33% of the rows are for testing, the rest for training. You choose the ratio.
• The split is random, on purpose. Taking the top rows would be wrong if the file happens to be sorted (for example, by salary), because then training and testing would not represent each other.
• random_state is any fixed number (42, 1, 2…). The same number gives the same split every time.

You get four variables:

x_train has 8 columns: experience, projects, certificates and five job-title columns.

8. Train with many features

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(x_train, y_train)

model.coef_         # 8 values: one weight per feature
model.intercept_    # 1 value: a single b for the whole model

With several features, the model learns one weight (slope) per feature and a single bias:

salary = m₁·x₁ + m₂·x₂ + m₃·x₃ + ... + b

9. Test: predict, compare, score

predictions = model.predict(x_test)          # 33 predictions, one per test employee

We also have the real salaries of those 33 employees, the truth, in y_test. The closer the predictions are to the truth, the better the model. scikit-learn can score it:

from sklearn.metrics import r2_score
r2_score(y_test, predictions)                # about 0.75

A one-by-one check shows the gap: the first test employee really earns 67,000 and the model predicted about 72,000.

Good to know: an R² of 0.75 does not mean “75% of predictions are correct”. It means the model explains about 75% of the variation in salary. 1.0 is perfect, and 0 is no better than always guessing the average. In class we read it as “about 75% accurate”, which is fine as a first feel.

This model is not ready for production. It predicts a salary that is too high, so the score must improve.

10. What happens next: experimentation

1. Look for what to change: the data, the features, the training.
2. Train again and compare the score.
3. Repeat until you are happy, then save the model and deploy it.

One strong lever is trying another algorithm on the same data. The data is not perfectly linear, so linear regression may not be the best choice. Try three or four models and compare their scores. (There are hundreds, so use your judgement to pick a few.) The next session trains the same data with a different model.

Common questions

Can I run all notebook cells at once? Yes: use the Run menu → Run All Cells. But a notebook is for experiments, not for building the product. Once an experiment works, move the code into your IDE.

How much server capacity does ML need? Decide it the way you size any application: start with a reasonable guess (for example 2 GB and 1 core), monitor it, and tune it. A performance-testing team runs load tests and gives you numbers. All of this happens in the lower environments, so you reach production already knowing the capacity.

Recap

• Workflow: analyse → engineer features → split x/y → train.
• Plot the data and use corr() to see which features matter. Drop the weak ones.
• Models need numbers: encode categorical columns (one-hot with get_dummies).
• Never test on training data. Use train_test_split.
• Multiple features give one weight per feature and one bias.
• r2_score compares predictions with the truth. About 0.75 means we still have work to do.

An assignment will follow this week for hands-on practice.

Quick reference

from matplotlib import pyplot as plt                      # plot the data
data.corr(numeric_only=True)                              # feature weight as numbers
data.nunique()                                            # unique values per column
data_encoded = pd.get_dummies(data, columns=['Job_Title'], dtype=int)

X = data_encoded.drop(columns=['Salary'])
y = data_encoded['Salary']

from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(x_train, y_train)
predictions = model.predict(x_test)

from sklearn.metrics import r2_score
r2_score(y_test, predictions)