Session 13 – Agentic AI DevOps: Embeddings Introduction

Agentic AI DevOps: Embeddings Introduction

Recap of the previous session

• Statistical ML models (from scikit-learn) suit simple data. Neural networks (built with PyTorch) let you customise the layers, loss function, learning rate and optimizer for complex data.
• Both need numeric input. In the PyTorch flow the data went from a DataFrame to NumPy, then to a scaled tensor.
• A neural network is trained, evaluated on test data, then saved (.pt weights and the scalers) and served as an API.
• So far, the problems were regression and classification on structured data, with a single-word category (like Graduate) encoded as 0 and 1.

In this session

• The new problem: classifying text written in natural language
• Why one-hot or label encoding does not work for text
• Tokenization: sentence, word, sub-word and character
• Token IDs and the vocabulary table
• Embeddings and vectors: what they are and how they are trained
• Dimensions, and why more dimensions capture more meaning
• Static embedding models, pre-trained models and Hugging Face

In one sentence: an embedding turns text into a list of numbers (a vector), and the numbers are chosen so that words with similar meanings end up with similar numbers.

1. The new problem: ticket routing

When an application fails, a ticket is raised with a title, for example “NetworkPolicy denies traffic from cart to auth” or “kubectl unauthorized: credentials rejected by the API server”. Each ticket also has a category (the team or area responsible), such as network or access.

The goal: a model reads the ticket text, understands its meaning, and routes it to the right team automatically, instead of a person acting as a dispatcher. It is still a classification problem, but the input is natural language, not a number or a single word.

You cannot use one-hot encoding or a label encoder here. Those encodings turn “Graduate” into 1 without caring what it means. Text needs more:

1. It must become numbers, because machines work on numbers.
2. The numbers must carry the meaning of the text, so the model can understand the context.

Embeddings do both. They are also the core idea behind LLMs. When you type a question into an AI chat, the text is converted into embeddings before it is processed. You can use the title, the description or both. It depends on your problem, and you find out by checking the accuracy.

2. Step 1: tokenization

Tokenization: sentence, word, sub-word and character

Tokenization splits the text into pieces called tokens. Take: “I run every day. He runs on weekends. We were running yesterday.”

There is no hard rule: the tokenizer decidesSub-word…, week, ##ends, …

LLMs use sub-word tokenization automatically in the background.

3. Step 2: tokens to numbers

Each token is converted into a number by an embedding model. This is a different kind of model from the ones you trained earlier.

You do not train it. A big company has already trained it on a very large collection of text. The task is general (turn any text into numbers), not specific to your business, so one model can serve everyone.

How an embedding model is built (once, by its builder)

Training an embedding model: two tables, saved and reused

1. The training text (billions of words) goes in. The model finds all the unique tokens and builds a vocabulary table: each token gets a unique token ID.
2. The model trains as usual (start with random weights, predict, compute the loss, adjust). What it learns is a numeric form for every token. This is the embedding table, which maps a token ID to its numbers.
3. The finished model saves both tables.

How you use it (every time)

1. The tokenizer looks each token up in the vocabulary table and returns a list of token IDs. These IDs are just labels with no meaning.
2. The embedding model looks each ID up in the embedding table and takes the vector for it.
3. The vectors are combined (a step called pooling) into the final numbers for your text.

Because the tables were fixed during training, this kind of model is called a static embedding model. Nothing is learned when you use it.

4. Embedding and vector

The result is a list of numbers, such as [0.9, -0.3, ...], with many more entries than shown. In maths, a list of numbers is a vector. In machine learning, the same thing is called an embedding. Embedding and vector mean the same here.

The number of values in the list is the number of dimensions. A 300-dimension model gives 300 numbers for every token. A four-token sentence therefore starts as 4 × 300 = 1,200 numbers before it is combined.

Embedding models and their dimensions: Word2Vec, GloVe, FastText, BERT, GPT-3

Contextual models are explained later in the course.

5. Why dimensions matter

How can numbers capture meaning? Take four words: man, king, queen and woman.

Vector space: 2D, 3D and n dimensions

• One number each (man 1, king 2, queen 3, woman 4) tells you nothing about how the words relate.
• Two numbers each can be drawn as points on a graph: man (0, 0), king (1, 0), queen (1, 1), woman (0, 1). Now relationships show up. Man and king sit close together. Woman and queen sit close together. King and queen are opposite, as are man and woman.
• Three numbers give more room, and n numbers give far more.

A real model uses hundreds or thousands of dimensions. We cannot draw that, but the computer works on the numbers and calculates which tokens are near and which are far. The numbers in these vectors are learned during training, not chosen by hand.

A famous example of meaning captured by maths: take the vector for king, subtract man, add woman, and you get the vector for queen.

import numpy as np

man, king, queen, woman = map(np.array, ([0, 0], [1, 0], [1, 1], [0, 1]))
print(king - man + woman)        # [1 1] -> the same as queen

(The numbers here are toy values. Real models do this with hundreds of numbers per word and matrix calculations.)

The trade-off: more dimensions capture the meaning better, but need more compute, storage and time (latency). If a 300-dimension model gives good results, use it. If not, try a bigger one. Different models have different dimensions, and you choose by your requirement.

6. Pre-trained models and Hugging Face

Hugging Face is a hub for open-source models from many vendors. For example, the word2vec-google-news-300 model is a pre-trained model: Google trained it on a news dataset of about 100 billion words, so it already knows about 3 million words and phrases. It is not realistic for us to train anything like this, so we download it and use it.

7. How you use the embeddings

Once your text is in numbers, you give it to a model, as you gave numbers before. For the tickets, each ticket’s text is replaced with its vector, and a classifier learns to predict the category from it. Embeddings are not only for classification. They are the starting point for almost everything language models do, and the base for RAG, which comes a little later.

The next session does this in code: load a pre-trained embedding model from Hugging Face, give it text, and read the vector.

Summary

Key takeaways

• Text must become numbers that carry meaning, and one-hot or label encoding cannot do that for natural language.
• Tokenization splits text into tokens (sentence, word, sub-word, character).
• A vocabulary table maps token to token ID, and an embedding table maps token ID to a vector. Both are saved when an embedding model is trained.
• Embedding models are pre-trained by large companies. You use them and do not train them.
• A vector (embedding) is a list of numbers. The number of values is the dimension count.
• More dimensions capture meaning better but cost more compute, storage and time.
• Static embeddings are fixed after training. Contextual ones come later.

Quick revision

Remember: tokens, then IDs, then vectors.