Session 14 – Agentic AI DevOps: Embeddings in Action

Agentic AI DevOps: Embeddings in Action

Recap of the previous session

• Text must become numbers that carry meaning. One-hot and label encoding cannot do that for sentences.
• Tokenization splits text into tokens. A vocabulary table maps token to token ID, and an embedding table maps token ID to a vector (a list of numbers).
• Embedding models are pre-trained by large companies. You use them and do not train them.
• Embedding (ML term) and vector (maths term) mean the same thing. The number of values in it is the dimension count.

In this session

• Static and contextual embeddings, in one example
• Loading a pre-trained static model (GloVe) and looking inside it
• Cosine similarity: measuring how close two words are
• Words the model has never seen, and why the training data matters
• Hugging Face transformers: a tokenizer and a model
• Reading the vocabulary and the embedding table of a real model
• Choosing a model, and the vector database idea

In one sentence: an embedding model gives each word a list of numbers, and comparing those numbers tells you which words, or later which documents, are close in meaning.

1. Static vs. contextual embeddings

There are two kinds of embedding models:

Take the word bank. In “money in the bank” and “the river bank”, the meaning differs. A static model gives both the same vector. A contextual model gives different ones. Most real use cases use contextual models. This session starts with static models to see the basics.

2. A pre-trained static model: GloVe

Gensim is a Python library that downloads pre-trained static embedding models such as GloVe (Stanford) and Word2Vec (Google).

!python3 -m pip install gensim
import gensim.downloader as api

glove = api.load("glove-wiki-gigaword-50")     # trained on Wikipedia
print("vocabulary size:", len(glove.key_to_index))   # 400000 words
print("vector size    :", glove.vector_size)         # 50

• The name tells you the training data (Wikipedia). The 50 is the number of dimensions: each word has 50 numbers. It is a small model, good for learning. The vocabulary holds about 400,000 words.
• Look up a word. pod is row 22460 (its token ID), and its vector starts [1.11, -0.07, 0.8, ...]. The rest of the 50 numbers are hidden.

glove.key_to_index["pod"]       # 22460
glove["pod"][:6]                # the first 6 of 50 numbers

Words the model does not know

for word in ["docker", "kubernetes", "crashloopbackoff"]:
    print(word, word in glove.key_to_index)    # True, False, False

docker is in the vocabulary, but kubernetes is not. A word that is missing gets a generic “unknown” vector, so the model cannot tell it apart from other unknown words.

Which words exist depends on what the model was trained on. GloVe learned from Wikipedia, so the nearest words to pod are pods, larva, legume, larvae, worm: the biology meaning, not the Kubernetes one. A model trained on Google News (Word2Vec) has a different vocabulary. If your domain words (DevOps terms) are missing, that model is a poor fit, so choose a model by comparing it against your own use case.

3. Why we want embeddings: similarity

Similar words get similar vectors. That lets a machine tell which words are alike or different. The usual measure is cosine similarity, which compares the direction of two vectors.

import numpy as np

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

cosine(glove["server"], glove["machine"])    # 0.43

• 1.0 means the same direction (identical meaning). 0 means unrelated. A negative score means opposite directions.
• The closer to 1, the more similar the words.

You can also ask for the nearest words:

glove.most_similar("server", topn=5)
# servers, user, desktop, functionality, interface
glove.most_similar("outage", topn=5)
# outages, blackouts, glitch, blackout, shutdown

(The scores for the top five are close to 1, with the word itself, in its plural form, first.)

4. Hugging Face transformers

The static models above are the early generation. In practice, people use the transformers library from Hugging Face. It gives you tokenizers and models (including embedding models and LLMs) from one install. A model is loaded by its name.

!python3 -m pip install transformers
from transformers import AutoTokenizer, AutoModel

model_name = "sentence-transformers/all-MiniLM-L6-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)   # turns text into token IDs
model = AutoModel.from_pretrained(model_name)           # looks up and builds the vectors
model.eval()                                            # use mode (not training mode)

• Two steps: the tokenizer makes tokens and token IDs, then the model turns the IDs into embeddings. You can ask for the embedding in one call, but it does the same two steps inside.
• eval(): these models are neural networks built with PyTorch (so PyTorch must be installed), and you set them to evaluation mode for use.
• Loading the weights means loading the model’s learned parameters into memory. This model has about 22 million of them.

Setup tips

• This library needs a recent PyTorch (version 2.6 or newer). If the versions clash, reinstall PyTorch and restart the notebook kernel.
• Pin versions (for example torch==2.6.0) so that every install gives the same result.

5. Looking inside the model

Tokenizer looks up IDs, the model looks up vectors

vocab = tokenizer.get_vocab()
len(vocab)                       # 30522 tokens in the vocabulary
vocab["pod"], vocab["deploy"]    # 17491, 21296 (token IDs)

tokenizer.convert_ids_to_tokens([17491, 8241, 21296])   # ['pod', 'server', 'deploy']

Token IDs are only labels. The vectors live in the model’s embedding table:

embedding_matrix = model.get_input_embeddings().weight.detach()
embedding_matrix.shape           # (30522, 384)  -> 30,522 tokens, 384 numbers each

Each of the 30,522 tokens has 384 numbers: its static embedding. To get the vectors for a sentence, take the rows for its token IDs:

ids = tokenizer("Pod crashed after deploy")["input_ids"]
vectors = embedding_matrix[ids]  # shape (6, 384): one row per token

The six tokens are the four words plus two special markers ([CLS] and [SEP]) that the tokenizer adds to the start and end. This lookup table is the static part. The layers of the model sit on top of it and add the context, which is the next topic.

6. Choosing an embedding model

Hugging Face has many models. All of them give you embeddings, so how do you choose?

• What it was trained on. A legal project suits a model trained on legal text, and a healthcare project suits one trained on medical text. That way the vocabulary covers your domain.
• Size. The number of parameters shows the size. This one has 22 million, which is light and fast, so it suits learning and small machines. Others have hundreds of millions or 7 billion, which are better for demanding retrieval work but need much more compute.
• Documentation. Read the model card for its strengths, weaknesses and training data, short-list two or three, and test them on your own data.
• Cost. These open-source models are free to use. You pay only for the compute you run them on.
• Resources. There is no single rule. Look at the number of parameters and the download size, estimate, run, and tune, as you do for any application. The documentation often states a minimum: a 7-billion-parameter model, for example, needs about 15 GB of memory just to load. Large models are usually run on managed services such as Amazon SageMaker instead of your own machine.

Tools such as Ollama run LLMs locally. They are not embedding models.

7. Vector databases and what embeddings are for

A vector often has to be kept for later. Where you store embeddings is called a vector database. The usual flow is: convert the text into embeddings, store them in a vector database, and let the model work from there.

Example: finding a document. A user asks for “the policy document”. None of the documents contain the word policy, but one contains claim. The question and the document words are both turned into embeddings, and the cosine similarity shows they are close. So the system returns that document. This is how a semantic search engine works, and it is the heart of the RAG architecture, the next topic.

Summary

Key takeaways

• Static embeddings give a word one fixed vector. Contextual embeddings change with the sentence.
• A pre-trained model such as GloVe has a vocabulary and a vector for every word. Words outside the vocabulary are not represented properly, so check that the model covers your domain.
• Cosine similarity measures how close two vectors are: 1 is the same, 0 is unrelated, negative is opposite.
• With Hugging Face transformers, a tokenizer makes token IDs and a model looks up the vectors.
• You can inspect the vocabulary (30,522 tokens) and the embedding table (30,522 by 384).
• Choose a model by training data, size and tests on your own data. Open-source models are free.
• Embeddings are stored in a vector database and used for semantic search and RAG.

Quick revision

Remember: embeddings turn “similar meaning” into “similar numbers”.