https://genai.gitpull.in/2-intro-genai/2.1-key-concept/2-embeddings/index.html Embeddings Embeddings are numerical representations of tokens in a high-dimensional space. They capture the semantic meaning of tokens, allowing models to understand relationships between words. Why are Embeddings Important? Words with similar meanings have similar embeddings. Embeddings enable models to generalize and understand context. Example: The embeddings for "king", "queen", "man", and "woman" might satisfy the relationship: king - man + woman ≈ queen Hands-On: Embeddings from transformers import AutoTokenizer, AutoModel import torch # Load the tokenizer and model tokenizer = AutoTokenizer.from_pretrained("gpt2") model = AutoModel.from_pretrained("gpt2") # Tokenize input text text = "Hello, how are you?" inputs = tokenizer(text, return_tensors="pt") # Generate embeddings with torch.no_grad(): outputs = model(**inputs) # Extract embeddings for the first token embeddings = outputs.last_hidden_state print("Embeddings shape:", embeddings.shape) print("Embeddings for 'Hello':", embeddings[0, 0, :5]) # First 5 dimensions Output: Hugo en-us