Book

Description

Every modern language technology, from search and translation to question answering and conversational agents, begins by turning words into numbers. How that conversion is done determines what a machine can understand. This book traces the evolution of word embeddings from sparse One-Hot and TF-IDF vectors, through static neural models such as Word2Vec, GloVe and FastText, to the contextual representations produced by Transformer models.   Static embeddings assign each word a single vector, so the word “bank” means the same thing beside a river and beside a loan. The Transformer resolves this limitation with subword tokenization, positional and rotary encodings, and multi-head self-attention. The book derives each step mathematically, including the scaling of attention and the flow of gradients, and verifies the key results with hand-checkable examples and numerical experiments.   Readers will see how attention heatmaps separate the senses of an ambiguous word, how representations change from layer to layer, and how contextual embeddings compare with earlier methods on standard benchmarks. Practical chapters explain how to extract, pool and use embeddings, and how they support question answering over domain-specific document collections.   Written for students, researchers and practitioners of artificial intelligence and natural language processing, the book combines rigorous mathematics with clear worked examples and review questions.