Vectors and similarity of meaning
A vector is a list of numbers that gives a word a position on a plot. Words used in similar ways get nearby positions, so small distance stands for similar meaning.
You type "how do I sign up for a class" into the help search of the learning management system. The top result is a page called "Course enlistment process". The page title shares no word with your question.
Each word here has two numbers, which give its position on the plot. Real models use hundreds of numbers per word. Each button adds the first word's position, subtracts the second and adds the third. The circle marked result is where the sum ends up, and the nearest word is the answer. These positions were placed by hand.
More detail
The rule that is often left out
The well-known answer appears only when the three input words are removed from the candidates. Turn the option off to see which word is nearest to the result.
Why the bias example is disputed
A 2016 paper by Bolukbasi and colleagues reported the programmer analogy using word2vec trained on Google News, and it became a standard illustration of bias in embeddings. Nissim and colleagues showed in 2020 that the analogy method produces part of the result. The search must exclude the input words, so it has to return something other than programmer, and a gendered-looking answer is partly a product of that rule.
What holds up is smaller. An embedding is built from a corpus, and patterns in that corpus become geometry. A prompt cannot remove that, because it is built into the positions the model uses. Anyone who claims one tidy number for how biased a model is should be asked how it was measured.
One vector per use of a word
On this plot, bank would get one point, placed between its river sense and its money sense. Newer models give a word a different vector depending on the sentence around it. That change is a main reason retrieval works as well as it does today.
A sentence also gets a position, worked out from its words. How alike two sentences are depends on the angle between their arrows. Pick two sentences and compare the similarity score with the number of words they share.
More detail
What the score is
The score is the cosine of the angle between the two arrows. It is 1 when the arrows point the same way and 0 when they are at right angles. It measures direction only, so a long sentence and a short one can score highly when they are about the same thing.
The help search turns your question and every help page into positions. A keyword search would find nothing useful, because the question and the title share no words. The two positions point almost the same way. Finding the text with the nearest position, called retrieval, returns the enlistment page.
You can now explain how a search by meaning finds a page that shares no words.
More detail
Retrieval in AI assistants
Retrieval happens when you point an AI assistant at your documents. A student asking how to stop a double enlistment can be matched to a page about idempotency keys, which stop a repeated request from being processed twice, without the two sharing a single word.
The reverse case
How do I drop a class shares five words with the sign-up question and means the opposite, and the score reflects that. Two sentences can share many words and mean opposite things.