1 / 4
From text to training examples
Choose a center word. Words inside the window become real pairs; sampled
noise words become negative pairs.
—sentences
—words
24Dthen PCA for display
Show the training text
PCA is a linear compression: it preserves vector directions, so it works for the analogy.
Choose the center word in the sentence
Positive pairs · raise the dot product
Negative samples · lower the dot product
Training: — pairs · moving loss:
—
Ready
milk
←
raise: boy, drank
+
lower: random noise
Ready
Follow the direction from man to king, then transfer it to woman
king − man + woman
≈
?
Waiting to run
Nearest candidates among the mapped words:
People
Food
and drink
Places
and countries
Roles
Transport
Language
and meaning