Lesson 6 of 38 · Written learning
Vectors, matrices, tensors, and shape
Read the basic language of numerical arrays and understand why shape matters.
A scalar is one number. A vector is a one-dimensional sequence, a matrix is a two-dimensional grid, and a tensor generalizes this idea to more dimensions. A batch of token representations might have dimensions for examples, positions, and features. Shape names are more useful than memorizing a fixed number of axes.
Operations encode relationships
The dot product multiplies corresponding vector components and adds the results. It can measure alignment: a large positive value means two vectors point in a similar direction, while a small or negative value indicates weaker alignment under the chosen representation. Cosine similarity divides the dot product by the vectors' lengths, making direction easier to compare when magnitudes differ.
batch = 8
sequence_length = 16
features = 64
# token representations: (8, 16, 64)- Check dimensions before multiplying matrices.
- A transpose swaps matrix axes; it does not change the underlying values.
- A zero-length vector has no defined cosine direction.
- Similarity in an embedding space is useful evidence for retrieval, not proof that two passages mean exactly the same thing.
Return to the Mathematics for LLMs stage · Suggested timing is flexible; move at your own pace.