How AI works · A plain-English guide

The Zipper, the Mask, and the Lost Write: How AI Picks the Right Numbers

A plain-English guide to one of the most common operations in machine learning — and the silent mistakes hiding in it.

By Siddharth Tiwari · AI strategy & analytics · 23 min read

Every time a streaming service suggests a film, a phone recognises its owner’s face, or a chatbot answers a question, a computer is doing something surprisingly humble: picking numbers out of grids. Underneath it all, modern AI is a vast collection of numbers in tables, and almost everything it does begins by pulling the right ones out. That act of picking is called indexing.

AI is made of grids of numbers

A tensor is a grid of numbers, like a spreadsheet: rows, columns and a number in every cell. Each direction you can move in, down the rows or across the columns, is called an axis (engineers also say dimension), and the tensor’s shape lists its size along each axis. Three rows and four columns make shape (3, 4).

Almost anything can become a tensor. A photo is a grid of tiny squares called pixels, each stored as a brightness number; a colour photo stacks three such grids, for red, green and blue. Sound is a long list of numbers. Text becomes numbers too: every word gets an ID and a row of numbers describing its meaning. An AI model is itself a giant stack of grids, often billions of numbers in all.

Indexing is how AI picks what it needs

To index a tensor is to point at positions in it and take what is there: “row 2, column 3”, or “every cell above 100”. The position numbers are called indexes, and computers count them from 0, so the first row is row 0.

Picking is everywhere. A film recommender looks up the row of numbers that describes you and picks your best-scoring films. A translator swaps each word for its ID and picks that word’s row from a vocabulary table. Face recognition picks out the block of pixels holding a face, then the closest match among known faces. A chatbot writes one word at a time, scoring every possible next word and picking one. Indexing is one of the most common operations in AI code, and often the first thing a model does with your data.

Small rules, quiet mistakes

Indexing follows precise rules, and some are surprising. Engineers who misread them rarely see an error message. They get answers that look right and are not: a model that learns from the wrong numbers, a correction that silently changes nothing, a step that copies gigabytes of data and burns costly time on GPUs, the specialised chips that run AI. Understanding these rules is understanding how AI handles information.

Each idea below comes in four layers: plain English with an everyday comparison, the precise rule, code for engineers, and why it matters. The code is optional.

A quick testFour questions that fool most people

Before the rules, a test of intuition. We will use a tiny tensor: twelve numbers in three rows of four, numbered 0 to 11 so that every cell is easy to find. The code is written in PyTorch, one of the most popular toolkits for building AI. In it, x is the grid; t is a second tensor made of two such grids, stacked like the pages of a book; and rmask and cmask are lists of yes/no answers (True or False). rmask says yes to rows 0 and 2, cmask says yes to columns 1 and 3, and lists like these are called masks. Square brackets after a name, as in x[...], are how code asks for cells.

>>> import torch
>>> x = torch.arange(12).reshape(3, 4)
>>> x
tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]])
>>> t = torch.arange(24).reshape(2, 3, 4)            # two 3 x 4 layers
>>> rmask = torch.tensor([True, False, True])        # rows 0 and 2
>>> cmask = torch.tensor([False, True, False, True]) # columns 1 and 3

Here is what PyTorch 2.14 answers:

>>> x[[0, 2], [1, 3]]          # 1. not a 2 x 2 block
tensor([ 1, 11])
>>> x[x % 3 == 0]              # 2. not a 3 x 4 grid with holes
tensor([0, 3, 6, 9])
>>> x[rmask][:, cmask]         # 3. the chain builds a grid...
tensor([[ 1,  3],
        [ 9, 11]])
>>> x[rmask, cmask]            #    ...one bracket builds pairs
tensor([ 1, 11])
>>> x[rmask][:, cmask] = 0     # 4. no error...
>>> x                          #    ...and no change
tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]])

In plain English: asking for two rows and two columns returned two numbers, not four. Asking for the multiples of three returned them in a flat line, with the grid gone. Doing the same selection in two steps or in one gave different answers. And the attempt to set cells to zero ran without complaint and changed nothing at all.

None of this is a bug. Every result follows from a handful of rules that PyTorch applies with complete consistency, nearly all of them inherited from NumPy, the library that set the conventions for numerical computing in Python. We will build those rules in eight stops, and by the end all four answers should look obvious. Every snippet on this page was run on PyTorch 2.14, and each one starts from the setup above.

The route

1. Slots: how each position inside the brackets changes the shape
2. Broadcasting: stretching a row or a column to fit
3. Views and copies: a window or a photocopy
4. Lists of positions: the zipper
5. Masks: the highlighter
6. Masks beside other indexes: pairs, never grids
7. Chained brackets: every step renumbers the grid
8. Writing through an index: the lost write

Stop 1 · SlotsEach position in the brackets controls one direction

In plain English

Think of the square brackets as a form with one box per axis. In x[1, 2:4], the first box holds 1 and talks about the rows; the second holds 2:4 and talks about the columns. We will call each box a slot. What goes in a slot decides what happens to that direction of the grid.

A single number, such as 1, means “row 1 only”, and the row direction disappears: we are left with one line of four numbers. A range, called a slice, such as 1:2 (“from row 1 up to, but not including, row 2”), means “this stretch of rows”, and the row direction survives, even when the stretch is a single row. It is the difference between tearing one line out of a spreadsheet and keeping a spreadsheet that happens to have one line. Two special markers complete the set: None adds a new direction of size 1, like slipping a single page into a folder, and ..., the ellipsis, means “every direction I have not mentioned”.

For engineers

>>> x[1]                    # integer: the row axis is gone
tensor([4, 5, 6, 7])
>>> x[1:2]                  # slice: the row axis survives with length 1
tensor([[4, 5, 6, 7]])
>>> x[None].shape           # None: a new leading axis
torch.Size([1, 3, 4])
>>> x[:, None].shape        # ...or a new axis between rows and columns
torch.Size([3, 1, 4])
>>> y = torch.arange(120).reshape(2, 3, 4, 5)
>>> y[0, ..., None].shape   # drop axis 0, keep (3, 4, 5), append 1
torch.Size([3, 4, 5, 1])

The difference between x[1] and x[1:2] is the first thing worth internalising. Both select the same four numbers, but one returns a vector of shape (4,) and the other a matrix of shape (1, 4). The data is identical; the contract is not. Integers are for when we are finished with an axis. Slices are for when that axis still means something downstream.

The edges of the rule are just as consistent. An integer must point at a position that exists, a slice is clamped to whatever exists, and a bracket cannot name more axes than the tensor has:

>>> x[5]
Traceback (most recent call last):
  ...
IndexError: index 5 is out of bounds for dimension 0 with size 3
>>> x[5:9].shape            # past the end: empty, not an error
torch.Size([0, 4])
>>> x[0, 0, 0]
Traceback (most recent call last):
  ...
IndexError: too many indices for tensor of dimension 2

Stop 2 · BroadcastingBroadcasting stretches rows and columns to fit

In plain English

Suppose we want to add a column of three numbers to a row of four. They are different shapes, so PyTorch stretches each one until they fit: the column is repeated across four columns, the row is repeated down three rows, and every cell now holds one number from each. This stretching is called broadcasting. Picture laying the same one-row ruler across every row of a table. It only works when the pieces are compatible: in each direction the sizes must match, or one of them must be 1, so that it can stretch.

There is one catch. Ask PyTorch to put the answer back into the column itself, an in-place update that changes the original instead of making something new, and it refuses: the column has room for three numbers and the answer has twelve. Stretching works on the way in, never on the way back.

For engineers

>>> a = torch.arange(3).reshape(3, 1)   # a column
>>> b = torch.arange(4).reshape(1, 4)   # a row
>>> a + b                               # (3, 1) + (1, 4) gives (3, 4)
tensor([[0, 1, 2, 3],
        [1, 2, 3, 4],
        [2, 3, 4, 5]])
>>> a += b                              # but a must stay (3, 1)
Traceback (most recent call last):
  ...
RuntimeError: output with shape [3, 1] doesn't match the broadcast shape [3, 4]

Nothing is copied to perform the stretch. PyTorch rereads the same values along the stretched axis (internally, with a stride of 0), so broadcasting a column across a million columns costs no extra memory for the input. The output is another matter: a + b really is a new 3 × 4 tensor.

Assignment follows the same one-way rule. The left-hand side fixes the shape, and the right-hand side must broadcast to it. That makes some writes pleasantly compact:

>>> x[[0, 2]] = torch.tensor([100, 200]).reshape(2, 1)  # fills (2, 4)
>>> x
tensor([[100, 100, 100, 100],
        [  4,   5,   6,   7],
        [200, 200, 200, 200]])

Stop 3 · Views and copiesA window or a photocopy: views and copies

In plain English

When we ask for part of a tensor, PyTorch can answer in one of two ways. It can hand us a view: a window onto the original. No numbers move; we simply see a region of the same data, so changing a number through the window changes the original. Or it can hand us a copy: a photocopy of the numbers we asked for, on fresh memory, so writing on it never touches the original.

Which one we get depends on how we asked, not on what we asked for. Single numbers and ranges describe a regular pattern of positions, so they give a window. Lists of positions and masks can name any cells in any order, so they give a photocopy, even when they happen to pick a perfectly regular block. Engineers call the first kind basic indexing and the second advanced indexing.

For engineers

The four slot types from stop 1 share a property: their result can always be described as a starting offset plus a stride per axis into the original memory. So PyTorch moves no data at all. It returns a view, a new tensor header that points into the same storage. Python lists, integer tensors and boolean masks name arbitrary positions, which in general cannot be expressed as strides, so PyTorch gathers the selected values into fresh memory and returns a copy. Here ptr reports the address where a tensor’s memory block starts:

>>> ptr = lambda p: p.untyped_storage().data_ptr()
>>> ptr(x[1:3]) == ptr(x)                    # basic: shares x's memory
True
>>> ptr(x[torch.tensor([1, 2])]) == ptr(x)   # advanced: new memory
False

The difference becomes visible the moment we write through the result:

>>> v = x[1:3]                    # a view
>>> v[0, 0] = 99
>>> x[1, 0]                       # the write went through to x
tensor(99)
>>> c = x[torch.tensor([1, 2])]   # a copy of the same rows
>>> c[0, 0] = -1
>>> x[1, 0]                       # x never sees it
tensor(99)

Stop 4 · The zipperTwo lists of positions zip together

In plain English

Now the first surprise. Give PyTorch a list of rows and a list of columns, and it does not take every combination. It pairs them like the two sides of a zipper: the first row with the first column, the second row with the second column, and so on. Each pair is the address of one cell, its coordinates, and each address yields one number. Rows 0 and 2 zipped with columns 1 and 3 give the cells (0, 1) and (2, 3): two numbers, 1 and 11, not four.

To get every combination, the two lists must be arranged so that they stretch into a grid, one standing as a column and the other lying as a row, using the broadcasting of stop 2.

For engineers

>>> x[[0, 2], [1, 3]]          # coordinates (0, 1) and (2, 3)
tensor([ 1, 11])

The zipper is a special case of a more general rule. All the index tensors in one bracket are first broadcast against each other, exactly as in stop 2, and the broadcast shape becomes the shape of the result. Two lists of shape (2,) broadcast to (2,): two coordinates, two values. To get the grid, we give the indexes shapes that broadcast to a grid, a column of row numbers and a row of column numbers:

>>> x[torch.tensor([[0], [2]]), torch.tensor([1, 3])]  # (2, 1) by (2,)
tensor([[ 1,  3],
        [ 9, 11]])

Stop 5 · MasksA highlighter marks the cells to pick

In plain English

A mask is a grid of yes/no answers that marks the cells to pick, like a highlighter. When you index with a mask, PyTorch reads the highlighted cells in order, row by row, and returns them as a single flat line. The grid’s layout is gone: you get the selected values, not a picture of the original with gaps.

The order matters. PyTorch reads the highlighted cells the way you read a page: left to right along the first row, then the second, and so on. Turn the grid on its side and the same cells come out in a different order, because the rows are different.

For engineers

>>> x[x % 3 == 0]        # flat: the layout is gone
tensor([0, 3, 6, 9])
>>> x.T[x.T % 3 == 0]    # read in x.T's own row order
tensor([0, 9, 6, 3])

Row-major order is a property of the tensor being read, not of the values. Transpose it and the same four numbers come out in a different order, because x.T has four rows of three and is read row by row like everything else.

A mask does not have to cover every axis. A two-dimensional mask over the first two axes of t collapses those two axes into one and leaves the last axis alone:

>>> layrow = torch.tensor([[True, False, True], [False, True, False]])
>>> t[layrow].shape        # 3 trues, each picks a whole row of 4
torch.Size([3, 4])
>>> diag = torch.zeros(3, 4, dtype=torch.bool)
>>> diag[0, 0] = diag[1, 1] = diag[2, 2] = True
>>> t[:, diag].shape       # 3 trues, one per layer
torch.Size([2, 3])

>>> torch.where(x % 3 == 0, x, torch.tensor(-1))
tensor([[ 0, -1, -1,  3],
        [-1, -1,  6, -1],
        [-1,  9, -1, 11]])   # shape preserved

Stop 6 · Masks beside other indexesMasks pair up, they never multiply

In plain English

A mask in a slot turns into the positions of its True values, so a one-dimensional mask becomes a single list of positions. From then on it behaves exactly like the lists of stop 4: it zips with other index tensors, broadcasts against integers, and leaves slices alone.

This is where intuition imported from other tools does the most damage. In a spreadsheet or a data table, selecting “these rows and these columns” usually means every combination: the Cartesian product. PyTorch does not do that. Two masks in one bracket are two lists of positions, and lists zip. They never multiply.

For engineers

>>> x[rmask, cmask]      # two masks: zip, not grid
tensor([ 1, 11])
>>> x[rmask][:, cmask]   # chain: grid
tensor([[ 1,  3],
        [ 9, 11]])

Stop 7 · Chained bracketsEvery step renumbers the grid

In plain English

A chain such as t[:, 1][0] is two separate indexing operations. The first produces a new tensor with its own directions, numbered from 0, and the second bracket indexes that tensor knowing nothing about where its directions came from. A single bracket, by contrast, addresses the original directions all at once. The same characters can therefore land on different directions.

For reading, a chain is legitimate and often the clearest way to say “these rows, then these columns”. For writing, everything depends on what the first bracket returns. A chain of basic indexes is windows all the way down, so a write at the end of it reaches the original.

For engineers

>>> t[:, 1][0]     # chain: first bracket drops rows, so [0] hits layers
tensor([4, 5, 6, 7])
>>> t[:, 1, 0]     # one bracket: 0 hits the column axis
tensor([ 4, 16])

Renumbering also explains the third surprise from the other side. Chaining two masks applies them one after the other, rows first and then the columns of that result, which is how we get the grid that a single bracket refuses to build.

For writing, everything depends on what the first bracket returns. A chain of basic indexes is views all the way down, so a write at the end of it reaches the original:

>>> x[1][0] = 99        # chain of basic indexes: write reaches x
>>> x[1, 0]
tensor(99)

Stop 8 · Writing through an indexThe write that goes nowhere

In plain English

Every indexing form we have met can also appear on the left of an assignment, and the core rule is refreshingly simple. x[idx] = v is a single call that writes into the memory of x itself, whatever kind of index idx is. Masks, lists and index tensors all write in place, and the value broadcasts to the shape of x[idx] under the one-way rule from stop 2.

The traps come from statements that look like one write but are not. There are two of them.

The chained write that goes nowhere

Python evaluates x[rmask][:, cmask] = 0 as x.__getitem__(rmask).__setitem__((slice(None), cmask), 0). The getitem builds a copy (stop 3), the setitem dutifully writes zeros into that copy, and the copy, referenced by nothing, is thrown away:

>>> x[rmask][:, cmask] = 0      # no error, no warning
>>> x
tensor([[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]])

The fix is to put the whole selection into one bracket, so that the only call is a setitem on x. Two masks would zip (stop 6), so we either convert them to position lists shaped into a grid (stop 4) or combine them into one two-dimensional mask:

>>> x[rmask.nonzero().squeeze(1)[:,None], cmask.nonzero().squeeze(1)[None,:]] = 0
>>> x
tensor([[ 0,  0,  2,  0],
        [ 4,  5,  6,  7],
        [ 8,  0, 10,  0]])
>>> x2 = torch.arange(12).reshape(3, 4)
>>> x2[rmask[:, None] & cmask[None, :]] = 0   # one 2-D mask
>>> torch.equal(x, x2)
True

Duplicate positions are written once

b[idx] += 1 expands to three steps: read b[idx], which gathers a copy; add 1 to the copy; and write the copy back with b[idx] = .... If idx repeats a position, the increments never accumulate. index_put_ with accumulate=True, or its siblings index_add_ and scatter_add_, adds every contribution:

>>> b = torch.zeros(3)
>>> b[torch.tensor([0, 0, 1])] += 1
>>> b
tensor([1., 1., 0.])
>>> b = torch.zeros(3)
>>> idx = torch.tensor([0, 0, 1])
>>> b.index_put_((idx,), torch.ones(3), accumulate=True)
tensor([2., 1., 0.])

Without accumulation, PyTorch’s documentation is explicit that writing to duplicate positions gives an undefined result. Our example only looks predictable because every write carries the same value.

Putting it togetherThree sentences that explain every result

Eight stops reduce to three sentences. Engineers can carry them in this compact form:

Indexes become coordinates, masks become true positions, position lists broadcast together.

In plain English:

Indexes become coordinates. Everything inside the brackets is an instruction for one direction of the grid, and together the instructions spell out addresses. Single numbers and ranges describe a regular pattern, so they give a window onto the original; anything that lists positions gives a photocopy. A new set of brackets means a new tensor, with its directions numbered afresh.

Masks become true positions. A highlighter is shorthand for the addresses of the highlighted (True) cells, read like lines on a page. That is why the multiples of three came back as a flat line, and why the grid turned on its side gave a different order.

Position lists broadcast together. All the lists of positions in one set of brackets are stretched to a common shape and read in step, one address per position: the zipper. That is why two lists gave two numbers, why two masks gave pairs, and why arranging the lists as a column and a row produces a grid.

The fourth surprise needs one more fact, about Python rather than PyTorch: an assignment writes into whatever the final brackets are attached to. If an earlier step made a photocopy, that is where the write goes, and it is thrown away with the copy.

The field card

Before writing the bracket

• Want every combination of some rows and some columns? Arrange the two position lists as a column and a row so they stretch into a grid: rows[:, None] and cols[None, :].
• Want to keep the grid’s layout while using a mask? Use torch.where(mask, x, fill), which blanks cells instead of removing them.
• Changing values? Use a single set of brackets on the tensor you mean to change. In a chain, any list or mask before the last bracket sends the write to a copy.
• Might positions repeat? Use index_put_(..., accumulate=True), index_add_ or scatter_add_ so that every update counts.
• Merging two sets of brackets into one? Recount the directions first.
• Keeping a small piece of something large? Call .clone(), or the whole original stays in memory.

None of this requires memorising special cases. It requires reading brackets the way PyTorch reads them: one direction at a time, one list of positions at a time, one result at a time. Once that habit forms, the four answers at the top of this page stop looking like surprises and start looking like the only possible answers. And the next time a recommendation lands just right or a translation reads naturally, it is worth remembering how much of that depended on picking exactly the right numbers.

Pocket glossary

Further reading for engineers

• NumPy: Indexing on ndarrays, the rulebook PyTorch follows.
• PyTorch: Broadcasting semantics.
• PyTorch: Tensor views, the list of operations that return views.
• PyTorch: torch.Tensor.index_put_, on accumulation and duplicate indices.