Course Classroom

Using Transformers — Batch two maintenance notes without changing their meaning

Lesson summary

A dispatcher adds a long inspection note to a batch containing one short note. The labels for the short note should remain stable within normal numerical tolerance. How can you check whether padding changed the input that the model actually sees?

Lesson 2 / 2 · 4 min · Beginner · English

Learning route

Model configuration and weights, tokenization, subwords, padding, masks and batched inference.

The course follows a sentiment pipeline through tokenization, model inference and decoding of the class scores. It then combines sequences of different lengths. A tensor must be rectangular, so shorter rows gain padding. The attention mask tells the model which positions carry input and which positions contain padding.

Work through the case

  1. Use two invented sequences of token identifiers, already including their boundary tokens. The short sequence is [11, 12, 13] and the long one is [21, 22, 23, 24, 25]. Reserve zero for padding in this exercise. Real tokenizers define their own padding identifier.

  2. Right-pad both rows to five positions. The short row gains two zeros. Give its first three positions a mask of one and its last two positions a mask of zero. The sum of the mask must equal three. For the long note the sum must equal five.

  3. With a real encoder, load the tokenizer and model from the same checkpoint, set evaluation mode and compare the short note alone with the same note inside the batch. Pass the entire tokenizer output into the model. Compare logits with a tolerance and examine the predicted label separately.

Five identifiers and a mask that retains the first three positions.

Original GuideWit diagram. Five identifiers and a mask that retains the first three positions.

Worked example

Input IDs

Attention mask

Visible tokens

[11, 12, 13]

[1, 1, 1, 0, 0]

3

[21, 22, 23, 24, 25]

[1, 1, 1, 1, 1]

5

Try it with Python 3

rows = [[11, 12, 13], [21, 22, 23, 24, 25]]
width = max(map(len, rows))
for row in rows:
    pad = width - len(row)
    ids = row + [0] * pad
    mask = [1] * len(row) + [0] * pad
    print(ids, mask, sum(mask))

Expected output

[11, 12, 13, 0, 0] [1, 1, 1, 0, 0] 3
[21, 22, 23, 24, 25] [1, 1, 1, 1, 1] 5
Python code and its actual output. The mask sums are three and five, although both padded rows have five positions.

GuideWit local Python experiment. The mask sums are three and five, although both padded rows have five positions.

If padding is correct but a critical word disappears, inspect truncation. An attention mask cannot recover words removed before inference. Test with a note that places the fault description after a long equipment history and examine the decoded retained tokens.

Troubleshooting

Symptom

Check

A short row has five visible positions.

Inspect the attention mask. The padding positions need zeros in this example.

The fault at the end never appears in the tokens.

Inspect the maximum input length and truncation policy before changing the model.

Check your understanding

  1. How many padding positions appear across these two rows?
  2. Can a matching predicted label prove both runs used the same input?

Answer key

  1. Two. Only the shorter row requires padding.
  2. No. Different logits can have the same winning class. Check the tokens, masks and logits.

Glossary

Logit — A raw class score before conversion into a probability distribution.

Sources and chapter units

The case, Python check and diagram are original teaching examples. The chapter sources are listed below.