Lecture 11: Prompting and Decoding | CMSC 25700/35100 NLP

Chain-of-Thought Prompting

Ask the model to reason step by step before giving an answer.

Wei et al. (2022)

Zero-Shot Chain-of-Thought

Simply add "Let's think step by step" to the prompt.

No need to provide worked examples.

Kojima et al. (2022)

Zero-Shot Chain-of-Thought

Kojima et al. (2022)

Beyond Prompting

Greedy: When It Goes Wrong

Open-ended generation can produce repetition loops:

Example from The Curious Case of Neural Text Degeneration by Holtzman et al.

But note: for modern aligned models, greedy works much better than it used to.

Top-k and Nucleus (Top-p) Sampling

## Beam Search: Example ($B = 2$)

**Step 1**: Start with `<s>`, compute $P(w_1)$
                        - Candidates: "The" (0.4), "A" (0.3), "It" (0.15), ...
                        - Keep top 2: **"The"** (0.4), **"A"** (0.3)

**Step 2**: Expand each beam, compute $P(w_2 | w_1)$
                        - From "The": "The cat" (0.4 x 0.3 = 0.12), "The dog" (0.4 x 0.25 = 0.10)
                        - From "A": "A big" (0.3 x 0.3 = 0.09), "A small" (0.3 x 0.2 = 0.06)
                        - All 4 candidates: 0.12, 0.10, 0.09, 0.06
                        - Keep top 2: **"The cat"** (0.12), **"The dog"** (0.10)

**Step 3**: Expand again, compute $P(w_3 | w_1, w_2)$
                        - From "The cat": "The cat sat" (0.036), "The cat is" (0.024)
                        - From "The dog": "The dog ran" (0.030), "The dog is" (0.020)
                        - Keep top 2: **"The cat sat"** (0.036), **"The dog ran"** (0.030)

Note: greedy would pick "The" then "cat" then "sat" and miss "The dog ran" entirely.

Chain-of-Thought Prompting

Zero-Shot Chain-of-Thought

Zero-Shot Chain-of-Thought

Beyond Prompting

Greedy: When It Goes Wrong

Top-k and Nucleus (Top-p) Sampling

Beam Search: Problems