Examples of NLP over time

Increase Decrease

Studying the History of Ideas Using Topic Models
David Hall, Daniel Jurafsky, Christopher D. Manning

Examples of NLP over time: Parsing

Parsing

Examples of NLP over time: Question Answering

Once there was a boy named Fritz who loved to draw. He drew everything. In the morning, he drew a picture of his cereal with milk. His papa said, "Don't draw your cereal. Eat it!" After school, Fritz drew a picture of his bicycle. His uncle said, "Don't draw your bicycle. Ride it!" ...

What did Fritz draw first?

A) the toothpaste
B) his mama
C) cereal and milk
D) his bicycle

Example from MCTest (Richardson et al., 2013)

Examples of NLP over time: Modern benchmarks

## NLP has gone a long way!
## The first goal of this course is to understand the fundamentals and building blocks towards these advances.

Survey Results: Python Experience

Survey results for Python experience

Survey Results: NLP Knowledge

Survey results for NLP knowledge

Roadmap Comparison

Our Roadmap

  1. Introduction & Word embeddings
  2. Text Classification & Language modeling
  3. Overall recipe & Attention
  4. Transformers & Modern NLP pipeline
  5. Benchmark & evaluation & midterm
  6. Decoding LLMs & Prompting
  7. Post-training, Agents & reasoning
  8. Research in NLP
  9. Alignment & safety & presentations

Generated by ChatGPT

  1. Introduction to NLP
  2. Text Preprocessing & Tokenization
  3. Language Modeling (n-grams)
  4. Word Embeddings (Word2Vec, GloVe)
  5. Sequence Models (RNNs, LSTMs)
  6. Neural Networks & Deep Learning
  7. Attention & Transformer Models
  8. Pretrained Models (BERT, GPT)
  9. Information Extraction & NER
  10. Machine Translation & Seq2Seq
  11. Text Generation & Summarization
  12. Advanced Topics / Projects