NLP and LLMs - Transformers
About
Transformer architecture has enabled AI tools such as ChatGPT, Google Search, and Google Translate. In this module we will dive how the transformer architecture works and apply select models from the transformer family to solve a real-world problem.
Learning Objectives
You will be able to:
- Explain how probabilistic language models generate text.
- Describe transformer architecture: encoders, decoders, and encoder-decoders.
- Evaluate which architecture to use for common NLP business applications.
- Apply enocoder architecture to a Named Entity Recognition (NER) task.
- Apply decoder architecture to a text generation task.
- Apply enocoder-decoder (seq2seq) architecture to a translation and summarization task.
Content
| Topic | About |
|---|---|
| Full Lesson Deck | Intro to LLMs and transformers |
| encoder-ner | Encoder example on NER task |
| decoders-generation | Explore text generation |
| seq2seq-translation-summarization | Explore translations and summarization |
Prerequisites:
- Python
- Core data science libraries
numpyandpandas - Understand, run, and evaluate classification metrics in
scikit-learn - Comfort with Hugging Face Pipeline API for NLP tasks.
References
📖 Reference Materials
Important note
Please ignore the _layouts folder and the config.yml file in the root of this repo. They are needed to make sure the repo renders well; please don’t change them or delete them and your repo will work fine :)