Deep Learning for NLP:
from the Perceptron to the Transformer

Build the architecture behind modern language models from scratch, with full mathematical rigour.

Level: intermediate43 lessons

Start the course View the syllabus

01 — Before you start

Prerequisites

This course is mathematically rigorous. Before you start, you'll need:

Intermediate Python
Functions, classes, basic NumPy.
Linear algebra
Vectors, matrices, matrix multiplication.
Calculus
Partial derivatives and the chain rule.

02 — Content

Syllabus

5 blocks · 43 lessons · ~18h reading

01NLP FundamentalsTokenisation, vocabularies and the first vector representations of text.8 lessons · 2h 44m

02The Multilayer PerceptronFrom the perceptron to a multilayer network trained with gradient descent and backpropagation.10 lessons · 4h 50m

03Recurrent Neural NetworksSequences, memory and the problem of the vanishing gradient.8 lessons · 3h 39m

04The Bridge to AttentionThe context bottleneck and the alignment that solves it.6 lessons · 2h 42m

05The TransformerSelf-attention, multiple heads and positional encoding.11 lessons · 4h 45m

03 — About Gustavo

Your instructor

Gustavo Torres Guerrero

Holds a B.S. in Computer Science and a Master's in Mathematics and Computing from the University of Cantabria, Spain. Following four years as a software developer and five years in university teaching and research, I have spent over seven years working as an independent consultant and instructor, specializing in programming, mathematics, and artificial intelligence.

Author's note

This course was built with the help of AI tools, which I used to review, correct and polish the writing of my notes and materials, greatly speeding up the process. The content, the approach and the structure are guided by me, and every explanation, paragraph and equation has been carefully reviewed to ensure its clarity, coherence and reliability. Many lessons also include interactive explorables that reinforce the learning. I hope you find it useful. And if you ever get stuck or want to move faster, book a session with me: I'll be glad to help.

04 — Frequently asked questions

Frequently asked questions

How much does it cost?

Nothing. The course is completely free.

What do I need to know before starting?

Intermediate Python, linear algebra (vectors and matrices) and calculus (partial derivatives and the chain rule). They're listed in the prerequisites section.

How long will it take?

Around 40 hours if you do the exercises. Go at your own pace; there are no deadlines.

What do I need to install?

Nothing. Everything — the Python code included — runs directly in your browser.

Start today

Ready to build a Transformer?

Start with the first lesson. You don't need an account to read.

Start the course

Deep Learning for NLP:from the Perceptron to the Transformer