Deep Learning for NLP:
from the Perceptron to the Transformer
Build the architecture behind modern language models from scratch, with full mathematical rigour.
Prerequisites
This course is mathematically rigorous. Before you start, you'll need:
- Intermediate PythonFunctions, classes, basic NumPy.
- Linear algebraVectors, matrices, matrix multiplication.
- CalculusPartial derivatives and the chain rule.
Syllabus
5 blocks · 43 lessons · ~18h reading01NLP FundamentalsTokenization, vocabularies and the first vector representations of text.8 lessons · 2h 48m
- El problema de representar el lenguaje8 min
- Tokenización: palabras, caracteres, subpalabras20 min
- Vocabulario, frecuencia y el problema OOV20 min
- One-hot encoding y la maldición de la dimensionalidad22 min
- Bolsa de palabras y TF-IDF24 min
- Representaciones densas: la idea central23 min
- Word2Vec: skip-gram y CBOW26 min
- GloVe y los límites de los embeddings estáticos25 min
02The Multilayer PerceptronFrom the perceptron to a multilayer network trained with gradient descent and backpropagation.10 lessons · 4h 50m
- La neurona artificial22 min
- Funciones de activación y no linealidad30 min
- XOR: por qué necesitamos capas ocultas25 min
- El forward pass en forma matricial30 min
- Funciones de pérdida: MSE y entropía cruzada30 min
- Descenso de gradiente30 min
- La regla de la cadena, en serio30 min
- Backpropagation: la derivación completa35 min
- Implementar un MLP desde cero30 min
- Proyecto: un clasificador de sentimiento28 min
03Recurrent Neural NetworksSequences, memory and the problem of the vanishing gradient.8 lessons · 3h 37m
04The Bridge to AttentionThe context bottleneck and the alignment that solves it.6 lessons · 2h 41m
05The TransformerSelf-attention, multiple heads and positional encoding.11 lessons · 4h 45m
- Adiós a la recurrencia22 min
- Auto-atención: la secuencia se mira a sí misma29 min
- La raíz que faltaba en la atención28 min
- Varias cabezas, varios repartos29 min
- Cada posición, un puñado de relojes29 min
- Apilar sin perder lo de abajo28 min
- Escribir no es leer: el decoder y su máscara28 min
- Quince cajas, cinco números24 min
- Proyecto: un Transformer desde cero30 min
- BERT y GPT: una columna cada uno20 min
- Fine-tuning en la práctica18 min
Your instructor
Forged between code and equations
Graduated in Computer Science with a Master's in Mathematics and Computer Science from the University of Cantabria. After several years as a software developer, I spent five years in research and university teaching before working independently, combining education and consulting.
Frequently asked questions
How much does it cost?
Nothing. The course is completely free.
What do I need to know before starting?
Intermediate Python, linear algebra (vectors and matrices) and calculus (partial derivatives and the chain rule). They're listed in the prerequisites section.
How long will it take?
Around 40 hours if you do the exercises. Go at your own pace; there are no deadlines.
What do I need to install?
Nothing. Everything — the Python code included — runs directly in your browser.