LLM Explorer

Table of Contents

All 85+ interactive visualizations for exploring modern Large Language Models

1 Transformer 2 Architectures 3 Reasoning 4 Memory 5 Prompting 6 Training 7 Trends 8 Tools

1

Transformer Basics

From Tokens to Attention: The building blocks of modern LLMs

15 Pages

1.1 – 1.3 Input Processing

1.1 BPE Tokenization Animation 1.2 Embedding Space 2D Interactive 1.3a Sinusoidal Position Encoding Interactive 1.3b Position Encoding Matrix Interactive

1.4 – 1.5 Self-Attention

1.4 Q·K Matrix Multiplication Animation 1.5a Multi-Head Attention Interactive 1.5b Head Concatenation Animation

1.6 – 1.7 FFN & Residuals

1.6a Activation Functions Interactive 1.6b FFN as Memory Interactive 1.6c SwiGLU Gate Interactive 1.7a Residual & LayerNorm Animation 1.7b Gradient Flow Animation

1.8 – 1.9 Complete Picture

1.8a Transformer Block Scrollytelling 1.8b LayerNorm Live Interactive 1.9 Architecture Comparison Comparison

→ Chapter Summary: Transformer Basics → Modern Architecture Variants

2

Modern Architecture Variants

MoE, GQA, Flash Attention and Multimodal

11 Pages

2.1 Mixture of Experts

2.1a MoE Router Simulation Interactive 2.1b Load Balancing Problem Interactive

2.2 – 2.3 Attention Optimizations

2.2 Grouped Query Attention Interactive 2.3 Flash Attention Animation

2.4 Sparse Attention

2.4a Sparse Attention Heatmap Interactive 2.4b Lightning Indexer Animation 2.4c Sparsity Slider Interactive 2.4d Dense vs. Sparse Retrieval Comparison

2.5 Native Multimodal

2.5a Native Multimodal Interactive 2.5b Early Fusion Animation Animation 2.5c Interleaved Context Interactive

→ Chapter Summary: Modern Architecture Variants → Reasoning & Test-Time Compute

3

Reasoning & Test-Time Compute

Chain-of-Thought, o1/o3, DeepSeek R1, and Flexible Inference

11 Pages

3.1 Chain-of-Thought

3.1a Chain-of-Thought Demo Demo 3.1b Model Size vs. CoT Interactive

3.2 – 3.3 Reasoning Models

3.2 o1/o3 Hidden Reasoning Interactive 3.3 DeepSeek R1 & GRPO Interactive

3.4 Test-Time Compute

3.4a Compute Allocation Interactive 3.4b Test-Time Scaling Interactive 3.4c Scaling Strategies Comparison

3.6 Flexible Inference

3.6a Flexible Inference Interactive 3.6b Effort Parameter (Claude) Demo 3.6c Thinking Budget (Qwen3) Interactive 3.6d Dual-Mode Comparison Comparison

→ Chapter Summary: Reasoning & Test-Time Compute → Optimizations & Memory

4

Optimizations & Memory

KV-Cache, Positional Encodings, and Context Extension

12 Pages

4.1 KV-Cache

4.1a KV-Cache Animation Animation 4.1b KV-Cache Calculator Calculator 4.1c Memory Growth Chart Interactive

4.2 Positional Encodings

4.2a RoPE vs. ALiBi Comparison 4.2b RoPE Rotation Animation 4.2c ALiBi Bias Heatmap Interactive 4.2d Sliding Window Attention Interactive 4.2e Context Extrapolation Test Interactive

4.3 – 4.4 Memory Management

4.3 Ring Topology Animation Animation 4.4a Paged Attention Animation 4.4b RAG Pipeline Interactive 4.4c Embedding Retrieval 2D Interactive

→ Chapter Summary: Optimizations & Memory → In-Context Learning & Prompting

5

In-Context Learning & Prompting

ICL, System Prompts, and Few-Shot Learning

10 Pages

5.1 In-Context Learning

5.1 In-Context Learning Demo Demo

5.2 System Prompts

5.2a System Prompt Attention Interactive 5.2b Attention Analysis Interactive 5.2c System Prompts Demo Demo

5.3 Attention Patterns

5.3a Lost-in-the-Middle Interactive 5.3b Attention Distribution Interactive 5.3c Attention Heatmap Interactive

5.4 Few-Shot Learning

5.4a Format vs. Content Interactive 5.4b N-Shot Plateau Scaling Interactive 5.4c Few-Shot Learning Demo Demo

→ Chapter Summary: In-Context Learning & Prompting → Training & Inference

6

Training & Inference

RLHF, Sampling, Quantization and Speculative Decoding

9 Pages

6.1 Training

6.1 Training Loss Curves Interactive

6.2 RLHF & DPO

6.2a RLHF Training Interactive 6.2b RLHF vs. DPO Comparison 6.2c RLHF Pipeline Scrollytelling

6.3 Sampling

6.3a Sampling Settings Guide Interactive 6.3b Top-K & Top-P Interactive

6.4 Inference Optimization

6.4a Quantization Comparison Comparison 6.4b Speculative Decoding Animation

→ Chapter Summary: Training & Inference → Trends & Future

7

Trends & Future

Benchmarks, Emergence and Attention Scaling

4 Pages

7.1 – 7.3 Developments

7.1 Benchmark Evolution Timeline Interactive 7.2a Emergence Timeline Animation 7.2b DeepSeek Emergence Interactive 7.3 Attention Complexity Scaling Interactive

→ Chapter Summary: Trends & Future → Tools & Glossary

8

Tools & Glossary

Calculators, References and Lookup Tools

15 Pages

8.1 Glossary & Basics

8.1a Glossary Reference 8.1b Causal Mask Interactive 8.1c Induction Heads Interactive 8.1d Data Composition Interactive 8.1e Chinchilla Scaling Laws Interactive 8.1f Parameter vs. Compute Interactive

8.2 Vocabulary & Embeddings

8.2a Vocabulary Explorer Interactive 8.2b Word Analogies Demo 8.2c Embedding Lookup Interactive 8.2d Next-Token Prediction Demo

8.3 Model Database

8.3a Parameter Calculator Calculator 8.3b Model Database Reference

8.4 – 8.5 Research & Tools

8.4a Research Timeline Interactive 8.4b Benchmark Explorer Interactive 8.5a Inference Cost Calculator Calculator 8.5b Token Economy Calculator Calculator