Every LLM is 8 layers of ideas stacked on top of each other which you can implement on tensortonic:
Layer 1: Gradients
Write a tiny autograd engine. Value nodes, expression graphs, a neuron.
loss.backward() stops being a black box.
Layer 2: Tokens
Train BPE yourself and apply the merges by hand.
Layer 3: The block
RMSNorm, SwiGLU, RoPE, causal grouped-query attention.
Layer 4: Training at scale
FLOP estimates, mixed-precision memory, gradient accumulation.
Layer 5: Attention variants
MHA → MQA → GQA → MLA, then FlashAttention.
Layer 6: Kernels
Softmax, GELU and reductions in CUDA, then again in Triton.
Layer 7: Fine-tuning
QLoRA on Qwen3-4B with real medical Q&A.
Layer 8: Serving
Build your own Qwen3 inference server.
All 8 are graded, runnable problems on TensorTonic.
link in comments.