Hi there! While this looks cool, we don't feature links that are solely to github repos or crates.io/ pages (here are our guidelines github.com/rust-lang/this-we…)
Would you be open to writing up on how you created this and what you learned about Rust in the process?
I ported nanochat to Rust:
Model Architecture
Multi-layer Transformer: N stacked decoder blocks with pre-norm residual connections
Rotary Position Embeddings (RoPE): Replaces learned positional encodings with rotary embeddings for better length generalization
Multi-Query Attention (MQA): Reduces KV cache size by sharing key/value heads across query heads
RMSNorm: Parameter-free normalization for stability (instead of LayerNorm)
QK-norm: Normalizes queries and keys before attention to prevent numerical instability
ReLU² MLP: Uses ReLU(x)² activation for better gradient flow on GPUs
Softcap Logits: Bounds output logits using tanh(x/15)*15 to prevent extreme values
#LLMs#llmstackgithub.com/tuned-org-uk/nano…
Hi there! While this looks cool, we don't feature links that are solely to github repos or crates.io pages (here are our guidelines github.com/rust-lang/this-we…)
Would you be open to writing up on how you created this and what you learned about Rust in the process?
Hi there! This is certainly cool and we would love to feature it, but we need a little more Rust specific content. Would you be open to writing up how you created this game - assuming it's written in Rust?
Thanks for submitting this, apologies that I missed it last week. Could you consider opening a pull request with this link to make sure that we see it? github.com/rust-lang/this-we…