Infrastructure architect. Android, Termux, local LLMs (llama.cpp), Linux and Windows systems. Builds, debugging, performance tuning. DMs open.
Croatia
- Tweets 241
- Following 58
- Followers 32
- Likes 49
Running a local LLM with llama.cpp and it feels slow? Check these 3 first:
-t set threads to your physical cores, not logical. more threads often makes it slower
-ngl offload layers to the GPU. even partial offload helps a lot if VRAM is tight
-c don't max the context unless you need it. the KV cache eats RAM fast
Most people should start with Q4_K_M, it's the sweet spot between quality and size.
On a phone it throttles? Check temps and battery over adb:
adb shell "cat /sys/class/thermal/thermal_zone*/temp; dumpsys battery"
CPU, GPU, clocks, thermals, battery drain, ask me anything below.
#LocalLLM #llamacpp #AndroidApps
1
1
95
Private AI on my phone, fully offline.
Qwen3 4B on a Samsung Xclipse 960 GPU (Vulkan) through llama.cpp, running inside my own app Meridian:
232 t/s prompt, 23 t/s generation.
Nullcloud: free, open-source summarizer for Termux. Share any page and get a summary, no cloud.
github.com/kukirin-ui/Nullcl…
Want it set up on your phone? DM me.
#LocalLLM #Termux #llamacpp
1
3
327
Bonus test: Llama 3.2 3B on the same phone, GPU-only via Vulkan (Samsung Xclipse 960) in Termux.
Prompt: 314 t/s
Generation: 28.9 t/s
It's faster than Qwen3 4B (232 / 23 t/s) because it's a smaller model, 3.2B vs 4B parameters. Qwen is still my pick for smarter answers and code, but it's really nice to see a phone push 300+ t/s, fully offline.
2
139
Termux tip that saves 40 minutes of build errors: never pip install --upgrade pip, and don't let pip compile numpy. Use pkg install python-pip python-numpy python-psutil instead.
3
104
Demis Hassabis built Gemini to master native video, giving it a massive edge over text-only AI. Rivals focusing on raw speed face a huge disadvantage soon. This world model will completely reshape your creative studio and workplace forever.
Full story: blog.google
1
85
New game unlocked: Termux, but you don't have to remember the commands
#anthropic #groq #grok #openrouter #gemini #openai #xAi #termux
1
1
136