An experimental end-to-end on-device LLM platform for constrained Android devices.
Can a constrained Android device support an end-to-end local LLM lifecycle, from parameter-efficient fine-tuning through deployment and resource-aware inference?
MobileFineTuner → LoRA fine-tuning → checkpoint creation → native model merge/export → native C++ SafeTensors → GGUF conversion → llama.cpp → Android / Termux inference
Training plane for native C++ on-device model adaptation.
Model artifact validation, checkpoint inspection, merging/export, and native GGUF conversion.
Inference/runtime plane for persistent and resource-aware mobile LLM execution.
GPT-2 Small:
- 12 transformer layers
- 768 hidden dimension
- 12 attention heads
- 124M parameters
- LoRA rank 8
- WikiText-2 fine-tuning
- Native F32 GGUF deployment
- llama.cpp inference on Android / Termux
Observed generation performance on the test Android environment was approximately 10–12 tokens/second for F32 GPT-2 Small.
v0.1 is an experimental research prototype.
The current release demonstrates end-to-end execution but does not yet claim numerical equivalence against a reference GGUF conversion.
This project builds on and integrates existing work including:
- MobileFineTuner
- llama.cpp
- GPT-2
- LoRA
- SafeTensors
- GGUF
- WikiText-2
- prior research on mobile LLM inference and memory management
This repository does not claim ownership of those underlying technologies.
Apache-2.0