Inference engine
Custom graph executor with operator fusion and paged KV-cache reuse; unified scheduling across ARM CPU, GPU and NPU backends.
GUANGZHOU HUYU TECHNOLOGY CO., LTD.
HUYU builds on-device LLM deployment for mobile. With our own inference engine, extreme quantization and mobile agent framework, large models run offline on ordinary phones — no network, no cloud, no data leaving the device.
Hu (戽) is an ancient tool for lifting water — scoop by scoop, farmers carried water uphill into the fields.
A frontier model in the cloud is like a great river: abundant, but behind a dam of latency, traffic cost, data residency and privacy rules. We build the scoop that carries that intelligence out of the river and into every device in hand.
The fish in our name is the device itself: small, fast, agile — swimming in its own waters. HUYU brings the water of models to the palm.
From operators and weights to memory and scheduling — closing the gap between large models and small phones, layer by layer.
Custom graph executor with operator fusion and paged KV-cache reuse; unified scheduling across ARM CPU, GPU and NPU backends.
2/3/4-bit mixed precision with group-wise quantization; weights packed offline, dequantized at runtime — balancing size, speed and accuracy.
On-demand weight paging under a hard memory budget; temperature- and battery-aware dynamic scaling for long conversations.
Text, vision and speech encoders run together on the device — photo Q&A, screenshot understanding and offline voice interaction.
Local tool calling, structured output and device-capability orchestration, with on-device retrieval for private knowledge bases.
Chunked model packages with resumable downloads, per-device rollout, hot updates and rollback — shipped inside your app.
On-device inference SDK for Android and iOS: streaming generation, multi-turn chat, structured output and multimodal APIs — a few lines of code.
A complete pipeline from raw weights to distributable model packages: quantization calibration, operator adaptation, size estimation and device compatibility testing.
An agent framework that runs inside the device: intent routing, tool calling, local knowledge base and memory — entirely offline.
A system-level AI assistant that keeps working in airplane mode, weak coverage and underground garages.
Clinical notes, legal and financial documents never leave the device — data minimization by design.
Glasses, earbuds, car systems and robots gain language and vision understanding at low power.
Offline manual Q&A, equipment fault diagnosis and voice logging where there is no network.
GUANGZHOU HUYU TECHNOLOGY CO., LTD. is based in Guangzhou, focused on on-device LLM deployment and mobile AI application development. We believe the default form of the next generation of intelligent applications is not in the cloud, but in the user's pocket.
Our work centres on three lines — inference engines, model compression and on-device agents — giving handset makers, enterprises and developers everything from model adaptation to shipped applications.
| Company | 广州戽鱼科技有限公司 |
|---|---|
| English name | GUANGZHOU HUYU TECHNOLOGY CO., LTD. |
| Business | On-device LLM deployment / mobile AI applications |
| Brand | HUYU |
| Location | Guangzhou, China |
| Website | localai.mobi |
Technology partnerships, SDK trials and custom deployment — write to us.
xielaobanxxx@gmail.com