Ferro - A lightweight Inference engine for Intel N100
retained text
Hi everyone, since most generic engines do not target this type of hardware i though I could make a small inference engine for budget/low power NAS builds with this SoC. It may not be blazing fast but it can definitely deliver for 24/7 applications ! What we run today: LFM2.5-D1-omni a decision model at about 500tok/s LFM2.5-1.2B Thinking and LFM2.5-350M at \~15tok/s and 35tok/s respectively Embedding Gemma 2 - about 800tok/s You can see more performance figures in the github repo: [https://github.com/paragontasx/ferro](https://github.com/paragontasx/ferro) Some idea on how you could use this and why i chose the above models: EG2 - multimodal embedding text video images audio all in one place D1 - Small decision model for very fast decision making Chat Models - In case you want interaction with the system. Possible use cases: 1. **Camera event verification fully local.** 2. **Search everything by describing it.** Because EG2 puts text, photos, CCTV clips and audio into the same vector space, one query like “white van in the yard at night” or “the dog barking” searches all of them at once. 3. **Find sensitive data in file shares.** 4. **Clean up photo libraries.** d1 sorts each photo as person, pet, receipt, screenshot, document or scenery. Receipts go to your document intake, screenshots get moved out of the camera roll, and blurry duplicates get flagged by EG2 similarity plus a d1 “is this the better shot?” check. 5. **Voice commands for Home Assistant without speech recognition.** 6. **A complete local loop in one process: retrieve, decide, write.** EG2 retrieves, d1 judges, and LFM2.5 writes. For example, a morning digest: overnight camera events, log triage and backup results, each filtered by d1, then summarised in a short paragraph by the 350M model. Speed doesn’t matter here because it runs as a batch job. 7. **Explain only what was flagged.** d1 triages every log line or alert, and LFM2.5 writes a one-line explanation only for the few that score high. You get human-readable alerts without paying generation cost on every event. Fun fact this was developed mostly by a locally hosted Qwen3.8 Flash Next with major assistance from Sonnet 5.5 using Hermes Agent.
document fetched 2026-10-22 00:39 · canonical https://reddit.com/r/homelab/comments/1xbkom8/ferro_a_lightweight_inference_engine_for_intel