← sources

Senior/Lead AI engineers: what portfolio project actually makes you say "this person knows production"?

source: reddit (reddit:AI_Agents)published 2026-09-23 10:29ingested 2026-09-24 02:36

retained text

Everyone and their mom has a portfolio full of "Chat with your PDF" Streamlit apps or basic LangChain wrappers. Obviously, that stuff isn't getting anyone hired anymore.



I'm aiming for AI Engineer / Applied ML roles and want to spend the next 2–3 months building one solid, end-to-end system from scratch instead of 4 toy projects. The goal is to prove actual engineering discipline—handling failure states, latency, cost, and reliability—rather than just gluing APIs together.



I'm bouncing between three directions and want a sanity check on what's most respected right now:



1. **A proper LLM routing/gateway service:** Semantic caching (Redis), token budget management, fallback routing across cheap small models vs frontier models depending on query complexity, and full observability (Langfuse / OpenTelemetry) to track cost and latency.

2. **A deterministic agent system with actual guardrails:** Multi-step workflow with a strict state machine (LangGraph or custom), tool calling with Pydantic validation, human-in-the-loop fallback, and an automated eval harness (DeepEval/Ragas) running synthetic test cases on PRs.

3. **Local inference microservice + fine-tuning:** QLoRA fine-tuning a small open model (e.g. Llama/Qwen) on a niche domain task, deployed via vLLM with quantized weights, backed by latency/throughput benchmarking against a baseline.



If you review candidates or run a team:



\- Which of these feels closest to the actual headaches you deal with day-to-day?

\- When you open someone’s GitHub repo, what's an immediate green flag that screams "this person understands production" versus "they copied a tutorial"? (Is it eval pipelines, Docker/CI setup, integration tests, error handling?)

\- What are the common gaps you keep seeing people miss?



Roast the ideas if they're off the mark—just want to make sure I build something that genuinely moves the needle.

document fetched 2026-09-24 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1wo286p/seniorlead_ai_engineers_what_portfolio_project