Senior/Lead AI engineers: what portfolio project actually makes you say "this person knows production"?
retained text
Everyone and their mom has a portfolio full of "Chat with your PDF" Streamlit apps or basic LangChain wrappers. Obviously, that stuff isn't getting anyone hired anymore. I'm aiming for AI Engineer / Applied ML roles and want to spend the next 2–3 months building one solid, end-to-end system from scratch instead of 4 toy projects. The goal is to prove actual engineering discipline—handling failure states, latency, cost, and reliability—rather than just gluing APIs together. I'm bouncing between three directions and want a sanity check on what's most respected right now: 1. **A proper LLM routing/gateway service:** Semantic caching (Redis), token budget management, fallback routing across cheap small models vs frontier models depending on query complexity, and full observability (Langfuse / OpenTelemetry) to track cost and latency. 2. **A deterministic agent system with actual guardrails:** Multi-step workflow with a strict state machine (LangGraph or custom), tool calling with Pydantic validation, human-in-the-loop fallback, and an automated eval harness (DeepEval/Ragas) running synthetic test cases on PRs. 3. **Local inference microservice + fine-tuning:** QLoRA fine-tuning a small open model (e.g. Llama/Qwen) on a niche domain task, deployed via vLLM with quantized weights, backed by latency/throughput benchmarking against a baseline. If you review candidates or run a team: \- Which of these feels closest to the actual headaches you deal with day-to-day? \- When you open someone’s GitHub repo, what's an immediate green flag that screams "this person understands production" versus "they copied a tutorial"? (Is it eval pipelines, Docker/CI setup, integration tests, error handling?) \- What are the common gaps you keep seeing people miss? Roast the ideas if they're off the mark—just want to make sure I build something that genuinely moves the needle.
document fetched 2026-09-24 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1wo286p/seniorlead_ai_engineers_what_portfolio_project