I am a software engineer with 9+ years of experience designing and building production AI platforms, distributed backend systems, search infrastructure, and reusable engineering foundations.
My work focuses on the engineering required to make AI systems reliable in production: retrieval, orchestration, structured outputs, asynchronous processing, evaluation, observability, model serving, failure recovery, and cost-aware architecture.
| Area | Technologies & Experience |
|---|---|
| Production AI Systems | Production LLM applications, AI platform architecture, structured outputs, prompt design, prompt versioning, model routing, guardrails, evaluation, observability, and failure-tolerant workflows |
| Agentic AI & Orchestration | Agentic workflows, tool calling, durable state, human-in-the-loop systems, multi-agent coordination, Agno, LlamaIndex, and LangChain |
| RAG & Retrieval Systems | Enterprise RAG, semantic retrieval, hybrid search, reranking, metadata filtering, chunking strategies, conversational memory, Pinecone, Qdrant, and pgvector |
| Model Adaptation | Supervised fine-tuning, QLoRA, LoRA, Unsloth, dataset preparation, structured extraction, annotation workflows, and model evaluation |
| Model Serving & Inference | vLLM, llama.cpp, quantized models, CPU-oriented inference, model-loading trade-offs, latency, throughput, memory optimization, and local model deployment |
| Backend Architecture | Node.js, TypeScript, NestJS, Python, FastAPI, REST APIs, microservices, asynchronous workflows, reusable backend frameworks, and API platform design |
| Distributed Systems | Apache Kafka, Redis, event-driven architecture, consumer groups, idempotent processing, retries, dead-letter queues, backpressure, and high-volume ingestion pipelines |
| Search Engineering | Algolia, Apache Solr, vector search, keyword search, hybrid retrieval, ranking, indexing pipelines, catalogue ingestion, and search relevance |
| Databases & Data Systems | PostgreSQL, MongoDB, JSONB, EAV modelling, Pinecone, Qdrant, data-ingestion pipelines, schema design, indexing, and query optimization |
| Cloud & Platform Engineering | AWS, GCP, Azure, Kubernetes, Docker, GKE, CI/CD, observability, production reliability, and developer experience |
| Technical Leadership | Architecture reviews, technical strategy, engineering standards, mentoring, roadmap decomposition, incident ownership, and stakeholder alignment |
- Designed and delivered production AI platforms for research, education, and enterprise workflows.
- Built enterprise RAG systems using advanced retrieval pipelines, vector databases, reranking, and memory-aware conversational workflows.
- Fine-tuned a 3B-parameter language model on more than 1,000 annotated research papers using QLoRA and Unsloth.
- Built document-processing pipelines using Docling, custom preprocessing, structured parsing, model inference, schema validation, and evaluation workflows.
- Optimized model-serving workloads for CPU-constrained environments using llama.cpp and vLLM.
- Built Kafka-based ingestion pipelines processing approximately 5 million commerce events per day.
- Designed search and catalogue-discovery systems using Algolia and Apache Solr.
- Architected reusable backend capabilities for authentication, workflows, notifications, dynamic entities, and shared service infrastructure.
- Led engineering initiatives covering AI-enabled products, backend platforms, observability, production reliability, and developer experience.
I approach AI features as production distributed systems rather than isolated model calls.
Some of the principles I follow:
- Treat model output as untrusted input.
- Validate structured outputs against explicit schemas.
- Evaluate retrieval separately from generation.
- Prefer deterministic workflows when agent autonomy is unnecessary.
- Keep authorization and tool permissions outside the model.
- Version prompts, models, schemas, retrieval configurations, and evaluation datasets.
- Design retries, fallbacks, timeouts, and human escalation before production rollout.
- Measure correctness, groundedness, latency, cost, and business outcomes.
- Trace the complete request across retrieval, model calls, tools, validation, and persistence.
- Build for reproducibility, auditability, and safe rollback.
I write about the architecture and engineering behind production AI systems.
Topics include:
- Production LLM architecture
- AI system design
- RAG and retrieval systems
- Agentic workflows
- Model gateways
- Structured outputs
- AI evaluation
- Prompt and model versioning
- AI observability
- Failure-tolerant workflows
- Distributed systems
- Backend and platform engineering
- Model inference and serving
Read my articles at:
- Reliable agentic systems
- Durable multi-agent workflows
- AI evaluation and observability
- Retrieval and reranking architectures
- Long-context inference
- KV-cache optimization
- Local and edge inference
- Quantization and model-serving efficiency
- AI platform governance
- Human-in-the-loop systems
- Responsible AI architecture
- Production-grade LLMOps
| Platform | Link |
|---|---|
| Technical Blog | hiteshsharma.dev |
| linkedin.com/in/hiteshsharma-v2 | |
| Twitter / X | twitter.com/ihiteshsharma |
| hitesh9sharma@gmail.com | |
| Professional Resume | View Resume |


