Fixed scope
Internal knowledge assistant
Hybrid retrieval across scattered internal documentation, with citation enforcement and an eval harness that gates every prompt change in CI.
Services / AI, ML & MLOps
Retrieval and LLM systems that hold up in production.
Most AI pilots die in month two. We build the parts that decide whether yours survives.
- Typical first engagement
- 6–10 weeks, fixed scope
- Starts with
- An eval set built from your real questions
- Ends with
- Runbooks, eval harness and cost model in your repo
The problem
What this practice is actually for.
The demo works because someone asked it three friendly questions. Production breaks on the fourth kind.
So we build an evaluation set from real user questions first, then measure every retrieval, prompt and model change against it. You get a number, not an opinion.
- Grounding on your own corpus — documents, tickets, code, CRM records, database rows
- Guardrails that refuse rather than invent when retrieval comes back thin
- Citations traced to source spans, so a reviewer can verify an answer in seconds
- Cost per resolved query modelled at design time, not discovered on the first invoice
In the work
What AI/ML code looks like when we write it.
Running
0.0s
pipeline
main
● live run
Retrieval that refuses rather than invents.
Dense and sparse retrieval fused, re-ranked, with a floor below which the system declines to answer. The eval gate blocks the deploy when faithfulness drops.
Capabilities
AI, ML & MLOps — the full stack.
9 capability groups
- RAG systems
- Retrieval pipeline design, pgvector, Pinecone, Weaviate, chunking and metadata strategy, prompt/response evaluation, output guardrails
- Retrieval quality
- Hybrid dense + sparse/BM25 search, re-ranking models, multi-modal document and image retrieval
- Agentic systems
- LangGraph, multi-agent workflows, tool and function-calling design, ReAct-pattern agents, human-in-the-loop checkpoints
- Orchestration
- LangChain, LlamaIndex, Semantic Kernel
- Serving & inference
- vLLM, Triton Inference Server, SageMaker and Vertex AI endpoints, batching and quantisation, inference cost optimisation
- Fine-tuning
- LoRA, QLoRA, parameter-efficient fine-tuning, instruction tuning, dataset construction
- Evaluation
- Automated RAG evaluation harnesses, LLM tracing and debugging, regression gates in CI, custom eval pipelines
- MLOps
- Deployment pipelines, model versioning and registry, CI/CD for ML, drift monitoring, experiment tracking
- Platforms
- MLflow, Kubeflow, Weights & Biases, feature pipelines with Airflow and dbt
Typical engagements
How this work usually starts.
Indicative scope and duration
Fixed scope
Agentic back-office workflow
A multi-step agent that reads a request, gathers what it needs from your systems, drafts the action, and stops for human approval where the cost of error is real.
Advisory
AI feature audit
Independent review of a pilot that is not converting: retrieval quality, prompt architecture, latency budget, unit economics, and whether the use case is winnable at all.
What you get
Deliverables, every time.
01
Evaluation harness
A versioned eval set with pass thresholds wired into CI, so quality regressions fail a build rather than a customer conversation.
02
Retrieval architecture
Documented chunking, embedding, hybrid search and re-ranking decisions with the trade-offs and the benchmark numbers behind each.
03
Cost and latency model
Cost per query and p95 latency at your projected volume, with the levers that move them ranked by effort.
04
Operating runbook
What to do when quality drops, retrieval goes stale, or a provider deprecates a model — written for your on-call engineer.
Questions
What clients ask before they commit.
Usually not first. In most engagements retrieval quality, prompt architecture and re-ranking move accuracy far more than fine-tuning, at a fraction of the cost and with none of the retraining burden. We fine-tune when there is a measured ceiling that retrieval cannot lift — typically format adherence, domain vocabulary, or latency-driven use of a smaller model. We will tell you which case you are in before you spend on it.
Yes. We deploy open-weight models on your own infrastructure with vLLM or Triton when data residency, contractual restrictions or cost make hosted APIs unworkable. The trade-off is real — you take on GPU capacity and evaluation burden — and we will quantify it before you commit rather than after.
Three layers, none of which is a prompt asking it politely. Retrieval returns scored passages and the system refuses when scores fall below a floor. Answers must cite retrieved spans, and uncited claims are stripped. The eval harness includes questions with no correct answer, and refusing them correctly is a passing result.
Also from Windsor Harlow
Other practices
→
Salesforce Development
Governor-safe Apex, tested triggers, automation a new admin can read.…
→ →Cloud, DevOps & Infrastructure
Cost and operability designed in as requirements, not cleaned up after the invoice arrives.…
→ →Backend, Web & Distributed Systems
Distributed patterns that survive growth, without the premature architecture that sinks a product first.…
→Have a AI/ML problem worth a senior pair of eyes?
Tell us the system, the constraint, and what happens if it is not solved. A senior engineer replies within one business day.