How a Green AI Evaluation Hid Broken Retrieval

The first version looked better in a report than it felt in use. It could search conversations, project records, repositories, deployments, documents, and meeting notes. It returned citations and connected people to projects and projects to decisions.

Current-state questions exposed the gap. The system would find real fragments and assemble the wrong story. It could answer a question about this week with evidence from months ago. It could split one project into several entities, or merge unrelated context because a name looked similar. It could cite a sentence that existed without proving the sentence supported the claim it had just made. The citations were real, and the answer was still nonsense.

The job shifted from searching company files to exposing incomplete retrieval and refusing unsupported conclusions.

Obsidian fixed file sprawl, not operational recall

We started where a lot of internal knowledge projects start: Markdown and Obsidian.

That solved a real problem. Documents could live in one human-readable place. Notes could link to clients, projects, meetings, and concepts. Raw source material could remain immutable while a curated wiki layer organized what mattered. Compared with files spread across laptops and cloud folders, this was a large improvement.

A clean folder did not provide operational recall. The team's actual questions were time-sensitive and relational: what changed, who owned the next step, and which source should win when a meeting note, a task, and a repository disagreed.

Obsidian could hold the answer. It could not guarantee that the right evidence would be found, scoped to the right entity, checked for freshness, and assembled without invention. The problem had moved from file organization to recall.

So the internal knowledge base grew an ingestion layer, synthesized records, semantic retrieval, typed relationships, account resolution, citations, and query surfaces. Raw events stayed available for replay. Derived indexes made the material searchable. A knowledge graph handled questions that vector similarity alone was bad at, such as ownership, decisions, and relationships over time.

The synthesis layer introduced a new failure: it could fall behind the raw evidence.

More sources created more ways to be wrong

Multi-source ingestion did not create one accuracy problem. It created several problems that happened to produce the same symptom: a confident bad answer.

A source could be missing entirely. Raw events could arrive while their searchable synthesis lagged. An account alias could resolve to the wrong scope. A project could fragment into multiple entities. A source hint inferred from the question could filter out the only useful evidence. A deleted upstream document could leave a derived episode behind. The final model had no way to repair those gaps with better prose.

Our causal chain looked more like this:

1. A source event arrived, or did not.
2. The event was normalized and attributed to an entity.
3. A synthesized record became searchable, or stayed in a backlog.
4. Retrieval applied account, person, source, and time constraints.
5. Structured and semantic evidence were combined.
6. A model wrote an answer from whatever survived.

If step two assigned the wrong entity, step six could produce beautifully cited fiction. If step three was stale, "latest" meant "latest available to this index," which is not the same thing. If step four over-filtered, the system could miss the answer while the raw source still contained it.

We had to evaluate the path, not just the paragraph.

Why citations did not solve hallucinations

A citation proves that retrieval returned something. Entailment, freshness, scope, and completeness still need separate checks.

One of our historical evaluation runs made that painfully clear. Most questions returned citations, yet the report still contained operationally bad answers. Current-state questions leaned on old fragments. Ambiguous snippets were promoted into firm conclusions. A projection could be mistaken for a present total. The evaluation itself was also too generous in places because it tested the stores more directly than the user-facing query path.

The report looked healthy while the user-facing path remained wrong.

We changed the grounding layer to inspect the answer after synthesis. It can remove citation references that point outside the supplied evidence, flag factual sentences with no citation, and catch unsupported currency figures. Unit tests cover synthetic hallucination cases, including a changed amount, a flipped sign, an out-of-range citation, and an honest refusal when the sources do not contain the answer.

That last test matters. A trustworthy system needs to get credit for saying, "The sources do not support that." If the evaluator punishes refusal, the model learns to fill the silence.

The grounding check is built and tested. It is not a universal truth machine. It catches specific failure classes, and current documentation still treats some warnings as warnings rather than a hard veto. Dates, percentages, counts, and whether a cited passage truly entails a sentence remain harder. We would rather state that limit than rename a score "confidence" and hope nobody asks what it measures.

Grounded AI evaluation needs more than one grader

We now keep machine checks and human judgment separate.

The system produces deterministic grounding signals. A regression suite checks known retrieval behavior with code. A separate local quality evaluator can score dimensions such as faithfulness and hallucination risk. None of those is recorded as human approval.

Human grades live in their own feedback path. When a person marks an answer wrong, the system records the answer, the cited evidence, the correction, and the reason. Repeatedly bad evidence can be downweighted on later retrievals. Those penalties decay rather than following an episode forever.

The correction then enters a review queue. A human can promote it into the golden regression set, where future retrieval changes must keep returning the expected signal. Automatic promotion is deliberately constrained. A model should not be able to write a correction, grade itself, and canonize the result in one loop. That is self-certification with extra steps.

This separation gave us a much cleaner operating model:

• Deterministic checks catch known mechanical failures.
• Model-based evaluation contributes a second opinion and carries no approval authority.
• Human grades decide which mistakes should change future behavior.

The improvement came from keeping those signals separate instead of collapsing them into one flattering number.

We added negative tests to the suite

A knowledge system should be tested on what it must not claim.

Our earlier evaluations focused heavily on whether retrieval found something. That can reward a system for returning plausible material even when the correct result is "unknown," "not in this source," or "not current enough to answer."

The suite now checks more of the path users actually hit: account resolution, person filtering, time windows, exact structured facts, unsupported numbers, missing entities, malformed citations, and source-qualified refusals. The retrieval harness runs through the same composition layer used by the query surface, so ranking changes, feedback penalties, and post-retrieval filters are visible to the test instead of bypassed.

That design came from an embarrassing discovery: an older harness could stay green while user-facing retrieval was broken. It was testing ingredients, not the meal.

Every serious failure should become one of two things: a deterministic test when the failure is mechanical, or a human-reviewed golden question when the right answer depends on company truth. If it becomes neither, the system has learned nothing.

Cleanup and automation had to become reversible

A company brain accumulates derived state. Sources get renamed, merged, superseded, or deleted. Entity resolution improves. Rankings change. If cleanup is a one-way delete, one bad heuristic can erase the evidence needed to prove it was wrong.

We built detection before destruction: the current orphan-cleanup path reports candidates and unknown references without deleting them. Its own documentation calls the local existence check a heuristic. A future apply mode is planned to re-check the live source, remove derived records across stores, preserve an audit trail, and require explicit confirmation. That destructive path is not shipped, and we do not describe it as shipped.

Entity reconciliation follows the same pattern. Ambiguous merges become proposals. High-confidence cases can be narrowed by conservative rules, while risky full-name merges require a person to decide. Operational proposals can be accepted or dismissed, but the Brain does not silently close work or rewrite an external system based on its own inference.

That boundary costs a few clicks. It also keeps one false positive from becoming five downstream facts.

Recovery can change the answer

If raw records survive but the searchable layer misses recent material after a crash, the system can return an old answer with perfect syntax. If the graph and semantic indexes cannot be rebuilt from durable sources, the Brain's memory depends on derived state nobody can reproduce.

The built recovery design keeps append-only source ledgers as the durable record, takes snapshots of operational state, and treats search and relationship indexes as rebuildable. Backup creation and artifact checks have automated tests. The operating plan also calls for restore drills that rebuild into an isolated environment and run the evaluation suite against the result.

The backup machinery is built. A recurring, evidenced restore-drill history is still an unmet operational requirement. Untested recovery is a plan.

Brain proposes, humans decide.