The reader problem

Utrecht produces useful public information every day, but much of it is hard to access for residents who do not read Dutch confidently. Official notices, local reporting, cultural listings, and practical guidance are fragmented across sources with different formats and publishing rhythms.

The opportunity was not simply to translate pages. It was to create a local publication that could find what matters, preserve its provenance, and make it understandable in English without obscuring where each claim came from.

The product

The Utrechter Post combines daily briefings, reported stories, analysis, agenda entries, practical guides, and a continuously updated Live Desk. The public experience is designed as a newspaper first: the AI is useful because it supports the editorial system, not because it dominates the interface.

Live Desk stream combining newsroom updates, agenda items, and published stories
The Live Desk brings different publication types into one time-ordered local stream.

The agent system

I separated the system into bounded roles. Product agents prepare source candidates for editors, help readers retrieve published work, and assemble one explicitly automated product: the Daily Five. Development agents help research, design, implement, and review the software. Authority is assigned per workflow rather than implied by the model.

Candidate-assistance AI cannot independently verify or publish. The separate Daily Five workflow publishes bounded headline summaries through a named automation editor, with schema checks and run logs.

Inside the product

  1. Approved sourcesOnly sources with an explicit editorial policy enter automation.
  2. Source radarIngest, normalize, and deduplicate permitted source material.
  3. Editorial assistance agentTranslate, extract claims, flag uncertainty, and preserve names, numbers, and dates.
  4. Evidence-backed candidateKeep generated working material connected to the supplied evidence.
  5. Human verification and editingCompare claims with sources, resolve uncertainty, edit, and approve.
  6. Published newsroomRelease editor-verified work from this source-to-story workflow.
  7. Reader discovery agentRetrieve relevant documents and answer with citations.

While building the product

  1. Product intentStart with the reader need, editorial rule, and acceptance boundary.
  2. Research and design agentsChallenge assumptions, compare approaches, and draft specifications.
  3. Implementation agentsWork from bounded plans, add tests, and make traceable changes.
  4. Review agentsinspect code · content · evidence · failure modes
  5. Automated verificationtypes · tests · builds · evaluations
  6. Human decisionrevise · accept · publish

From source to story

The source radar ingests approved material, normalizes headlines and canonical URLs, and removes duplicates. An assistance agent can translate permitted text and extract claim-level working material. Each claim carries source attribution and an uncertainty state for the editor to inspect.

Private newsroom candidate prepared for editorial review
A source candidate retains its Dutch headline, canonical source, language, dates, and permitted excerpt before assistance begins.

Generated output is schema-validated before it is stored. The workflow records prompt and model versions, checks names, numbers, and dates, and rejects unsupported structures such as invented direct quotations. Invalid output fails closed; it does not silently become publication copy.

Private newsroom review showing an AI-generated internal translation and extracted claims
The AI-generated internal translation and extracted claims keep machine-generated working material visible for editorial review.

Ask the Post

Ask the Post helps readers find an answer across a controlled corpus of the publication's checked Guide listings, published pathways and articles, agenda entries, and trust pages. Retrieval handles time-sensitive questions, supplies only relevant documents to the answer step, and resolves every citation back to a document the reader can open.

It is a retrieval-augmented generation pipeline: retrieve from the corpus, supply those documents to the model as evidence, generate an answer that cites them. Retrieval is lexical rather than dense — BM25 rather than embeddings — which suits a corpus of this size and the questions readers actually ask, where street names, institution names and exact terms carry the meaning. The decision is recorded together with the threshold at which it should be revisited.

Three constraints separate that from a pipeline which passes its top results to a model and hopes. Retrieval floors — coverage 0.5 and a score floor of 1 — mean weak matches never reach the answer step at all, which is where confidently wrong answers usually begin. Citations are an identifier contract: the model returns document ids rather than prose references, answers run to at most 120 words, ids outside the supplied evidence are dropped, and an answer left with no surviving citation collapses into a refusal rather than being quietly trimmed. And evidence travels as input, never as instruction — retrieved content cannot issue directions to the model.

When corpus retrieval returns no candidates, a bounded fallback can consult a hand-curated allowlist of roughly thirty public-body and non-commercial domains, extendable only by code review and never at runtime. It labels them separately. If the system cannot resolve supporting citations, it returns a safe no-answer response. Corpus answers cache for seven days and are invalidated per cited document, so an unrelated edit does not flush everything and a changed source cannot leave its answer silently stale. Outside-tier answers use a one-day TTL. A ceiling of 500 answers a day bounds what a runaway loop can cost; past it, cached answers keep serving while everything else refuses, so the failure is a spending limit rather than an outage.

reader questionnormalisereject <4 or >300 charsrefuse — no spendcache lookupsha256(q + corpusVersion) · 7dserve cached answerdaily ceiling500 answers / dayrefuse · cache still servesBM25 retrievecoverage 0.5 · score floor 10 hitsweb fallback~30 allowlisted domainsgenerateevidence in · document ids outvalidate ids≤120 words · ids ⊆ evidenceinvented idrefuse — never trimmedanswer + provenance tierchecked · reported · outside
Four exits before the model — a malformed question, a cache hit, the daily ceiling, and retrieval below the floors. Anything the model invents at the end collapses the answer into a refusal rather than being trimmed.
Ask the Post answer with links to the cited publication documents
A corpus-first answer states what the checked evidence cannot establish, then links readers to the documents it used.

Building with agents

I used agents throughout development, but gave them work shaped by written design specifications and implementation plans. Assignments were narrow, acceptance criteria were explicit, and changes remained small enough to inspect. Research and content-review agents assembled evidence ledgers and challenged drafts; coding agents proposed tested implementation; review agents looked for missed requirements, unsafe authority, and failure modes.

Fixed fixtures and editorial question sets make evaluation repeatable and agent behavior reviewable before release. The harness is a release method, not a claim of measured model superiority. Code checks and human acceptance still control software changes.

Designing against misplaced trust

Trust is encoded in the architecture rather than delegated to prompt wording. A restricted identity limits candidate assistance to private working fields, and candidate state transitions limit who can verify and publish. The Daily Five uses a separate, named automation editor for its narrower publishing exception. Schemas constrain shape; attribution and uncertainty keep evidence visible; prompt and model versioning make behavior traceable.

Failure handling protects the reader-facing product too. Unsupported answers refuse safely, failed generation does not create filler, and publication outages preserve the last valid public material where possible.

Recovery is bounded rather than open-ended. When the selection agent returns duplicate or unusable candidates, it gets a single repair attempt — one further search that excludes what it already produced — and then stops. A retry budget of one keeps a failing run cheap and prevents an agent from working indefinitely against a problem it cannot solve.

Architecture and operations

A public Next.js application runs on Cloudflare. The Payload newsroom runs on Railway with PostgreSQL, scheduled editorial jobs, and role-restricted access. Media is served from R2. Versioned shared contracts keep the two applications aligned, while signed, retryable revalidation messages update the public cache after a publication change.

  • Public reading surface: Cloudflare and R2 media delivery.
  • Editorial surface: Payload, Railway, and PostgreSQL.
  • Integration: shared contracts and signed revalidation.
  • Operations: scheduled ingestion jobs with explicit failure paths.
  • Discovery: schema.org structured data — the publication as a news organization, and each story as a news article with its author, section, and imagery — so machines read the same claims as readers.

Evidence of quality

For a current project, mechanisms are more honest evidence than invented impact numbers. I tested the rules that would otherwise exist only as claims in a diagram.

Authority

Workflow transition tests

Exercise the boundary that lets assistance prepare candidates while only editors verify and publish.

Grounding

Citation and refusal tests

Drop citations outside the retrieved set and refuse answers when no supplied document resolves.

Operations

Publication contract checks

Validate public payloads, signed revalidation, retry behavior, and safe handling of unavailable private infrastructure.

Published Utrechter Post story in the public reading interface
A published history story in the public reading interface, with an original photograph by Hatef.

What I learned

Agentic products need more engineering discipline, not less. Useful autonomy comes from explicit authority, observable decisions, grounded retrieval, repeatable evaluation, and a graceful way to say no.

Agents accelerated breadth and iteration, but increased the importance of precise specifications, review boundaries, and verification. Most outputs remained proposals; the narrow automated publishing path required its own explicit authority, validation, audit trail, and failure behavior.