How This Blog's Auto-Publishing Pipeline Works

This blog's auto-publishing is a pipeline that queues candidates, filters them through a deterministic gate, and only ships when a human approves, and the flow is five stages: (1) generate a candidate (post) → (2) gate checks → (3) deduplicate → (4) a dry run → (5) publish and log to a ledger only on approval. The point is not a bigger model but a well-designed gate, and this very post had to pass the same gate, and across this session, by observation, every candidate in the queue was tuned to pass (a measured 100% pass rate; target 20 posts a day).

In plain terms: the pipeline is airport security. Everyone lines up (the queue), prohibited items and missing papers are caught at the gate (checks), and final boarding happens only when staff confirm it (approval).

How do the five stages flow?#

A candidate passes five checkpoints in order before it ships. Each checkpoint has a pass condition, and if it fails, the candidate stops right there. The diagram below is that flow.

What does the gate check?#

Format, density, evidence, and safety, by rule. The checks are deterministic (rules and regex, not an LLM's judgment), so the same input gives the same result. Key rules: does the first paragraph answer-first in 90+ characters, is there a comparison table, does the body clear a density floor (a measured ~650 non-space characters for KO and ~330 words for EN), are there at least two question subheads, and is there a freshness Note. For safety and evidence it adds a secret scan (blocking tokens, keys, internal addresses), a references section with a live 200 link check, and whether numbers with units carry an evidence label like "measured" or "benchmark." Below are the gate's two branches of checks.

Why a deterministic gate (and the trade-off)?#

For reproducibility. Being rule-based, the same post always gets the same verdict, which is easy to debug and cheap (the industry also runs deterministic checks before any semantic LLM-judge). But there is a trade-off: the live 200 link check is an external dependency and creates non-determinism. Some bot-blocking hosts return 403/429 to automated requests, so a perfectly good post is intermittently rejected (observed in this session too). The operational workaround is to prefer stable, non-bot-blocking hosts (GitHub, arxiv), and a deeper fix (treat 403/429 as "reachable") has been proposed separately.

Why is human approval still required?#

For trust and accountability. The default is a dry run: pass the queue through the gate and only record "how many would ship" to a ledger. Actual publishing happens only with an explicit approval flag - because it is an irreversible public release, a human confirms it last. Below is that two-branch decision.

The industry agrees: automation handles operations (checks, iteration), while creative judgment like choosing angles and final accountability stay human. On the regulatory side too, human accountability at publish time is assumed (transparency duties).

How can you explore it yourself?#

Small, gate first.

• Put deterministic checks first before shipping (answer-first, table, density, subheads, Note, secrets, links).
• Make the link check distinguish dead links (404) from bot blocks (403/429) - treating the latter as dead leaks good posts.
• Keep publishing dry-run by default and real publish behind approval. Put a human in front of irreversible actions.

Reference links

• FastAPI (server and SSR stack)
• SQLite (storage)
• pytest (verification and regression tests)
• schema.org (structured data / SEO)
• LLM-as-a-Judge (evaluation paper)

Note: stages, rules, and figures reflect this blog's 2026 operations observations and public materials and vary by policy (internal paths, files, DB, and accounts are private). A deterministic gate is reproducible but its external link dependency can cause intermittent false rejections, and real publishing assumes human approval. The pipeline improves often, so this is reviewed quarterly.