Semantics Delivery Network · HotNets '26

Decisions at the Edge

Agents consume evidence in coherent passages or chunks; however, today the web delivers in pages for human readers. We propose a new network infrastructure serving evidence chunks to answer the query, unifying the utility of traditional search engines and CDNs in one round. Instead of a finer-grained CDN delivering semantic-oblivious bytes, a semantic-aware network infrastructure creates new opportunities as well as challenges. This artifact guides the audience to step through the decisions an edge has to make.

One request, three paths

Compare the API rate limits of Halden and Quillbase

The middle path is Cloudflare's per-site AI Index¹. AI Search is the query path over that index². AISTRAL's Agentic Twins³ also give each site its own index, and add a resolver that picks which sites to ask. Ranking across sites is still left to the agent.

What one query costs today

Median of 100 live queries through a SERP API and its top-5 URLs⁴. The no-JS fetch moves 107× what the model reads; a full render, 2,532×. Bars use a log scale.

Two systems, split down the middle

	Search engine	CDN	SemDN
Keyed on	Query	URL	Query
Returns	URLs and snippets	Whole objects	Ranked chunks with provenance
Sees the query	Yes	No	Yes
Serves the origin	No	Yes	Yes
Paid by	Client	Origin	Both, for different services

The unit it returns is called a chunk. It can be a text passage⁵, a table block^{6, 7}, an image region^{8, 9}, or a page screenshot¹⁰. We currently focus on passages and tables. Tessera rebuilds a web page into evidence units and completes a table row with its headers⁷. Other work explores images and screenshots. ColPali and CLIP retrieve image regions^{8, 9}. Computer-use agents operate on the rendered page^{11, 12, 13, 14, 15}, and PixelRAG retrieves from the screenshot itself¹⁰. The same idea extends to an audio clip¹⁶ or a video segment¹⁷.

text passage ⁵table block ^{6, 7}image region ^{8, 9}page screenshot ¹⁰audio clip ¹⁶video segment ¹⁷

References

Cloudflare. An AI Index for all our customers. 2025.
Cloudflare. AI Search.
Yixi Chen, Muhammad Bilal, Tie Ma, Zafar Ayyub Qazi, Marco Canini, and Suhaib Fahmy. From Crawled to Published: A Publisher-Controlled Web Architecture for an AI-Native Internet. arXiv:2511.18354v2, 2026.
Peichun Hua and Yunming Xiao. Semantics Delivery Network: Rethinking Web Retrieval Infrastructure for LLM Agents. HotNets 2026. The byte measurements.
Vladimir Karpukhin et al. Dense Passage Retrieval for Open-Domain Question Answering. EMNLP 2020.
Jonathan Herzig et al. Open Domain Question Answering over Tables via Dense Retrieval. NAACL 2021.
Peichun Hua, He Sun, Zirui Zhuang, Qi Qi, Yiming Qiu, Shinan Liu, and Yunming Xiao. Tessera: Structure-Completed Evidence Units for Web Retrieval-Augmented Generation. arXiv:2610.37080, 2026.
Manuel Faysse et al. ColPali: Efficient Document Retrieval with Vision Language Models. arXiv:2407.01449, 2024.
Alec Radford et al. Learning Transferable Visual Models From Natural Language Supervision (CLIP). ICML 2021.
Yichuan Wang et al. PixelRAG: Web Screenshots Beat Text for Retrieval-Augmented Generation. arXiv:2606.28344, 2026.
Anthropic. Introducing computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku. 2024.
Google DeepMind. Introducing Gemini 2.0: our new AI model for the agentic era. 2024.
Jing Yu Koh et al. VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks. ACL 2024.
OpenAI. Computer-Using Agent. 2025.
OpenAI. Introducing ChatGPT Agent: Bridging Research and Action. 2025.
Yusong Wu et al. Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation (CLAP). ICASSP 2023.
Huaishao Luo et al. CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval and Captioning. Neurocomputing, 2022.

Further reading (11 papers, click to expand)

Where this idea comes from

Patrick S. H. Lewis et al. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. The paper that named retrieval-augmented generation: fetch passages, then let the model answer.
Reiichiro Nakano et al. WebGPT: Browser-assisted question-answering with human feedback. arXiv:2112.09332, 2021. An early agent that browses the live web to answer questions, the workload SemDN serves.
Jason Wei et al. BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents. arXiv:2504.12516, 2025. A benchmark of hard browsing tasks, showing how many pages agents touch per answer.
Van Jacobson et al. Networking named content. CoNEXT 2009. Named-data networking: route requests by what they ask for, not by host. SemDN keys on the query instead.

CDNs and the network side

Erik Nygren, Ramesh K. Sitaraman, and Jennifer Sun. The Akamai Network: A Platform for High-Performance Internet Applications. ACM SIGOPS Operating Systems Review, 2010. How a large CDN is built, the system SemDN is compared against throughout.
Waris Ali et al. A survey on the state-of-the-art CDN architectures and future directions. Journal of Network and Computer Applications, 2025. A recent survey of CDN architectures.
Yihua Cheng et al. Do Large Language Models Need a Content Delivery Network?. arXiv:2409.13761, 2024. Proposes a delivery network for LLM knowledge in the form of KV caches, a neighbouring idea.
Siddhant Ray et al. An Internet for the KV Cache: Rethinking Classical Infrastructure Boundaries in the LLM Inference Age. arXiv:2608.01526, 2026. Asks which Internet infrastructure should carry KV caches between models.
Muhammad Bilal et al. Toward an AI-Native Internet: Rethinking the Web Architecture for Semantic Retrieval. arXiv:2511.18354, 2025. Argues for an AI-native web that serves semantic content, close in spirit to SemDN.
Yazhuo Zhang et al. Rethinking Web Cache Design for the AI Era. ACM SoCC 2025. Revisits web cache design when AI crawlers and agents are the clients.
Cloudflare. Introducing Markdown for Agents. Cloudflare blog, 2026. A CDN serving Markdown instead of HTML to agents: the first step from bytes toward text.

Four structural mismatches

The stack was built for human readers

Similar is not useful

Pick the passage you would hand to the agent. Scores are query-passage similarity.

Chunking is a budget decision

The reader has a context budget. Spend it on chunks or on pages?

Context budget

Paper Fig. 3c²: CRAG Task-3¹, 600 questions, Qwen3-32B reader. With the same LLM reranker, chunks beat pages by 3.5–8.2 accuracy points from 256 to 6,144 tokens.

Same question, same 256 tokens, same reader

Page-first cannot reach what it never downloads

The tiles are pages in a page ranker's order. Marked tiles hold a chunk from the global top-5. Download more pages and see how far you get.

Pages downloaded

16–29% of the global top-5 chunks lie on pages a whole-page ranker skips, even at 20 pages². Chunks match or beat a client that downloads ten pages (0.456 vs. 0.427 accuracy with e5), which moves about 1,200× the bytes.

Every client runs the same fragile pipeline

Reach, extract, de-boilerplate, segment, embed. Run it on 40 result URLs, then multiply by the number of operators doing the same work.

Operators fetching these URLs

Over half the text is lost without JavaScript on at least 39% of sampled URLs, and about 19% are bot-blocked. Across 1,430 live URLs, a cloud renderer gets usable clean text from at most 75%³.

Real queries, real top-5 URLs

References

Xiao Yang et al. CRAG — Comprehensive RAG Benchmark. arXiv:2406.04744, 2024.
Peichun Hua and Yunming Xiao. Semantics Delivery Network. HotNets 2026. Fig. 3c and the page-reach probe.
Cloudflare. Browser Rendering: Markdown endpoint.

Further reading (14 papers, click to expand)

What unit to retrieve

Tong Chen et al. Dense X Retrieval: What Retrieval Granularity Should We Use?. EMNLP 2024. Shows that retrieving smaller units (propositions) can beat passages and documents.
Zijie Zhong et al. Mix-of-Granularity: Optimize the Chunking Granularity for Retrieval-Augmented Generation. COLING 2025. Picks the chunk size per query instead of fixing one size.
Renyi Qu et al. Is Semantic Chunking Worth the Computational Cost?. Findings of NAACL 2025. Finds that semantic chunking often does not pay for its extra compute.
André V. Duarte et al. LumberChunker: Long-Form Narrative Document Segmentation. Findings of EMNLP 2024. Uses an LLM to split long documents where the topic changes.

Similar is not useful

Orion Weller et al. On the Theoretical Limitations of Embedding-Based Retrieval. arXiv:2508.21038, 2025. Proves there are relevance patterns a single embedding vector cannot represent.
Florin Cuconasu et al. The Power of Noise: Redefining Retrieval for RAG Systems. SIGIR 2024. Shows that a passage that looks relevant but lacks the answer can hurt more than random text.
Lu Dai et al. SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction. ICLR 2025. Measures a passage's value by how much it helps the reader, not by similarity.
Manish Chandra et al. LURE-RAG: Lightweight Utility-driven Reranking for Efficient RAG. arXiv:2601.19535, 2026. Reranks passages by their utility to the answer at low cost.
Nelson F. Liu et al. Lost in the Middle: How Language Models Use Long Contexts. TACL 2024. Models use the start and end of a long context best, so where evidence sits matters.

Cleaning the web for models

Jiejun Tan et al. HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems. WWW 2025. Keeps some HTML structure instead of flattening pages to plain text.
Feng Wang et al. ReaderLM-v2: Small Language Model for HTML to Markdown and JSON. arXiv:2503.01151, 2025. A small model that converts HTML to clean Markdown.
Mengjie Liu et al. Dripper: Token-Efficient Main HTML Extraction with a Lightweight LM. KDD 2026. Extracts a page's main content with few tokens.
Murrough Foley. WCXB: A Multi-Type Web Content Extraction Benchmark. arXiv:2605.21097, 2026. A benchmark for web content extraction across page types.
Aaron Steiner et al. MCP vs RAG vs NLWeb vs HTML: A Comparison of the Effectiveness and Efficiency of Different Agent Interfaces to the Web. WWW 2026. Compares MCP, RAG, NLWeb and raw HTML as ways for agents to read sites.

Difference 1 · Misses

Your index always answers. Is the answer good enough?

A URL lookup tells you when it misses. Top-k search returns k results even when better evidence exists elsewhere. Thirteen queries arrive at your PoP. Decide what each one gets.

Edge PoPHot content and index state

Regional parentLarger, colder shards

Global tierThe distributed catalog

External discoverySERP results, filtered by the enrollment registry

More context does not fix old evidence

Correct answers on CRAG¹ with reranked chunks, by how fast the truth changes². Spending twelve times the budget helps stable questions. Real-time questions barely move, because their answer is missing from the pool.

The truth changes	Questions	512 tokens	2,048 tokens	6,144 tokens
Never	340	65.0%	68.5%	70.0%
Slowly	135	57.8%	56.3%	60.0%
Quickly	55	36.4%	41.8%	45.5%
In real time	70	35.7%	35.7%	37.1%

Now hand the job to a controller

A semantic coverage miss² is when the estimated risk that an external search would change the top-k exceeds a threshold. The buttons are the signals that estimate that risk. Each one is imperfect. For example, a sufficiency judgment checks whether the evidence holds enough to answer before any generation³. Turn them on and move the threshold to see the tradeoff in a simulation of 2,000 queries.

What each signal measures (5 signals, click to expand)

Score distribution: The scores on the chunks just returned, and the gap between the first and the rest. A high score with a steep drop is read as confidence. In the queries above, the seat-limit table scores weakly and still holds the answer, and Halden's chunks score 0.91, 0.87, and 0.84 while stating an old limit.
Result coherence: Whether those chunks agree with one another. Agreement is read as a stable answer. For HTTP 429 the chunks agree and the meaning is unchanged. For Halden they agree on the limit from last spring.
Sufficiency judgment: A check that reads the query and the chunks and decides whether they contain enough to answer, before a model generates anything³. It asks whether the evidence supports an answer, which is separate from how similar a chunk is to the query.
Cross-tier agreement: Whether this edge's top-k matches the regional parent and the global tier. A mismatch means a larger tier may hold chunks this PoP evicted. A match means the gap, if there is one, is outside the network.
Freshness metadata: The age and version stored with each block. The same age means different things for different queries. Six hours is stale for a status page, and ordinary for a standards document.

Risk threshold

Discovery runs when the combined risk is above this value. Lower it and more real misses get a search. More queries that were already covered get searched too.

Content drift: a breaking-news week

Pages are changing faster than usual. The same signals then separate covered queries from real misses less cleanly. How to recalibrate when that happens is open.

The first bar counts coverage misses the controller did not search. The second counts searches that would not have changed the top-k. The third is how often an external search is paid for.

References

Xiao Yang et al. CRAG — Comprehensive RAG Benchmark. arXiv:2406.04744, 2024.
Peichun Hua and Yunming Xiao. Semantics Delivery Network. HotNets 2026. The coverage miss and the context table.
Suting Chen, Peichun Hua, and Yunming Xiao. Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG. arXiv:2609.37469, 2026.

Further reading (8 papers, click to expand)

Knowing when the evidence is not enough

Hailey Joren et al. Sufficient Context: A New Lens on Retrieval Augmented Generation Systems. ICLR 2025. Defines sufficient context: does the retrieved text actually contain enough to answer?
Shi-Qi Yan et al. Corrective Retrieval Augmented Generation. arXiv:2401.15884, 2024. Corrective RAG: grade the retrieved passages and fall back to web search when they are poor.
Akari Asai et al. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. ICLR 2024. Trains the model to decide when to retrieve and to critique what it got.

Predicting how good a search was

Maria Vlachou and Craig Macdonald. Coherence-based Query Performance Measures for Dense Retrieval. ICTIR 2024. Predicts retrieval quality for dense retrievers without knowing the answers.
Fangzheng Tian et al. Predicting Retrieval Utility and Answer Quality in Retrieval-Augmented Generation. ECIR 2026. Predicts whether retrieval will help the RAG answer before generating it.
Aparajita Sinha and Kunal Chakma. Adaptive Query Performance Prediction for Retrieval-Augmented Generation: Bridging Retrieval Quality and Generation Relevance. ACM TOIS 2026. Query performance prediction adapted to retrieval-augmented generation.
Fangzheng Tian et al. Am I on the Right Track? What Can Predicted Query Performance Tell Us about the Search Behaviour of Agentic RAG. IR-RAG workshop, 2025. Asks what predicted query performance can tell a search agent about its progress.
Robin Aly et al. Taily: Shard Selection Using the Tail of Score Distributions. SIGIR 2013. A classic: decide which index shards to ask from the tail of their score distributions.

Locality

Three kinds of chunk reuse

A URL cache sees low locality. Within a task, 70–100% of a search agent's URLs are new^{1, 2}. Chunks are different.

Already seen, or on a page already opened

An exact repeat is a chunk the agent already saw. That is 41% of later chunks on Search-R1 over Wikipedia, and 70% on CRAG^{2, 3}. Our guess: the first search missed the passage that holds the answer, so the agent only tweaks the query. Most of the same chunks come back, with a few new ones. Exact same queries are removed.

	Search-R1 / wiki-183,075 traces	CRAG1,500 tasks
Exact repeat	41%32% → 55% by turn	70%
Sibling, on a page already opened	11%about 1 in 10, every turn	10%
New page	48%	20%

A sibling is the second kind. The edge has already fetched, cleaned, and chunked the page, so it can keep every chunk on it. About 1 in 10 later chunks is one the agent has not been shown, on a page already opened. The rest are pages the agent has not opened. These traces are Search-R1 with Qwen2.5-7B, an older agent¹. We expect both effects to be stronger with a better agent and a better retriever.

The first search missed the answer

Where was the music festival Billy McFarland was charged with wire fraud for supposed to take place?

The question is where the festival was supposed to be. Turn 1 asks about McFarland and the fraud. The three passages the agent sees are the charge and the guilty plea. None of them says where it was, so the agent tweaks the query and asks for the location. Great Exuma was already on that page, at rank 40 of the first search, and was not sent. Turn 2 is when the agent receives it. That passage is both a sibling and the third kind, below.

Already retrieved, then cut off

The third kind is a chunk an earlier search already returned and did not show. The agent only sees the top 3. If the edge keeps the rest of that list, it can have the next chunk ready before the agent asks, which is the prefetch, or serve it from cache when the agent does ask.

What sport is played by both Justin Gimelstob and Angelique Kerber?

Query: Justin Gimelstob and Angelique Kerber

Justin Jeremy Gimelstob is a retired American tennis player.Justin Gimelstob1

He established himself as a doubles specialist, winning 12 titles.Justin Gimelstob2

Another passage on Justin Gimelstob.Justin Gimelstob3

The 2018 Angelique Kerber tennis season began at the Hopman Cup. She entered the season ranked 21st.Angelique Kerber, not shown4

Passages like the one just shown are more passages on Justin Gimelstob. The next query adds the word "sport", and the Kerber passage moves into the top 3.

Between Polytechnic University of the Philippines and California Polytechnic State University, which was founded as a vocational high school?

Query: Polytechnic University of the Philippines and California Polytechnic State University founding

Polytechnic University of the Philippines was founded on October 19, 1904 as the Manila Business School.Polytechnic University of the Philippines1

Another passage on the same university.Polytechnic University of the Philippines2

The University of the Philippines was founded by the American colonial government.University of the Philippines3

School was founded as a vocational high school when California Governor Henry Gage signed the Polytechnic School Bill on March 8, 1901.California State Polytechnic University, Pomona, not shown4

Passages like the one just shown stay on universities in the Philippines. The next query asks for the California founding, and this passage comes back first.

Of the chunks that first appear in the next turn's top 3, 25% were already in ranks 4–13 of the search just served. Ranks 4–50 of that same search cover 38.5%. Separately, 12% of those same new chunks are among the ten passages most similar to the single top passage that was sent.

References

Bowen Jin et al. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. arXiv:2503.09516, 2025. The traces use Qwen2.5-7B.
Peichun Hua and Yunming Xiao. Semantics Delivery Network. HotNets 2026. The 32–55% repeat metric and the trace replay. The sibling split uses the same cut on those traces.
Xiao Yang et al. CRAG — Comprehensive RAG Benchmark. arXiv:2406.04744, 2024.

Further reading (10 papers, click to expand)

Caching for retrieval and LLMs

László A. Belady. A Study of Replacement Algorithms for a Virtual-Storage Computer. IBM Systems Journal, 1966. The classic study of which item a cache should evict, the baseline for any cache policy.
Chao Jin et al. RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation. ACM TOCS 2026. Caches the model's internal state for frequently retrieved documents.
Fu Bang. GPTCache: An Open-Source Semantic Cache for LLM Applications Enabling Faster Answers and Cost Savings. NLP-OSS workshop, 2023. A semantic cache that reuses answers to similar prompts.
Shai Bergman et al. Leveraging Approximate Caching for Faster Retrieval-Augmented Generation. Middleware 2025. Reuses retrieval results for queries that are close in embedding space.
Sukjoon Oh et al. Aker: Density-Aware Approximate Caching for Vector Search (Extended Version). arXiv:2609.03712, 2026. Approximate caching for vector search that takes the density of the data into account.
Peizhen Guo et al. FoggyCache: Cross-Device Approximate Computation Reuse. MobiCom 2018. Reuses computation across devices when inputs are similar, an early approximate cache.

How agents search

Seungjin Lee et al. Characterizing Locality in Large-Scale Vector Search Workloads of Agentic AI Systems. VLDB 2026 Workshop on Vector Databases. Measures locality in vector-search traces from agentic AI systems.
Wuyang Zhang and Shichao Pei. Predictive Prefetching for Retrieval-Augmented Generation. arXiv:2605.17989, 2026. Prefetches retrieval results before the agent asks for them.
Huatong Song et al. R1-Searcher++: Incentivizing the Dynamic Knowledge Acquisition of LLMs via Reinforcement Learning. arXiv:2505.17005, 2025. Another RL-trained search agent, alongside Search-R1.
Wenhan Liu et al. Agentic-R: Learning to Retrieve for Agentic Search. arXiv:2601.11888, 2026. Trains the retriever for agentic search rather than for one-shot queries.

Refresh under coverage risk

Two days at your PoP

Twelve blocks across three documents, 48 hours

The edge is holding chunks it already fetched. The pages they came from keep changing, and serving the old copy gives the agent a stale answer. Each row is one block: four from a status page, four from pricing, and four from documentation. Each column is one hour. Red means the origin changed that block and this PoP still holds the old copy. The last color in the legend is a query that was answered with that old copy. The buttons are three ways to go get a new copy.

TTL

FreshStaleQuery, hit freshQuery, got stale evidence

Three ways the query stream misleads placement

The same queries that tell you what to keep at the edge can also push the index the wrong way.

A page almost nobody asks for. A rare-disease reference gets about one query a month, so it falls out of the edge index. The next question that needs it misses.

One tenant's queries, every tenant's index. A hedge fund's agents keep requesting chunks on one small biotech. Reweighting the shared index from those queries makes that company faster for everyone, and it shows the other tenants what the fund is working on.

A chunk that used to be right. Passages that produced correct answers get boosted. A 2024 passage on a tax rule has a long record of being right. The rule changed last week. The boost, and an old benchmark, keep serving that passage.

All three are open.

Further reading (9 papers, click to expand)

Keeping copies fresh

James Gwertzman and Margo Seltzer. World-Wide Web Cache Consistency. USENIX Annual Technical Conference, 1996. Compares TTLs, polling and server invalidation for web caches, the same three options as this page.
Junghoo Cho and Hector Garcia-Molina. Synchronizing a Database to Improve Freshness. SIGMOD 2000. How a crawler should schedule re-fetches when pages change at different rates.
Muhammad Mansoor et al. Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs. arXiv:2607.04281, 2026. A semantic cache for web RAG that bounds the risk of serving stale results.
Rui Li et al. EviDex: Provenance-Weighted Evidence-Path Indexing for Fresh and Auditable Retrieval under Continuous Updates. POMACS 2026. Evidence indexing weighted by provenance, aimed at answers that are fresh and auditable.

When answers go stale

Tu Vu et al. FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation. Findings of ACL 2024. A benchmark of questions whose answers change, and search augmentation to keep up.
Jie Ouyang et al. HoH: A Dynamic Benchmark for Evaluating the Impact of Outdated Information on Retrieval-Augmented Generation. ACL 2025. Measures how outdated documents in the pool mislead RAG systems.
Adam Liška et al. StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models. ICML 2022. Questions asked over a stream of news, testing adaptation to new facts.
Angeliki Lazaridou et al. Mind the Gap: Assessing Temporal Generalization in Neural Language Models. NeurIPS 2021. Shows that language models degrade on events after their training data.
Youngeun Nam et al. TimelyRAG: Semantic-Temporal Hybrid Retrieval for Time-Critical Question Answering in Overlapping-Evolving Documents. arXiv:2609.11572, 2026. Combines semantic and time-aware retrieval for time-critical questions.

Differences 2 and 3 · Tenancy and state

Four tenants, one substrate. What can they share?

A cached page serves every client. An index serves only the clients that compute vectors the same way.

The origin's contract stays the same in all of these. Enrollment, permitted transformations, freshness bounds, and purge. A tenant's encoder never appears in it. Its utility function and labels stay private too.

Now scale the tenants

Tenants

Our Zipfian tenant model over real task-to-page sets². Savings track overlap: the top-10 domains draw 34% of live results, while four deliberately diverse benchmarks share only 22 of 4,618 pages.

Bring your own encoder to a canonical index

The canonical index only has to place a tenant's true top-k inside its top-K. The tenant's encoder then re-encodes those candidates from normalized blocks and ranks them exactly. Hot chunks' vectors get cached and grow into a model-specific index.

Candidates K Encoder drift from canonical Re-encoded vectors already cached

Re-encoding cost uses a measured 392 chunks per GPU-second (e5-base-v2 on an A100).

Store vectors or recompute them?

wiki-18 is 13.4 GB of text and needs a 64.6 GB flat index. Recompute-based indexes such as LEANN¹ keep the text and materialize vectors during search. Where does your PoP sit?

Embeddings stored at this PoP

References

Yichuan Wang et al. LEANN: A Low-Storage Vector Index. arXiv:2506.08276, 2025.
Peichun Hua and Yunming Xiao. Semantics Delivery Network. HotNets 2026. The tenant model.

Further reading (11 papers, click to expand)

Sharing one index across encoders

Yantao Shen et al. Towards Backward-Compatible Representation Learning. CVPR 2020. Trains a new embedding model so its vectors stay comparable with the old index.
Vivek Ramanujan et al. Forward Compatible Training for Large-Scale Embedding Retrieval Systems. CVPR 2022. Prepares today's embeddings to be compatible with future models.
Harshil Vejendla. Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases. EMNLP 2025. Maps new embeddings onto an old index to upgrade without re-embedding everything.
Rishi Jha et al. Harnessing the Universal Geometry of Embeddings. arXiv:2505.12540, 2025. Translates between embedding spaces of different models without paired data.
Jun Woo Chung and Weijie Zhao. On Reranking Space for Multi-Tenant Retrieval with Adapted Queries. RecSys 2026. Studies reranking when many tenants with adapted queries share one retrieval system.
Aditya Kusupati et al. Matryoshka Representation Learning. NeurIPS 2022. Embeddings that can be cut to shorter lengths, trading accuracy for storage.

Vector indexes and their cost

Yury A. Malkov and Dmitry A. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE TPAMI, 2018. HNSW, the graph index most vector databases use.
Hervé Jégou et al. Product Quantization for Nearest Neighbor Search. IEEE TPAMI, 2011. Product quantization: compress vectors so the index fits in memory.
Suhas Jayaram Subramanya et al. Rand-NSG: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node. NeurIPS 2019. DiskANN: billion-scale search from SSD on one machine.
Matthijs Douze et al. The Faiss Library. IEEE Transactions on Big Data, 2026. The library behind many of these indexes, including ours.
Niklas Muennighoff et al. MTEB: Massive Text Embedding Benchmark. EACL 2023. The standard benchmark for comparing embedding models.

A two-sided service

A reverse proxy and a forward proxy in one

Whose decision is it?

An origin pays a CDN so requests for its pages do not all land on its own servers. The list on the left is the condition for holding that copy. The client pays for retrieval.

Origin contract

Enrollment
Permitted transformations
Freshness and version bounds
Purge and invalidation
Access control
Provenance

Settled with each client

Encoder
Chunking policy
Utility reranker
Outcome labels

Open: can the origin refuse embedding?If the origin says no, a client can still fetch the page the way the web works today. A CDN serves the bytes, and the client embeds them. If SemDN refuses, the client leaves and does that, and SemDN serves less. The origin cannot enforce the ban, and honoring it inside SemDN hurts the service.

Pushback: freshness is the client's problem.The client knows when a question is time-sensitive. The Halden limit changed, and an origin bound can still serve the old number. The client can refuse those chunks and pay for a search. Only the origin knows the page changed, so the bound stays with the origin.

Pricing steers architecture

Charge a registered representation at its unshared cost and discount the canonical one, and tenants have a reason to use shareable state. Set a month's bill.

Representation

Chunks consumed per month

A sample price sheet: $0.10 per 1,000 chunks delivered, plus each representation's share of a $4,000 monthly index and re-embedding cost.

Neither side pays merely because the other uses SemDN

Will anyone show up? Over 36 months, the origins line is how many origins have enrolled, out of all origins. The clients line is how many clients have shown up, out of all clients. Up the side, zero is none of that group and 100 percent is all of it.

Origins enrolled at launch Clients' discovery fallback

Further reading (8 papers, click to expand)

Who pays

Jean-Charles Rochet and Jean Tirole. Platform Competition in Two-Sided Markets. Journal of the European Economic Association, 2003. The economics of platforms that charge two sides, like SemDN charging origins and clients.
Cloudflare. What is Pay Per Crawl?. Cloudflare docs, 2025. Lets sites charge AI crawlers per request, one existing settlement model.
The Associated Press. AP, OpenAI Agree to Share Select News Content and Technology in New Collaboration. AP press release, 2023. A direct content licence between a publisher and an AI company.

What crawling costs origins today

Wikimedia Foundation. How Crawlers Impact the Operations of the Wikimedia Projects. Wikimedia Diff blog, 2025. Wikimedia on how AI crawlers drive up its bandwidth and operations load.
Fastly. Q2 2025 Threat Insights Report: The Rise of AI Crawlers and Fetchers. Fastly, 2025. CDN data on the growth of AI crawlers and fetchers.
Shayne Longpre et al. Consent in Crisis: The Rapid Decline of the AI Data Commons. NeurIPS 2024. Documents sites restricting AI crawlers through robots.txt and terms.
Taein Kim et al. Scrapers Selectively Respect robots.txt Directives: Evidence From a Large-Scale Empirical Study. IMC 2025. Measures which scrapers actually obey robots.txt.
Enze Liu et al. Somesite I Used To Crawl: Awareness, Agency and Efficacy in Protecting Content Creators From AI Crawlers. IMC 2025. Asks whether site owners know about AI crawlers and can block them.

Trust

Why would anyone trust this edge?

A courier versus a librarian

A CDN is a courier. It delivers the site's page sealed, and HTTPS shows the seal is intact. SemDN is a librarian. It cuts pages into passages, picks which ones you read, and is paid by both the site and you. An intact seal says nothing about whether the librarian picked fairly. So a client should be able to ask three questions.

1. Is this really what the site said?The site signs what it publishes, so every passage traces back to the site's own text, at a stated version. This catches made-up and outdated passages.

2. Did I get the best passages, or the ones someone paid for?The edge commits to its index and proves that what it returned really is the top of that index. This catches paid ranking and passages quietly dropped. A courier never had to answer this, because it never chose anything.

3. Was anyone left out?A ranking can be honest over an index that silently dropped a site. So the list of sites that joined is public, the way browsers check a public log of certificates³.

Spot checks, like a ticket inspector

Proving a ranking takes about a second per query¹. No agent will wait for that every time. So check a random sample²: if the edge cannot tell which answers will be checked, cheating is still caught. But every answer that is checked waits for its proof.

Answers checked Proof time Answers the edge cheats on

References

Nikolay Avramov et al. Atlas: Efficient Verifiable Semantic Search. arXiv:2609.11841, 2026.
Zipeng Qiu et al. V3DB: Audit-on-Demand Zero-Knowledge Proofs for Verifiable Vector Search over Committed Snapshots. arXiv:2603.03065, 2026.
Ben Laurie, Adam Langley, and Emilia Kasper. Certificate Transparency. RFC 9162, 2021.

Further reading (9 papers, click to expand)

Proving where content came from

Ralph C. Merkle. A Digital Signature Based on a Conventional Encryption Function. CRYPTO 1987. Merkle trees: prove one piece belongs to a signed whole without sending the whole.
Annabelle Backman et al. HTTP Message Signatures. RFC 9421, 2024. A standard way for a site to sign HTTP messages, one way to sign what it publishes.
Richard Barnes et al. Delegated Credentials for TLS and DTLS. RFC 9345, 2023. Lets a CDN serve TLS on a site's behalf, which is why HTTPS alone cannot vouch for the content.

Proving a search result is honest

Roberto Tamassia. Authenticated Data Structures. European Symposium on Algorithms (ESA), 2003. A survey of data structures whose answers come with proofs anyone can check.
HweeHwa Pang and Kian-Lee Tan. Authenticating Query Results in Edge Computing. ICDE 2004. Proves that answers from an untrusted edge server are correct and complete, the closest older idea to this page.
Qian Chen et al. Authenticating Top-k Queries in Location-based Services with Confidentiality. PVLDB 2013. Proofs that a top-k answer is really the top k.
Rosario Gennaro, Craig Gentry, and Bryan Parno. Non-interactive Verifiable Computing: Outsourcing Computation to Untrusted Workers. CRYPTO 2010. General proofs that outsourced computation was done right, and why they are expensive.

What can still go wrong

Wei Zou et al. PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. USENIX Security 2025. Planting a few crafted passages is enough to steer a RAG system's answers.
Chenchen Gu et al. Auditing Prompt Caching in Language Model APIs. ICML 2025. Detects shared prompt caches from response timing, the same side channel as on this page.

At the poster

Concluding remarks. Come and provide your ideas, suggestions, and comments!

Our preliminary experiments show promising results, but an end-to-end implementation is not complete yet. In principle, SemDN could sit on top of an existing CDN or be a separate infrastructure. Either way, keying delivery on queries changes three things: (1) misses must be estimated instead of keying on exact URL matches; (2) clients choose representations that origins do not see; and (3) acquisition and normalization are shared by all, creating opportunities for cache reuse via locality.

When is cached evidence good enough to serve? Two things decide it: timeliness (is the chunk still current?) and coverage (does the evidence hold everything the query needs?). A coverage miss leaves no trace in a URL or a timestamp. How can an edge estimate both, cheaply, before it answers?
Why not keep the cache on the client? A session cache on the client catches much of what one task re-reads. But a larger cache works better and costs the client more, and a client-side cache exposes how the provider indexes, chunks, and embeds, which may be what it competes on. Clients still need to see enough to trust the results. What belongs on the client, what belongs in the network, and what must be shared across clients to justify a network?
Where should the encoder boundary sit? When does re-encoding stop paying off, and when do you materialize a model-specific index?
How much query state can tenants safely share? Reweighting from one tenant's trajectories can expose its task structure to others.
Why would anyone join first? Origins gain offloaded traffic and fresher copies at the edge. Clients gain one round instead of many. How do you get origins and clients to enroll while the network is still small? And when an origin refuses, how does the edge still serve its content well to clients?
Why trust the edge? A client sees the chunks the edge returns, not the index behind them. Can the edge show that it returned the best chunks its index holds, without revealing the index? A simpler version, and maybe not an easy one either: can it show that each chunk is a faithful, processed copy of the origin page, with no injected misinformation, bias, hidden instructions, or ads?
How does one query find its sources across the web? Cloudflare's AI Index^{1, 2} and AISTRAL's Agentic Twins³ give each site its own index. AISTRAL adds a resolver that picks which sites to ask. Its evaluation uses 10 sites, and the authors leave routing at web scale open. An answer may come from one source or need several together, and the web has millions of sites. Should a query be routed to sites one by one, or should one shared index rank chunks across sites in the client's representation? What does each cost at web scale?

References

Cloudflare. An AI Index for all our customers. 2025.
Cloudflare. AI Search.
Yixi Chen, Muhammad Bilal, Tie Ma, Zafar Ayyub Qazi, Marco Canini, and Suhaib Fahmy. From Crawled to Published: A Publisher-Controlled Web Architecture for an AI-Native Internet. arXiv:2511.18354v2, 2026.

Peichun Hua and Yunming Xiao, CUHK-Shenzhen. Paper on arXiv · Code on GitHub