The headings a keyword search of the tariff schedule can never find

Measured 2026-09-19 on 140 U.S. Customs and Border Protection binding rulings, in two disjoint samples of 100 and 40. For 4% and 10% of products respectively, the heading CBP assigned scores zero against the product's own description — so no depth of keyword list ever reaches it.

If you are building HTS classification, or buying it, the usual first move is a keyword search of the tariff schedule: score each heading by how well its official text overlaps the product description, take the best ones, work from there. It is cheap, deterministic, and it explains itself.

We measured its ceiling. Reading further down the list keeps helping, with diminishing returns — and then it stops, because for some products the correct heading was never on the list at any depth.

Method

We took CBP binding rulings — the classification requests importers file when the answer is not obvious — and used only CBP's description of the merchandise, never the ruling's reasoning or its answer. For each one we ranked every 4-digit heading in the schedule by keyword overlap with that description, and asked a single question: where does the heading CBP actually assigned appear in that ranking?

The scorer is stemmed, stopworded and IDF-weighted, and a heading scores as its best-matching line. No model is involved anywhere in this measurement, and no product's accuracy is reported on this page. Two samples, drawn by the same method and disjoint: 100 and 40 rulings.

Where the correct heading ranks

The two samples agree on the shape and differ on the size of the unreachable group — 4 of 100 against 4 of 40, 8 of 140 overall. We publish both rather than the more striking one.

Why those score zero

They are not exotic goods. They are ordinary products whose correct heading is written in vocabulary the product description does not use:

“Wine” is not “drink.” An “urn” is not an “ornament.” A “harvest decoration” is not “festive.” Each is obvious to a person and invisible to a term-matching scorer, because the schedule and the seller are describing the same object in different registers — one legal and categorical, the other commercial and specific.

Some of them turn on a fact the description does not contain at all. The gasket's heading depends on which engine it is for; nothing in “100% FKM rubber” says. That is a second and harder problem — not a vocabulary gap but a missing-premise one — and no retriever of any kind closes it from text it was not given.

What this means if you are building it

1. Measure your retriever separately from whatever chooses among its candidates. They fail differently and the fixes are different. A single end-to-end accuracy number cannot tell you which half is costing you.
2. Depth is cheap and has a known ceiling. The curve above keeps rising to the top 120 and then flattens against the unreachable group. If your recall is short of that ceiling, more depth is the cheapest fix available; once you are at it, more depth buys nothing.
3. The remainder needs a different signal, not a longer list — embeddings over the schedule text, a synonym or hypernym layer, or a person. Budget for it as a separate problem.
4. Keyword retrieval and semantic understanding solve different failure classes. That is the practical content of this measurement: the two are not competing implementations of one step.

Limitations

• This describes these samples, not tariff classification everywhere. Binding rulings are hard cases by construction — an importer requests one when the answer is not obvious. The unreachable share on ordinary merchandise is very likely lower, and we have not measured it.
• The two samples disagree on that share (4% and 10%). With 100 and 40 rulings, treat the shape of the curve as the finding and the exact percentage as an estimate with a real interval around it.
• One scorer. A different keyword scorer moves the curve. It does not move the zero-overlap cases, which is the part that matters: those share no vocabulary for any term-matching method to find.
• Merchandise descriptions only. A real importer knows more about their own goods than a ruling's description states.
• One schedule snapshot, the USITC Harmonized Tariff Schedule of 2026-09-14.

Data

Every row is published: the ruling, the heading CBP assigned, and the rank at which a keyword search finds it. Recompute rather than trust us.

• results.csv
• summary.json
• README.txt

Rulings come from CBP's Customs Rulings Online Search System, collection ny, newest first, taking only rulings titled “The tariff classification of…” with exactly one code. Descriptions were cut before CBP's reasoning, and any description mentioning a classification term or an HTS-shaped number was dropped rather than cleaned — cleaning is where an answer leaks into a question. CBP rulings are works of the U.S. Government in the public domain; CBP is cited as the source and does not endorse this analysis.

Related: Six digits against CBP's own rulings, where we grade our own classifier on the same kind of data, and is the 6-digit HS code enough?

Corrections. If a figure here is wrong we correct it on this page with the date and what changed, and say so in the changelog. We have withdrawn a published finding before when a larger sample killed it.

Aervik Labs (2026). The headings a keyword search of the tariff schedule can never find [Data set]. Measured 2026-09-19. CC BY 4.0. https://aerviklabs.com/research/keyword-search-hts-ceiling-2026/

@misc{aervik_keyword_search_hts_ceiling_2026,
  author       = {{Aervik Labs}},
  title        = {The headings a keyword search of the tariff schedule can never find},
  year         = {2026},
  howpublished = {Data set, CC BY 4.0},
  note         = {Keyword retrieval study; 140 binding rulings in two disjoint samples; measured 2026-09-19},
  url          = {https://aerviklabs.com/research/keyword-search-hts-ceiling-2026/}
}