Original research · tariff classification

Six digits against CBP's own rulings

U.S. Customs and Border Protection publishes binding classification rulings: CBP's own description of a product, and the tariff number CBP assigned to it. That is ground truth we did not write and cannot influence. We sent 100 of those descriptions to our own HS classifier; 98 came back with an answer and 2 returned HTTP 503 with none. Of the 98 answered, it matched CBP's 6-digit subheading 23.5% of the time (23 of 98; 95% confidence interval 16.2–32.8%). Every miss is listed below, and the whole run is downloadable.

The result

The shape of the error matters more than the headline. Of the 75 misses, 35 landed in the correct chapter — the classifier usually knows what kind of thing it is looking at and then picks the wrong drawer within it. CBP's answer appeared among our returned alternate subheadings in 8 of the misses, so offering alternates recovers very little.

What the confidence score is actually worth

Each answer carries the model's own 0–1 confidence. It ranks — the mean was 0.817 when the answer was right and 0.618 when it was wrong — but it is badly calibrated: a 0.74 does not mean 74%.

Correction, same day. This page first published at n=45 and reported that none of the 17 answers below 0.7 was correct — a clean "do not rely on this" line. At n=100 that is 2 of 32, and the band table shows the 0.70–0.80 group behaving almost like the low group rather than like the rest of the "confident" answers. The rule did not survive the larger sample, so it is withdrawn rather than restated. Nothing was built on it; it is corrected here, on the product page and on the free tool.

What a refuse-below-X rule would really do, computed over the 98 answered rows — the reason we did not ship one:

Even the best cut here answers a minority of cases and is still wrong more often than a buyer would expect. A threshold does not make this product safe; it only makes it quieter.

Where the error actually lives

The classifier proposes candidate subheadings and then picks one. In 67 of the 75 misses, CBP's code was not among the candidates we returned at all; in only 8 was it present and passed over. That points at candidate generation, not the choice step, as the place the accuracy is lost — so a better prompt at the choosing stage could address at most a small share of these cases. Stated as the inference it is: this study records the alternates we publish, not the full internal candidate list, so absence is consistent with never-retrieved rather than proof of it.

By CBP chapter, over the 98 answered rulings — thin per chapter, listed because the pattern is where any fix would start:

What we did, exactly

1. Frame, fixed before any result was seen. CROSS collection ny, sorted newest first, subject beginning "The tariff classification of", exactly one tariff code of at least six digits. Rulings dated 2026-07-07 to 2026-08-14.
2. The search API requires a search term, which could have selected the sample. It does not: a, the and of each return the same total and the identical first five rulings, so the term is inert. Rulings were taken in date order, never chosen.
3. Only the merchandise description was used. A ruling prints its tariff number in the header and argues its way to it in the body, so the description was cut out between CBP's request sentence and the first line of its legal reasoning.
4. Then a leak gate disqualified the ruling if that description still contained an HTS-shaped number, the words HTSUS, heading, subheading or classification, or the answer's own six digits. 47 rulings were dropped by that gate and none was edited to save it — editing is exactly where a leak would survive. Other drops: 91 whose subject was not a classification, 40 carrying more than one code, 21 where no description could be isolated.
5. Each surviving description was sent once to the live product, and the 6-digit answer compared to CBP's. No retries, no second opinions, no prompt tuning between calls. Two calls returned HTTP 503 with no classification, and a call that returns no answer is not a wrong answer — so accuracy is computed over the 98 that answered, and these two are reported rather than counted as misses. One of them is the product working as designed: its model reply stopped at the output token limit, and this API refuses a cut-off reply rather than serving the partial answer inside it. The other never reached the model at all. Neither was retried, because the retry would have cost more than the approved budget had left.

Limitations, including the one that flatters us

• These are hard cases by construction. An importer requests a binding ruling precisely when the classification is unclear or contested. This set is therefore not a representative product catalog, and 23.5% should not be read as accuracy on ordinary merchandise. It is the most important caveat on this page and it cuts in our favour, which is why it is stated plainly rather than left for a reader to find.
• n = 98, and the interval is still wide: 16.2–32.8%. Per-chapter counts are thinner still and should be read as direction, not rate. This began as 45 rulings and was extended to 100 the same day; the first 45 were not re-run.
• The input is CBP's prose, not a merchant's product title. It is more detailed than what most users type, and in that respect easier than real traffic.
• One call per ruling. The model is not deterministic; a rerun would not reproduce every row exactly.
• 6 digits only. The product never returns 8 or 10 digits by design, so nothing here says anything about the U.S. tariff line, which is what an entry actually requires.

Every miss

All 100 rows, including the exact text the classifier was given and the two that returned no answer, are in results.csv. The 77 misses:

Metadata correction, September 19, 2026. The machine-readable dataset description still described the initial 45-ruling sample after the study expanded. It now derives the attempted and answered counts from the same summary as this page. The benchmark results and earlier correction remain unchanged.

Run it yourself, against anything

• results.csv — every row, with the description given to the classifier
• summary.json — every figure on this page
• README.txt — method, licence and caveats in plain text

The descriptions and CBP's answers are public domain, so this test set works against any classifier, including ones that are not ours. A benchmark only its author can run is a brochure.

Source: U.S. Customs and Border Protection, Customs Rulings Online Search System, read 2026-09-17. CBP's Copyright Notice places its website information in the public domain and asks that CBP be cited as the source; it is. CBP did not participate in this study and does not endorse it. The product graded here is Aervik Labs' Landed Cost / HS Classifier, and the figures are what it returned on 2026-09-17.

Aervik Labs (2026). Six digits against CBP's own rulings [Data set]. Measured 2026-09-17. CC BY 4.0. https://aerviklabs.com/research/cross-six-digit-2026/

@misc{aervik_cross_six_digit_2026,
  author       = {{Aervik Labs}},
  title        = {Six digits against CBP's own rulings},
  year         = {2026},
  howpublished = {Data set, CC BY 4.0},
  note         = {Historical classifier benchmark; 100 attempted rulings, 98 answered; measured 2026-09-17},
  url          = {https://aerviklabs.com/research/cross-six-digit-2026/}
}