Define what “good enough” means
| Method | Evidence | Accuracy | Cost / 1K | Median latency | Auditable | Meets requirements |
|---|
A high overall score can hide a weak class. Inspect recall (how many real cases are caught) and precision (how often a prediction is right) before deciding which errors you can tolerate.
Fairness has not been established. These are label-level diagnostics, not demographic groups. The published artifacts do not establish performance across protected groups, languages, or your production traffic. Deterministic rules can also produce biased outcomes.
Keyword RulesInspect class errors
Published run: comment-spam-detection--rules--20260906T213659Z · youtube-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 59.37–77.22% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 52 | 63.29% | 96.15% | 76.34% |
| spam | 48 | 90.48% | 39.58% | 55.07% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
TF-IDF + Logistic RegressionInspect class errors
Published run: comment-spam-detection--ml--20260906T213659Z · youtube-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 96.30–100.00% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 52 | 100.00% | 100.00% | 100.00% |
| spam | 48 | 100.00% | 100.00% | 100.00% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Small hosted · GPT OSS 20BInspect class errors
Published run: comment-spam-detection--small-model--20260906T213659Z · youtube-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 88.82–97.85% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 52 | 96.08% | 94.23% | 95.15% |
| spam | 48 | 93.88% | 95.83% | 94.85% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Large hosted · GPT OSS 120BInspect class errors
Published run: comment-spam-detection--frontier-llm--20260906T214122Z · youtube-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 87.52–97.22% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 52 | 94.23% | 94.23% | 94.23% |
| spam | 48 | 93.75% | 93.75% | 93.75% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Keyword RulesInspect class errors
Published run: sentiment-classification--rules--20260906T212659Z · sst2-sentiment v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 48.56–65.98% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| negative | 58 | 88.89% | 13.79% | 23.88% |
| positive | 62 | 54.95% | 98.39% | 70.52% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
TF-IDF + Logistic RegressionInspect class errors
Published run: sentiment-classification--ml--20260906T212659Z · sst2-sentiment v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 79.44–91.62% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| negative | 58 | 85.00% | 87.93% | 86.44% |
| positive | 62 | 88.33% | 85.48% | 86.89% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Small hosted · GPT OSS 20BInspect class errors
Published run: sentiment-classification--small-model--20260906T212700Z · sst2-sentiment v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 91.74–98.70% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| negative | 58 | 95.00% | 98.28% | 96.61% |
| positive | 62 | 98.33% | 95.16% | 96.72% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Large hosted · GPT OSS 120BInspect class errors
Published run: sentiment-classification--frontier-llm--20260906T213149Z · sst2-sentiment v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 91.74–98.70% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| negative | 58 | 96.55% | 96.55% | 96.55% |
| positive | 62 | 96.77% | 96.77% | 96.77% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Keyword RulesInspect class errors
Published run: spam-detection--rules--20260906T211650Z · sms-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 83.33–94.19% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 105 | 96.97% | 91.43% | 94.12% |
| spam | 15 | 57.14% | 80.00% | 66.67% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
TF-IDF + Logistic RegressionInspect class errors
Published run: spam-detection--ml--20260906T211650Z · sms-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 95.43–99.85% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 105 | 99.06% | 100.00% | 99.53% |
| spam | 15 | 100.00% | 93.33% | 96.55% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Small hosted · GPT OSS 20BInspect class errors
Published run: spam-detection--small-model--20260906T211651Z · sms-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 79.44–91.62% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 105 | 100.00% | 84.76% | 91.75% |
| spam | 15 | 48.39% | 100.00% | 65.22% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Large hosted · GPT OSS 120BInspect class errors
Published run: spam-detection--frontier-llm--20260906T212141Z · sms-spam v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 88.45–97.15% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| ham | 105 | 100.00% | 93.33% | 96.55% |
| spam | 15 | 68.18% | 100.00% | 81.08% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Keyword RulesInspect class errors
Published run: support-request-routing--rules--20260906T205923Z · cfpb-complaint-routing v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 36.39–53.92% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| bank-account | 18 | 34.38% | 61.11% | 44.00% |
| cards | 17 | 100.00% | 23.53% | 38.10% |
| credit-reporting | 17 | 40.63% | 76.47% | 53.06% |
| debt-collection | 17 | 33.33% | 52.94% | 40.91% |
| loans | 17 | 37.50% | 17.65% | 24.00% |
| money-transfer | 17 | 100.00% | 17.65% | 30.00% |
| mortgage | 17 | 78.57% | 64.71% | 70.97% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
TF-IDF + Logistic RegressionInspect class errors
Published run: support-request-routing--ml--20260906T205923Z · cfpb-complaint-routing v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 69.24–84.05% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| bank-account | 18 | 68.75% | 61.11% | 64.71% |
| cards | 17 | 93.33% | 82.35% | 87.50% |
| credit-reporting | 17 | 73.68% | 82.35% | 77.78% |
| debt-collection | 17 | 80.00% | 70.59% | 75.00% |
| loans | 17 | 86.67% | 76.47% | 81.25% |
| money-transfer | 17 | 70.00% | 82.35% | 75.68% |
| mortgage | 17 | 75.00% | 88.24% | 81.08% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Small hosted · GPT OSS 20BInspect class errors
Published run: support-request-routing--small-model--20260906T205924Z · cfpb-complaint-routing v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 56.97–73.71% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| bank-account | 18 | 50.00% | 55.56% | 52.63% |
| cards | 17 | 60.00% | 70.59% | 64.86% |
| credit-reporting | 17 | 65.22% | 88.24% | 75.00% |
| debt-collection | 17 | 68.75% | 64.71% | 66.67% |
| loans | 17 | 63.16% | 70.59% | 66.67% |
| money-transfer | 17 | 75.00% | 52.94% | 62.07% |
| mortgage | 17 | 100.00% | 58.82% | 74.07% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.
Large hosted · GPT OSS 120BInspect class errors
Published run: support-request-routing--frontier-llm--20260906T210802Z · cfpb-complaint-routing v1.0.0. This fixed snapshot is independent of the live comparison above and never changes its recommendation.
Accuracy uncertainty: 58.69–75.22% · approximate 95% Wilson interval on this sample. This is not a bound for your production traffic and does not change the ranking rule.
| Actual class | Examples | Precision | Recall | F1 |
|---|---|---|---|---|
| bank-account | 18 | 45.45% | 55.56% | 50.00% |
| cards | 17 | 65.00% | 76.47% | 70.27% |
| credit-reporting | 17 | 66.67% | 94.12% | 78.05% |
| debt-collection | 17 | 78.57% | 64.71% | 70.97% |
| loans | 17 | 65.00% | 76.47% | 70.27% |
| money-transfer | 17 | 77.78% | 41.18% | 53.85% |
| mortgage | 17 | 100.00% | 64.71% | 78.57% |
Small classes carry more sampling uncertainty. These point estimates do not establish an error bound or a guarantee. Zero exact train/test overlap does not exclude semantic duplication or provider pretraining exposure.