Skip to content

Update GPTTrace (Ettin-1B) entry to v8s - #233

Merged
liamdugan merged 2 commits into
liamdugan:mainfrom
andrewnakas:gpttrace-v8s-update
Nov 4, 2026
Merged

liamdugan merged 2 commits into
liamdugan:mainfrom
andrewnakas:gpttrace-v8s-update

Conversation

@andrewnakas

Copy link
Copy Markdown
Contributor

This updates our existing entry, gpttrace-ettin1b, in place instead of adding a new one, so the leaderboard doesn't get crowded with GPTTrace versions.

  • Model: GPTTrace Ettin-1B v8s, open weights at https://huggingface.co/nakas/gpttrace-v8s-1b (CC BY-SA 4.0). Same base model (jhu-clsp/ettin-encoder-1b) as the previous entry.
  • What changed: v8s is trained to be robust beyond RAID: humanizer and paraphrase rewrites, 2026 generators, and far fewer false positives on everyday human web text. On our held-out RAID-train titles it scores the same as the previous model (TPR@5% 99.94 vs 99.93), so we expect a similar leaderboard score.
  • RAID data use: RAID train only (held-out titles never used). No RAID test data was used for training or calibration.
  • Files: predictions.json (672,000 rows, same ids and [{id, score}] format as before; score = P(machine)) and metadata.json. results.json is left for the CI to regenerate.

Thanks for maintaining RAID!

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

GPTTrace (Ettin-1B v8s)

Release date: 2026-11-01

I've committed detailed results of this detector's performance on the test set to this PR.

On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 99.94 and a TPR of 99.93% at FPR=5% and 99.54% at FPR=1%.
Without adversarial attacks, it achieved AUROC of 99.97 and a TPR of 99.95% at FPR=5% and 99.74% at FPR=1%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

@liamdugan

Copy link
Copy Markdown
Owner

Congrats on the strong performance @andrewnakas let me know if you'd like me to merge your submission

@andrewnakas

Copy link
Copy Markdown
Contributor Author

Thanks @liamdugan yes please merge the submission!

@liamdugan
liamdugan merged commit 133c153 into liamdugan:main Nov 4, 2026
2 checks passed

This branch was successfully deployed

1 active deployment
raid-main — 43538724 Deployed Nov 4, 2026 by liamdugan via evaluate #415
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants