GitHub uncovers secrets with the compact ModernBERT model: higher accuracy, greater speed, and cost under one percent

When a developer pushes a change to a repository, the scan is supposed to uncover real keys, not random strings that look like them. For years, the solution lay in regular-expression rules that cast a wide net, catching secrets but also dragging along thousands of false alerts: documentation values, test variables, checksum collections, placeholder strings. The result is familiar to every security team: developers ignore the alerts, and real secrets slip out amid the noise.

GitHub announced this week that it has replaced its pipeline built on a large language model with a precise, compact classifier based on ModernBERT, and the result measured in production rather than in the lab: owners' acceptance of alerts rose by 11.5 %, post-push scan time fell by 86 %, and service cost dropped by more than 98 % compared with the previous pipeline. In other words, the scan became faster, more accurate, and tens of times cheaper at the same time.

The detector's paradoxThe problem is not finding something that looks like a secret, but separating what “appears” to be a secret from what actually is one. The same text string might be a production password next to a database URL, or it might be the exact value in a local test file. Traditional rules add an exception for each case, expanding the exception list and hiding real secrets. The compact model learns from context, file path, variable names, surrounding words, and distinguishes an environment variable in deployment settings from a similar string in an experimental-settings directory, without having to program each exception individually.

Hybrid engineering, not a single modelThe regular expression (regex) remained in the first stage as a fast, transparent candidate filter, after which each candidate was passed to the compact ModernBERT, which reads the candidate with its limited context and assigns a probability score. The thresholds were chosen based on offline evaluation and then validated on production traffic before wide rollout. The model was released in ONNX format with 8-bit quantization, making inference fast enough to scan every push before it is accepted, enabling push protection to operate on public secrets, not only after a commit.

Training on ambiguous casesTraining drew on three sources: examples derived from GitHub’s own product with human augmentation; high-recall regular-expression candidates from repository data with human documentation and weak labels to broaden coverage; and synthetic examples targeting rare and difficult cases, documentation secrets, test fixtures, public keys, checksum collections, near-duplicate pairs. The core principle: let the model see the ambiguous examples so it learns not only how a secret looks, but also how a false positive appears in context.

What does this mean for developers in the region?Development teams in the Gulf, Egypt and the Levant that rely on GitHub will see fewer noisy alerts, faster response in daily workflow, and broader coverage of secret types without the bill climbing. As AI-driven code generation accelerates, the volume of generated code rises, and so does the chance of key leakage, meaning scanning tools must keep pace rather than become a bottleneck. GitHub’s hybrid approach offers a model of what built-in security tools can be: precise, fast, and cost-effective, instead of massive models that consume resources and slow developers.

What’s worth followingGitHub announced two next steps: building a continuous improvement loop that turns production feedback into higher-quality training data, and exploring architectures that perform detection and extraction in a single step to simplify the pipeline while preserving accuracy and scale. If successful, the next time a developer pushes code they will not see a false alert, a further step toward genuine confidence in built-in security tools.