GitHub Secret Scanning Adopts a New Classifier That Uses Context to Tell Real Secrets from Lookalikes; GitHub Says False Positives Fall and Scans Get Faster

What happened

GitHub published an article on its official blog describing a context-aware secret classifier trained for secret scanning. Here, a "secret" means a credential that should not be public, such as a password, API key or connection string; secret scanning is the feature that finds such content in code. GitHub says secret scanning works at two points: push protection scans changes before a push (uploading code changes to a repository, the project space that stores code) is accepted, while post-push scanning continuously scans repository content after code is committed and creates an alert when it detects a potential secret.

GitHub says that, compared with its earlier LLM (large language model) based flow, the new classifier cut median post-push scan runtime by 86%. This efficiency let generic secret detection extend to push protection, returning results before developers push.

Why generic secrets are hard to judge

GitHub says generic secrets do not always have a distinctive prefix, length, character set or checksum (a string of characters used to verify data is correct). Their syntax can overlap with non-sensitive strings such as build identifiers, checksums, asset hashes and sample values, so context is the deciding signal. For ordinary readers, this means the same string of characters can mean something entirely different in a production configuration file than in a test example.

How the new flow works

GitHub says the new scanner uses a two-stage flow. The first stage uses regular expressions (regex, a method of matching text patterns with fixed rules) to cast a wide net for candidate strings, aiming for "high recall", meaning real secrets are missed as little as possible. In the second stage, a fine-tuned (retrained for this task) ModernBERT classifier evaluates each candidate along with its surrounding context; GitHub describes the model as small and efficient. The model's input includes the candidate value, file path, secret type and a limited range of surrounding context, and its output is a classification score that the scanner uses to decide whether the candidate proceeds.

GitHub says the classifier can filter out placeholder values, test fixtures (fake data used in tests) and similar strings such as checksums, and only high-confidence findings become alerts. GitHub also admits the classifier is only a best effort at distinguishing "looks like a secret" from "likely a real secret".

Results published by GitHub

In the table, "precision" is how many of the alerts raised are truly secrets, and "recall" is how many of the real secrets are found; an "A/B experiment" assigns the old and new flows to different customers and compares the results. GitHub explains that acceptance is weighted by repository owner to reduce the distortion caused by customers who resolve many alerts at once.

Training and deployment details

GitHub says the training data combines three sources: product-derived examples from GitHub repositories (including feedback from the secret detection feature and human validation), high-recall regex candidates (including human labels and weak labels, the latter meaning rougher automatic tagging), and synthetic examples built specifically for hard cases such as secrets in documentation, test fixtures, public keys and checksums. Before training, rows sharing the same repository, candidate value or example were grouped, to keep near-duplicate data from appearing in both the training and evaluation sets.

GitHub also says the model was exported in ONNX, a model file format, with 8-bit quantization (storing the model's numbers at lower precision) to speed up inference, the process by which the model actually makes judgments. Next steps focus on two directions: using production feedback to build a continuous improvement loop, and exploring new model architectures that can detect and extract secrets in a single step.

What it means for ordinary readers

GitHub says this should let developers and security teams see fewer false positives and get faster feedback in commits, pull requests and everyday development workflows. GitHub's reasoning is that higher precision means fewer distracting alerts, higher recall means more real secrets caught, and lower cost lets it keep improving scanning coverage.

Browse the latest news in this topic

• Tech news