What the score is
The score is an estimated probability that a repository becomes inactive within the next 12 months, checked against what actually happened to held-out repositories the model never trained on. It is calibrated: across many repositories, a group scored around 30% really does go inactive roughly 30% of the time. So the number is meant to be read as a probability, not a verdict or a grade.
Where it is more or less reliable
Aggregate accuracy hides real differences between kinds of repositories. The evaluation slices the held-out set so these differences are visible:
Widely-used repositories
The most reliable: they emit rich, consistent public signals, so the score is well-supported.
Obscure / low-signal repositories
Weaker: with little public activity there is less evidence, so treat these scores more cautiously — the per-repository evidence support will usually reflect this.
Cold-start (no reconstructed history)
Less precise than full-history by design, and measured on seed-like repos, so a brand-new submission can be out-of-distribution. Use it as a first read, not a final word.
Already-quiet repositories
Part of the signal is simple persistence — a repo that is already silent tends to stay silent. The model still adds value on repos that are active at observation time, which is the harder, more useful case.
What this means when you score a repo
- Read the probability together with the evidence support and margin — a well-supported, decisive score is far more actionable than a borderline one with low coverage.
- Low evidence support usually means missing or unusual signals, not that the repo is fine — gather more evidence before acting.
- Use it to prioritize review, not to auto-reject dependencies. It is conservative triage support.