5 Lessons from GitHub's Rate Limiting Overreach: When Defenses Become the Problem

Building robust defenses is crucial for any large-scale platform like GitHub. Rate limits, traffic controls, and protective measures are essential to maintain availability during abuse or attacks. However, there's a hidden danger: these same protections can silently outlive their purpose and start blocking legitimate users. This happened recently at GitHub, leading to user frustration and an important internal cleanup. Here are five key lessons from that incident.

1. The Problem: Overprotective Defenses That Outlive Their Purpose

When a platform faces abuse, emergency responses often involve adding broad protections quickly. These measures are designed to stop malicious traffic, but they can become outdated. Over time, user behavior changes, and what was once a strong signal of abuse may no longer be relevant. At GitHub, protection rules added during past incidents were left in place long after the threat had passed. These rules continued to block legitimate users, causing 'too many requests' errors for normal browsing. The lesson is clear: defenses need regular reviews to ensure they remain effective and don't become a liability.

2. What Users Reported: False Blocks During Normal Browsing

Users took to social media to report receiving 'too many requests' errors while performing routine actions—like clicking a link from a third-party app or casually browsing GitHub pages. These were not heavy users or automated scripts; they were individuals making a handful of normal requests. The errors were confusing and disruptive, as there was no obvious pattern of abuse. GitHub’s support team quickly correlated these reports with a spike in false positives, confirming that legitimate traffic was being unfairly limited. This feedback was the catalyst for investigation.

3. Root Cause: Outdated Incident Rules Still in Place

Investigating the reports, GitHub’s engineering team discovered the root cause: protection rules added during past abuse incidents had never been removed. These rules were based on patterns that had been strongly associated with abusive traffic when they were created. However, those same patterns were now also matching some logged-out requests from legitimate clients. The rules were essentially 'leftover' from emergency responses, highlighting a gap in lifecycle management for security measures. No one had audited these rules after the incidents were resolved.

4. Composite Signals: Why False Positives Happen

The protection system used composite signals—combinations of industry-standard fingerprinting techniques and platform-specific business logic. This approach helps distinguish legitimate usage from abuse by analyzing multiple attributes. However, composite signals can produce false positives. In this case, only requests that matched both the fingerprinting and the business-logic rules were blocked. About 0.5–0.9% of fingerprint matches triggered the full block. While this seems small, it meant that real users who happened to match the outdated business logic were incorrectly limited. The system was working as designed, but the design was no longer accurate.

5. The Impact: Small Percentage but Unacceptable for Affected Users

Overall, false positives represented only 0.003–0.004% of total traffic—a tiny fraction. But for the customers who experienced the error, any incorrect blocking is unacceptable and disruptive. GitHub recognized that even a low false-positive rate can erode trust and hinder productivity. The company apologized publicly and reinforced that observability is critical not just for features but also for defensive systems. The incident led to a cleanup of outdated mitigations and a commitment to regularly re-evaluate protection rules. The lesson is that scale doesn't excuse negative user impact; you must proactively manage your defenses.

In conclusion, this incident underscores that security measures require ongoing maintenance. Just as features evolve, so should the rules that protect them. GitHub’s experience shows the importance of auditing emergency responses as soon as the crisis ends, and of listening to user feedback to catch problems early. The goal is to keep defenses effective without becoming the enemy of legitimate usage. Moving forward, regular reviews and better observability will help prevent similar overreach.