Data leak
Cloudflare Cloudbleed Memory Leak β OAuth Tokens, Passwords, Private Keys Exposed
Primary Source βIncident Details
On 22 September 2016, Cloudflare deployed a change to its HTML parsing pipeline that introduced a buffer overread bug (named ‘Cloudbleed’ by researcher Tavis Ormandy, in reference to Heartbleed). The bug caused Cloudflare’s edge servers to include uninitialized memory β containing data from other Cloudflare customers’ HTTP requests β in HTTP responses. This data included session cookies, authentication tokens, OAuth tokens, passwords in cleartext, private API keys, TLS private key material, and full HTTP request/response bodies from numerous Cloudflare-proxied websites. Tavis Ormandy of Google Project Zero discovered the issue on 17 February 2017 while investigating memory corruption in search results and notified Cloudflare. Cloudflare patched the bug within hours of notification (18 February 2017) but did not publicly disclose until 23 February. The period of maximum leakage was 13-18 February 2017, though lower-rate leakage occurred from September 2016. The leaked data had been cached by Google, Bing, Yahoo, and other search engines in their caches. Cloudflare worked with the major search engines to purge approximately 770 unique URLs containing leaked data from public caches. Approximately 3,438 Cloudflare customer domains were identified as having been directly active in triggering the overflow. Potentially affected downstream user data spanned millions of users across thousands of websites. The incident affected major Cloudflare customers including Uber, OKCupid, 1Password, FitBit, and others. Despite the severity, no confirmed large-scale exploitation by malicious actors was discovered, though the search engine caching meant sensitive data was potentially publicly accessible for months. Cloudflare estimated the leak rate was approximately 1 in every 3,300,000 HTTP requests.
Technical Details
- Initial Attack Vector
- A bug in Cloudflare's HTML parser (introduced 22 September 2016) caused the parser to read past the end of a buffer when processing certain HTML constructs (including server-side includes, email obfuscation, and automatic HTTPS rewrites); the overrun memory contained data from other Cloudflare customers' HTTP requests including authentication tokens, session cookies, passwords, and private messages β this data was served in HTTP responses to users and cached by Google, Bing, and other search engines
- Vendor / Product
- Cloudflare reverse proxy / CDN / security service
Timeline
- 2016-09-22 Breach occurred
- 2017-02-23 Publicly disclosed
- 2017-02-23 Customers notified