Curated / Rust
Cloudflare outage on November 18, 2025 - Caused by single .unwrap()
r/rust subreddit · Published November 19, 2025
Reddit user
Summary
On November 18, 2025, a global Cloudflare outage stemmed from a failure in their bot management service, internally named FL2, which was partially written in Rust. According to Cloudflare's technical post-mortem, the root cause was a database permission alteration that caused a metadata query to return over 200 items, exceeding a strictly pre-allocated memory buffer of 200 slots. The service attempted to append these items via a function returning a Result, which failed and produced an Err. This error was met with a .unwrap(), causing the thread to panic and the service to crash.
The subsequent discussion on r/rust, centered on Cloudflare's blog post, debated whether the unwrap was the fundamental cause or a symptomatic failure that exposed deeper systemic issues. Many commenters argued it revealed insufficient validation of internally-generated configuration and a lack of canary deployment practices, with the panic serving as a beneficial circuit breaker that prevented silent corruption. Others contended that while expect() would have provided marginally better logs, the core failure was a flawed assumption about database query results.
Concrete outcomes from the analysis include Cloudflare's public commitment to treat internally-generated configurations with the same rigor as external inputs. The incident also fueled community advocacy for enforcing lints like clippy::unwrap_used and clippy::expect_used in production CI pipelines, and reconsidering panic strategies (e.g., panic = abort) for faster failure detection in distributed systems.
From the source
The Rust community's discussion on r/rust immediately focused on the full failure chain rather than the .unwrap() itself. Key themes:
- the database query returning duplicates was the root cause;
- the panic prevented silent corruption that the previous C system exhibited;
- the real failures were deployment processes (no canary, delayed alerting);
.unwrap()has legitimate uses for invariants.
Notable pushback against 'ban all unwraps' narratives, with emphasis on proper tooling (clippy lints) and development practices. The community positioned this as 'down vs. catastrophic security hole', referencing 2017's Cloudbleed for context.