What came first: the CNAME or the A record?

Summary

On January 8, 2026, a routine update to 1.1.1.1 aimed at reducing memory usage accidentally triggered a wave of DNS resolution failures for users across the Internet. The root cause wasn't an attack or an outage, but a subtle shift in the order of records within our DNS responses. While most modern software treats the order of records in DNS responses as irrelevant, we discovered that some implementations expect CNAME records to appear before everything else. When that order changed, resolution started failing. This post explores the code change that caused the shift, why it broke specific DNS clients, and the 40-year-old protocol ambiguity that makes the "correct" order of a DNS response difficult to define. All timestamps referenced are in Coordinated Universal Time (UTC).TimeDescription2025-12-02The record reordering is introduced to the 1.1.1.1 codebase2025-12-10The change is released to our testing environment2026-01-07 23:48A global release containing the change starts2026-01-08 17:40The release reaches 90% of servers2026-01-08 18:19Incident is declared2026-01-08 18:27The release is reverted2026-01-08 19:55Revert is completed. Impact ends While making some improvements to lower the memory usage of our cache implementation, we introduced a subtle change to CNAME record ordering. The change was introduced on December 2, 2025, released to our testing environment on December 10, and began deployment on January 7, 2026. How DNS CNAME chains work When you query for a domain like www.example.com, you might get a CNAME (Canonical Name) record that indicates one name is an alias for another name. It’s the job of public resolvers, such as 1.1.1.1, to follow this chain of aliases until it reaches a final response:www.example.com → cdn.example.com → server.cdn-provider.com → 198.51.100.1As 1.1.1.1 traverses this chain, it caches every intermediate record. Each record in the chain has its own TTL (Time-To-Live), indicating how long we can cache it. Not all the TTLs in a CN...