Enlarge / The internet this week, more or less (credit: torange.biz (modified))
Last week, Verizon caused a major BGP misroute that took large chunks of the internet, including CDN company Cloudflare, partially down for a day. This week, the rest of the internet has apparently asked Verizon to hold its beer.
Cloudflare went down again for half an hour yesterday, and this time, it was the company’s own fault—we’re still waiting on a full post-mortem, but the short version is that a firewall regular expression rule targeting malicious Javascript spiked the firewalls’ CPU usage, crippling throughput and causing widespread HTTP 502 errors. Microsoft’s Office365 also seems to have experienced a multi-hour partial outage yesterday, with the service working over some ISPs and routes but not others for about four hours.
Office365 partial outage, on Jul 2 reported by third-party users at downdetector.com (credit: downdetector.com)
Facebook and its properties WhatsApp and Instagram have suffered widespread outages relating to image display for most of today. The problem seems to be bad timestamp data being fed to the company’s CDN in some image tags; when I looked into the broken images littering my own Facebook timeline, I discovered different timestamp arguments embedded in the same URLs. Loading an image from fbcdn.net with bad “oh=” and “oe=” arguments—or no arguments at all—results in an HTTP 403 “Bad URL timestamp”. If you dig into the code, Facebook refers to the same image URL with different arguments in its anchor tags and its image tags; in the posts with broken images, the URLs in the anchor tags work, but the ones in the image tags don’t. (On the plus side, now everybody’s suddenly noticing the AI-driven hidden alt tags the company’s been embedding in its images since 2017.)
Read 2 remaining paragraphs | Comments
Source: Ars Technica – Facebook, Cloudflare, Microsoft, and Twitter suffer outages