Whatsapp messages can neither be sent nor received, and behaves in the same way it up on facebook. Publishers like t3n cannot even publish new news there at the moment. The server API returns an error. Instagram users have problems loading feeds and stories – in short: the Zuckerberg empire is completely down.
Both the Facebook Messenger and Whatsapp have now completely stopped the service
The German portal allestören.de is also literally flooded with error messages. For the Whatsapp messenger service alone, the portal received more than 20,000 fault reports in just over half an hour. The Facebook Messenger has completely stopped the service. He reports “no server connection”. In the meantime even the status page is down, which is supposed to show which errors there are on Facebook.
Whatsapp has in the meantime reported via Twitter, but without offering significant information or a perspective.
“Some people” would have problems accessing Whatsapp, it says there, while actually no access to the service – be it via the app or via the web. The most telling error message is currently provided by Instagram. Whoever calls up the website receives a 5xx error after a considerable waiting time. In general, it describes errors that are caused by the operator’s server infrastructure. So this is not just a misdirected DNS entry, as we were able to find out a few days ago at Slack.
In the meantime, Facebook has also dared to make a similar statement on Twitter. The best way to get information from the Zuckerberg empire seems to be on Twitter on Monday evening.
Zuckerberg servers report errors
5xx errors can affect the server itself, but also upstream gateways or proxies. Facebook has all of it in action. So we only know: The problem lies with Facebook and with some certainty not with a service provider such as a CDN. In this context, we will not forget the recent failure of large parts of the network due to problems at Fastly.
On Hacker News is about DNS problems speculated. But that would have to affect the domain name services in the Facebook infrastructure. It goes without saying that they would be down if the servers themselves were also down. Shortly before 7 p.m., more and more details become known, even if there is still no meaningful statement from the Zuckerberg empire itself. It is now clear that all Facebook name servers are not only down, but are no longer entered in the name services worldwide. Queries to facebook.com and whatsapp.com do not bring any result, i.e. they do not return the actual IP addresses, just as if the names were not assigned an address at all.
This means that the Zuckerberg services could no longer find each other – speaks for this: Oculus is no longer available either.
Journalist Dylan Byers claims to have leaked an internal Facebook memo. It reads like this:
In essence, the memo confirms that the problem affects both the internal and external resolution of IP addresses. This is strange in that internal name resolution would not go through external DNS servers. Speculations about a much larger problem seem justified.
DNS problems become more obvious, but cause unclear
A Reddit user wants a source on the technical team investigating the issues. This source is said to have said the following:
As many of you know, the DNS for FB services is affected. This is probably just a symptom of the real problem, which is that the BGP peering with Facebook’s peering routers has gone down. This is most likely due to a configuration change that took effect shortly before the outages (from around 1540 UTC).
There are people now trying to gain access to the peering routers to make corrections. But the people who have physical access are separate from the people who know how to authenticate to the systems and the people who know what to do. So it is now a logistical challenge to combine all this knowledge.
This is partly due to the minor Staffing in the data centers due to the pandemic measures.
Shortly after 8 p.m. on Monday evening, there are increasing indications that the problem could be a serious error in the DNS configuration of the Facebook domains. The security expert Brian Krebs even wants confirmation of this, but does not say where from.
In any case, Krebs writes that all Facebook DNS entries were removed from the global routing tables on Monday morning in California. Since Facebook has sole control over its DNS records, it is still unclear how this could happen. An internal, system-wide update is conceivable, which could have failed.
The network specialist Doug Madory specifies that the deleted routes are so-called BGP routes that are sent via remote Connection can be configured. If you now delete these entries, you take remote access. So it could be that Facebook actually has to manually access the affected hardware, which can be difficult because hardly anyone is on site.
This statement is quite close to the one above, which is said to have been posted on Reddit by the alleged member of the investigation team. From the point of view of the CDN service provider Cloudflare, a connection with a BGP error is obvious:
Faults in other services apparently randomly correlated
In the meantime, malfunction en also reported with other services, such as signal or telegram. In the opinion of an expert in network infrastructure, however, this has only a very indirect connection. The head of technology at the cloud service provider Cloudflare, John Graham-Cumming, pointed out that users and software were still trying to access Facebook services. This would fundamentally increase the load on DNS services worldwide, which could then lead to disruptions which, however, have nothing to do directly with the services concerned. In fact, no extensive interference with other services has been verified so far.
The New York Times asked two cybersecurity experts whether a hacker attack appears likely. They said no, because the technology behind the individual apps was so different that they couldn’t be brought offline with a single cyberattack.
NYT Reportering Sheera Frenkel reports on Twitter from a phone call with a Facebook employee. He described a real chaos to her. If the workforce tried to enter the building on Monday morning, their digital keys would not work. Apparently, Facebook really has broken down into all its individual parts. There are apparently no areas that are not affected.
And the internet so …
The Facebook share price was almost even faster than the servers:
Which is no wonder, because such a failure is not cheap, as a current overview from the Statista service shows. After that, Facebook alone makes a turnover of around 202,000 US dollars per minute (!):
Twitter insisted on welcoming “literally everyone”:
McDonald’s felt addressed directly and offered its delivery services:
hi what can i get u
– McDonald’s (@McDonalds) October 4, 2021
Of course, we also let e first jokers are not long in coming. This Twitter user is hoping that the hashtag #deletefacebook might have worked:
Also at Netflix it has meanwhile arrived that the stable communication service will be called Twitter on Monday:
Fans of Facebook, Instagram and Whatsapp may turn out to be more flexible than expected:
Meanwhile this one wants Twitter user Mark Zuckerberg took pictures at work:
What doesn’t work today is this:
We are constantly expanding the article with new information and findings.