A major Rogers outage has cut off 25 percent of Canada’s internet traffic

Canadian telecom Rogers is suffering a major outage affecting landline phones, cellular connections, and internet connectivity throughout Canada that started early this morning. Downdetector listed thousands of reports for the issues that flooded in as people started to get up around 5AM ET and couldn’t get online.

Rogers first addressed the outage in a tweet from its official support account just before 9AM ET and then went silent for a couple of hours. On Friday afternoon, the company tweeted that its technical teams are working to restore services “alongside our global technology partners, and are making progress.” There is still no ETA for restoration, even after services have been available for about 12 hours nationwide. In a follow-up tweet, the company promised that it would be “proactively crediting all customers,” and said that it had “every technical resource and partner” working on getting its network back up.

Image: Cloudflare

Cloudflare Radar shows communication between its load-sharing network and Rogers in Canada exhibiting its normal nightly pattern before dropping to essentially zero at around 3–4AM ET and registering a flat line ever since. At that same time, there was a massive spike in BGP (Border Gateway Protocol) announcements for the network that indicate changes in routing.

In a blog post published Friday afternoon, Cloudflare personnel speculated that the outage is the result of an internal error at Rogers, as opposed to a cyberattack or other cause. They cited that pattern of BGP announcements, noting that with the Rogers network now failing to announce its presence, the rest of the internet can’t find it. BGP is a fundamental part of the technology that helps move information from one place on the internet to another, and a problem with BGP routing information was the cause of a massive Facebook network outage last fall.

The outage has also disconnected Interac, the network Canadian banks use, disabling debit cards, ATMs, and the e-Transfer services that recently notched over 1 billion transactions in one year.

Passport offices and Canada’s tax-collecting Revenue Agency are among the government services that are unavailable today due to the outage. Both agencies also warned users the outage is cutting off multifactor authentication codes sent by voice or text message, so people who are logged out may not be able to log in at this time.

Even the Canadian Radio-television and Telecommunications Commission (CRTC), which regulates broadcasting and telecoms, says its phones aren’t working due to the outage.

Internet network disruption tracker NetBlocks shared real-time data showing Canada’s national connectivity has dropped to just 75 percent of its normal levels.

A report by CBC News noted Rogers subsidiary Fido is having problems, too. The article says problems have affected card payment processing and ATMs across Canada. A CBC radio station in Kitchener, Ontario, is also offline, and some telephone services for Ottawa’s transit agency have been knocked out. Pictures from Toronto show people crammed into coffee shops or heading to the library to access Wi-Fi now that their other options are offline.

TekSavvy customer service has told customers the outage extends to “Ontario, Quebec and the Eastern Provinces,” with no ETA for resolution, while the CBC reports that Thunder Bay operator Tbaytel says the Rogers outage is nationwide.

Toronto police tweeted early in the morning after some people in the city experienced connection issues calling 911. “The Rogers Network is experiencing some technical difficulties,” said the Toronto Police Operations Twitter account. “We are working to resolve these issues.” In a follow-up tweet, the police confirmed their 911 center is operational and advised anyone who has to call to stay on the line if they connect and to retry calls if they don’t.

Update 9:52AM ET: Added Bank of Montreal information

Update 11:20AM ET: Noted ongoing outage, added information from Cloudflare Radar, TekSavvy, and Tbaytel

Update 11:33AM ET: Added statement from Rogers

Update 12:40PM ET: Added latest update from NetBlocks

Update 1:31PM ET: Added additional information about Interac banking network, — thanks, mbarriault.

Update 2:13PM ET: Added information about CRTC, passport offices, and tax services outage.

Update 3:39PM ET: Added Cloudflare notes about BGP as a possible cause of the outage, as well as Rogers’ statement that it is “making progress” in fixing the issue.

Update 5:56PM ET: Added statement from Rogers about crediting customers.

Source