Data issues 28/10/2023

As some of you may have noticed, the RailChecker app was unable to show live information for most of the day on Saturday 28th October 2023. This was due to the National Rail data feeds being shut down twice. CACI (who provide the open data service that RailChecker uses on behalf of National Rail) have since issued updates to explain the issues, and I have put them down here.

On Friday 27th October, a major security vulnerability was found with the software that CACI use to allow open data users (like RailChecker) to connect and receive data from National Rail. CACI therefore decided to shut down the server with immediate effect without telling anyone at 08:23 on Saturday 28th October to undertake remedial work - the reason behind this was not known until 12:06. 

CACI restored the data connection at about 13:30, and apart from an issue with the RailChecker server (possibly caused by the thousands of update messages backlogged coming at the same time) causing an unusually long emergency restart at about 15:00, there were no issues until 17:20. Unfortunately, the National Rail server then stopped working again, due to another security breach.

Normal service was restored at 01:04 29/10/2023. On behalf of National Rail, I can only apologise for any difficulties you experienced in using the app on Saturday.

A history of updates from National Rail is available below.

Status information is available on the Status page.

UPDATE 29/10/2023 01:33

Incident resolved - from National Rail:

CIS Pushports down due to critical vulnerability
Resolution Details:
New servers built and patched to address the vulnerability
Incident Start Date / Time: 19:37hrs
Incident Resolution Date / Time: 01:04hrs

UPDATE 29/10/2023 01:05

From National Rail:

Testing with public connectivity is taking place and looking positive
Who Is Impacted: CIS
Incident Reference: [redacted]
Incident Start Date / Time: 9:40 hrs
Next Update Due: 02:00 hrs

UPDATE 29/10/2023 00:57

Messages are being received again.

UPDATE 29/10/2023 00:09

From National Rail:

Primary server is under going final testing, with no issues being experienced, Services connectivity is currently being assesses whilst secondary server is currently being patched and configured.
Who Is Impacted: CIS
Incident Reference: [redacted]
Incident Start Date / Time: 9:40 hrs
Next Update Due: 01:00 hrs

UPDATE 28/10/2023 23:14

From National Rail:

New servers are in being built and currently undergoing testing before deployment
Who Is Impacted: CIS
Incident Reference: [redacted]
Incident Start Date / Time: 9:40 hrs
Next Update Due: 00:01 hrs

UPDATE 28/10/2023 22:04

From National Rail:

Please be aware that all Push Ports are currently down due to a Security breach on active MQ
Please note, Live Departure boards are not affected, only CIS Push ports
Who Is Impacted: CIS
Incident Reference: [redacted]
Incident Start Date / Time: 9:40 hrs
Next Update Due: 22:00 hrs