This meeting will receive an update on the performance of the Shared Technology Service (STS) that provides IT services to Brent, Lewisham and Southwark Councils. It will also consider approving the Shared Technology Service Strategy 2024-2026, the Shared Technology Services Cyber Security Strategy 2024-2026 and will discuss a cybersecurity incident that took place in July of 2024.

Shared Technology Service Update Report

This report updates the committee on how the Shared Technology Service is performing. It shows that the STS responded to 57,889 incidents and requests between July and October 2024, an increase of almost 5000 tickets on the previous 4 months.

It highlights the fact that the STS is trialling a new approach for people needing in-person support in one of the three boroughs. It will no longer be necessary for visitors to Brent Civic Centre, Laurence House or 160 Tooley Street to have already logged a ticket before arriving. Visitors will be greeted by an STS engineer who will log a ticket on their behalf. The report also highlights plans to introduce new signage to improve visitors’ experience and explains that a reception service will be introduced in the future to triage visitors' needs more quickly.

The report also highlights that the contract for the Service Desk’s telephone support is due for renewal on 1st April 2025. The report notes that market engagement has been completed and that the tender documents are being prepared.

The report discusses a number of projects that are underway to improve the IT infrastructure across the three boroughs. This includes a plan to replace all of the Councils' laptops by the end of 2025, with Windows 11. In preparation for this, the report shows that a contract for 4,500 Lenovo laptops has been awarded to CDW. It highlights that a pilot scheme will begin in November 2024 in Brent and the Local Government Association. A separate pilot scheme is underway in the LGA for Microsoft InTune, which will be used to deploy software and updates to laptops. The report notes that Southwark Council is currently in the process of developing a business case for the second phase of the project in their borough, and that Lewisham Council is also drafting a business case for the project in their borough.

The report also discusses a project to replace the network infrastructure across all three boroughs. The report shows that Virgin Media circuits with an SD-WAN overlay will be used to connect council offices to the network and that the project is already underway in all three boroughs.

The report also discusses plans to implement a managed XDR service. A contract for the service has been awarded via the G-Cloud 13 framework to an unnamed supplier. The report explains that the service will provide an additional layer of security for the councils' laptops, and will allow the supplier to help the STS to respond to cybersecurity incidents on the councils' behalf.

Shared Technology Services Cyber Security Update Report

The agenda indicates that a separate report, not available to the public, will be considered in the private part of the meeting. It will update the committee on the cybersecurity status, threats and mitigations for the Shared Technology Service.

Briefing note for JC for CrowdStrike incident 2024-07-19

This briefing note describes an incident that took place on 19th July 2024.

The note explains that all three councils use a CrowdStrike product on their servers to protect them from cyberattacks. It explains that, at 5:30am on 19th July 2024, the CrowdStrike agent installed on 932 of the councils’ 934 servers received a routine software update. It goes on to explain that a logic error in the code caused the servers to crash and that the servers could not be recovered without manual intervention or data restoration from backups.

The note explains that the crash meant that almost all of the councils' online services were unavailable because it was not possible for people to log in. It says that some services that are hosted outside of the councils' infrastructure, and therefore did not have the CrowdStrike agent installed, may have still been available. It also explains that the councils' websites were still accessible, although some of the functionality on the websites was not available. The report explains that Southwark's website was down until 11:00am because of its reliance on services that were affected by the outage, and that it was not until 2:00pm that most of the websites' functionality had been restored for all three councils.

The note goes on to explain that the STS became aware of the issue at 6:30am and assembled a team to deal with the incident. The note explains that the team focused first on the underpinning infrastructure that allows the rest of the councils' IT systems to operate. They contacted CrowdStrike to get information about how to recover the servers, and managed to get these services back online by 10:30am. The note explains that they then worked to restore the most important applications such as social care, housing, revenues and benefits and ERP, which they had working by around 2:00pm. After that they restored less important services such as libraries, street cleaning and planning systems. The note explains that this work took until around 6:30pm, and that almost all services were back online, although some work continued over the weekend and the following Monday.

The note explains that they used a variety of methods to recover the systems. It says that they used some of the methods provided by CrowdStrike, but that in some instances, where data integrity was not important, they found it was quicker to restore from a backup. It goes on to explain that they were lucky because the crash happened shortly after a scheduled backup was taken. The note concludes by saying that they were also fortunate that they had installed the CrowdStrike agent on laptops belonging to councillors and senior managers. Because most of these laptops were offline at the time that the update was released they avoided a much wider problem.

Laptop Refresh Project – Procurement process review

The agenda indicates that a separate report, not available to the public, will be considered in the private part of the meeting. It describes the procurement process that was used to purchase laptops for the councils.