Europe’s flagship payments system died for 11 hours

Over a decade ago, I curated a book about Europe’s Singe Euro Payments Area (SEPA). I was quite boring ten years ago, before FinTech made me cool. However, I still watch SEPA, the PSD regs, MiFID and more. I’m still a bit boring underneath it all. I don’t blog about it, because the audience for this blog has changed over the last decade since I started doing it daily back in February 2008, but today is one of those days. In fact I just realised I haven’t blogged about this particular subject for around five years.

Anyways, for the regular audience, switch off now if you aren’t into detailed infrastructure discussions around payments and risk.

On Friday, Target2 – the main payments system for Europe – went down for 11 hours. During that period, zero cross-border payments could be made. What happened? It’s difficult to know – as it’s all running over in Germany in a special relationship between the Bundesbank and the ECB – but the gist seems to be this (with thanks to Christian Kirchner of Finanz Szene, Germany).

Everything about the 11-hour (!) Failure of EU payment transactions

When Target2 is mentioned in public, it is usually about the fact, once popularized by Professor Sinn, that the Bundesbank has amassed billions in claims against the ECB. Those familiar with the subject know, of course, that the key word Target2 stands for something else. Namely not for an economic debate. But for large parts of the payment traffic in the euro zone.

The Target2 system is used to process payment transactions, liquidity transfers and securities transactions totalling almost two trillion euros every day or, rather, at least on normal days. Last Friday, however, everything was different. The ECB’s Target2 system suffered a technical collapse at around 2.40pm, which became public on Monday morning thanks to exclusive research by FinanzSzene.de. The total failure lasted almost 11 hours, with some banks transactions accumulating to the level of tens of billions of euros. Insiders are using descriptions like an “unprecedented breakdown” and “unbelievable embarrassment”. But above all, it could have been worse as the cause of the mega-collapse is said to be completely unclear.

Here is everything that we know so far about the incident.

What exactly happened?

On Friday at 2:40 p.m., the Target2 system collapsed. The IT damage was largely repaired on Saturday morning at around 1:20am. In the meantime - for almost 11 hours - the commercial banks in the Eurozone could neither process payment transactions nor transfer liquidity nor settle securities transactions. In order to give the institutions the opportunity to catch up on at least some of the transactions, the Target2 system was kept open until 3:30am on Saturday morning (the normal “cut-off time” would have been 6pm on Friday evening).

In its memo on the incident (in English), the ECB literally speaks of a “major incident” and of “internal issues”. However, it has not yet revealed any technical details about the incident. It is also clear that all security systems and “backups” did not work when they were needed.

Even the official ECB announcement speaks of the fact that all safety systems that should have started, in the event of technical difficulties in the Target2 system, failed: “Neither the failover to the secondary site in the same region nor the opening of the contingency module were possible” it says.

Oh dear. So much for risks and controls, backups and disaster recovery, mission critical systems planning and processing.

Christian’s excellent article in German gives a lot more background and the ECB posted an update this morning saying it’s all fine now, as such, with no explanation of what happened.

Meantime, an interesting comment from Peter Grosskopf, Geschäftsführer / CTO at Börse Stuttgart Digital Exchange GmbH, on LinkedIn:

“Wow! A major incident that affected the TARGET2 and SEPA payments system. Customers across Europe were not able to move their money. Are IT risk management rules and procedures not strict enough? Well, NO! Centralized systems like in traditional banking infrastructure simply are prone to regularly fail, even if you invest endless amounts of money and effort into redundancy and failover. Do you remember last time when the internet was down? It never was, cause it’s decentralized. There is a bunch of decentralized systems like Bitcoin, Ethereum, Stellar, Tezos that show us ways how to deliver value from A to B in a robust and stable way as the internet is delivering messages and information. The internet (of information) was probably the most life-changing invention of our generation. The ‘Internet of Value’ called Blockchain will step into its footsteps and be life-changing for all next generations.”

There’s an interesting book about that.

Finally, this proves Jack Ma's comments, which I blogged about earlier today, that the world's financial markets infrastructure is way to complicated, and that the banks and bank regulators are "like and old people's club".

 

UPDATE

An update on Thursday from the ECB states that “the root cause has been found in a software defect of a third party network device used in the internal network of the central banks operating the Target2 service on behalf of the Eurosystem.”

It adds: “The Eurosystem has taken measures to prevent this from happening in the future and will discuss it with the vendor.” The ECB has not yet addressed why the failover nor contingency modules failed to operate. Data published by the central bank indicates that the glitch resulted in a €416 billion queue in the use of its deposit facility.