Skip to main content

After years of investment in front office technology to streamline workflows in the face of squeezed trading margins and higher trading costs, investment banks and brokers now are focusing on enhancing the efficiency and efficacy of middle office systems by investing in resilient software that offers fast and effective processing. The often forgotten but essential middle office is now front and centre of the capital markets industry’s change management plans.

By GreySpark’s Rachel Lindstrom, Senior Manager

It is no secret that the middle office has been lacking in investment for many years in most banks/brokers. Much of the little investment in the middle office to date was driven by regulation and the need to comply. In Europe, ESMA’s revised Markets in Financial Instruments Directive and Regulation (MiFID II and MiFIR) highlighted the need for firms to focus on internal processes and systems to ensure that they were robust and resilient. In effect, front offices became more dependent on effective post-trade processing to ensure they met the new regulatory obligations.

Although, trading firms are still dealing with the runoff from MiFID II, the middle office is also addressing changes required by the third phase of the EU’s Central Securities Depositories Regulation (CSDR) known as the Settlement Discipline Regime (SDR). This regulation aims to harmonise operational aspects of securities settlement and incentivises quicker settlement, with mandatory buy-ins and cash penalties to lessen both the impact and the number of settlement fails. In June 2020, the UK announced that it will not implement SDR, signalling perhaps the beginning of a regulatory divergence and creating both uncertainty and opportunity for firms operating in the UK. Meanwhile, The Depository Trust & Clearing Corporation (DTCC) is pushing for the shortening of the US settlement cycle to T+1 by 2023. Across the Asia Pacific region T+2 settlement is the norm, but there too voices are calling to move to shorter settlement periods, perhaps invigorated by China’s T+0 settlement requirement for certain trades. All these factors are increasing the global appetite for straight-through processing (STP) across the trade lifecycle.

The scope of middle office responsibility evolved idiosyncratically for banks and brokers, and this, coupled with the complexity of middle office workflows, meant that commoditised middle office solutions were not a commercially viable proposition for financial technology vendors to offer at the time when third-party front office and back office solutions began to appear. The lack of an off-the-shelf middle office-specific solution left a vacuum that was partly filled by back and front office solutions which strayed into the middle office space. The middle office functionalities that these front or back office solutions did not incorporate were either backfilled by in-house development or retained as manual processes for middle office staff. Many of these manual processes became bottlenecks across the post-trade lifecycle. It is against this middle office technology backdrop – or lack thereof – that shortening of the settlement cycle is being addressed.

A renewed focus on middle office technology – if backed by an appropriate budget – presents firms with the opportunity to think strategically, rather than tactically, in the middle office and to transform existing tactical middle office change projects into business transformation programmes that not only eradicate the persistent middle office issues but also arm the middle office with technology fit for the future. In this article, GreySpark examines how firms are addressing the key middle office issues common across the industry and is based on interviews and survey responses from individuals in a range of middle office roles in a variety of banks and brokers across the globe. Figure 1 describes the research sample, and anonymised commentary is presented in Data Panels through the article.

Figure 1: Description of Interviewed Individuals and Survey Respondents

(Click to enlarge image)

In capital markets banks and brokers, manual workarounds emerge when either of two situations arise: a lack of business model conformity or the need for frequent bespoke workflows. The middle office space, where upstream client-driven processes and systems require frequent amendment, is a prime example. The immense complexity of achieving reconciliations, allocations and settlements – workflows that require data from both internal front and back office systems, as well as external parties’ systems – becomes extremely complex and introduces significant key person risk.

The Settlement Issue

Post-trade processing has many moving parts and the failure to settle trades can occur for multiple reasons. The most prevalent reasons for settlement failures include incomplete matching of settlement instructions, the inability to borrow or recover and operational or technology issues, as shown in Figure 2. Approximately, 60% of the research sample stated that settlement failures are still a significant problem in their organisation.

Figure 2: Main Reasons for Settlement Failure According to the Research Sample

(Click to enlarge image)

Source: GreySpark analysis

A decade ago, the idea of reducing the settlement cycle in the EU from T+3 to T+2 was met with reticence by trading firms. Their concern centred around two issues: the quality of trade data and the time taken for matching. However, after the remediation of some software and process issues, the transition was less of a challenge than anticipated for many firms.

Figure 3: Potential Benefits of Reducing the Settlement Cycle

(Click to enlarge image)

Source: GreySpark analysis

While a T+1 settlement cycle would reduce much of the settlement risk, it would introduce some weighty operational challenges for banks and brokers. Firms would have to confirm and match trades at T+0, to leave one day to resolve the issue or issues that caused the settlement failure: a problematic timeline for many. While typically the larger firms are capable of matching intraday, many smaller firms would need considerable development work to be able to do so. Additionally, a T+1 settlement cycle would, by definition, shorten the timeframes for stock borrowing, meaning that to meet the T+1 deadline firms would need an auto-borrowing capability in their post-trade processing workflows.

Stock borrowing is used when a firm is short of a stock, which can occur unexpectedly because of failed trades. Currently, if the firm is short for an exchange trade, it could be subject to a daily fine, and a mandatory buy-in as well as an admin fee – an expensive mistake, in other words. If the trade is with another broker, then no charge or ‘buy in’ is demanded. However, CSDR will make the charge and buy in mandatory for every trade. So, were the settlement cycle to be shortened imminently, the cost to a firm of settlement fails and the likelihood of settlement failures would both increase.

In essence, shortening the settlement period may benefit investors, buyside firms and broker-dealers in some ways, but there are challenges that must be addressed if some very detrimental impacts on banks and brokers are to be mitigated. Despite a commonly held view that the challenge for firms lays in implementing amendments to systems to facilitate the settlement change, this research reveals that the challenge is in mitigating the risk that some counterparties will not be able to fulfil their obligation to send matching details and settle trades in a timely manner. The reason for this is, quite simply, data connectivity.

Unblocking Data Flows

Problems relating to incorrect or missing data can without a doubt be said to plague the middle office. They can lead to the allocation of trades to incorrect legal entities, failed settlements and the need for significant manual intervention to rectify issues. Many of the matching, allocation and settlement issues begin and end in the reconciliation of front office, middle office and back office data.

An inefficient flow of data from the back office to the front office can lead to settlement failures that not only impact the specific trade but ripple out to create further problems. If the front office is not notified of failed trades, traders have no real sight of their actual cash positions and capital requirements. The ability to flag any post-trade status to the trading desk relies on middle and / or back office technology that can integrate into internal messaging systems that traders can see. Typically, however, the front office is not adopting new messaging and adaptive desktops and so there is no automated process that ensures the front office is aware of failed trades.

The evolution of trading in investment banks led to parallel technology stacks developing by asset class. Historically, the people and processes to undertake and process trades were siloed by asset class and as technology was brought in to automate parts of the trade lifecycle, this too was approached separately by asset class. Thus, technology, teams and processes for each asset class were segregated from other parts of the organisation. Clearly, this is an oversimplification; cross-asset class front office technology was developed in pockets of the industry, but segregation is still very much the norm, and this leads to duplication of processes where more than one front office system is in place. Operational staff that process middle office issues must access different middle office platforms with varying levels of functionality and automation to find and correct errors, which is inefficient, time-consuming and costly.

Figure 4: Data Flows Through the Trade Lifecycle and Typical Relative Efficiencies

(Click to enlarge image)

Source: GreySpark analysis

Passing data on unsettled trades back to the front office is a largely manual process, which is undertaken intraday at best in many firms. The front office needs to understand the status of their trades – especially when undertaking time-sensitive trading – and, should there be settlement and cash issues further down the line, data needs to be readily available to traders so that they can take appropriate action to avoid detrimental effects to both the business and its client base.

It is not only the data that flows out of the bank / broker that can be an issue, but also internal information flows. Real-time transfer of static, trade and reference data from the back office to the front office and vice versa is unavailable in some firms, due to inefficient processes and inadequate software. Unclogging data flows to make real-time data available at the point of need is something that can help a firm avoid many of the pitfalls that cause raised eyebrows in compliance teams.

Just over a third of the research sample were unsatisfied with the speed of data transit from the back office to the front office. Whether the issue is due to inadequate software, bad data or poor connectivity, the only way out is automation – preferably STP.

The Search for Simplicity

Straight-through processing and automation of the trade lifecycle would be a remedy for many of these middle office maladies. Operationally, processes are faster and more accurate when automated. Allocations and settlement instructions can flow freely from back office to front office, unhindered by manual intervention, allowing firms to offer clients the best service possible. The question of why STP is rarely achieved in banks and brokers is rooted in the evolution of the electronic financial platforms.

In the early days of digitalisation of the financial markets, firms relied primarily on the in-house platform development, but as functionalities evolved and matured technology vendors were able to carve out portions of functionality and offer commoditised platforms to bank and broker clientele.

Some groups of functionalities became cheaper to buy as third-party platforms than to build in-house and a mottled landscape – a mishmash of in-house and third-party systems – developed. Commoditised third-party front office and back office solutions became widely available, and the middle office emerged as the centre ground over which platform providers for either end of the trade lifecycle fought. So, in 2021, there are four competing options open to firms to facilitate STP.

Figure 5: Four Options for the Creation of STP Across the Trade Lifecycle

(Click to enlarge image)

Source: GreySpark analysis

Option 1: Front / Middle Plus Back; or Option 2: Front Plus Middle / Back

Option 1 and Option 2 are the approaches taken by the large majority of firms. For the vendor to extend the reach of one of either the front or middle office platform deeper into the middle office space, or for the bank / broker to fill the functionality gap in the middle office themselves, is possibly the least disruptive of the options available to firms.

Where more than one front office system is in place in a firm – for instance, separate solutions for each asset class – operational staff that process middle office issues must access different middle office platforms to find and correct errors; a situation that is as inefficient as it is time-consuming and costly.

The disadvantage of Option 1 and Option 2 is that many of the vendors of these platforms already have significant negotiating power when contracts need to be renewed. Extending their reach will only lessen the bank / broker’s ability to limit price hikes.

Option 3: Front-to-back Solution

Option 3 could be the optimal approach from a technology standpoint, but it is not viable for the enormous enterprises that make up typical Tier I or Tier II banks or brokers today. Where this may be a realistic scenario is where technology is developed in house by the bank / broker. However, the inclusion of one or more vendors in the trading technology stack of today’s banks and brokers is almost universal, so this scenario is very rare.

The advantage of these solutions is that the flow of data can be as efficacious from front-to-back as from back-to-front, and trades can be tracked using one system identifier across its lifecycle.

Option 4: Discrete Front, Middle & Back Office Solutions

The lack of a discrete middle office solution was a result of the way the trading technology space evolved and the variation in scope of middle office responsibility from firm to firm. However, the acceptance of cloud-based solutions into almost every corner of the financial services sector and the utilisation of microservices architecture that can replace legacy monolithic technology stacks means that vendors can now offer commercially viable modular solutions that cater specifically to the purchasing organisation’s middle office scope.

The independent middle office solution brings with it the benefit of managing the workflow and exceptions in one platform, streamlining and focusing operational staff on a single data set and workflow process, and so reducing time to resolution for errors and removing duplication cost.

With the option of a middle office offering, banks and brokers can introduce a buffer that limits the reach of any single vendor within its enterprise; effectively diminishing any unreasonable price hikes from vendors that stretch the BAU budget.

The disadvantage of this approach is that there is more integration work to do, but if the vendor providing the middle office solution is chosen with an eye on the flexibility of their APIs, and the functional capability across multiple asset classes, this potential challenge on change budget can be mitigated.

Of the research sample, 28% were planning upgrades to their front office systems, compared with over 45% planning change to their middle office systems. In the third-party trading technology space change is the name of the game.

Middle Office Automation is Key

As banks and brokers realise that there can be no more delay to the automation of their systems – be it because of regulatory or operational / business drivers – the middle office can expect an overhaul. Automation will remedy the data connectivity challenges that front, middle and back offices struggle to manage, and it will allow banks and brokers to manage delivery time expectations of trade data notifications and confirmations to clients. 

As banks and brokers prepare for CSDR, the prospect of mandatory buy-ins for all failed trades and the consequential rise in costs is focusing minds on the problems of settlement.

Automation may help to reduce settlement risk, depending on specific capabilities, and it may prepare banks and brokers for the potential shortening of the settlement cycle. An STP (front-to-back and back-to-front) system, creates the potential for matching on T+0. 

The open question for banks and brokers is how they should approach the creation of a full STP platform. The issues of vendor lock-in and the increased negotiating power of embedded vendors is a thorny one that haunts the industry. While the extended offerings of front and back office vendors into the middle office space may be an easy solution, there are potential commercial disadvantages. 

The availability of a discrete middle office platform that has a modular offering is an obvious solution, should they be able to integrate with upstream and downstream platforms that provide the front- and back office functionalities. The middle office – an often-neglected technology space – is holding its breath. Technological change is starting to happen there and the goal of most firms to automate trading and trade processing could soon be achieved for banks and brokers, large and small. 

Torstone Technology’s middle office solution provides extensive cross-asset functionality providing a holistic view of operations through to the back office seamlessly. Trade operations staff can work together across the operational spectrum to improve client service and deal with the most pressing issues, while the technology digitises and automates much of the workflow, saving valuable time and resource. The solution significantly improves operational efficiency by allowing business functions to be performed across asset classes. For more information visit:

Timothy Diggines joined GreySpark having gained nearly 30 years’ experience within the trading and investment industry. He has a strong skillset covering capital markets from technology to operational workflows. Experienced and with a comprehensive knowledge of the front-to-back trade lifecycle, he also commands expertise across connectivity and operational and control procedures.