Read time: 6 min

Overcoming Latency and Capacity Issues

It is a truth universally acknowledged that the proliferation of high frequency trading activity has resulted in a dramatic rise in volume in the financial markets.

Data from the Bank for International Settlements (BIS) shows that most of the recent foreign exchange market growth – about 20% in volume terms between 2007 and 2010 – was due to the increase in trading from HFT players. In its review of the market late last year, the BIS estimated that 25% of spot FX activity, or as much as $375 billion per day, was from that trading type. In a recent report, research company Celent estimates that 32% of daily spot FX volumes will be HFT-driven in 2013.

As the FX markets become more automated, managing holistic end-to-end latency has become increasingly important. Effective capacity planning is also an imperative for financial institutions. Intrinsically linked with latency, where every millisecond matters, accurately aligned capacity is fundamental for both managing trading performance and realising efficiencies. In this environment, firms are increasingly turning to innovative solutions to help them overcome capacity constraints and reduce latency levels.

A beneficiary of this focus is UK-based Sumerian, which sets out to answer big IT change questions specific to latency and trading optimisation for its clients through its IQ services. The company touts itself as an expert in analysing big data.

“If a bank is going to invest in high frequency trading, which means it is going to double the amount of volume it will see through its trading platforms, where does it need to invest from a technology infrastructure perspective and from an application development or a business process standpoint?” says Joanne Kinsella, managing director, Americas, at Sumerian.

Sumerian answers these questions by capturing and analysing data from across a client’s IT environment – for example, bandwidth information to see how heavily links or switches and routers are being used, as well as unstructured data and trade volume. The data is transferred to Sumerian’s data warehouse for modelling and analysis using algorithms and interpretation from Sumerian’s dedicated team of analysts and statisticians and then the findings and recommendations are delivered back to the client.

“We give the client a programme that allows them to make optimisation changes to their environment. We give them either a very detailed piece of analysis that will give them prioritised recommendations as to what they need to do to optimise their trading environment or we will give them a front-end thin client portal where they can see all the analytic results on a frequent basis. Most of our clients want to see this level of detail weekly, monthly or quarterly,” Kinsella explains.

Sumerian offers a light touch approach with no software to install and IQ is delivered as a monthly subscription service. Typically, clients upload data to Sumerian securely after the trading day or during the weekend. As Sumerian is data agnostic, Kinsella says it does not matter what the asset class is, and as such the company has worked for banks’ FX and equities teams and has conducted studies around capacity headroom across an entire trading environment.

The company also carries out ‘what if?’ scenarios, for instance assessing what the impact could be to a client if the number of trades passing through its systems rise from 50 million to 150 million trades a day.

Kinsella says: “With the volatility we have seen in the market recently, clients are grappling with the same challenge, which is ‘how do I right-size my environment for the increase in volume and the increase in speed?’ and with the fact that they may have architected a particular system for a certain number of trades per day. Clients are looking all the way from the front to the back office to see if each part of their system is optimised to be able to support the trading volume growth that we are seeing.

“There has been a huge growth in FX – it is one of the fastest growing, if not the fastest growing, market. The increasing uptake of electronic trading means that the amount of volumes that banks now have to process is considerably higher. With our service, we can tell the CTO of an FX team exactly where they need to invest money and resources to make sure that their FX environment is the right size to support business growth.”

One bank that has used Sumerian’s IQ services is Bank of America Merrill Lynch, with which Sumerian is still working on an on-going basis. BAML was concerned about scalability to meet its business growth goals and deployed Sumerian IQ to capture low-level granular data from its distributed IT environment and geographically dispersed servers and create a model of its systems, including its market data interfaces and risk management systems. BAML also used Sumerian to model correlations between quote and trade volumes, and server use and latency.

As a result of its deployment of IQ, Sumerian says BAML has reduced FX latency by over 75% by identifying limiting components for end-to-end latency. It has also increased headroom growth from 11% to 30% by identifying capacity bottlenecks and horizontally scaled existing applications without the need for any additional investment, says Kinsella.

Database Technology

Market participants are also seeing a need to upgrade their database technology to store and query the ever-increasing data volumes. This is particularly pertinent as new regulations are forcing banks to store numerous years’ worth of market data history.

Simon Garland, chief strategist at Kx Systems in Zurich, says: “If you look back a few years ago data volumes were laughable compared to what we are seeing today. If you’ve got to store data, say for seven years, that seven year chunk is going to get bigger and bigger as the early, small years ‘drop off the front’.”

Kx Systems’ main products include kdb+, an ultra high-performance database for massive data volume that manages real-time and historical data in one platform and allows for low latency communication between kdb+ processes. Kx Systems’ second product, kdb+ tick, captures and analyses billions of real-time ticks and retrieves millions of records per second.

The Kdb+ database is often used by firms to analyse, process and retrieve large data sets at high speed for algorithmic strategy development and testing.

“With a significant rise in FX volumes and a new HFT mentality of needing to do new things very quickly, banks need a different development environment and that’s where we come in,” says Garland. “Traditional sorts of systems aren’t really set up for handling enormous volumes and volatility, and this has been a driver towards technology like kdb+, where programmes can be changed more quickly.”

Garland describes kdb+ as ‘one stop shopping’. “If a bank is dealing with billions of records, they don’t want to be taking the data from one application and sending it to another – that would be too slow. But once the data is in kdb+, the intra-day/real-time and historical data is all immediately available.”

An advantage of kdb+ is its speed, says Garland. “Banks can back-test years’ worth of data in just a few minutes. They don’t have to run tests overnight and wait for the results the following morning.”

He says the company is seeing demand for its products from clients who have already adopted the technology for other asset classes.
“We particularly noticed demand after the start of the financial crisis when people moved out of equities to elsewhere within the same bank. Those who went into FX, brought with them experience in handling huge data volumes and understood different ways of analysing data at real-time speeds to produce a significant competitive advantage,” Garland says. “Many of the big FX players already have an enterprise license and kdb+ deployed somewhere within the bank so there is a lot of potential for expansion.”

Competing in the same space as Kx Systems is OneMarketData, a provider of tick data management services that encompass real-time and historical market data and the collection, capture and analysis of the data.

In September, the company announced that Société Générale’s foreign exchange trading division had deployed its OneTick database and complex event processing platform to support data processing and analysis for algorithmic strategy development and testing.

Speaking at the time, Richard Chmiel, vice president at OneMarketData, said: “With OneTick, Société Générale’s FX team will be able to tap into the platform’s speed and scalability to manage growing data volumes and deliver robust and precise analytics. They will also be able to lower their total cost of ownership while still maintaining the speed and performance necessary to compete in this market.”

Specifically, the bank’s FX division is using OneTick for pre- and post-trade analysis and data storage as well as real-time production monitoring and back-testing of strategies and algorithms. OneTick stores and analyses Société Générale’s internally generated prices and data while also providing access to various vendors’ full and conflated feeds.

Built from inception as a single integrated system, OneTick Database and CEP allows users to write a single set of code for historical analysis and real-time signal generation, run queries that span historical and real-time data, and apply OneTick analytics to these data processes.

According to Louis Lovas, director of solutions at OneMarketData: “As data volumes increase, clients are discovering that older technologies, many of which have relational engines, just don’t cut it. They don’t have the collection capacity. They may have storage capacity but when it comes to querying and analysing that data that’s where they start to fall apart.”

Like Kx Systems, officials at OneMarketData say that users can reduce costs associated with licensing, installing and maintaining separate data management and CEP systems by leveraging a single product that is able to collect and analyse both historical and real-time data.

As data levels increase in the foreign exchange markets, so opportunities abound for companies competing on infrastructure, speed and low latency technology.

Profit & Loss

Share This

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on reddit

Related Posts in