Making cloud-based market data make sense
Where one goes, the rest will surely follow. With more senior executives with strong profiles in cloud solutions joining financial institutions in recent years, there has been a shift towards adopting cloud-shared infrastructure as the standard for data delivery exchanges, trading systems and data providers.
This has caused a ripple effect in the market, as those that do not have a cloud presence will ultimately lose their competitive edge and become unviable. However, despite the popularity of the cloud, it is not without its limitations, as a vast number of these companies are struggling to initiate the migration of market data distribution due to a variety of business, monetary and technical reasons. Only by introducing event-driven architecture (EDA) into the equation will financial institutions truly experience the benefits of live data exchanges in the cloud.
The high cost of cloud technology
One of the main barriers is the significant costs associated with implementing and maintaining cloud technology. Adopting cloud-based infrastructure requires a long-term view from a cost-saving perspective. Maintenance costs must also be considered. Cloud providers meter by data volume, so, given the immense volumes of data messages being exchanged (100 billion messages per day), the costs add up over time. Nonetheless, investing in the cloud is worthwhile, but only if it runs alongside a platform that allows for a real-time market data service in the cloud.
The promise of cloud-delivered market data
Cloud-delivered market data holds many promises for those willing to make the switch. Its sophisticated capabilities better support servicing smaller clients as cloud-based technology does not require large and costly infrastructure. Likewise, it can easily offload data to third parties or partner suppliers that can offer managed services at scale, ultimately making it easier to work out solutions to unforeseen challenges. Furthermore, businesses are increasingly adopting cloud solutions for specific use cases such as cryptocurrency trading or outsourcing data for commoditization purposes.
The challenges with cloud-delivered market data
The foundations of the financial industry are built on fast, accurate and resilient market data exchanges and flows. Therefore, when implementing cloud-based solutions it is worth noting some of its delivery challenges.
One of the main concerns in market data delivery is latency; the speed at which the data is being transferred. Recent research suggests that using native cloud services, cloud-delivered data does not deliver in the latency stakes in comparison to non-cloud-based solutions. Native cloud services posting latencies with an average of around 30 – 100ms. This indicates that using the cloud does not overcome the industry’s challenge with latency.
Likewise, when reviewing the structure of message delivery in the cloud, what it offers in terms of scale, it lacks in terms of message order. This is because cloud models provide flexible scale by deploying message pathways horizontally. If the user requires more throughput, the service application creates an additional pathway, or partition, and pushes traffic across the multiple paths. While this scaling model provides vast capacity, the model suffers from order problems. As, when an additional pathway is created, the order of the message is no longer guaranteed. Cloud providers have attempted to resolve this by offering an ‘in-order’ delivery option, but this has been prohibited to single message pathways and as such, scalability is sacrificed.
While cloud-based solutions offer businesses scalability, they fail to meet the industry’s requirements for speed, accuracy and robustness. Consequently, for financial services to maximize the cloud’s utility, it must be used in conjunction with event-driven architecture to enable live data exchanges.
Using data to deploy a real-time market data service in the cloud
Event-driven architecture (EDA) is a software design pattern that can decouple applications asynchronously to publish and subscribe to events via an event broker (modern messaging-oriented-middleware). It can enhance the key variables in cloud-delivered market data such as robustness, scalability and latency as it ties data together in an ‘on premises’ messaging style. This prevents data from being compromised by native cloud networks. The deployment of EDA provides a real-time market data service in the cloud that overcomes latency issues as it can sustain latencies of < 1.5ms at the 99.9th percentile within a cloud region with average latencies below 1ms.
Implementing EDA into the cloud also enhances the resilience of data transfer as its capacity to offload data to third-party or partner supplies at scale makes it easier to resolve issues that may arise. Therefore, resolving unexpected issues and planning or implementing new initiatives becomes easier with cloud technology.
EDA can also support inter-broker routing, enabling users to connect brokers together to form a distribution network that supports publishing on one broker and subscribing to another. This then enables messages to flow from the publisher across to the subscriber. This can then support traffic between regions of a cloud using the event brokers and allows data to move between clouds simply by deploying a PoP in another cloud and forming the inter- broker adjacencies.
Finally, implementing EDA ensures the accuracy of data exchanges as it overcomes message order disruption while delivering the performance required for market data delivery. EDA can provide better message latencies and preserve message order. The EDA platform can send to multiple subscribers with an equal latency spread, meaning that data flows fairly and in real-time.
Overall, real-time data in the cloud is not only possible but necessary. In today’s exchanges, trading systems and data providers real-time market data is vital so that financial services can succeed by staying ahead of the competition and retaining customers.
Author: By Mathew Hobbis, Chief Architect FSI, Solace.
