Data is at the heart of every buy-side firm. Traders are awash in a sea of data and currently the central challenge for most firms is to find more efficient ways to aggregate and rapidly access the data required for insights that support decision-making and improve speed to market on the trading desk.
As a SaaS (Software-as-a-Service) provider of front office solutions including trade order management (OMS) and portfolio management, at INDATA we spend a lot of time finding new and better ways to help our clients solve their own unique data challenges. After all, the old industry maxim “garbage in, garbage out” prevails in the world of front office software systems that rely on accurate and timely data from multiple and wide-ranging sources.
Aggregating the data
As the world of data is constantly changing in terms of the volume, variety and velocity, the concept of data aggregation is one of the bigger technical challenges for most firms. At root level, data is often contained in different silos in terms of separate systems that are used such as market data terminals/providers, software systems (i.e. trading, portfolio management), and internally stored data consisting of research, proprietary databases and spreadsheets that don’t relate to each other or that do so in a very limited fashion.
The practice historically was to try to “stitch” together this patchwork of systems by taking advantage of vendor supported interfaces/downloads or by building proprietary interfaces between them, often at great cost and based on internal resources. This approach presents many challenges, not the least of which is the scalability of the resourcing required to support this model, which is not tenable in a business environment that consists of market volatility accompanied by the downward fee pressures faced by most investment managers.
Data warehouses vs. data lakes
As an alternative to the “patchwork” approach, the traditional data warehouse followed by data lakes became the ways in which many firms looked to solve the challenge of data aggregation in their own firms. The data warehouse or data lake model presents its own set of challenges. The first is the amount of resources required to build and then maintain these types of solutions. In the case of data warehouses, some vendors in this space have not kept up with technology and have become legacy type solutions themselves with outdated tech accompanied by an outdated business model, which becomes too costly for many firms. In the case of data lakes, quality and oversight surrounding the data can become an issue and therefore important data can become unusable on a practical basis for decision-making purposes. An even bigger challenge arises in storing the data that powers these solutions.
Storing the data
Many data warehouse type solutions have been on the market for many years and started in a world where on-premises systems were the norm. On prem software solutions require significant in-house IT resources to support them. A fundamental IT issue involves the resources required to store the data itself, which keeps growing. Another issue is the scalability of the resources required to work with the data, since as the data gets larger it becomes “slower” and more difficult to access effectively.
The importance of a cloud-based model
As a result, many providers have moved to a cloud-based model to address the issues of scalability. While the cloud solves a number of issues, of equal importance is the type of cloud model used, whether hosted or SaaS. A number of vendor solutions fit into the former category, which often requires additional IT that can result in challenges with accessing the data and also integrating it with other upstream and downstream systems effectively.
Accessing the data
The evolution of BI reporting
To overcome the limitations imposed by traditional data warehouses and data lakes, BI reporting, which makes use of better back-end technology usually based on an in-memory computing model to access larger data sets more quickly via web-based reporting dashboards has come into focus. BI reporting makes use of the cloud and can be deployed with a SaaS approach, which improves the ability to access mission-critical data effectively and quickly without additional IT resources.
The main challenge that these systems pose is that they are poorly integrated with the underlying software used by buy-side firms for trading and portfolio management. Ultimately, this means that firms must implement elaborate solutions that can reliably bring in the necessary underlying data to create BI reports. Problems with data integrity, data lineage, and other data management challenges can become numerous. As a result, trying to integrate stand-alone BI reporting solutions frequently becomes a difficult endeavor with the same challenges that traditional data warehouses and data lakes have.
The role of APIs
To address the challenges involved with aggregating, storing, and accessing data in cloud-based environments, APIs have come into focus. APIs (Application Programming Interfaces) are not a new concept in the world of the buy-side front office. Defined simply, an API is a set of definitions, communication protocols, and tools for building software. An API may be for a web-based system, an operating system, a database system, computer hardware, or a software library.
With the emergence of the cloud and the internet as the way most software interactions will take place, web-based APIs become the most important type of API, with their key purpose of connecting systems and sharing data. APIs have numerous uses that can effectively connect mission-critical systems in a plug-and-play fashion. Some examples include OMS/trading, compliance, portfolio modeling/optimisation, risk/analytics engines, and back-office systems. This is where open APIs can offer a competitive advantage by powering data analytics for investment management firms.
The rise of data analytics powered by APIs
In a world of cloud-based software dominating the front office, fully integrated data analytics driven by APIs will become an even more essential part of the buy-side technology stack. And, because most data will be hosted in the cloud, open APIs are essential for modern data analytics solutions that will be used by investment management firms.
To solve the data challenge in the front office, buy-side firms should look to modernise and streamline their underlying front office and data management systems by selecting vendors who support cloud-native, SaaS-based approaches that integrate data analytics powered by APIs directly into their underlying solutions. Firms should look to replace legacy and silo-based systems as quickly as possible and pare down providers who are not modernising their solutions quickly enough.
Any vendor partner selected should also be experienced in both traditional data management tools like data warehouses, and also more modern tools like BI reporting, since overall data expertise is required whether in the case of replacing legacy systems or working with the specialised and individual data challenges for buy-side clients on an ongoing basis.
Additionally, an understanding of how AI will be used on the buy-side trading desk is also an important requirement, as AI stands to play a significant role going forward for data analysis for firms looking to future proof their front offices.
To learn more about our INDATA solves the data challenge for its buy-side clients read Future Proofing the Front Office: Data Analytics
INDATA’s innovative cloud-native, SaaS-based OMS & Portfolio Management System helps leading global buy-side firms to solve their data challenges while delivering increased automation, efficiency, and transparency. Visit www.indataipm.com for more info.
