It’s fair to say that open source software is enjoying a moment. Once viewed with scepticism over its ability to deliver results, open source has become the dominant model for technology firms in every industry – including those in Finance.
Author: James Harrison – COO & Co-Founder, Genesis Global
To offer some perspective on how businesses are buying into it, the market for open source services is growing at a breath-taking rate and is expected to be worth $3.95bn by 2022. And recent acquisitions of leading open source firms like RedHat for $34B and GitHub for $7.5B offer further proof of the market buy-in. Beyond the numbers, we’re also seeing actions from large firms that would have seemed unfathomable just a few years ago. For example, the likes of Goldman Sachs, J.P. Morgan Chase and Deutsche Bank are all open sourcing specific programs and releasing code into the public domain.
Many more are embracing the culture of open source, using solutions to meet their business goals – even preferring it to proprietary software. This comes as no surprise either as firms face the shifting challenges of a rapidly changing digital world. Customer expectations continue to rise, markets become ever-more competitive, and regulations grow tighter. This all pushes businesses to go further and faster with their innovation, and open source software has become the accelerator of first choice for development teams.
In this sense, open source is the original Low Code platform. Allowing developers to build their applications around code they didn’t have to write, it provides IT teams with the building blocks to create new software and solutions at speed. So, it is only logical that we at genesis would use open source as the bedrock of our own Low Code platform for Finance.
However, open source doesn’t come without its own challenges. With so many, different projects and technologies available, choosing the right one can be risky. What if it was the wrong choice for the problem you need to solve? What if a better technology comes along afterwards? And, if you need to change technology, how much will it cost to migrate when needed?
A good example of this in action can be found in the NoSQL database space. There are many great open source offerings to choose from and they each have their own characteristics and APIs. So, if you find that you need to migrate from one NoSQL DB to another, the acceleration provided from leveraging an open source project can suddenly result in a deceleration, as there is no simple plug-and-play solution.
The genesis platform was originally built using FoundationDB as its main database backend. Creating our solution around FoundationDB became a major problem when Apple acquired it to power the iCloud database infrastructure. Suddenly, the technology ceased to be available (until a few years later when Apple decided to open source it). The open source world is littered with many similar stories, from suddenly revoked NPM modules to projects that just stagnate or fragment into obsolescence.
FoundationDB taught us that having proprietary technologies as a core base of our framework represents a significant risk. From that moment onwards, we worked to make sure we never found ourselves in this position again by engineering our platform with abstraction layers for each part of the system.
The genesis platform serves as an interface to the open source ecosystem underpinning it. Should one database system become unavailable (like it did for us before), it’s a non-event. We can simply connect to another and use the new database to seamlessly drive the functionality of the platform.
This approach eliminates the risk of choosing the wrong open source technology, while preserving the technology and ecosystem benefits, for downstream consumers of our platform. Having layers of abstraction allows you to make use of several different technologies as you need them. For example, our abstracted database API harnesses the power of multiple systems (FoundationDB, Aerospike, Postgres, etc.) as each of them can solve separate problems.
There are three areas in which we use different open source technologies that demonstrate how this works in practice:
Event processing in the frameworkReactive Extensions (RX) is our tool of choice for composing asynchronous and event-based observable sequences in the genesis framework. There are many advantages of using RX, but, for us, it’s important for handling layers of complexity without passing any impact onto users. It also makes our code less prone to bugs, while offering compatibility with a broad range of other technologies.
Event/notification bus
In the early stages of the genesis platform, we relied heavily on ZeroMQ as an open source library. However, we hit an issue when working in the cloud, which saw us implement our internal event bus using Aeron. As well as offering the highest throughput with the lowest and most predictable latency, it integrates with Simple Binary Encoding (SBE) to drive the best possible performance.
Database layers
As mentioned before, you can use the genesis framework with different database layers depending on the problem you are trying to solve, or your need for scalability. Our main three open source layers of choice are FoundationDB, Aerospike and Postgres, but we also connect to others. We achieved this through the creation of a common abstraction API called ‘GenesisDb’, which allows you to code an application and reuse the exact same code with different database layers just by changing a configuration setting.
The important aspect of this is that it makes the genesis platform far more agile, removing the risk of losing access to the underlying ecosystem. I often wonder if we were quite fortunate to have faced the challenge of losing access to FoundationDB when we did – even if we didn’t realise it at the time. It pushed us to rethink the way we worked. Acknowledging the challenges open source technologies can present led us to the idea of compatibility through abstraction. Ultimately, investing in this approach to open source has enabled us to create a successful Low Code platform that provides our clients with the building blocks they need to develop software at speed.
---
About the author
James Harrison
James has over 20 years’ experience in senior technology, development & implementation roles for some of the worlds leading investment banks including Credit Suisse, Merrill Lynch, Bear Stearns, HSBC and vendors incl. Fidessa. During his career James has lived in New York, London, & Hong Kong.
Prior to genesis global, James was regarded as one of the leaders in this field, managing global technical teams in supporting & delivering mission-critical, real-time systems for tier 1 global banking institutions needing their trading businesses to run near 24-7.
Interested in FINOS open source projects? Click the link below to see how to get involved in the FINOS Community.
