Building a scalable digital banking solution delivering personalised engaging content to millions of customers raises multiple technical challenges. Read on to see how we’re tackling those with the modern digital banking technology stack of the Moneythor solution.
—
At Moneythor, we are continually designing, developing, and redefining our digital banking platform to meet the needs of financial institutions globally. Our solution is a fully configurable orchestration engine and services that sit between data sources and customer-facing digital banking channels to deliver highly personalised and engaging experiences in real-time.
Our mission is not only to build a robust solution, but to also invest in research and development and to go beyond the limits of traditional architectures to better scale and address new challenging use cases like processing and analysing a large amount of data in real-time.
Each solution has its own advantages and drawbacks especially considering that we often run systems on legacy network and disk and keep CPU and memory cost under control. Without going all the way up to Alan Turing era, the characteristics of the problems the Moneythor solution deals with have long since been classified into two distinct categories, online transaction processing (OLTP) and online analytical processing (OLAP).
OLTP
The Moneythor solution offers an engine deployed between the financial institutions’ systems of record and their digital channels to power engaging and tailored experiences for end users. It is therefore expected that our solution can digest a large volume of data in real-time in a reliable and scalable way.
Relational database management systems (RDBMS) have historically been proven to be an efficient and reliable way to implement such OLTP system as they also offer an extremely valuable benefit of being ACID compliant.
Our solution also includes Personal Financial Management (PFM) features driven by APIs that can create, read, update, and delete personalised objects like budgets, goals, subscriptions and more. It is therefore crucial that the digital channels calling our API can unconditionally rely on their responses and for that purpose, our solution heavily takes advantage of the Atomicity, Consistency, Isolation and Durability (ACID) properties of the database whether it’s a RDBMS or not.
- Atomicity guarantees that all the modifications requested by an API call are completed successfully as a whole. That is, when our response is successful, one can trust that all the requested modifications have been processed or none when the response is not successful. Calling an API to create 3 budgets will create 0 or 3 budgets but never only 1 or 2.
- Consistency preserves the internal consistency of the database. No API call even if they are cancelled will ever let the database in an undesirable state. Calling an API to delete a goal will always delete not only the goal itself but its dependencies as well.
- Isolation ensures that one can call multiple concurrent API at the same time and the result will be the same as if the calls were processed one at a time. When calling an API to create a budget and at the same time, an API to modify a goal, both will be processed taking into account all these changes. All RDBMS use some sort of locking mechanism for that purpose and it’s worth mentioning that Moneythor chose an optimistic lock approach to avoid any database deadlock.
- Durability guarantees that after an API returns a successful response, no data will be lost in case of a failure. With API over HTTP, a common problem arises in case of a network failure when the client never get a response. For that reason, Moneythor implements idempotence to help clients retry without the risk of creating duplicates.
OLAP
The Moneythor solution enables the delivery of data-driven insights, recommendations and nudges tailored to each customer. This is where the analytical part of the solution shines and allows our clients to imagine a wide range of contextual and personalised nudges for their customers.
Such data analysis typically relies on a OLAP system optimised for that task and it is where friction can emerge when trying to build a solution on top of both OLTP and OLAP.
While being smart with what they can do with the data, OLAP systems work well when the data does not change a lot which is not a typical scenario with a digital banking solution that receives millions of transactions a day, in real-time and when data analysis is expected to happen at that time. Also, data can be changed by users themselves and this can lead to a completely different outcome when it comes to personalised nudges.
OLAP systems are not necessarily designed with fast response time as a top priority, whereas new user experiences built on top of the Moneythor API are served over HTTP and therefore synchronous. Waiting more than a few seconds for a screen to be rendered nowadays feels an eternity.
Distributed systems
Taking these constraints into account, the Moneythor solution leverages the best of both types of systems and mitigates their shortcomings through a distributed architecture with the characteristics below.
i) When the benefits greatly outweigh the cost of redundancy and additional processing, data in the persistent store is de-normalized on a case-by-case basis. This allows us to continue processing transactional data while keeping analytical processing time under control.
ii) The Moneythor solution sits between other systems and natively integrates with distributed event streaming platforms. For example, Apache Kafka is a very popular tool and together with the Moneythor solution greatly improves overall performances and simplifies implementation and scalability, because both systems are distributed by design. While APIs heavily rely on the ACID properties presented above and guarantee a response, when it comes to a stream of messages, Kafka and others usually offer 3 different message delivery modes: a message may or may not be delivered, a message can be delivered multiple times, or a message is delivered exactly once. The first mode is an obvious no-go when dealing with financial transactions, and while it might be appealing to look for an “exactly once” delivery, this comes at an extra performance and complexity cost. For this reason and because messages in Moneythor are idempotent, our solution is built around an “at least one” delivery concept which guarantees that no message will be lost, and consumers may simply control duplicates thanks to the idempotent flags.
ii) Finally, we cannot discuss distributed systems without paying attention to the CAP and PACELC theorems. The first one states that any distributed data store can only provide two of the following three guarantees:
Consistency (C): every read, every API call in the present context, receives the most recent version of the data or an error.
Availability (A): every API request receives a (non-error) response, without the guarantee that it contains the most recent data.
Partition tolerance (P): the system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes.
In most instances, the Moneythor solution is deployed in an environment where there is only 1 logical instance of the database (with this database system generally running in a cluster for high availability). The number of partitions in this context is 1 but it would be a mistake to imagine that the system could therefore provide both consistency and availability at the same time. Indeed, in the absence of partitioning, the PACELC theorem refines those rules by stating that one must choose between Latency (L) and Consistency (C).
The Moneythor solution is designed to scale vertically by adding processing power and horizontally by adding additional instances in its cluster. It also exposes a set of front-end APIs used to enable rich interactive features to the end-users with the minimum response time possible for optimal user experience.
For these reasons, the solution is optimised for latency. While this choice may sound counterintuitive especially after stating that APIs rely on the ACID model, the definition of consistency is different here and does not apply to the data manipulated and computed by an API that must be consistent no matter what but determine when the most up to data will be available, immediately or eventually.
On the contrary, targeting consistency first would also mean that the system would have to wait for many instances in the cluster to be synchronised before returning a response since the solution uses in-memory caches to speed up processing and API response times. Despite the best in-class network and CPU, it would be extremely difficult to guarantee first class response times in that case.
Essentially, there are simple principles that can be implemented to dramatically reduce the risk of reading out-of-sync data. A properly configured load balancer routing requests in the cluster to the same instance where the data for the current context has already been processed and cached before is one example.
Conclusion
Building a highly scalable and real-time solution that combines transactional and analytics data is a trade-off between what features can be provided, what are the response times expectations and of course the cost of deploying and operating such a system.
Distributed and decentralised architectures can be complex to build, and it is necessary to understand their characteristics to keep their promises. Concurrently, technology is evolving, and high-performance computing is increasingly accessible, especially with Cloud platforms. These are the reasons why we have chosen to take this direction, now allowing us to serve successfully the data-driven personalisation needs of digital banks and financial institutions of all sizes.
To learn more about the Moneythor solution and our technology, please contact us.
