4 technologies that are shaping the future of finance — an explanation for non-technical mortals

02 June 2019
Big Data
Microservice Architecture
Open Banking

Whether you are attending this week’s Money2020 conference or not there is no denying that the future in financial services will belong to those who understand the tech behind it. This guide will help you to make sure you have a crystal clear picture.

4 technologies that are shaping the future of finance — an explanation for non-technical mortals

In this article we focus on the following four key technologies that are changing the face of the financial industry. You probably already encounter them on a daily basis — if not, trust us, you will.

1. The tech behind Open Banking and PSD2
2. Microservices and Containerization
3. Big Data
4. Artificial Intelligence and Machine Learning

This is the moment to take time and understand

It’s paramount for decision makers to understand how technologies that shape the financial services industry really work. Technology and business have become completely intertwined, and that is how things will stay. Of course technology has been widely used in financial services for decades, but its importance was nothing like it’s become over the last ten years.

Not long ago, you sent messages to the engine room and hoped that the IT guys would figure out what to do. More specifically, you sent “business specs” or “demand requirement documents” and hoped that you’d get software that resembled something you’d imagined. Once you finished contributing to the spec you could return to your real job — the one not revolving around computers but around finances and banking. But as consumers started to pass the barrier between physical and digital space the importance of IT solutions skyrocketed.

You probably started to spend more and more time at meetings making decisions about technologies, infrastructure and other seemingly exotic topics. There is no engine room anymore: now your whole company is the engine room! Just as technology permeated your private life (think of the last time you took a walk without your smartphone) the same has happened to your workplace. This marriage of tech and business won’t dissolve; the union will just grow stronger and stronger. To accommodate this new reality business decision makers have to develop a deeper understanding of the IT world. Only this can lead to fruitful conversations about the future of your company with engineers and decisions which are rooted in the understanding of each other’s domains.

1. The tech behind Open Banking and PSD2

Meet your customer Joe Smith. Joe has a savings account here. And a Forex account at another bank. And a money market account way over there. And for that faroff rainy day, he has a brokerage account far off in the other direction. (He’s never mentioned that one to his wife.) Just imagine Joe’s grief when he has to figure out how much money he has. Or which account to access to buy that scooter he’s been eyeing, especially when he really wants to buy it just before rent is due. Luckily for Joe, the EU’s PSD2 Directive is going to make his life a whole lot easier.

PSD2 and Open Banking open the doors to a wide variety of new services to be built on top of existing, robust banking infrastructure. The EU created the PSD2 directive to boost competition between financial service providers. The EU also aims to open up possibilities for newcomers to build banking services without the hassle of acquiring a banking license. While there is plenty of discussion of the ensuing opportunities, let’s focus on the technology that’s behind this revolutionary movement.

PSD2 Service Providers

Put simply, PSD2 basically does two things:

First, it forces banking institutions to create a secure way for 3rd parties to access account and transaction details. Second, it makes it possible for the 3rd parties to initiate payment transactions.

PSD2 hence makes possible two kinds of access, resulting in two new kinds of providers, namely:

1. Account Information Service Providers (AISP): These are authorised to retrieve account data provided by banks and financial institutions. This could benefit Account Aggregators or Personal Finance Managers. An Account Aggregator could access your balances from several accounts at several banks to tell you how much money you have in total and what are you spending on (and here you were, thinking that your don’t overspend on street food!).

2. Payment Initiation Service Providers (PISP): These are authorised to initiate payments into or from a user’s account. A good potential service here would be a Savings apps, which could chip off a little from your main account every day and invest the money without you having to do a thing. And presto! You get notified that you now have the money for that scooter you put on your bucket list a year ago.

Figure 1: The Logic Behind PSD2

It is important to note that 3rd parties must go through a rigorous FCA (Financial Conduct Authority) authorization process before getting permission to provide these types of services. They are required to have certain insurance and initial capital, and must comply with data security and privacy regulations. This will give Joe the assurance that his money will be as safe with the new provider as with his bank.

Ok, let’s say we want to create an app that aggregates account balances for customers who have multiple accounts in multiple banks. First of all we need to get access to the data the customer has at the various banks he or she is using.

Banking APIs

For a start we need to somehow connect to the bank and all the information stored there. To provide information and initiation, banks have to create communication interfaces that allow third party access to data and payments. For this reason, according to the PSD2 directive, banks have to provide open APIs (Application Programming Interface) that can be used to retrieve data.

What is an API? API is a set of functions for a given service that is made public for outside access. For example a user of the API can say that I need the transaction history for a given user’s account and the API returns a list of transactions for that account. Or he can say that Joe wants to send 100 euro to a Bill and the API executes the payment.
Unfortunately the PSD2 regulation does not standardize APIs, meaning every bank can decide how it wants to implement its API independently.

This could mean that the ~5000 banks in the EU will use 5000 different APIs. For us to provide services for every customer and for every bank we would need to integrate all 5000 of them!

Figure 2: The flow of customer’s data with open APIs

Fortunately there is hope. There are some initiatives aiming to standardize APIs for banks:

This means that if a 3rd party can speak the language of NextGenPSD2 it could reach data and payments from all 40 banks that provide NextGenPSD2 standard APIs.

It is unlikely there will be one standard that will rise above the rest and be adopted by everyone. But there is a good chance that 4–5 standards will become popular and be adopted by most banks. So to tell your customer how much money she has over several bank accounts in several countries, you won’t need to learn 5000 languages. Four or five will do.

Strong Customer Authentication

So your customer loves the comfort you promise. Bring all their banking into one easy to use system. But what about the customer’s sense of security? Surely Joseph Smith does not want to accidentally pay for Guiseppe Fabbro’s weekend in Sardinia. PSD2 regulates authentication procedures related to payments. This is called Strong Customer Authentication, or SCA. This means that in order to initiate a payment transaction the user needs to authenticate with at least 2 of 3 authentication factors:

  • Something the customer knows (e.g., password, PIN);
  • Something the customer has (e.g., mobile, hardware token);
  • Something the customer is (e.g., fingerprint or facial recognition).

This is meant to reduce fraud and make online payments even more secure than is currently the case. Joe Smith, rest assured, the only Sardinian holiday you’ll be paying for is your own.


With the rise of open APIs everyone will want to get their piece of the financial market pie. It will be easier than ever to create new and innovative fintech solutions; therefore we expect to see some new global contenders. To stay on top and keep their customers satisfied bankers will need to understand the ins and outs of APIs and the opportunities they create. The first step to compete with disruptors is to understand the tech behind the scenes.

Would you like to hear more? Get in touch!

2. Microservices and Containerization

Imagine transferring €1000 to a friend with a banking app. Now imagine how it feels when that app takes minutes to load. Not good. Maybe next time you’d rather pay with PayPal — and that is not something that’s going to make a bank happy. A recent survey of financial institutions found that about

85 percent consider their core technology to be too rigid and slow. Consequently, about 80 percent are expected to replace their core banking systems within the next five years.

These core systems are usually monolith backends (we’ll get to that later), sometimes still running on mainframes! (Btw, mainframes are specialized hardware for running a specific software).

These days, with new fintech startups cropping up every day, releasing new features every month, we can’t accept taking years to develop new core banking features. Truth be told, these old systems are written in programming languages that are almost extinct (read about COBOL on a museum’s website), making it incredibly hard and expensive to assemble a development team that can do the job. As a result, your teams cannot react fast enough to the growing number of financial transactions and changing customer needs.

You can do two things in this situation:

1. “Simply” — meaning through blood and sweat — replace the old core system with a new one;
2. Replace certain parts or functionalities one by one with microservices.

The first solution is tricky. Moving to a new core, migrating all the data while preserving business continuity is very risky. We’ve seen some banks introducing new cores from close up. It’s nasty. It can result in multiple days of downtime and embarrassing bugs after restart. Your customers will not like it. And for some big players that’s way too much risk.

Here the second solution comes into play. Instead of tearing down everything, we start replacing core functionalities with microservices. One by one.

Monolith architecture vs. microservices

Figure 3: How the monolithic and microservice architectures differ
A monolith means that every service a software provides runs within the same application.

Imagine a core banking system where authentication, customers, accounts, transactions and payments are all handled by one application connected to one huge database. Should something happen with the database connection, or should the application crash, everything can go down. The customer will record no transaction history, and will be very upset at the inability to initiate payments.

Another common problem with this architecture is that if the demand for certain services grows, we are largely unable to scale only that part of the application. We have to scale everything which results in slow, painful and costly projects. And in the end, we might not even solve the real issues.

Microservice architecture comes to the rescue! Every service — accounts, transactions, etc.. — runs as a standalone, separate entity, creating a loosely coupled system. There can be communication between each service and they can rely on each other, but they are run separately, connecting to separate databases. This allows us to scale them one-by-one when loads grow. Think about it. Should one of our services go down, this is a much better scenario.

All of the other services — which do not depend on the failing service — can keep working properly. Another great advantage is that every microservice can be implemented with the technology that’s best suited for the given use-case, using an optimized storage structure and optimized processing solution. Separate teams can work on the microservices, implementing a well-defined, bounded set of functionalities.

Microservice infrastructure — Containerization and Orchestration

As always, magic comes with a price. More resources have to be put into the infrastructure, DevOps side in exchange for increased flexibility, scalability and resilience. There can be hundreds of applications deployed and running independently in a microservice architecture. This results in increased in-application operation responsibilities.

Normally there are four parts to offering baking apps. A server (1) runs the operating system (2) that runs the application server (3) that runs our application (4). We have to make sure that these 4 participants are always compatible with each other. When we use only 1 monolith application this isn’t a big deal. But imagine when we have to make sure that hundreds of microservices with dozens of application servers are still compatible with the operating system and every other dependency. This would be a nightmare. You could lose a lot of sleep.

Figure 4: Container orchestration

Docker containers are designed to solve this issue. All your application developer needs to do is package the microservices inside a Docker container and then hand it over to the DevOps team. They will make sure works just fine. All the DevOps team needs to do is support running Docker containers, which will ensure that everything will be compatible. Developers can write the microservices in any programming language of their choice, such as Java Node.JS or Scala. Until they package it into a Docker Container the microservices will be able to run on the infrastructure.

Docker containers are a great tool to run your applications in a very flexible way. However, they cannot solve the scaling issue on their own. When we need to run 100 microservices, 4 instances each on 20 servers, well, to put it lightly, it can be a challenge.

It means starting and stopping 800 apps, observing which are running how much, and switching around servers endlessly. Sounds ineffective, doesn’t it? This is where Container Orchestration, such as Kubernetes, OpenShift or Docker Swarm can help. It removes the burden of starting, stopping and moving the microservices to new servers. All we need to do is state how many instances we need from the microservices. The Container Orchestration takes care of running them and balancing them between the servers. This gives us a very powerful setup to manage scalability and reliability issues.

The new infrastructure makes it possible to easily introduce new features and new services without risking the stability of the system as a whole.


Microservices, containers and orchestration enable us to introduce cutting-edge new financial services next to old core systems, extract functionalities one-by-one and make them scalable and more reliable. This not only gives us a safer way to replace old core systems but also introduces state-of-the-art technologies that give us the opportunity to respond to customer needs in a truly agile way. Wait four minutes for your app to upload a €1000 transfer? By using Microservices and Containerization, your customer will never need to wait long.

Would you like to hear more? Get in touch!

3. Big Data

Banks have been gathering a huge amount of data about us in recent years. They know all about where we spend our money and what we spend it on, how much we spend on travel or gambling and how frequently we reach our credit limit. This data usually sits in silos tied to specific services. Accessing or using the silos to build new services is an enormous amount of work. Also, these older-type services store only the most important data to minimize the amount that needs to be handled. Oftentimes the systems cannot handle more than a few years of it, so older chunks are backed up every now and then, making them entirely inaccessible to other services.

Do you like maple syrup? This is just like making maple syrup. Imagine you own a forest of maples. To make syrup you need to tap the trees. In fact, you need to concentrate on tapping the best trees. Old-style data systems are like the syrup producer who can only access the first few rows of trees. What a loss of potential! What a crime against pancakes!

This is a classic missed opportunity! We cannot build smart, real-time services with inaccessible and incomplete data. What we need is the ability to make predictions based on lots of good data. Who is thinking about getting a loan? Who is apt to buy travel insurance after buying their airplane ticket? With such information at hand we can show customers relevant products and services at the exact moment they need them.

In essence, what our maple syrup producer really needs to do her job is the ability to see all the trees in the woods and to get to any of them (perhaps the best ones) quickly.

These are the kinds of problems where Big Data comes in and makes cutting edge applications possible.

Data processing pipeline

Figure 5: How all that data is processed and the technologies used meanwhile

Big Data solutions aim to capture as much data as possible, enrich it with all possible information (e.g., make connections to already existing data) and store it in a place where it can be easily retrieved, processed and visualized. It can also make sense to save data in its raw native format, creating a data lake.

Let’s go through the steps of how data goes through a system and what technologies we can use along the way. We’ll use open source Apache technologies for our examples, but there are many alternatives at every step in other data processing ecosystems.

Data producers

This is where data is created. This can entail analytics events from a mobile phone, payment transactions from a POS terminal, an authentication log event from the auth server or literally anything else. When data is created we need to make sure it reaches the processing pipelines so it will be stored somewhere. In the maple syrup industry, the data producers would be the maple trees.


This is the layer and protocol for how the data gets into the processing pipeline. It’s usually moves through some kind of network: internet, lan, etc… In IoT use-cases MQTT is a standard protocol for data transport, while HTTP is another popular protocol for web-based applications. So for our maple syrup producer, this step is the same as tapping the trees and carrying the buckets of sap to the cooking house.

Data processing

There are two main ways we process data. The first is Batch Processing,which means that a certain process reads a large chunk of data, processes it and publishes the results. Then it retrieves the next chunk of data and processes that as well. This is repeated until all the data is processed. Hadoop is one of the most famous batch processing softwares. It can process huge amounts of data, but not necessarily in real-time. The usual use cases are report generation and analyses of unstructured data.

The second type of processing is called Stream Processing. It processes the data on the go in real-time. Spark streaming or Kafka streams are two regular players here. The usual use cases are data transformation, firing actions on certain events or machine learning model updates. Stream Processing is used when we need immediate actions upon data arrival. What would a maple syrup producer be doing in this stage? Cooking the sap, of course!


Selecting the right data storage is critical in terms of solutions we can build on top of the data. Every storage solution has advantages and disadvantages specific to certain types of data and certain types of data retrieval mechanisms. We need to select the right storage for the right use-case. Without going into detail, let’s look at a couple of examples: if we want to store large amounts of unstructured data and process them with Hadoop, most probably we should go with HBase as the underlying datastore. If we want to categorize payment transactions for a customer and save it as a database which we can later read in real-time we should use Cassandra. Should we want to search our data with a lot of dimensions we would be wise to use ElasticSearch.

As you can see, there are plenty of possible solutions, so we need to choose wisely. Sometimes it can also make sense to save the same data into multiple databases so we can always query the one that best fits our use-case. Of course, these are issues you’ll need to discuss with your data engineers. And if you don’t have a data engineer and you’re sitting on this much data, maybe it’s time to make that call. I mean, could you imagine a maple syrup producer without a warehouse for storing syrup, and without a system to manage all those barrels?

Presentation and visualization

After we’ve processed the data, generated the insights and stored them in a database, we would like to animate it through visualization. There are specialized visualization tools for certain types of data, like Kibana to display logs or Grafana to display time series data. These tools come with Enterprise-grade features so we don’t need to worry about authorization and security.

Should the above-mentioned tools not be sufficient for our use case we can easily create our own custom visualization tool with web technologies like React or d3.js. In this case we also need to think about the security aspects of the visualization application. For fans of maple syrup, this step is akin to knowing what you need to know about the warehouse, and to putting fancy labels on elegant jars of your syrup.


This future has arrived. For example, TrueMotion has already started collecting and using driving telemetry data to help reduce risk for car insurance companies. Seon helps pinpoint fraudulent activities in your data streams to make your system a safer place for your customers.

Utilizing these big data solutions can create immediate business value. Imagine having the ability to capture any event happening in your system and being able to take immediate actions on them. This event-stream is set to become the backbone of modern IT infrastructures. To sum it all up, in order to offer the very best services and apps, it is essential we make the most of our data. With its accessibility, speed and comprehensiveness, big data makes sure you are never left in the dark. Or without the best syrup on your pancakes.

You know what I’m going to say. Good data makes good syrup.
Would you like to hear more? Get in touch!

4. Artificial Intelligence and Machine Learning

Like in all areas of life — fintech being no exception — both end-users and service providers show an ever increasing need to have “intelligent” solutions to practical problems, like how to deal with fraud or mitigate risks of becoming complicit in money-laundering. Intelligence, whether artificial or not, leads to higher efficiency and lower costs, better customer experiences, high levels of compliance and security, and the ability to solve problems quickly.

Most often such problems would be relatively simple for a truly intelligent human to solve, but they pose a much bigger challenge for programs and programmers alike.

Nowadays handling Personal Finance Management (PFM) is a very common use-case for financial institutions. Imagine you bought your shiny new TV at your local Tesco the other day. You expect your banking app to categorize it as a Home Spending, I mean it’s clearly not Groceries, right?

Or let’s take fraud detection as another example. For every $1 lost in fraud, banks pay almost $3 for recovery. The crystal clear goal then is to detect the highest number of frauds without unnecessarily annoying customers with false alarms. Because obviously that credit card charge from last Friday evening in Amalfi is because you took your well-deserved weekend off, and not because someone’s up to no good.

Well, it’s not that clear and not that obvious — at least not from the program’s perspective.

AI Definitions 101

For an outsider, diving into the AI world can be quite confusing. Abstract concepts and two-letter abbreviations abound, so first let’s clarify what we mean when discussing terms like Artificial Intelligence, Machine Learning or Deep Learning.

Figure 6: AI 101
Artificial Intelligence is the broadest term which denotes the whole field in computer science which deals with solving problems traditionally associated with intelligent “learning” or “problem solving”.
Under the big umbrella of Artificial Intelligence, Machine Learning is the study of algorithms which (with proper training) can learn to solve new tasks without being explicitly programmed to do so.
The term Deep Learning covers a subset of Machine Learning methods which combine different “standard” ML techniques to effectively adapt to and make decisions from vast amounts of data.
On the same hierarchical level as ML are the subfields of Natural Language Processing or Computer Vision. Although in articles fintech problems are almost exclusively associated with ML, typical fintech use-cases like the automation of Customer Service or introducing Digital Assistants rely heavily on other areas of AI, like NLP.

The duck test for AI

So this new thing my company is so obsessed with is labelled as AI, promoted as AI, and has an AI price tag, but is it really AI?

Figure 7: AI Duck Test — Just as in the case of the proverbial duck, a lot of things can look and behave like an AI — and actually be nothing more then a big pile of “IF this, then that” functions or a human operators in a far off location carefully sorting data to categories
It’s not that easy to tell. Speaking more in layman’s terms: when talking about “AI solutions” experts generally mean some program which is not based on the “if Tesco: then Grocery” scheme, but which is able to get smarter over time and learn from its own mistakes.

Be aware! Having an AI-based solution doesn’t necessarily mean that it would be inherently better than a “traditional” one. In situations where you find it difficult to identify the problem you’re facing, AI is generally a “good fit.” This is one of the reasons many companies find it tough to take the first step toward adopting these new technologies.

Why is it so hard to jump into AI?

First of all, let’s be realistic: for the vast majority of software vendors working with AI/ML technologies is not part of their well-established day-to-day operations. Developing a custom AI solution certainly requires a fair amount of R&D effort. And it’s not just the core algorithmic part that can cause headaches… A custom-built AI project calls for a very diverse skill set: having someone on the team who understands AI is just a small — but inarguably vital — part of the big picture.

Figure 8: ML Pipeline with pre and post processing steps
A fundamental issue is the importance of data — both in quality and quantity. You can’t train an ML model with thin air! You’ll need data in order to teach your machine. Probably a lot of it.

To be able to train your PFM (Personal Finance Manager) categorizer to identify that “tricky” kind of TV purchase at Tesco you need to have a large number of correctly categorized examples.

And who will categorize those millions of transactions, you ask? Well, that’s the — sometimes literally — million dollar question. We’ll get back to it later.

In terms of data quantity the deal is fairly simple: the more the better. Keep in mind that no matter how much good quality data you piled up, you can only use about 60% of it for the actual training set. You need to put aside about 20% for a validation set which is used to prevent the algorithm from fitting too much on the training set (the whole point of using an AI-based solution is to be able to solve “unknown” problems, not just the ones you trained for). And finally, how can we objectively determine how good our new ML model is? The remaining ~20% data points are used as a test set — they are evaluated after completing training in order to see how the model would perform “in the wild”, but still in a controlled and measurable environment.

Figure 8: How data is used in an ML pipeline

Another factor that doesn’t seem unique to AI projects is the fact that it’s extremely hard to spot and fix bugs. Isn’t this true for all kinds of software development? Generally it is, but in a typical ML pipeline there are so many moving parts which need to be perfectly aligned with each other that the pipeline is particularly sensitive to any small disturbance. When something goes south, what causes it? Is it the data? Is it a simple bug in the preprocessing pipeline? The training parameters? The ML algorithm itself?

Finally I’d like to mention one more important factor from the business side of things, which is the need for new or updated KPIs that function with the non-deterministic nature of AI solutions. In a fraud detection system it probably doesn’t make sense to count only on the hard number of actual fraud attempts caught by the system. A good KPI would factor in the additional benefit an intelligent system would bring to the table by reducing the number of false alarms and thus the load on customer support.

On pitfalls and how to avoid them

Let’s do a quick recap of the pitfalls of working on an AI project we collected in the previous sections:

  1. Possible massive R&D work
  2. Data, data, data
  3. Particularly hard to fix bugs
  4. Need for a new way of thinking about the right KPIs

Undertaking these might cost a lot of time and money. Luckily, we happen to have some pieces of advice to help guide you through the process of kicking off an AI project.

First of all — with the help of domain and AI experts — try to gather as much information as possible to make sure you understand the complexity and extent of the problem. There’s no need to reinvent the wheel. See if your use cases match those that are already implemented by one of the tech giants like Google or Amazon. Try to use an existing ML model if you can, leveraging the time other companies put into a refined solution. Also be cautious with random ML models on GitHub implemented by a single developer. The lack of proper testing and validation might cause you more headaches than you save on the implementation costs.

Whether you’re going to use an existing model or create a new one you need to identify the steps of the pipeline you should build, and then assemble a team of experts for each field. Trust me, when implementing all the Big Data preprocessing steps it’s better to start with sanitized and good quality data at your disposal.

Speaking of data, we already covered the fact that you’ll need a lot of it. If you’re sitting on a big pile of good quality data (e.g., a lot of transactions already categorized for a PFM) then you’re in luck. If not, investigate the option to buy data — but be prepared, it can be very pricey. An alternative could be to build a dataset yourself. You can consider using your existing user base to help you build the necessary dataset for you.

(Sidenote: Remember the million dollar question? You’re already solving it for Google every time you mark images with traffic signs or vehicles in CAPTCHAS.)

Figure 9: A Google CAPTCHA example

In order to minimize the number of bugs and the time spent on finding and fixing them, utilize an iterative approach, especially for the actual ML algorithms. Start with the simplest ML model you can think of to filter out bugs in the pre- or post-processing steps. Only increase complexity when you’re already seeing good results.

And last but not least, on the business side try to set up realistic KPIs and expectations that can take advantage of the AI approach. Even if it feels a bit unnatural at first, try to embrace the potential strengths of the AI-based approach.

For example, when setting KPIs up for your new fraud detection system think of it as someone with actual intelligence. It might make a few errors on even simple tasks, but in the grand scheme of things you’ll benefit from its ability to deduce hidden logical connections and learn from its mistakes.

There is no denying that AI and ML are coming into the fintech world. Some will be ready for it, others not. By embracing AI, you will be nipping emerging problems and issues in the bud, ultimately raising efficiency and revenue, giving you the edge and more time to watch that new TV of yours.

Would you like to hear more? Get in touch!

The most important conversation

We believe that companies that are able to successfully integrate people, tech and business are the ones that will prevail . This can only happen through fruitful conversations between designers, engineers and decision makers. To start, everyone needs to take time to understand basic concepts and their interconnections in today’s tech landscape. We hope that our article helped you gain some clarity — from infrastructure to smart recommendation engines.
Let’s start that conversation!

At Supercharge we build high impact digital products that make life easier for millions of users. If you liked this article, check out some of Supercharge’s other articles on our blog, and follow us on LinkedIn, and Facebook. If you’re interested in open positions, follow this link.

4 technologies that are shaping the future of finance — an explanation for non-technical mortals was originally published in Supercharge's Digital Product Development Guide on Medium, where people are continuing the conversation by highlighting and responding to this story.