Deploying and Managing Cloud Microservices in GxP Environments - Part 2

Last time, we set the scene for microservices. Now that we’re familiar with what they are and why they’re important, let’s explore deploying and managing cloud microservices in GxP environments. Let’s dive right into it.

Approaches to deploying cloud microservices in GxP environments

When you’re working within regulated environments, a common approach is to deploy a copy of the microservice for each customer. It’s a very risk-conscious approach that really ticks the box with the classic regulated approaches of previous years, but it does come at a cost. If you have 400 customers, then you have 400 services to manage.  

And that’s assuming just a single service per customer (to put it into perspective, Monzo bank operates 2000 services).  

How do you handle this ever-growing number of services? You can automate, you can scale out by hiring more DevOps staff, but over time this can cause an impedance to your ability to innovate. If you take the Log4Shell incident in 2021 as an example, the more services you have, the more you could be vulnerable, and the more you must patch against exploits like this.  

And this is where using shared services come into play. But before we talk about shared services, let’s take a small segue to talk about tenancy, because when you start talking about shared services, people typically assume you’re talking about a multi-tenancy system, which may raise concerns for regulated companies.  

Multi-tenancy  

One concern often raised regarding multi-tenancy is the potential risk for data leakage, due to storing everyone’s data in the same database (a valid concern, but one that can be addressed by following some standard design patterns and best practices). But multi-tenancy covers more than just databases – it simplifies down to “a single software instance serving multiple tenants”.  

If the concerns are around the risk of data leakage, then you can easily mitigate that risk by taking data out of the picture, and a good example of this is a stateless service. These are services that do not store any data – they are given an input, process it, and return a response (like the malware service). These types of services are a prime candidate for being a shared service in a regulated environment.   

In fact, if you think about it, every service is inherently multi-tenant at the user level. If Karen and Virendra both submit a job to a service, then Karen should get her result and not Virendra’s, and vice-versa.  

So, what changes if you move that service from being called by 100 people from company A, to being called by 100 people from company A and 100 people from company B? Nothing – a user will submit a job and get their response (well, you may need to scale the service, but that should be automatic!).  

The takeaway here is that multi-tenancy can occur at many levels and its presence should not be seen as an alarm bell. The challenge faced by companies such as IDBS is ensuring that it is fit for purpose, especially for our regulated customers. 

Shared Services  

So, let’s jump back to our shared service approach, where we have a single service that is supporting multiple customers:  

By sharing services like this, we further reduce the operational cost of running, maintaining and securing the service. Should a defect be found, or a new capability added, then the service can easily be upgraded, and all customers will benefit from that.   

Let’s look at a couple of upgrade scenarios:  

Security patch to the service runtime 

Applying security patches is a key part of the contracted SaaS (Software as a Service) service and those security fixes should go out to all customers to ensure their systems are secured. These types of changes are low risk as the system code is not changing and therefore do not require any application requalification activities.  

New feature added to the service 

Adding new features to the codebase does require an impact assessment and potentially revalidation (albeit typically limited as the change is limited to the service, not the whole application), and not every regulated customer wants to do that. This can be handled by introducing versioning to the services. When an upgrade is available an updated version of the service is deployed, alongside the existing version. A regulated customer can, when ready, choose to swap across to this new service, performing whatever validation is required.   

And eventually, when everyone has been migrated, the older version of the service can be retired. The key here is that the customer is involved in this upgrade journey – it’s not being forced on them. They see the benefits of a consistent system and you see the benefits of operating shared services.  

How to manage cloud microservices in GxP environments

At this point, I hope you’re excited by microservices, understand the value they can bring, and how you can operate them efficiently, even in regulated environments. The challenge now is to determine how to use them. And the answer to that is to take it on a case-by-case basis. It can be all too easy to go overboard and microservice everything and that can be a route to failure. Our advice to you is to consider it on a case-by-case basis for each capability you add:  

• What does the usage pattern look like – are there peaks and troughs that could benefit from auto-scaling?  
• What does the resource usage look like? It could be big enough that it should be separated from your existing application to avoid stability issues.  
• Can it be shared across multiple applications and/or multiple customers?  
• Runtime costs – consider compute, maintenance and any other resources the service may be required and objectively use this information to make an informed decision.  
• Can the service benefit from a more optimized technology stack by carving out to meet the requirements? The beauty of microservices is that the tooling can be completely different from other services running as part of your platform. That is, choose the right tool for the job!  
• How quickly do you want to push new features out to production? A microservice architecture can lend itself to the rapid delivery of new capabilities.  
• What is the bounded context of the microservice? Consider its responsibilities and ensure that the boundaries are appropriate so that you do not go too small and resultingly distribute the domain across several microservices.  
• Leading on from bounded context, if the functionality is limited to a particular use case and none of the above tick any boxes, consider the overhead of distributing the domain logic rather than keeping the logic where it currently resides if pre-existing – this can lead to different challenges and costs.  

GxP Compliance  

As outlined above, the use of microservices can provide many benefits including improved performance, simplified maintenance, and flexibility of deployments and upgrades. For systems used in GxP-regulated environments, there is also the added benefit of introducing new features faster and with less revalidation time and effort as opposed to a full system upgrade due to the limited impact that microservices have on the entire application.    

As with all systems operating in a GxP-regulated environment, it is important to maintain the system in a validated state and introduce any changes to the application in a controlled manner. This applies to microservices as it would to any other component of the application. To maintain the validated state of any SaaS solution it is important to partner with a supplier that understands the needs of regulated customers.   

At IDBS, we understand the needs of our regulated customers and have demonstrated this by successfully delivering and maintaining SaaS-based applications to our regulated customers. All IDBS associates are required to train on GxP annually and GxP risk assessments are performed as part of our Engineering Software Development Lifecycle (SDLC).   

This blog accompanies our discussion panel on Deploying and Managing Cloud Microservices in a GxP Environment