Architecture Design 101 (From Monolith to Multi-Cluster with Istio)

Back then we hosting all our applications in a single Monolith server (we were using Exabytes at that time).

Single Monolith Server

Pros:

• Cheap

Cons:

• Not able to scale
• Single point of failure
• Very hard to isolate between applications
• Unable to handle a huge load
• No disaster recovery

In 2018, we migrated our monolith server to a single cloud Kubernetes cluster which contains 2 master and 3 worker nodes.

Single Kubernetes Cluster

Single kubernetes cluster architecture

Pros:

• Cheap
• Easy to manage especially having very few projects
• Disaster recovery

Cons:

• Single point of failure
• Unable to separate the billing based on the projects — we are unable to justify the actual server cost for specific project/product
• Very hard to manage by different teams
• Configurations conflict, required to set a standard naming convention

Since 2019, we started to create more and more clusters due to business needs.

Overflow with multiple Kubernetes cluster

Multiple Kubernetes Cluster (Multi-Cluster)

As you can see from the diagram below, we are unable to share the common applications such as Redis, NATS, etc between the clusters. Furthermore, this limitation also increase the cost of maintainability (Whenever a new version of Redis is released, we need to go into every cluster and upgrade it accordingly)

Mutliple kubernetes cluster

Pros:

• Separation of billing
• Easy to manage by different teams
• Separation of products
• Disaster segregation

Cons:

• Much higher cost
• Required more manpower on managing the clusters
• Wasting resources
• Unable to share common resources

Around September 2021, we decided to restructure it to Service Mesh.

Multiple Kubernetes Cluster with Istio (Service Mesh)

Multi-cluster with Istio

We move those sharable applications to a standalone cluster (we named it as shared in the diagram). In this way, we are no longer required to deploy Redis cluster in each of the Kubernetes clusters. Instead, we can just consume from the shared cluster.

Pros:

• A/B Testing
• Canary deployment
• Separation of products
• Much better version control
• High availability
• Cheaper than multi-cluster because some resources can be sharable

Cons:

• Slightly higher latency (around 2–5ms)
• Slightly higher memory consumption due to envoy proxy
• Hard to learn and setup

Currently, we only configured for the single region with multiple zone support. In the future, we probably will explore on multi-region cluster load balancing with Istio.

Thank you for reading.