If you’ve ever been told that Kubernetes is your golden ticket to seamless multi-cloud migrations, you’re not alone. Many engineers and enterprises adopt Kubernetes with the promise of portability—an ability to move workloads effortlessly across cloud providers. But does this promise hold up in real-world scenarios?
Let’s break down the complexities, trade-offs, and best practices of migrating a Kubernetes-based application from AWS to Google Cloud Platform (GCP). Using an intermediate-level case study featuring Federico Fregosi’s insights, we’ll explore whether multi-cloud portability is as straightforward as it seems and what you can do to get closer to that goal.
At Stack8s.ai, we take Kubernetes portability to the next level. Our platform provides seamless integrations with most CNCF-registered applications, allowing you to deploy, migrate, and manage workloads effortlessly—not just across multiple clouds, but also on-premises. Unlike traditional solutions that leave you wrestling with cloud-specific limitations, Stack8s.ai abstracts the complexity, offering built-in tooling to simplify migration workflows, storage abstraction, and service integrations.
But we don’t stop there. We’ve successfully built Kubernetes clusters spanning 15 different cloud providers, all controlled from a single unified control plane. This means you can deploy and orchestrate workloads across AWS, GCP, Azure, and beyond—all from one place. Ready to experience the next evolution of true multi-cloud Kubernetes? Get in touch for a demo via our website or reach out at support@stack8s.ai. Let’s redefine cloud portability—together! 🚀
Why Multi-Cloud Portability Seems Simple—but Isn’t
Kubernetes, often hailed as the "universal cluster manager," gives organizations hope for avoiding cloud vendor lock-in. In theory, you package your app as a container, run Kubernetes on any cloud, and voilà—easy migration!
But in practice, moving workloads across clouds involves much more than just Kubernetes itself. Dependencies like storage, databases, queues, monitoring, and other services often tie applications to specific cloud providers. Each switch brings a series of challenges beyond just the Kubernetes layer. Let’s unpack these through the lens of a hypothetical migration of a modern app.
The Migration Scenario: Moving an App from AWS to GCP
Imagine you’re part of a migration team tasked with moving a mobile app backend from AWS to GCP. The app consists of three microservices deployed on Kubernetes clusters using AWS’s Elastic Kubernetes Service (EKS). These microservices handle functions like image uploads, metadata storage, and notifications.
Here’s a high-level view of the AWS architecture:
- Microservice 1: Accepts image uploads, stores them in S3, and saves metadata in RDS (a managed PostgreSQL database).
- Microservice 2: Processes the images from S3, adds watermarks, and notifies users via email using third-party services.
- Monitoring: Uses Prometheus and Grafana.
- Secrets Management: Handles credentials via HashiCorp Vault.
On GCP, the goal is to recreate this setup using their managed services like Google Kubernetes Engine (GKE), Cloud Storage, and Cloud SQL. The question is: Can we simply repoint configurations and redeploy? Spoiler alert—it’s not that simple.
Dependency #1: Container Registry
The first hurdle in moving to GCP is where the container images for the microservices are stored. On AWS, they're in Elastic Container Registry (ECR). While GCP can technically pull images from ECR, this introduces complexity, including additional authentication steps and potential maintenance overhead.
Options for Resolving the Container Registry Issue:
- Keep images in ECR: Requires custom authentication setups, which increases operational costs.
- Move images to GCP Container Registry: A simple script can migrate images, but it introduces the need to modify deployment configurations.
The friction here shows how even the smallest dependency can ripple across the migration process.
Dependency #2: Databases
Next up is the managed PostgreSQL database (RDS) in AWS, which needs migration to GCP’s equivalent, Cloud SQL. But there’s a twist. While databases like PostgreSQL offer some consistency across providers, switching to Cloud SQL still involves reconfiguring connections and ensuring migrations don’t disrupt the application.
Alternative Solution:
Instead of relying on cloud-managed databases, some companies (e.g., Zalando) use PostgreSQL operators like Patroni to deploy databases directly inside Kubernetes clusters. This improves portability but drastically increases maintenance overhead.
Key Takeaway:
Shifting to self-managed services reduces vendor lock-in but comes with increased operational responsibility. It’s not just a technical choice—it’s a business decision.
Dependency #3: Storage
AWS S3 is central to many AWS-native apps, but GCP offers Google Cloud Storage, which isn’t fully compatible with S3’s APIs. This means application code interacting with S3 often requires modifications when switching to Cloud Storage.
Advanced Solutions:
- Container Storage Interface (CSI): Kubernetes makes it possible to abstract storage dependencies using the CSI driver. The app mounts a storage volume instead of directly interacting with the cloud provider’s APIs. While this improves portability, it may not perfectly replicate S3’s object storage behavior.
- Anti-Corruption Layers: By introducing a middleware between the application and storage (e.g., an adapter or facade), you can centralize changes in one layer rather than modifying all microservices.
Trade-Off:
Reducing cloud dependency at the storage level often adds complexity. Maintenance and compatibility issues, like limited file ownership support in CSI drivers, can arise.
Dependency #4: Queues
The app uses AWS’s Simple Queue Service (SQS) for communication between microservices. On GCP, Pub/Sub is the equivalent service, but their APIs are incompatible. Migrating queues is one of the trickiest challenges, especially if applications rely heavily on custom logic tied to the queuing system.
The Dapr Solution:
Dapr, a CNCF-backed project, provides a powerful abstraction. Acting as a sidecar, it enables microservices to interact with queues, storage, and other services through consistent APIs, regardless of the underlying provider.
With Dapr, you can configure it to switch from SQS to Pub/Sub without altering application code—an example of shifting from code dependencies to configuration dependencies.
Gotchas with Dapr:
- While Dapr accelerates portability, it may limit access to advanced features tied to specific cloud services.
- Its dependency on shared secrets or configurations requires careful planning when managing sensitive data.
Additional Considerations: Monitoring and Infrastructure as Code
When migrating, ensuring observability and infrastructure alignment is critical:
- Monitoring: By using tools like Prometheus and Grafana, which are not locked to specific clouds, our app was able to retain monitoring portability.
- Infrastructure as Code (IaC): Moving beyond Terraform, CNCF projects like Crossplane offer Kubernetes-native declarative IaC, further simplifying resource management across clouds.
Key Lessons from the Migration
1. The Ecosystem Plays a Big Role
Kubernetes itself doesn’t solve portability—it’s the surrounding ecosystem tools that make a difference, whether they’re CNCF-backed like Dapr, Prometheus, or Crossplane.
2. Shift from Code to Configuration
Portability improves when dependencies are moved out of application code and managed via configurations or sidecars like Dapr.
3. Portability Isn’t Free
Reducing vendor lock-in often comes at a cost—whether that's increased implementation time, operational complexity, or additional management overhead.
4. Trade-Offs Are Business Decisions
Bringing certain dependencies into the Kubernetes cluster might make migrations easier but will significantly raise operational maintenance levels.
Conclusion
Migrating applications across clouds with Kubernetes isn’t as straightforward as marketing promises suggest. While Kubernetes excels at running containerized apps, it isn’t a silver bullet for multi-cloud portability. Success depends on understanding and addressing dependencies at every layer—storage, databases, queues, and beyond.
By leveraging Kubernetes’ ecosystem, moving from code-based to configuration-based solutions, and making strategic trade-offs, you can get closer to true portability. However, remember that it's not just about technical excellence—it’s about making the right business decisions for your team and organization.
For more insights, check out CNCF-backed tools like Dapr and Crossplane, or explore further best practices in multi-cloud management.