Database Migration from GCP into AWS Using AWS Database Migration Service
Explore how we overcame multi-cloud management challenges by designing and implementing database migration from GCP to AWS.
Executive Summary
Simplifying Multi-Cloud Management
Our Customer
Greenfence is a blockchain platform eco-system developed for the consumer goods industry to enable cost-efficient, transparent and trustworthy commercial relationships at scale. Greenfence empowers stakeholders along the end-to-end value chain to create individualized and secured blockchain networks that can connect, collaborate and transact on a private or public basis.
The Obstacles They Faced
Greenfence encountered challenges in managing its multi-cloud environment due to the use of different providers. More precisely, Greenfence’s application layer was hosted on AWS as a containerized application running on an ECS cluster, while their database layer resided in Google Cloud (GCP) using MySQL as the database engine. The variety of cloud providers presented Greenfence with several obstacles:
- The complexity of application management across cloud platforms.
- The security concerns related to the public-facing data endpoints.
- High total costs of the multi-cloud IT infrastructure.
How We Helped
We analyzed the client’s existing infrastructure, identified pain points, and devised a migration strategy from GCP (Google Cloud Platform) to AWS using AWS Database Migration Service (DMS) to streamline their database layer.
The Challenge
Data Integrity and Synchronization During Database Migration
Greenfence’s primary challenge revolved around migrating the database layer from GCP to AWS while ensuring minimal downtime. The migration needed to maintain data integrity, synchronize data in near real-time, preserve the platform’s performance and functionality and avoid disruption to the live web platform. The secondary technical challenge was dealing with the limitations of AWS DMS service when migrating the AUTO_INCREMENT attribute.
Key challenges Greenfence faced included:
- Data synchronization
Managing data consistency and replication between the AWS and GCP environments was a significant complication. The lack of a unified database environment made it difficult to ensure real-time data availability and synchronization across the multi-cloud setup. - Network connectivity and security
Connecting the application layer in AWS to the DB layer in GCP requires establishing secure and reliable network connectivity between the two clouds. Configuring and managing virtual private networks (VPNs), firewalls, and security policies across different cloud providers can be complex, requiring careful planning and coordination. - Performance optimization
Greenfence’s teams needed to optimize the performance of their database layer to meet the growing demands of their application. However, the distributed nature of the database across different cloud providers made the implementation of efficient performance-tuning strategies demanding. - Cost management
Maintaining a multi-cloud setup can lead to increased operational costs due to the need for separate management and infrastructure resources. Greenfence sought a solution to optimize costs while ensuring the required functionality and scalability of its platform.
The Solution
Real-Time Data Sync and Enhanced Security
As we at Romexsoft settled on addressing the obstacles Greenfence faced and simultaneously meeting the no-downtime migration requirement, we implemented the following solution:
- Database migration
We recommended using AWS Database Migration Service (DMS) to perform a homogeneous migration from the existing MySQL database in GCP to Amazon RDS for MySQL in AWS. This approach allowed for a smooth transition of the database layer while minimizing downtime. - Target database engine
Considering the compatibility and familiarity with the existing MySQL engine, we used Amazon RDS for MySQL as the target database engine. This choice ensured a seamless migration process and minimized any potential compatibility issues. - Manual configuration
In order to overcome the limitations of DMS in migrating the AUTO_INCREMENT attribute, we assisted Greenfence in manually configuring the AUTO_INCREMENT attribute on the relevant columns in the target Amazon RDS for MySQL instance. This step was crucial in maintaining the continuity of auto-incremented primary keys. - Configuration for ongoing replication
We configured AWS DMS for ongoing replication to obtain near real-time data synchronization between the source GCP MySQL database and the target Amazon RDS for MySQL in AWS. The achieved seamless continuous replication minimized data latency and enabled the migration process without disrupting the live web platform. - Increasing high availability
In the target database in AWS RDS, we enabled data replication across different availability zones (multi A-Z). - Enhancing security
We implemented data encryption at rest in the AWS RDS database and configured network topology with private subnets and corresponding security groups to provide secured private access to the DB only from the entire VPC where the application cluster is located.
Database Migration from GCP into AWS Using AWS Database Migration Service (DMS) – Architecture Diagram
Amazon Web Services Utilized
The Results
Continuous Service and Cost Efficiency
Near-zero downtime
By utilizing AWS DMS for ongoing replication, we reached near-zero downtime during the migration process. Greenfence’s live web platform remained accessible and functional throughout the migration, ensuring uninterrupted services for its users.
Smooth database migration
The migration process from GCP to AWS was executed with minimal disruption to Greenfence’s operations. Data integrity was maintained throughout the migration, and near real-time synchronization enabled a seamless transition to the target environment.
Improved performance and scalability
With the migration to Amazon RDS for MySQL, Greenfence gained access to AWS’s scalable infrastructure, which allows optimization of the performance and scalability of the database layer. This adoption resulted in enhanced responsiveness and improved user experience.
Enhanced data security
The private subnets, protected by security groups and with disabled public endpoints, ensured that the Amazon RDS instances were shielded from direct access from the public internet. This approach significantly reduced the risk of unauthorized access and potential security breaches, providing a secure environment for storing and accessing sensitive data.
Cost optimization
Through consolidation of the database infrastructure onto AWS, Greenfence achieved tangible cost optimization: up to 40% of cost reduction. Such significant savings were achieved with benefits, such as no charges for DataTransfer, reserved RDS instances and reduced expenses on the IT staff. In addition, the consolidation reduced operational complexity, eliminated the need for separate management of multi-cloud environments, as well as allowed Greenfence to take advantage of AWS’s cost-effective pricing models.
Why Romexsoft
Have a Smooth Database Migration
Romexsoft is AWS Advanced Tier Services Partner, trusted Software Development Company and Managed Service Provider, founded in 2004. We help customer-centric companies build, run, and optimize their cloud systems on AWS with creative, stable, and cost-efficient solutions.
Our key values
- Delivery of quality solutions
- Customer satisfaction
- Long-term partnership
We have successfully delivered 100+ projects and have a proven track record in FinTech, HealthCare, AdTech, and Media industries.
Romexsoft possesses a 5-star rating on Clutch due to its strong expertise, responsiveness, and commitment. 60% of our clients have been working with us for over 4 years.
Related Success Stories
Database Migration from GCP into AWS FAQ
When undertaking database migration to AWS Cloud, organizations often encounter challenges such as:
Data inconsistency: Data stored without standardised measures can’t be transferred uniformly
Downtime: Database migration might cause unprecedented downtimes
Compatibility issues: Data formats might not be compatible during migration
It's crucial to conduct a thorough assessment before initiating the migration process to mitigate these risks.
Yes, AWS DMS limitations include constraints on certain data types, limited support for some database engines, and the inability to handle complex transformations natively. It's advisable to review the AWS DMS documentation to understand these limitations better before starting the migration.
AWS provides a cost calculator that can help you estimate AWS DMS costs. Factors that influence the cost include the amount of data to be transferred, the instance type selected, and the duration of the migration. It's essential to factor in these elements for budgeting accurately.
Yes, you can migrate from GCP to AWS using AWS DMS, although the process may require additional steps for compatibility. AWS DMS supports various source and target databases, making it a viable option for migrations from GCP to AWS. However, it's crucial to test the migration path thoroughly to ensure a smooth transition.
