Tags: lift-and-shift, MongoDB, PowerShell
An international healthcare company felt the urge to migrate core components of their application initially to Azure Infrastructure-as-a-Service (IaaS) with a long-term goal of transferring the solution to Azure Platform-as-a-Service (PaaS).
We came up with a solution that automates the ‘lift and shift’ migration of their existing Linux-based MongoDB Cluster (dozens of virtual machines) to Azure IaaS. We created the cloud automation script that allows not only to deploy the customer’s system but also to upgrade and scale it in the cloud, thus making the whole system cost-effective and improving its performance.
Watch the following video to find out more about our technical approach to the MongoDB Cluster Deployment automation.
Continue reading to learn more technical stuff about this interesting project.
Original Problem
As you may already know, the MongoDB clusters usually have a complex structure that includes shards for storing data, configuration servers and applications. The original customer’s system included dozens of shards, each stored in a separate database server instance.
We faced a complicated task of deploying this system to Microsoft Azure Cloud with many important factors. For example, any petty error in the settings of one of the shards may cause the overall system failure. So, we had to determine and eliminate all the errors before going live in the Azure cloud. Also, we had to consider cloud pricing because the customer pays for every minute of actual cloud resources usage. Another critical issue relates to the scalability of the system: we needed to ensure that it can be easily scaled or upgraded. Eventually, the customer wanted to have an option to backup the system at any moment.
Automated Solution
Generally speaking, Microsoft Azure provides a lot of templates for automatic deployment of various databases. There was no ready-to-use solution for automated deployment of a MongoDB cluster until the moment we have performed this database migration project. Therefore, we were the pioneers and have created the solution that was later used as a template for other similar projects.
Hence, we have created a PowerShell script able to automatically deploy a MongoDB sharded replica to Microsoft Azure cloud. You can see the example of the script on the image below.
This script uses settings from the XML files to configure the parameters of the cluster and the settings of the Azure resources. Apparently, you can easily update the system configuration at any time. We provide the example of this configuration file below.
Additionally, we have implemented the automated system to log the deploy process. Also, this automated system checks whether all of its stages were completed successfully or not. The PowerShell script creates a special Log folder for every virtual machine in the cloud. Thus, you can easily discover the errors and isolate them.
We store all versions of the configuration XML files in the History folder. Obviously, the customer can back up to any version of the system if necessary.
The PowerShell script works in 3 steps:
- Runs Linux virtual machine in Microsoft Azure cloud.
- Installs MongoDB servers to it according to the system’s determined parameters.
- Aggregates these servers into the cluster.
In addition, we have provided the reusability of developed scripts. Thus, we can use slightly modified version of the script to perform the cloud system upgrade.
Benefits and Results
The customer required that the deployment of the MongoDB sharded replica in Microsoft Azure Cloud should last no longer than 90 minutes. Our script performs automated deployment of the cloud cluster 3 times faster and completes the work in 30 minutes. You can never achieve the same result using manual MongoDB cluster deployment.
Apart from the high speed and impressively accurate performance of the automated script, we have grants some other valuable benefits. For instance, we are capable of keeping the customer’s data synchronized all the time. Moreover, we use automated tools to discover errors and we can back up the system at any point. What’s more, when the main script finishes the automated deployment, we run a special script to check that the system was deployed correctly.
Nevertheless, you have to consider that the developed solution does not include any graphic user interface. That said, you will have to master the command line to perform the automated deployment or upgrade of the cloud system. Furthermore, all the changes to the structure of the customer’s system should be made by editing the XML files. The lack of front-end might be not very obvious for inexperienced users, but the back-end seems to be a real masterpiece.
Feel free to contact us to propel your move to the cloud.
