Tags: AWS Schema Conversion Tool, AWS Snowball
One of the main concerns during large-scale database migrations to the cloud is how long will the data transfer last. When you need to move multiple terabytes of data, the migration process may last for weeks or even months. In addition, the bandwidth of your network connection becomes a limiting factor, and some security concerns may appear. So, the whole migration project becomes unsustainable, and many customers with heavy-weight databases had to abandon their cloud migration initiatives. Amazon came up with a physical solution called AWS Snowball Edge, which allows for fast and secure data transfer of up to 80 TB of data in a matter of days.
We at DB Best had a great opportunity to test the latest AWS Snowball Edge device in our data-center. Being half the size of the original AWS Snowball, the latest version of the appliance can store up to 83 TB of data. This allows for speeding up large-scale data transfers, even taking into account the shipping time of the device.
Managing data and applications anywhere, we often face the issues related to the migration of huge amounts of data for our customers. So, as the Amazon partner, we received the brand new AWS Snowball Edge for testing purposes and tried to migrate our Oracle Database to Amazon Aurora PostgreSQL. Watch the following video to learn more about our experience with AWS Snowball Edge.
About AWS Snowball Edge
AWS Snowball Edge is a physical appliance, which combines storage and compute power. This means that you can use your AWS Snowball Edge as a stand-alone storage without the Internet connection. For example, this may be useful at remote locations for data collection and analysis.
But of course, the main purpose of the AWS Snowball Edge device is the data transfer. You can use the Snowball appliance both for data transfers in to Amazon S3 and out of Amazon S3. Since the database migrations are the core of our business, let’s take a closer look at this feature. Generally, the database migration with AWS Snowball Edge looks like shown on the picture below.
You can rent a Snowball Edge appliance for $30 per day. When you order a Snowball Edge in the AWS Console, Amazon charges you $300. This fee includes first 10 days of onsite device usage. Please note that you can order a Snowball appliance only if you are the existing AWS customer.
Useful tips
AWS Snowball Edge arrives at your data center already configured, with S3 buckets, Lambda functions and clusters pre-installed. Literally, this device looks and feels for your database applications exactly as your Amazon cloud storage.
AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT) significantly simplify and expedite the database migration process. SCT helps you convert the schemas during the heterogeneous migrations. Also, the Schema Conversion Tool provides you with the interface to manage the AWS DMS replication agents, which basically extract the data from the source database and upload it to the Snowball Edge.
When you complete the data transfer into AWS Snowball Edge, you need to ship it back to Amazon. At this time new transactions that take place in your source database. You need to pick up this data after the Snowball Edge arrives at Amazon and the admins plug in the device to import data into the S3 bucket.
After the transfer of original data and the above-mentioned ‘delta’ is completed, we recommend to verify the migration. You can use the DB Best Database Compare Suite to run the validation task.
You can cluster several Snowball Edge appliances together for parallel data transfer. However, if you need to transfer petabytes of data, opt for ordering the AWS Snowmobile data truck. This is an enhanced version of the AWS Snowball device, which can store up to 100 petabytes.
Observations
AWS Snowball Edge proved to be a very convenient device. It’s really simple to set it up: you just need to use the power cord and Ethernet connection. However, as the work goes on, some specific features may emerge, so the experience of managing AWS devices should be useful.
You can be sure that the data in the AWS Snowball Edge appliance is secure, encrypted and tamper-resistant. Initially, the device is locked, so, nobody can access it. You can get the unlock code in the AWS console.
The E-Ink panel on top of the AWS Snowball device displays the shipping information. The main display on the front panel shows all the necessary information to connect and interact with the Snowball Edge appliance from your PC.
