Migrating Cassandra to Amazon DynamoDB

Are you looking for greater flexibility, speed, and scale for your Apache Cassandra solutions? You can consider Amazon’s DynamoDB as an alternative for your existing workloads.

The DB Best migration team has already used the new data extraction agents to migrate Apache Cassandra database to Amazon DynamoDB. So, here we will share our experience with you. Also, we will talk about the unusual migration approach implemented in the AWS Schema Conversion Tool for this pair of databases.

Background on Apache Cassandra

Apache Cassandra is a free and open-source distributed NoSQL database management system. Cassandra is a wide column store database. Rows are organized into tables and the first component of a table’s primary key is the partition key.

Initially, Cassandra was developed to handle large amounts of data across many commodity servers. Since July 2008, when Facebook brought Cassandra to the market, developers created quite a few versions of the Cassandra database. Actually, the code behind these versions is very different, and this brings many incompatibility issues when working with Cassandra workloads. So, you have to pay attention to the Cassandra database version and consider its limitations when you start a migration project.

Background on Amazon DynamoDB

Amazon DynamoDB is a fully managed proprietary NoSQL database service. DynamoDB allows you to create database tables that can store and retrieve any amount of data.

Our developers figured out that DynamoDB databases may serve almost any level of request traffic. You can easily scale up or scale down your DynamoDB tables’ throughput capacity without downtime.

Why choose Amazon DynamoDB instead of Apache Cassandra?

Sometimes Amazon DynamoDB can provide greater scale and performance over your existing Apache or Datastax Cassandra workloads. This very much depends on your database use cases. For example, DynamoDB works better with real-time bidding platforms, gaming applications, and recommendation engines. Cassandra has its own advantages too, however, now we will consider the benefits of Amazon’s solution.

• Performance. DynamoDB scans the data much faster, especially if you don’t have a primary key in your query.
• Consistency. DynamoDB provides you with strong consistency, while Cassandra can have issues with frequently updated data due to latency issues between distributed nodes. For example, Datastax support posted an example of this problem in a post titled – Dude! Where’s my data?
• Global reach. In addition to that, DynamoDB provides Global Tables for deploying multi-region, multi-master databases without you having to maintain your own replication solution.
• Security. This may also be a major concern for Cassandra’s data at rest, while DynamoDB encrypts the data at rest and in transit.

If you’re encountering similar issues, consider moving your workloads to Amazon DynamoDB! However, this may be a challenging task. Let’s discuss some of the key limitations we encountered while migrating Apache Cassandra to Amazon DynamoDB for our customer.

Despite the similarities in data models, Apache Cassandra and Amazon DynamoDB have some critical architectural differences. Generally speaking, Apache Cassandra is a column-oriented data store, while Amazon DynamoDB is a key-value and document-oriented store.

Cassandra’s table consists of rows. These rows may contain different numbers of columns. Opposed to that, DynamoDB considers rows as database items, and cells as attributes. Here you can define a schema for each item, rather than for the whole table.

Both Cassandra and DynamoDB databases require primary keys for your tables, and they both use partition keys to distribute your data. However, the meaning of partition is different in Cassandra and DynamoDB. In Cassandra, a partition is a set of rows with the same partition key. Therefore, Cassandra stores these rows on one node. In DynamoDB, a partition is a physical part of storage allocated for a particular chunk of a table.

Finally, we should note that Cassandra supports more data types than DynamoDB. So, during while migrating Cassandra to Amazon DynamoDB, you need to consider the right type mapping. Below, you can find additional challenges that emerge when migrating Cassandra to Amazon DynamoDB.