Amazon definitely piqued our interest by announcing their new Managed Cassandra Service (MCS) yesterday. As a database vendor, we’ve been asked by many about our view and even whether they’re running Scylla under the hood since they promise single digit latency. Here is our quick analysis while the discussion is still being hotly debated on Hacker News.
What Is it, Exactly?
Like many new services it’s hard to figure out what exactly being offered. An open source blog was published but it’s hard to sort out the architecture (and the limitations or advantages) of their implementation. This is about the 4th re-implementation of the Cassandra API (after Scylla, Yugabyte, CosmosDB), and certainly not the first cloud-hosted, managed Cassandra-as-a-service (Scylla Cloud, DataStax, Instaclustr, Bitnami, Aiven, and so on), so why keep it a secret? AWS may be doing themselves a disservice in being less transparent.
For AWS’ sake, we can say that they continue to surprise the industry with their rigor and their relentless execution. No service is immune from AWS. It doesn’t matter if it’s open or closed source, hardware or software — any worthy technology will eventually be offered as a service on AWS, even if they have an already robust alternate offering. Usually AWS prefers to go horizontal, broadening their portfolio like in the Cassandra/MongoDB case, instead of going vertically and focusing on a narrower set of better offerings. And they are always looking to learn and adapt, like the switch from Xen to KVM+Nitro.
That said, let’s take a look at what Amazon did here. MCS is a form of chimera: the front end of Apache Cassandra (including the CQL API) running on top of a DynamoDB back end (both for the storage engine and the data replication mechanisms). As confirmed on Twitter:
If you reverse engineer their pricing, you’ll discover it closely resembles the DynamoDB prices with an extra 16%, probably for the added translation layer. It’s compatible with a mix of Apache Cassandra 2.0 and 3.11 CQL features with many limitations we’ll discuss soon, but let’s be positive and start with the upside:
- It’s ‘serverless’ — As anyone who’s used to spin-up nodes and clusters before you can use them, this is quite refreshing. When you login to the MCS console, you can create a keystore and tables without spinning up anything, very convenient!
- It’s especially convenient for a complex service like Cassandra with all of its tuning challenges and administration overhead.
- It’s integrated with IAM, AWS’s authentication service. That’s very nice and secure too.
- The compute is detached from the storage. That’s a big upside for many Cassandra workloads.
- Fast Lightweight Transactions (LWT) – This is great actually, DynamoDB is based on a leader-follower model, thus simple transactions do not pay a penalty — nice!
Looking deeper, the functional differences are significant. There is no multi-region support, no UDT, no ALTER TABLE, no counters — all pretty fundamental for Cassandra users.
Since Cassandra is used, a JVM is used under the hood, and we are left to speculate if and how garbage collection (GC) may strike. Obviously GC mitigation and Cassandra tuning can be problematic in an “as a service” offering since the user has no control on their tuning.
There are no materialized views, no ability to load SSTables directly (a capability open source users have gotten used to). It’s unclear whether the DynamoDB storage limit of 1 megabyte per item exists here. Scylla for example can store objects at the size of 10s of megabytes.
Yes, there’s no doubt this is expensive, with $1.45 per million writes, a workload consisting of an average of 100k iops will cost 1.45×0.1×24×3600×365 = $4,572,720/year! To compare, Scylla runs 200k iops with three i3.2xlarge nodes. At $0.312 hourly per node, that’s 0.312×3×24×365 = $8,900 annually plus licensing.
Lastly, there is the flexible replication and consistency level. MCS allows you to use consistency level of one or local quorum (but no more than this), and no control over the replication factor (we assume its 3 base on DynamoDB architecture). Scylla and Cassandra allows you to have any replication factor and many more consistency levels.
Who Is This Aimed at?
As the MCS service is more expensive but less powerful than DynamoDB, Amazon must be aiming its new service at pure Cassandra migrators. However, the current set of limitations is quite a hurdle for those who wish to move to the convenience of serverless distributed Cassandra API. The service is still in its infancy and we’re sure it will get better, yet some basic limitations are inherent with their implementation model.
What’s ScyllaDB’s Take?
AWS continues to march on with impressive execution speed. Some products are very impressive, such as the new ARM chip, everything around s3 and Aurora. However, we find MCS to be as a hybrid, ‘chimera’ — half-Cassandra, half-Dynamo, semi-compatible, serverless but with a potential for GC, mostly proprietary with a dash of open source.
Yes, AWS promises to contribute back to Cassandra. The question is whether those contributions be more than self-serving ones such as the AWS IAM authentication or bigger contributions beyond pluggable backends. Time will tell but with the underlying DynamoDB database there are no incentives to make real progress. To Amazon’s credit, they already made a single major contribution in the form of the Dynamo paper, 12 years ago.
We at Scylla can certainly learn from how Amazon implemented MCS. Its integration is solid, the autoscale capability is great, and the GUI is well done. Competition is vital for our brains, for the whole industry, and for you, our users, developers and community members.
In that spirit of competition, if you’ve read this far, we invite you to have a look at our Scylla Cloud, which supports both CQL and our new DynamoDB-compatible API (Project Alternator) that offers 5x-20x price/performance gains over other NoSQL databases and killer features like workload prioritizations that guarantee per workload SLA.
The post Managed Cassandra on AWS, Our Take appeared first on ScyllaDB.