Our managed Apache Spark offering on Apache Cassandra now moves into full release.
With the release of Apache Spark 1.6 using the Spark Cassandra Connector v 1.6.0-M1, our Spark+Cassandra offering moves out of Preview status and into full release. With this release, based on learnings from supporting customers through the preview period, we have applied additional tuning to our standard Spark configurations to make them more robust in a wider variety of scenarios. The release also benefits from several enhancements to our console Spark functionality, including the ability to kill Spark processes directly from our console application.
In addition, Spark 1.6 and 1.5 include several significant improvements over Spark 1.4 (our previously supported version). Some highlights include:
- Unified Memory Management (shared memory pool for execution and caching)
- New Dataset API (best of RDDs + DataFrames)
- Backpressure control for Spark Streaming
- Full release of the Spark Streaming Direct Kafka API
For full details for new Spark features can be found at: