Aerospike Connect for Spark version 2.8.0 was released on July 14, 2021.
- Supported until October 14, 2022.
- Tested with Apache Spark 2.4.7, Scala 2.11.12, & Python 3.7.
- Minimum supported Aerospike Server version 5.0.
- [CONNECTOR-166] - Support batchget queries with digests in Spark Connector.
- [CONNECTOR-142] - Data Sampling using the Spark Connector using
- [CONNECTOR-142] - Support boolean bins in the Spark Connector (refer to
aerospike.booleanbinin the documentation).
- [CONNECTOR-211] - Support partial updates of records using the
- Migrated from queryPartiton() call to ScanPartitions().
- Updated Spark version to 2.4.7.
- Update Client version to 5.1.5.
- Migrated to Expressions for scans.
- Pushdown support for Float & Double datatypes.
- [CONNECTOR-205] - Filter out records that breach write block size in Aerospike via Spark Connector.
- [CONNECTOR-212] - Handle nulls in full record writes (REPLACE, REPLACE_ONLY, and CREATE_ONLY).
- [CONNECTOR-215] - Writes are slower in the Spark Connector v2 version. Introduced a new flag
aerospike.write.batchsizeto control write throughput.
- DataSource v2 API does not support the SQL statement
INSERT INTOa temp view. Use DataFrame syntax for equivalent functionality.
aerospike.write.modeflag overrides Apache Spark write mode.
- Spark connector stores spark DateType and TimestampType as long. In Aerojoin API calls convert these types to long.
- The default value of flag
aerospike.partition.factorhas changed from 12 to 8. Please update your application accordingly.