Aerospike Connect for Spark 3.0.2 (June 2, 2021)

Announcements Releases (Server, Client & Tools)
1

Aerospike Connect for Spark version 3.0.2 was released on June 2, 2021.

  • Tested with Apache Spark 3.0.0, Scala 2.12.11, & Python 3.7.
  • Minimum supported Aerospike Server version 5.0.
  • Supported until June 30, 2022.

Improvements

  • This library is an uber shaded jar.

Bug Fixes

  • [CONNECTOR-208] - Spark connector with default timeout settings is timing out after 1 second.
  • [CONNECTOR-205] - Filter out records that breach write block size in Aerospike via Spark Connector.
  • [CONNECTOR-212] - Handle nulls in full record writes (REPLACE, REPLACE_ONLY, and CREATE_ONLY).

Known Issues

  • This release does not support Aerospike 5.6 boolean bin and quota features.
  • DataSource v2 API does not support the SQL statement INSERT INTO a temp view. Use DataFrame syntax for equivalent functionality.
  • Streaming write does not work with Apache Spark 3.1.0.
  • Streaming update trait SupportsStreamingUpdate from Spark 3.0.0 has been renamed to SupportsStreamingUpdateAsAppend in Spark 3.1.0.
  • [CONNECTOR-312] - Update spark connectors with latest java clients to address CLIENT-1637 (“partition unavailable” errors occur). Fixed in version 3.0.3 .

Updates

  • The default value of flag aerospike.partition.factor have changed from 12 to 8. Please update your application accordingly.