FAQ Is there a way to get detailed debug information from the Spark connector?
When using the Aerospike Connector for Spark to move data between Aerospike and Spark, is there a way to get detailed debugging information on what is going on?
pyspark within a console window it is possible to set the logging level to debug using the following statement:
This will generate detailed debugging information on any subsequent statements that are executed. This can be useful when looking to see what data is being loaded where.
- Full documentation on changing log level in Spark.
- Reference documentation for the Aerospike Spark Connector.
SPARK CONNECTOR DEBUG LOG LEVEL