FAQ - Is there a way to get detailed debug information from the Spark connector?

FAQ Is there a way to get detailed debug information from the Spark connector?

Detail

When using the Aerospike Connector for Spark to move data between Aerospike and Spark, is there a way to get detailed debugging information on what is going on?

Answer

Yes, when pyspark within a console window it is possible to set the logging level to debug using the following statement:

sc.setLogLevel('DEBUG')

This will generate detailed debugging information on any subsequent statements that are executed. This can be useful when looking to see what data is being loaded where.

Notes

Keywords

SPARK CONNECTOR DEBUG LOG LEVEL

Timestamp

July 2020

© 2015 Copyright Aerospike, Inc. | All rights reserved. Creators of the Aerospike Database.