Setting Job Group and Job Description for Zeppelin Spark SQL Paragraphs
I’m having trouble setting a custom job group/description for the jobs triggered through spark.sql paragraphs on Zeppelin. I’m able to change this for the spark interpreter by executing sc.setJobGroup(…) in the editor, but not sure how to go about this for sql.
I have tried extending SparkInterpreter and SparkSqlInterpreter classes so that I can modify the sc.setJobGroup(…) call within the interpreter classes. But this also did not work for the sql version.