Zeppelin on Amazon EMR

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Zeppelin on Amazon EMR

Anders Hammar
Hi all,

I have created a script that can be used as a bootstrap action for installing Apache Zeppelin on an Amazon EMR cluster. I thought I would share it on this mailing list, hoping that someone will benefit from this.

You can find the script here:

The bootstrap script will parse the /home/hadoop/spark/conf/spark-defaults.conf configuration file and pass the YARN specific flags to the Spark interpreter in Zeppelin as ZEPPELIN_JAVA_OPTS so that the EMR cluster is fully utilized when notebook is executed. (Requires that Spark has been installed with the -x flag when the EMR cluster was created, see https://aws.amazon.com/blogs/aws/new-apache-spark-on-amazon-emr/ for details.)

Best regards,
Anders


Reply | Threaded
Open this post in threaded view
|

Re: Zeppelin on Amazon EMR

moon
Administrator
Hi Anders,

Thanks for sharing the script. Awesome!
This really helps!

Best,
moon

ps.
Please consider contribute code the to Zeppelin project if you're not planning to create repository for your code. Like spark-ec2 script in spark source tree. It'll help everyone improve the script.


On Fri, Jun 26, 2015 at 12:18 AM Anders Hammar <[hidden email]> wrote:
Hi all,

I have created a script that can be used as a bootstrap action for installing Apache Zeppelin on an Amazon EMR cluster. I thought I would share it on this mailing list, hoping that someone will benefit from this.

You can find the script here:

The bootstrap script will parse the /home/hadoop/spark/conf/spark-defaults.conf configuration file and pass the YARN specific flags to the Spark interpreter in Zeppelin as ZEPPELIN_JAVA_OPTS so that the EMR cluster is fully utilized when notebook is executed. (Requires that Spark has been installed with the -x flag when the EMR cluster was created, see https://aws.amazon.com/blogs/aws/new-apache-spark-on-amazon-emr/ for details.)

Best regards,
Anders