Zeppelin currently embeds all spark dependencies under interpreter/spark and loading them on runtime.
Which is useful because of user can try Zeppelin + Spark with local mode without installation and configuration of spark.
However, when user has existing spark and hadoop installation, it'll be really helpful to just pointing them instead of build zeppelin with specific version of spark and hadoop combination.
This PR implements ability to use external spark and hadoop installation, by doing
* spark-dependencies module packages spark/hadoop dependencies under interpreter/spark/dep, to support local mode (current behavior)
* When SPARK_HOME and HADOOP_HOME is defined, bin/interpreter.sh exclude interpreter/spark/dep from classpath and include system installed spark and hadoop into the classpath.
This patch makes Zeppelin binary independent from spark version. Once Zeppelin is been built, SPARK_HOME can point any version of spark.
Author: Lee moon soo <moon@apache.org>
Closes#244 from Leemoonsoo/spark_provided and squashes the following commits:
654c378 [Lee moon soo] use consistant, simpler expressions
57b3f96 [Lee moon soo] Add comment
eb4ec09 [Lee moon soo] fix reading spark-*.conf file
bacfd93 [Lee moon soo] Update readme
3a88c77 [Lee moon soo] Test use explicitly %spark
5a17d9c [Lee moon soo] Call sqlContext.sql using reflection
615c395 [Lee moon soo] get correct method
0c28561 [Lee moon soo] call listenerBus() using reflection
62b8c45 [Lee moon soo] Print all logs
5edb6fd [Lee moon soo] Use reflection to call addListener
af7a925 [Lee moon soo] add pyspark flag
5f8a734 [Lee moon soo] test -> package
a0150cf [Lee moon soo] not use travis-install for mvn test
cd4519c [Lee moon soo] try sys.stdout.write instead of print
6304180 [Lee moon soo] enable 1.2.x test
797c0e2 [Lee moon soo] enable 1.3.x test
8de7add [Lee moon soo] trying to find why travis is not closing the test
cf0a61e [Lee moon soo] rm -rf only interpreter directory instead of mvn clean
2606c04 [Lee moon soo] bringing travis-install.sh back
df8f0ba [Lee moon soo] test more efficiently
9d6b40f [Lee moon soo] Update .travis
2ca3d95 [Lee moon soo] set SPARK_HOME
2a61ecd [Lee moon soo] Clear interpreter directory on mvn clean
f1e8789 [Lee moon soo] update travis config
9e812e7 [Lee moon soo] Use reflection not to use import org.apache.spark.scheduler.Stage
c3d96c1 [Lee moon soo] Handle ZEPPELIN_CLASSPATH proper way
0f9598b [Lee moon soo] py4j version as a property
1b7f951 [Lee moon soo] Add dependency for compile and test
b1d62a5 [Lee moon soo] Add scala-library in test scope
c49be62 [Lee moon soo] Add hadoop jar and spark jar from HADOOP_HOME, SPARK_HOME when they are defined
2052aa3 [Lee moon soo] Load interpreter/spark/dep only when SPARK_HOME is undefined
54fdf0d [Lee moon soo] Separate spark-dependency into submodule
5de01c6800