### What is this PR for?
Change embedded Spark dependency in binary package for the release from 1.5 to 1.6.
### What type of PR is it?
Improvement
### Todos
### Is there a relevant Jira issue?
### How should this be tested?
build package using dev/create_release.sh and run created package with default configuration. And see output of 'sc.version' if it is based on spark 1.6
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes#618 from Leemoonsoo/create_release.sh and squashes the following commits:
dc4e2a3 [Lee moon soo] change build profile for binary package release from -Pspark-1.5 to -Pspark-1.6
This PR address https://issues.apache.org/jira/browse/ZEPPELIN-484.
We've got discussions recently about impasse of pullrequest and speeding up of accepting pullrequest.
This small utility can be used when reviewer want to test the pullrequest.
Usage is
```
dev/test.py [#PR]
```
for example
```
dev/test.py 500
```
will create local branch `pr500` and add remote repository of contribution, fetch and merge it for the test.
This script will help reduce small bit of time of reviewing.
Author: Lee moon soo <moon@apache.org>
Closes#513 from Leemoonsoo/ZEPPELIN-484 and squashes the following commits:
76f0ddf [Lee moon soo] Add description
e8a0e8e [Lee moon soo] Rename
1adb356 [Lee moon soo] Handle non-ascii char
39d81ba [Lee moon soo] Add utility for pr test
#407 addressed https://issues.apache.org/jira/browse/ZEPPELIN-406 and merged into branch-0.5.5
This PR applies it to the master branch
Author: Lee moon soo <moon@apache.org>
Closes#424 from Leemoonsoo/ZEPPELIN-406-master and squashes the following commits:
64b6550 [Lee moon soo] Add apache header and inline json message example into doc
ba72645 [Lee moon soo] ZEPPELIN-406 Handle license issue found in 0.5.5-incubating rc2
https://issues.apache.org/jira/browse/ZEPPELIN-397
Activate spark-1.5 and hadoop-2.4 by default
That'll enable use sc.textFile("s3...") with default configuration without specify SPARK_HOME.
Author: Lee moon soo <moon@apache.org>
This patch had conflicts when merged, resolved by
Committer: Lee moon soo <moon@apache.org>
Closes#397 from Leemoonsoo/ZEPPELIN-397 and squashes the following commits:
9855dde [Lee moon soo] Activate spark-1.5 and hadoop-2.4 by default
https://issues.apache.org/jira/browse/ZEPPELIN-396
Add -Ppyspark to enable pyspark in default configuration (without SPARK_HOME specified).
Author: Lee moon soo <moon@apache.org>
Closes#396 from Leemoonsoo/ZEPPELIN-396 and squashes the following commits:
4e599af [Lee moon soo] Add -Pypspark profile for release binary
This PR addresses https://issues.apache.org/jira/browse/ZEPPELIN-366
This patch creates single binary package instead of multiple binary package with different version of Spark / Hadoop combinations.
Also it's removing redundant directory structure from it's tar archive.
Author: Lee moon soo <moon@apache.org>
Closes#373 from Leemoonsoo/ZEPPELIN-366 and squashes the following commits:
366bbf4 [Lee moon soo] Create a single binary package with default build flags. Remove redundant directory structure in binary package.
Zeppelin currently embeds all spark dependencies under interpreter/spark and loading them on runtime.
Which is useful because of user can try Zeppelin + Spark with local mode without installation and configuration of spark.
However, when user has existing spark and hadoop installation, it'll be really helpful to just pointing them instead of build zeppelin with specific version of spark and hadoop combination.
This PR implements ability to use external spark and hadoop installation, by doing
* spark-dependencies module packages spark/hadoop dependencies under interpreter/spark/dep, to support local mode (current behavior)
* When SPARK_HOME and HADOOP_HOME is defined, bin/interpreter.sh exclude interpreter/spark/dep from classpath and include system installed spark and hadoop into the classpath.
This patch makes Zeppelin binary independent from spark version. Once Zeppelin is been built, SPARK_HOME can point any version of spark.
Author: Lee moon soo <moon@apache.org>
Closes#244 from Leemoonsoo/spark_provided and squashes the following commits:
654c378 [Lee moon soo] use consistant, simpler expressions
57b3f96 [Lee moon soo] Add comment
eb4ec09 [Lee moon soo] fix reading spark-*.conf file
bacfd93 [Lee moon soo] Update readme
3a88c77 [Lee moon soo] Test use explicitly %spark
5a17d9c [Lee moon soo] Call sqlContext.sql using reflection
615c395 [Lee moon soo] get correct method
0c28561 [Lee moon soo] call listenerBus() using reflection
62b8c45 [Lee moon soo] Print all logs
5edb6fd [Lee moon soo] Use reflection to call addListener
af7a925 [Lee moon soo] add pyspark flag
5f8a734 [Lee moon soo] test -> package
a0150cf [Lee moon soo] not use travis-install for mvn test
cd4519c [Lee moon soo] try sys.stdout.write instead of print
6304180 [Lee moon soo] enable 1.2.x test
797c0e2 [Lee moon soo] enable 1.3.x test
8de7add [Lee moon soo] trying to find why travis is not closing the test
cf0a61e [Lee moon soo] rm -rf only interpreter directory instead of mvn clean
2606c04 [Lee moon soo] bringing travis-install.sh back
df8f0ba [Lee moon soo] test more efficiently
9d6b40f [Lee moon soo] Update .travis
2ca3d95 [Lee moon soo] set SPARK_HOME
2a61ecd [Lee moon soo] Clear interpreter directory on mvn clean
f1e8789 [Lee moon soo] update travis config
9e812e7 [Lee moon soo] Use reflection not to use import org.apache.spark.scheduler.Stage
c3d96c1 [Lee moon soo] Handle ZEPPELIN_CLASSPATH proper way
0f9598b [Lee moon soo] py4j version as a property
1b7f951 [Lee moon soo] Add dependency for compile and test
b1d62a5 [Lee moon soo] Add scala-library in test scope
c49be62 [Lee moon soo] Add hadoop jar and spark jar from HADOOP_HOME, SPARK_HOME when they are defined
2052aa3 [Lee moon soo] Load interpreter/spark/dep only when SPARK_HOME is undefined
54fdf0d [Lee moon soo] Separate spark-dependency into submodule
Author: caofangkun <caofangkun@gmail.com>
Closes#186 from caofangkun/zeppelin-207 and squashes the following commits:
a2d155a [caofangkun] ZEPPELIN-207: travis-ci build log is too long to be displayed
713ce5a [caofangkun] ZEPPELIN-207: travis-ci build log is too long to be displayed
3ee6a15 [caofangkun] ZEPPELIN-207: travis-ci build log is too long to be displayed
aa2568d [caofangkun] ZEPPELIN-207: travis-ci build log is too long to be displayed
e5b0067 [caofangkun] ZEPPELIN-207: travis-ci build log is too long to be displayed
https://issues.apache.org/jira/browse/ZEPPELIN-174
* [x] change suffix to sha512
* [x] change to sha512sum command verifiable format
Author: Lee moon soo <moon@apache.org>
Closes#161 from Leemoonsoo/ZEPPELIN-174 and squashes the following commits:
7ba4577 [Lee moon soo] Create sha512sum command verifiable format checksum
fee7757 [Lee moon soo] sha512 suffix for sha checksum
It'll be nice Zeppelin projects has tool that helps making releases. https://issues.apache.org/jira/browse/ZEPPELIN-130
* [x] Ability to create a source release.
* [x] Ability to create binary releases.
Author: Lee moon soo <moon@apache.org>
Closes#119 from Leemoonsoo/release_tool and squashes the following commits:
4c55022 [Lee moon soo] Make and sign bin packages
0a48cfc [Lee moon soo] create source release
Since apache/incubator-zeppelin is a mirror, we can not simply click merge button for pull request.
Here's pullrequest merge tool, i modified little bit for Zeppelin, based on Spark's one.
Author: Lee moon soo <moon@apache.org>
Closes#3 from Leemoonsoo/merge_tool and squashes the following commits:
918e85f [Lee moon soo] Change JIRA project name to ZEPPELIN
1a70af5 [Lee moon soo] Add pull request merge tool
