### What is this PR for?
git repo infra have moved from incubator-zeppelin to zeppelin
### What type of PR is it?
Hot Fix
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Mina Lee <minalee@apache.org>
Closes#983 from minahlee/ZEPPELIN-980 and squashes the following commits:
7542c31 [Mina Lee] remove travis url after travis migration is done
3793b38 [Mina Lee] Move git repository from incubator-zeppelin to zeppelin
### What is this PR for?
This removes hive module and adds example setting for using Hive in a JdbcInterpreter by using loading dynamic dependencies. It reduces Zeppelin's binary size.
There's no codes' modification except removing hive directory and remove the module from pom.xml
### What type of PR is it?
[Feature]
### Todos
* [x] - Remove hive module
* [x] - Add an example for using Hive in JDBC
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-925
### How should this be tested?
Set the interpreter properties and test it
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#943 from jongyoul/ZEPPELIN-925 and squashes the following commits:
eb50c78 [Jongyoul Lee] Fixed typo
45ec667 [Jongyoul Lee] Updated style of jdbc.md
f6b94d6 [Jongyoul Lee] Update hive.md
6aa9c7b [Jongyoul Lee] Revert "Remove hive.md from docs/interpreter"
f6573e3 [Jongyoul Lee] Deleted hive module from pom.xml
f720ed5 [Jongyoul Lee] Updated Documentation
9c58640 [Jongyoul Lee] Remove hive.md from docs/interpreter
cdf1cfc [Jongyoul Lee] Remove hive module and its directory
a38dc28 [Jongyoul Lee] Update example for hive in jdbc.md
### What is this PR for?
Remove incubating from doc
### What type of PR is it?
Documentation
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Mina Lee <minalee@nflabs.com>
Closes#941 from minahlee/doc/removeIncubating and squashes the following commits:
dbf37ef [Mina Lee] Remove redundant doap.rdf
bcb6b54 [Mina Lee] Remove incubating from doc
c7e34e7 [Mina Lee] Remove incubating from doc
accec53 [Mina Lee] Remove incubating from doc
### What is this PR for?
Currently available interpreter list is not shown in `Creating New Interpreter` section. It seems this bug was generated after #835 was merged. So I temporally deactivated [3 SerializedName code lines](6d7f1bc434).
### What type of PR is it?
Bug Fix
### Todos
* [x] - Fix interpreter listing bug when creating new interpreter
### What is the Jira issue?
[ZEPPELIN-931](https://issues.apache.org/jira/browse/ZEPPELIN-931)
### How should this be tested?
1. Build latest master branch and browse Zeppelin home
2. Create new interpreter -> You can not see the available interpreter list in this step like below attached screenshot
3. Apply this patch
4. Build again and browse -> You can see the available interpreter list as normal
### Screenshots (if appropriate)
- **Before**
<img width="1273" alt="screen shot 2016-06-01 at 12 36 42 pm" src="https://cloud.githubusercontent.com/assets/10060731/15723066/9082435e-27f5-11e6-9783-df44638dbbec.png">
- **After**
<img width="1273" alt="screen shot 2016-06-01 at 12 33 06 pm" src="https://cloud.githubusercontent.com/assets/10060731/15723067/92bcc8ce-27f5-11e6-82f5-6c0db7b4342c.png">
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Author: Jongyoul Lee <jongyoul@gmail.com>
Author: Ah young <fbdkdud93@nflabs.com>
Closes#945 from AhyoungRyu/ZEPPELIN-931 and squashes the following commits:
711eb54 [Ah young] Merge pull request #2 from jongyoul/ZEPPELIN-931
6121f9b [Jongyoul Lee] - Fixed documentation
6e7dac9 [Ah young] Merge pull request #1 from jongyoul/ZEPPELIN-931
fed1b40 [Jongyoul Lee] - Fixed fieldName in interpreter-setting.json
6d7f1bc [AhyoungRyu] ZEPPELIN-931: fix interpreter listing bug
### What is this PR for?
Added documentation for new contributors
### What type of PR is it?
[Documentation]
### Todos
* [x] - Upadated docs
### What is the Jira issue?
N/A
### How should this be tested?
N/A
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#934 from jongyoul/doc-added-link-for-beginner and squashes the following commits:
e9d1aee [Jongyoul Lee] [DOC] Added the link of Apache's Jira for beginner
### What is this PR for?
This PR enable Zeppelin server register Interpreters without any dependencies of their own. For instance, we should build `spark` with `spark-dependencies` even we use our own Spark cluster because current initialisation mechanism needs to all of its dependencies.
### What type of PR is it?
[Improvement]
### Todos
* [x] - Add RegisteredInterpreter from interpreter-setting.json in a jar or interpreter/{interpreter}/interpreter-setting.json
* [x] - Adjust it to Spark*Interpreter
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-804
### How should this be tested?
1. Prepare your own spark cluster - e.g. standalone, Yarn, Mesos -
1. rm -rf interpreter
1. mvn clean package -DskipTests -pl 'zeppelin-display,zeppelin-interpreter,zeppelin-server,zeppelin-web,zeppelin-zengine,angular,jdbc,spark'
1. bin/zeppelin-daemon.sh start
1. Check error in log
1. apply patch
1. mvn clean package -DskipTests -pl 'zeppelin-display,zeppelin-interpreter,zeppelin-server,zeppelin-web,zeppelin-zengine,angular,jdbc,spark'
1. bin/zeppelin-daemon.sh start
1. run some paragraph with simple command like `sc.version`
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
### Description
#### This PR introduce three initialisation mechanism including current one.
* {interpreter_dir}/{interpreter_group}/interpreter-setting.json
* interpreter-settings.json in your interpreter jar
* Current static initialization
#### Initialization step
1. Get {interpreter_dir} from Configuration
1. Get the list of {interpreter_dir}/[{interpreter_group1},{interpreter_group2}...]
1. Find {interpreter_dir}/{interpreter_group1}/interpreter-setting.json
1. Find interpreter-setting.json in the resources of {interpreter_dir}/{interpreter_group1}/**/*.jar
1. Adopt static init
1. Repeat them from the second step with {interpreter_group2}
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#835 from jongyoul/ZEPPELIN-804 and squashes the following commits:
823321e [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Added documentation
25bc501 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Ignored while registering a new interpreter with existing interpreter key
312dd77 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Reverted log4j setting
81ab361 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Changed logger setting only for test. This will be reverted after test
e8f990f [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed some unicode characters in interpreter-setting.json
3ad41bb [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Checked if path exists or not
1b3cd0c [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Checked if path exists or not
844dccb [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed logic to check for supporting legacy mechanism
c5b7d54 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Extracted new initialization logic into another methods
5d63d91 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Removed static initialisation of Spark*Interpreter
0d720c0 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Removed a interpreter which fails to initialize
d343636 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Excluded SparkRInterpreter from interpreter-setting.json
519f057 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Excluded interpreter-setting.json from rat check
00f55a8 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Excluded interpreter-setting.json from rat check
1fa2e52 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed test environments
8a90fe4 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed test environments
d54f98e [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Changed Spark*Interpreter to use interpreter-setting.json
48ac41d [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed the style
ca7b96c [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Added a new initialization mechanism to use interpreter-setting.json - Adjusted new mechanism to SparkInterpreter for verification
### What is this PR for?
Update resources to use Zeppelin TLP links
### What type of PR is it?
[Bug Fix]
### What is the Jira issue?
[ZEPPELIN-898](https://issues.apache.org/jira/browse/ZEPPELIN-898)
Author: Luciano Resende <lresende@apache.org>
Closes#927 from lresende/incubator-links and squashes the following commits:
8fca07c [Luciano Resende] [ZEPPELIN-898] Update resources to use Zeppelin TLP links
### What is this PR for?
1. Fix some typos in docs.
2. Remove trailing white spaces for each line.
3. Remove leading white spaces if a line contains no content.
4. Add trailing new line for each file.
### What type of PR is it?
Improvement | Documentation
### Todos
None
### What is the Jira issue?
N/A
### How should this be tested?
Build the doc site and check.
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? *no*
* Is there breaking changes for older versions? *no*
* Does this needs documentation? *no*
Author: Cheng-Yu Hsu <m@cyhsu.me>
Closes#852 from cyhsutw/fix-typos-in-docs and squashes the following commits:
6f5f46b [Cheng-Yu Hsu] fix typos in docs
### What is this PR for?
Currently, all notebooks that binded to the same interpreter setting shares the same interpreter instance.
For example, let's say there're `spark-production` interpreter setting and all notebooks are using it.
Then all notebooks share a single instance of SparkInterpreter, and that result all the variables are being shared across the notebook.
But sometimes isolated name space for variables per notebook is preferred. In this case we can create per notebook interpreter instance.
This PR provides generalized facility to provides per notebook interpreter instance, not only for SparkInterpreter but for any other existing interpreter without modification.
**Design**
It provides toggle switch that user turn on / turn off this mode per interpreter setting.
When per notebook interpreter is turned on, it still creates a single remote interpreter process per interpreter setting. But inside of the process, each notebook will have it's own interpreter instance and the instance is terminated automatically on removal of notebook or unbinding note from interpreter setting.
**SparkInterpreter**
Because of the design keeps single remote interpreter process and creates multiple interpreter instance inside of it, SparkInterpreter become a problem. While multiple SparkContext can not be created on a single process. So this PR try to, create per notebook scala compiler instance, but share single SparkContext instance. In this way, each notebook can run job concurrently by leveraging fairscheduler of SparkContext without running multiple Spark application (SparkContext).
And this approach is inspired by https://github.com/piyush-mukati/incubator-zeppelin/tree/parallel_scheduler_support_spark, this PR trying to generalize it for all other interpreters.
### What type of PR is it?
Feature
### Todos
* [x] - Per note interpreter session
* [x] - Scala compiler instance per note and share a single SparkContext (SparkInterpreter)
* [x] - Handle Spark class server correctly
* [x] - Handle SparkInterpreter Progressbar correctly
### Is there a relevant Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-513
### How should this be tested?
Check "Per note session" of your Spark interpreter setting.
And create two different notebook and check scala variables are not shared.
### Screenshots (if appropriate)
Checkbox to enable
How it works for SparkInterpreter
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
This patch had conflicts when merged, resolved by
Committer: Lee moon soo <moon@apache.org>
Closes#703 from Leemoonsoo/notebook_interpreter_session and squashes the following commits:
222066c [Lee moon soo] Use nanoTime
8287eb9 [Lee moon soo] Fix style
45af2df [Lee moon soo] Prevent infinitely waiting loop
438b212 [Lee moon soo] Fix typo on the comment
b091b7e [Lee moon soo] Move numReferenceOfSparkContext.incrementAndGet() to the end of open() method
4d2533f [Lee moon soo] Per note session -> Separate Interpreter for each note
7b073f6 [Lee moon soo] Merge branch 'master' into notebook_interpreter_session
feabb5f [Lee moon soo] Handle NPE when closing interpreters in group spark.
5047217 [Lee moon soo] Restore angularObject correctly
1f03ebb [Lee moon soo] Merge branch 'master' into notebook_interpreter_session
7730cf3 [Lee moon soo] Add document for per note session mode
dedf93f [Lee moon soo] Nullcheck
72b2235 [Lee moon soo] Protect SparkContext creation
f2299d6 [Lee moon soo] Handle scheduler termination correctly when unbind, bind interpreter
269c27d [Lee moon soo] Fix unittest
4299b87 [Lee moon soo] Share a SparkContext across ScalaCompilers
cc33c25 [Lee moon soo] Add more unittests
6dd981a [Lee moon soo] update interpreter module test
1d1638c [Lee moon soo] Fix test
58e5f91 [Lee moon soo] Update spark interpreter
366b651 [Lee moon soo] Don't destroy interpreter on note memove or change bindings when interpreter.option.perNoteSession is false
fc9cb3f [Lee moon soo] interpreter session aware interpreter factory
ed1ab0d [Lee moon soo] zeppelin-interpreter note session support
5787984 [Lee moon soo] Add option for per note session interpreter
### What is this PR for?
Lately, an official location of Zeppelin was changed from **https://github.com/apache/incubator-zeppelin.git** to **http://git.apache.org/incubator-zeppelin.git**. So I changed the old location in several documentations to the latest one.
### What type of PR is it?
Documentation
### Todos
### Is there a relevant Jira issue?
[ZEPPELIN-657](https://issues.apache.org/jira/browse/ZEPPELIN-657)
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Ryu Ah young <fbdkdud93@hanmail.net>
Closes#700 from AhyoungRyu/ZEPPELIN-657 and squashes the following commits:
2144bcc [Ryu Ah young] ZEPPELIN-657: Change 'release' -> 'branch' in howtocontribute.md
813b438 [Ryu Ah young] ZEPPELIN-657: Change 'release' -> 'branch'
23646e5 [Ryu Ah young] ZEPPELIN-657: Fix the wrong locations
c4bef6c [Ryu Ah young] ZEPPELIN-657: fix a typo in howtocontribute.md
a5fb130 [Ryu Ah young] ZEPPELIN-657: Change the official location of Zeppelin
### What is this PR for?
Instead of going through a list of items one by one in PR review, we should have a checklist of TODOs for a potential interpreter contributor to review and follow
### What type of PR is it?
Documentation
### Todos
* [x] - Update doc
### Is there a relevant Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-632
### How should this be tested?
N/A
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? This is.
Author: Felix Cheung <felixcheung_m@hotmail.com>
Closes#675 from felixcheung/createinterpreterdoc and squashes the following commits:
0feb8fe [Felix Cheung] add link
2a4aa4b [Felix Cheung] add link to pom src
dc47a58 [Felix Cheung] small wording update
78d9127 [Felix Cheung] doc update
### What is this PR for?
It would be better if Zeppelin provides looking good documentation.
### What type of PR is it?
Improvement
### Todos
* [x] - Fix typos, grammars and Increase readability writingzeppelininterpreter.md
* [x] - Also spark.md
* [x] - tutorial.md
* [x] - cassandra.md
* [x] - elasticsearch.md
* [x] - flink.md
* [x] - hive.md
* [x] - lens.md
* [x] - markdown.md
* [x] - rest-interpreter.md
### Is there a relevant Jira issue?
No : )
### How should this be tested?
These documentations are in [here](http://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/).
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Ryu Ah young <fbdkdud93@hanmail.net>
Closes#578 from AhyoungRyu/Fix-Typo and squashes the following commits:
925f30e [Ryu Ah young] Fix the list of 'Configure your Interpreter'
9b768e9 [Ryu Ah young] Change .md -> .html in interpreter.md
785dcbb [Ryu Ah young] Fix odd space in spark.md
a5d26d7 [Ryu Ah young] Fix version 1.5.1 -> 1.5.2 in spark.md
ef77027 [Ryu Ah young] Fix Upper case -> Lower case in cassandra.md
0918f79 [Ryu Ah young] Fix grammar error and Increase readability interpreters.md
f3d0173 [Ryu Ah young] Add numbering for dividing the table list and Fix grammar error rest-interpreter.md
fe4dfff [Ryu Ah young] Fix image size unit px -> % and Increase readability markdown.md
11b32db [Ryu Ah young] Remove useless 'to' in lens.md
3eec4cf [Ryu Ah young] Remove numbering flink.md
aac2e01 [Ryu Ah young] Add short description about Apache hive and Increase readability hive.md
d08092a [Ryu Ah young] Add highlight for scala code and Increase readability flink.md
8690e02 [Ryu Ah young] Increase readability and remove useless <hr> tag elasticsearch.md
9c008cf [Ryu Ah young] Increase readability and Remove useless <br>, <hr> tag cassandra.md
3041b95 [Ryu Ah young] Change rebuild -> rebuilding in spark.md
99b1e9e [Ryu Ah young] Remove unnecessary space in writingzeppelininterpreter.md
c63e59f [Ryu Ah young] Increase readability and delete useless <br> tag tutorial.md
e867df3 [Ryu Ah young] Increase readability and delete useless <br>, <hr> tag spark.md
f72b1cb [Ryu Ah young] Add .sh next to .bin/zeppein-daemon
466f82c [Ryu Ah young] Fix some typos and grammars in writingzeppelininterpreter.md
https://issues.apache.org/jira/browse/ZEPPELIN-448
Addresses update documentation for Spark configuration.
And BASE_PATH change after #430, #431 is deployed.
(currently deployed website has this change applied)
Author: Lee moon soo <moon@apache.org>
Closes#448 from Leemoonsoo/update_install and squashes the following commits:
54e2edf [Lee moon soo] add newline
cbd95d6 [Lee moon soo] Add description for included version of spark when SPARK_HOME is not set
00099ad [Lee moon soo] Add description for bumping up version
cf99212 [Lee moon soo] Fix typo
ab6d267 [Lee moon soo] Update BASE_PATH
73a737c [Lee moon soo] Remove instruction for 0.5.0 as zeppelin has documentation per version
4369f3e [Lee moon soo] Fix link
dcdaa74 [Lee moon soo] Add more configurations possible
8c9361d [Lee moon soo] Change font style
de74e6b [Lee moon soo] Update install and configuring spark
