### What is this PR for?
The docs index must show the reference to the network visualization as for the the other types
### What type of PR is it?
[Improvement]
### Todos
* [x] - Add missing link
### What is the Jira issue?
[ZEPPELIN-3101](https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-3101)
### How should this be tested?
1. cd `docs/`
2. build: `bundle exec jekyll build --safe`
3. check the link is present
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Andrea Santurbano <santand@gmail.com>
Closes#2702 from conker84/graph-docs and squashes the following commits:
58d58cc4c [Andrea Santurbano] updated network label, added link to network display in index.md
### What is this PR for?
This PR is for the purpose of recover running interpreter process when zeppelin server is restarted. This would be useful when restarting zeppelin without interrupt current running interpreter processes, should be useful when admin do maintenance or upgrading.
Interface `RecoveryStorage` is used for storing the information of running interpreter process.
Currently it only has one implementation `FileSystemRecoveryStorage`, other implementation could be done later (such as zookeeper based). `InterpreterLauncher` is the component where to recover the running interpreter process.
Test:
* RecoveryTest.java
* FileSystemRecoveryStorageTest.java
Design Doc:
https://docs.google.com/document/d/1Plm3Hd40aGdNaXmjdsoY4ek3f-gTijTMGMkNjAZN39Y/edit?usp=sharing
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3051
### How should this be tested?
Unit test & Integration Test is added. Also manually verified.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2668 from zjffdu/ZEPPELIN-3051 and squashes the following commits:
a4c9b9c [Jeff Zhang] address comments
575b7b9 [Jeff Zhang] fix the pid of interpreter process id
02b118f [Jeff Zhang] address comments
da7cbb9 [Jeff Zhang] ZEPPELIN-3051. Support Interpreter Process Recovery
### What is this PR for?
Zeppelin's interpreter setting is shared by all the users and notes, if you want to have different setting you have to create new interpreter, e.g. you can create `spark_jar1` for running spark with dependency jar1 and `spark_jar2` for running spark with dependency jar2.
This approach works, but not so convenient. `ConfInterpreter` can provide more fine-grained control on interpreter setting and more flexibility.
`ConfInterpreter` is a generic interpreter that could be used by any interpreters. The input format should be property file format.
In the first paragraph, we use ConfInterpreter to make custom configuration of spark interpreter (set app name, yarn-client mode & add spark-csv dependencies). Then you can run the second paragraph which use spark-csv.
It can be used to make custom setting for any interpreter. `ConfInterpreter` would run in the zeppelin server side, it would update the interpreter properties before you launch the interpreter process, so it needs to run before interpreter process launched. And when interpreter process is launched is determined by interpreter mode setting. So users needs to understand the interpreter mode setting of zeppelin and be aware when interpreter process is launched. E.g. If we set spark interpreter setting as isolated per note. Under this setting, each note will launch one interpreter process. In this scenario, user need to put `ConfInterpreter` as the first paragraph as the above example. Otherwise the customized setting can not be applied (The paragraph using ConfInterpreter will report ERROR).
### What type of PR is it?
[Feature | Documentation]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3085
### How should this be tested?
* Unit test, System test is added, also manually verified it.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2692 from zjffdu/ZEPPELIN-3085 and squashes the following commits:
87ce20f [Jeff Zhang] ZEPPELIN-3085. Introduce generic ConfInterpreter for more fine-grained control of interpreter setting
Add `apt-get install` lines for R dependencies that are needed to build
successfully.
### What is this PR for?
This pull request adds installation instructions for building which relate to missing dependencies resulting in failing to build (if using R).
### What type of PR is it?
[Documentation]
Author: Jens Grassel <jens@wegtam.com>
Closes#2674 from jan0sch/docs-missing-build-deps and squashes the following commits:
aa581c1 [Jens Grassel] Add missing build dependencies.
### What is this PR for?
Added dynamic forms for Note. All paragraphs has access to note forms.
### What type of PR is it?
[Feature]
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1363
### How should this be tested?
Create global interpreter JDBC and Spark. Create note with 2 paragraphs
```
%pyspark
print("Textbox paragraph " + z.textbox('input', 'default'))
print("Textbox note " + z.noteTextbox('note_input', 'default_note'))
print("Select paragraph " + z.select("sel", [("1","opt1"), ("2","opt2"), ("3","opt3")]))
print("Select note " + z.noteSelect("sel_note", [("1","noteOpt1"), ("2","noteOpt2"), ("3","noteOpt3")]))
options = [("key1","Name1"), ("key2","Name2")]
print("Checkbox paragraph "+ " and ".join(z.checkbox("chk", options, ["key1"])))
print("Checkbox note "+ " and ".join(z.noteCheckbox("chk_note", options, ["key1","key2"])))
```
```
%jdbc
select '$${checkbox:chk_note=key1|key2,key1|key2} $${note_input} ${note_input=sameName}'
```
### Screenshots (if appropriate)
1) native forms
2) remove
3) simple forms
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: tinkoff-dwh <tinkoff.dwh@gmail.com>
Closes#2641 from tinkoff-dwh/ZEPPELIN-1363 and squashes the following commits:
3ee4826 [tinkoff-dwh] [ZEPPELIN-1363] autosave textbox
f30033a [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-1363
29eaca2 [tinkoff-dwh] [ZEPPELIN-1363] fix tests
bf8194e [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-1363
9b2f3e9 [tinkoff-dwh] [ZEPPELIN-1363] button to remove form
c566462 [tinkoff-dwh] [ZEPPELIN-1363] note dynamic forms (simple; native: spark, python)
### What is this PR for?
Change `docs/assets/themes/zeppelin/img/available_interpreters.png` file.
replace old `elasticsearch` logo to new one.
### What type of PR is it?
Documentation
### Todos
* [x] - replace old `elasticsearch` logo to new one on `available_interpreters.png` file.
### What is the Jira issue?
N/A
### How should this be tested?
N/A
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? N.
* Is there breaking changes for older versions? N.
* Does this needs documentation? N.
Author: Jongmin Kim <kimjmin@gmail.com>
Closes#2669 from kimjmin/master and squashes the following commits:
349194d9 [Jongmin Kim] available_interpreters.png file change.
b5e53410 [Jongmin Kim] Revert "available_interpreters.png file change."
61341759 [Jongmin Kim] available_interpreters.png file change.
### What is this PR for?
Sometimes need to see the difference between versions and to switch to another version and look for changes are not convenient (the page reloaded). This feature allows you to compare any two versions of the notebook.
### What type of PR is it?
Feature
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2813
### How should this be tested?
1 make some commits. Сchange the contents of paragraphs (delete, add, edit)
2 open Revisions comparator
3 compare revisions and check diff
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? yes (updated)
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: tinkoff-dwh <tinkoff.dwh@gmail.com>
Author: Tinkoff DWH <tinkoff.dwh@gmail.com>
Closes#2506 from tinkoff-dwh/ZEPPELIN-2813 and squashes the following commits:
acc624f [tinkoff-dwh] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2813
2fd89a8 [tinkoff-dwh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2813
8b8afcc [tinkoff-dwh] Docs edit
efa7ce2 [tinkoff-dwh] Merge branch 'master' into ZEPPELIN-2813
f530524 [tinkoff-dwh] zep-2813 anim off
0e866b2 [tinkoff-dwh] zep_2813 color change
310760e [tinkoff-dwh] zep_2813 UI for REVISIONS COMPARATOR.
3d4f86c [tinkoff-dwh] Merge branch 'master' into ZEPPELIN-2813_refactoring_2
dc67f8f [tinkoff-dwh] [ZEPPELIN-2813] refactoring
514b3f5 [tinkoff-dwh] small fixes, added documentation
4ce5286 [tinkoff-dwh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2813
b949814 [Tinkoff DWH] [ZEPPELIN-2813] license
a192b95 [Tinkoff DWH] [ZEPPELIN-2813] revisions comparator for note
### What is this PR for?
Documentation for LifecycleManager
### What type of PR is it?
[ Documentation ]
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2644 from zjffdu/Doc_Lifecycle and squashes the following commits:
5aa0c16 [Jeff Zhang] [Doc] Doc for LifecycleManager
### What is this PR for?
Simple PR to only apply dynamic forms for LivySqlInterpreter, this make the behavior consistent with zeppelin's built-in spark interpreter.
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3034
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2651 from zjffdu/ZEPPELIN-3034 and squashes the following commits:
39eff37 [Jeff Zhang] ZEPPELIN-3034. Only apply dynamic form for LivySqlnterpreter
### What is this PR for?
Allow an override of default Spark logic in building the URL path and redirect the Zeppelin user directly to a configured reverse proxy path
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2949
### How should this be tested?
* Set "zeppelin.spark.uiWebUrl" to http://localhost/spark-reverse-proxy-path
* Run Spark code
* Open Spark UI
* Expected to be redirected to http://localhost/spark-reverse-proxy-path
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? N
* Is there breaking changes for older versions? N
* Does this needs documentation? N
Author: Nelson Costa <nelson.costa85@gmail.com>
Closes#2596 from necosta/zeppelin2949 and squashes the following commits:
2ae8f2d [Nelson Costa] [ZEPPELIN-2949] Last amendments
118339f [Nelson Costa] [ZEPPELIN-2949] Updated documentation
826bad4 [Nelson Costa] [ZEPPELIN-2949] Updated documentation
0a56299 [Nelson Costa] [ZEPPELIN-2949] Allow custom Spark UI URL
### What is this PR for?
This PR will add tab as auto complete invoker if paragraph is non-md and user has not pressed the tab as a first character in the line
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-277
* https://issues.apache.org/jira/browse/ZEPPELIN-2736
### How should this be tested?
- Build: mvn clean package -Denforcer.skip -DskipTests -Drat.skip
- Open a paragraph
- Press tab with following options: first character, after space
### Questions:
* Does the licenses files need an update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Malay Majithia <malay.majithia@gmail.com>
Author: Lee moon soo <moon@apache.org>
Closes#2542 from malayhm/ZEPPELIN-277 and squashes the following commits:
436f22d [Malay Majithia] Added Tab auto completion flag for python sql and r
b37e084 [Malay Majithia] Fixed lint error
18fc814 [Malay Majithia] Merge branch 'master' into ZEPPELIN-277
b09730e [Malay Majithia] Merge branch 'master' into ZEPPELIN-277
63d69e1 [Malay Majithia] Merge branch 'ZEPPELIN-277' of github.com:malayhm/zeppelin into ZEPPELIN-277
a75f0fe [Malay Majithia] Improved the first character check logic
2ec879d [Malay Majithia] Merge pull request #1 from Leemoonsoo/ZEPPELIN-277-completion-key
77afdba [Lee moon soo] fix style
77b47b6 [Malay Majithia] If all the previous line characters are tab, don't show autocomplete on tab
46f612a [Malay Majithia] ZEPPELIN-277 Add Tab as Autocomplete for Notebook non-md paragraphs
865c0a6 [Lee moon soo] Set python and spark interpreter completionKey
05d5860 [Lee moon soo] Update doc
973068b [Lee moon soo] apply tab completion based on editor.completionKey
5f4d81c [Malay Majithia] If all the previous line characters are tab, don't show autocomplete on tab
655ba88 [Malay Majithia] ZEPPELIN-277 Add Tab as Autocomplete for Notebook non-md paragraphs
### What is this PR for?
Make Zeppelin support Ceph as a notebook storage.
Ceph has APIs which are compatible with AWS S3 APIs. However, it supports only AWS Signature V2 and GetObject requests of aws-sdk-java use V4 by default: https://github.com/aws/aws-sdk-java/issues/372
According to https://github.com/aws/aws-sdk-java/issues/372#issuecomment-137299691 , the Zeppelin configuration of `zeppelin.notebook.s3.signerOverride` is added to make the `signerOverride` field of a `ClientConfiguration` instance configurable.
### What type of PR is it?
[Improvement]
### Todos
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2950
### How should this be tested?
Tested manually.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Keiji Yoshida <kjmrknsn@gmail.com>
Closes#2598 from kjmrknsn/feature/ceph-support and squashes the following commits:
c66cb63 [Keiji Yoshida] Support Ceph as a notebook storage
### What is this PR for?
Follow up of #2577. Main changes on Interpreter
* Add throw `InterpreterException` which is checked exception for the abstract methods of `Interpreter`, this would enforce the interpreter implementation to throw `InterpreterException`.
* field name refactoring.
* `property` -> `properties`
* `getProperty()` --> `getProperties()`
* Introduce launcher layer for interpreter launching. Currently we only use shell script to launch interpreter, but it could be any other service or component to launch interpreter, such as livy server , other 3rd party tools or even we may create a separate module for interpreter launcher
* abstract cass `InterpreterLauncher`
* For now, only 2 implementation: `ShellScriptLauncher` & `SparkInterpreterLauncher`. We could add method in class `Interpreter` to allow interpreter to specify its own launcher class, but it could be future work.
### What type of PR is it?
[Improvement | Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2685
### How should this be tested?
Unit test is covered. `ShellScriptLauncherTest` & `SparkInterpreterLauncherTest`
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2592 from zjffdu/ZEPPELIN-2685 and squashes the following commits:
17dc2f1 [Jeff Zhang] address comments
e545cc3 [Jeff Zhang] ZEPPELIN-2685. Improvement on Interpreter class
### What is this PR for?
2 main changes.
* Introduce `interpreter-parent` module, so that all the interpreter can reuse the plugin defined in `interpreter-parent`
* Add new plugin for `copying interpreter-setting.json` to interpreter dir
### What type of PR is it?
[ Improvement | Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2982
### How should this be tested?
* First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2613 from zjffdu/ZEPPELIN-2982 and squashes the following commits:
ccccd32 [Jeff Zhang] update doc
8f643a7 [Jeff Zhang] ZEPPELIN-2982. Copy interpreter-setting.json to interpreter dir
### What is this PR for?
This PR is trying to allow user to add custom http headers when calling livy rest api. User just need to specify `zeppelin.livy.http.headers` in livy interpreter setting
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2953
### How should this be tested?
Outline the steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2600 from zjffdu/ZEPPELIN-2953 and squashes the following commits:
720d8d0 [Jeff Zhang] ZEPPELIN-2953 Allow custom http header for livy interpreter
### What is this PR for?
Trivial change for configuration name change.
### What type of PR is it?
[Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* No jira created
### How should this be tested?
* First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2620 from zjffdu/minor_refactoring and squashes the following commits:
c68ce76 [Jeff Zhang] [MINOR][Code Refactoring] Configuration name change for keytab and principal used for FileSystemNotebookRepo
### What is this PR for?
This contribution would to introduce Neo4j Cypher intepreter and at the same time provides base APIs that enable other graph databases (or graph framworks such as GraphX or Giraph).
### What type of PR is it?
[Feature]
### Todos
* [x] - Provide base APIs to manage graph results (under: org.apache.zeppelin.interpreter.graph.*)
* [x] - Create the Neo4j intepreter
### What is the Jira issue?
[[ZEPPELIN-2761]](https://issues.apache.org/jira/browse/ZEPPELIN-2761)
### How should this be tested?
Donwload and execute [Neo4j](https://neo4j.com/download/?ref=home) v3.x, you can also pull a [Docker image](https://neo4j.com/developer/docker/).
In order to execute test cases, if you are running Java 7, you need to also provide an environment variable telling the tests where to find Java 8, because Neo4j-the-database needs it to run.
```bash
export NEO4J_JAVA=<path/to/java/home>
```
Use this statement to create a dummy dataset
```bash
%neo4j
UNWIND range(1,100) as id
CREATE (p:Person {id:id, name: "Name " + id, age: id % 3}) WITH collect(p) as people
UNWIND people as p1
UNWIND range(1,10) as friend
WITH p1, people[(p1.id + friend) % size(people)] as p2
CREATE (p1)-[:KNOWS {years: abs(p2.id - p2.id)}]->(p2)
```
Then you can write some simple queries like:
```bash
%neo4j
MATCH (p:Person)-[r:KNOWS]-(p1:Person)
RETURN p, r, p1
LIMIT 10;
```
```bash
%neo4j
MATCH (p:Person)-[r:KNOWS]-(p1:Person)
RETURN p.id AS ID_A, p.name AS NAME_A, r.years AS YEARS, p1.id AS ID_B, p1.name AS NAME_B
LIMIT 20;
```
### Video
### Questions:
* Does the licenses files need update? **Yes**
* Is there breaking changes for older versions? **No**
* Does this needs documentation? **Yes**
Author: conker84 <santand@gmail.com>
Closes#2478 from conker84/master and squashes the following commits:
5fb97ce [conker84] review by felixcheung 18/09
1e3d611 [conker84] Review by felixcheung (09/08)
aaf618e [conker84] Review by 1ambda 26/07
b5c2c66 [conker84] Added docs
9eb568d [conker84] Final behavior after discussion
2f88e98 [conker84] Moved business logic to the connection manager
8e4690e [conker84] Create connection manager class
35b4e29 [conker84] First commit
### What is this PR for?
Support updating paragraph text/title via REST API
https://issues.apache.org/jira/browse/ZEPPELIN-1897
### What type of PR is it?
Improvement
### Todos
### What is the Jira issue?
[ZEPPELIN-1897]
### How should this be tested?
1. Create a note and paragraph
2. Hit REST API to update the paragraph, e.g. `curl -X PUT http://localhost:8080/api/notebook/NOTE_ID/paragraph/PARAGRAPH_ID -d '{"text": "some updated text"}'`
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, doc update included in the change.
Author: chilang <chilang@gmail.com>
Closes#2604 from chilang/master and squashes the following commits:
9cc86de [chilang] ZEPPELIN-1897 REST API - Update paragraph
### What is this PR for?
Support encrypting passwords using a private key
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2952
### How should this be tested?
- Set the env variable `ZEPPELIN_CREDENTIALS_ENCRYPT_KEY=something`
- Save a few credentials
- Check that the `credentials.json` file is storing encrypted passwords
- Restart server using the same env variable for `ZEPPELIN_CREDENTIALS_ENCRYPT_KEY`
- The credentials should still be decryptable
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
Yes
Author: Herval Freire <hfreire@twitter.com>
Closes#2599 from herval/encrypt-credentials and squashes the following commits:
e5857d8 [Herval Freire] missing license
1d4bc04 [Herval Freire] documentation....?
82ae8f4 [Herval Freire] added license
c3e0ead [Herval Freire] encrypt credentials.json with AES
### What is this PR for?
This is the first version for supporting yarn-cluster of `SparkInterpreter`. I just delegate all the function to `spark-submit` as yarn-cluster is natively supported by spark, we don't need to reinvent the wheel. But there's still improvement to be done in future, e.g. I put some spark specific logic in `InterpreterSetting` which is not a good practise. I plan to improve it when I refactor the `Interpreter` class (ZEPPELIN-2685).
Besides that, I also add `MiniHadoopCluster` & `MiniZeppelin` which help for the integration test of yarn-client & yarn-cluster mode, otherwise I have to manually verify yarn-client & yarn-cluster mode which would easily cause regression issue in future.
To be noticed:
* SPARK_HOME must be specified for yarn-cluster mode
* HADOOP_CONF_DIR must be specified for yarn-cluster mode
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
https://github.com/zjffdu/zeppelin/tree/ZEPPELIN-2898
### How should this be tested?
System test is added in `SparkInterpreterIT`.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2577 from zjffdu/ZEPPELIN-2898 and squashes the following commits:
9da7c4b [Jeff Zhang] ZEPPELIN-2898. Support Yarn-Cluster for Spark Interpreter
### What is this PR for?
This is a refactoring PR of ZEPPELIN-1515. Because hadoop's FileSystem API not only works with hdfs, but also other hadoop compatible filesystem. So in this PR I rename it to `FileSystemNotebookRepo`
### What type of PR is it?
[Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2933
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2588 from zjffdu/ZEPPELIN-2933 and squashes the following commits:
45d1e9b [Jeff Zhang] ZEPPELIN-2993. Code Refactoring of ZEPPELIN-1515 follow up
### What is this PR for?
Straightforward change for allowing pass non spark.* properties to spark app driver. See discussion here http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Setting-spark-config-properties-in-Zeppelin-0-7-2-td6143.html
### What type of PR is it?
[Bug Fix ]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2893
### How should this be tested?
Unit test is added
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2561 from zjffdu/ZEPPELIN-2893 and squashes the following commits:
1342527 [Jeff Zhang] ZEPPELIN-2893. Pass non spark.* of spark interpreter properties to spark app driver
### What is this PR for?
With ZEPPELIN-2841, it had changed the default working directory of Shell Interpreter from the relative path where Zeppelin is running to user-home. This is to make the configuration optional.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2903
### How should this be tested?
by default when the user runs shell interpreter and executes `pwd` will the path where Zeppelin server is running, but when `shell.working.directory.user.home` is set to true in Zeppelin's interpreter setting, it will point to the user's home directory by which the interpreter is running.
Author: Prabhjyot Singh <prabhjyotsingh@gmail.com>
Closes#2566 from prabhjyotsingh/ZEPPELIN-2903 and squashes the following commits:
9934df52a [Prabhjyot Singh] add doc for shell.working.directory.user.home
9164ed2c0 [Prabhjyot Singh] Make setting of working directory to user-home optional for shell interpreter
### What is this PR for?
The idea of this PR is to provide a new kind of user : Runner.
Basically, what it does is that it just removes write authorization and allow user to read and run note.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
[ZEPPELIN-2848] https://issues.apache.org/jira/browse/ZEPPELIN-2848
### How should this be tested?
- Log in as admin
- Create new notebook and create a paragraph with the interpreter you want
- Assign runner right to user1
- Log in as user1
- Try to run the paragraph (should work)
- Try to modify the paragraph (should fail)
- Log in as user2
- Try to run the paragraph (should fail)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? Yes
* Does this needs documentation? No
Author: Paolo Genissel <paolo.genissel-monsallier@1000mercis.com>
Author: gfalcone <paologenissel@gmail.com>
Author: Paolo Genissel <paologenissel@gmail.com>
Closes#2526 from gfalcone/new_type_runner and squashes the following commits:
96bba66 [gfalcone] Fix typo on notebook_authorization.md
8ab4512 [gfalcone] Update notebook_authorization.md
22a1eb3 [Paolo Genissel] Fixed typo
d621792 [Paolo Genissel] Fix NotebookSecurityRestApiTest
a67af0f [Paolo Genissel] Fix test
5c43ca9 [Paolo Genissel] Added new type of user
### What is this PR for?
This is the first step for implement IPython Interpreter in Zeppelin. I just use the jupyter_client to create and manage the ipython kernel. We don't need to care about python compilation and execution, all the things are delegated to ipython kernel. Ideally all the features of ipython should be available in Zeppelin as well.
For now, user can use %python.ipython for IPython Interpreter. And if ipython is available, the default python interpreter will use ipython. But user can still set `zeppelin.python.useIPython` as false to enforce to use the old implementation of python interpreter.
Main features:
* IPython interpreter support
** All the ipython features are available, including visualization, ipython magics.
* ZeppelinContext support
* Streaming output support
* Support Ipython in PySpark
Regarding the visualization, ideally all the visualization libraries work in jupyter should also work here.
In unit test, I only verify the following 3 popular visualization library. could add more later.
* matplotlib
* bokeh
* ggplot
### What type of PR is it?
[Feature ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2753
### How should this be tested?
Unit test is added.
### Screenshots (if appropriate)
Verify bokeh in IPython Interpreter
Verify matplotlib
Verify ZeppelinContext
Verify Streaming
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2474 from zjffdu/ZEPPELIN-2753 and squashes the following commits:
e869f31 [Jeff Zhang] address comments
b0b5c95 [Jeff Zhang] [ZEPPELIN-2753] Basic Implementation of IPython Interpreter
### What is this PR for?
Adding a section in Shiro Authentication about how to enable secure cookie via Shiro. Shiro do support configuring 'HttpOnly' flag in response cookie. A Zeppelin user, who is security conscious, should know how to enable this in Zeppelin's Shiro configuration.
### What type of PR is it?
Documentation
### What is the Jira issue?
ZEPPELIN-2873
### How should this be tested?
Doc changes. CI test should pass.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Vipin Rathor <v.rathor@gmail.com>
Closes#2545 from VipinRathor/ZEPPELIN-2873 and squashes the following commits:
c7b7995 [Vipin Rathor] Updated doc as per the review comments.
fec8d7e [Vipin Rathor] ZEPPELIN-2873 - Add documentation on Zeppelin Shiro's abliity to configure secure cookie
### What is this PR for?
This PR is trying to add hdfs as another implementation for `NotebookRepo`. There's another PR about using webhdfs to implement that. Actually hdfs client library is compatibility cross major versions. See http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_compatibility, if using webhdfs, the code become more complicated and may lose some features of hdfs.
This PR is also required for HA of zeppelin, so that multiple zeppelin instances can share notes via hdfs. I add hadoop-client in pom file. So zeppelin will package hadoop client jar into its binary distribution. This is because zeppelin may be installed in a gateway machine where no hadoop is installed (only hadoop configuration file is existed in this machine) And since the hadoop client will work with multiple versions of hadoop, so it is fine to package into binary distribution. Spark also package hadoop client jar in its binary distribution.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1515
### How should this be tested?
Unit test is added. Also manually verify it in a single node cluster.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2455 from zjffdu/ZEPPELIN-1515 and squashes the following commits:
b3e83ab [Jeff Zhang] ZEPPELIN-1515. Notebook: HDFS as a backend storage (Read & Write Mode)
### What is this PR for?
upgrade Beam interpreter to use the latest version of Apache Beam.
### What type of PR is it?
[Improvement]
### Todos
*
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2865
### How should this be tested?
* Start the Zeppelin server
* The prefix of interpreter is %beam and then write your code with required imports and the runner
Refer to `docs/interpreter/beam.md` for an example;
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes, updated `docs/interpreter/beam.md` and `README.md`
Author: mingmxu <mingmxu@ebay.com>
Closes#2541 from XuMingmin/ZEPPELIN-2865 and squashes the following commits:
520f0fd7 [mingmxu] restore the notice message of scala-2.10
93b3e24d [mingmxu] upgrade to Apache Beam 2.0.0
### What is this PR for?
Current screenshot images for "Settings" and "About Zeppelin" is outdated since UI is updated. So it'll be better update them with latest one.
### What type of PR is it?
Documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2856
### How should this be tested?
Just seeing file changes will be enough :)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: EomJeongyeon <lune6482@naver.com>
Closes#2537 from EomJeongyeon/ZEPPELIN-2856 and squashes the following commits:
153a891 [EomJeongyeon] update screenshot images with new about_menu.png and settings_menu.png
### What is this PR for?
To change the order of content in index documentation. Now the last two contents is "External Resources" and "Available Interpreters". This PR change the order to "Available Interpreters" and "External Resources".
### What type of PR is it?
[Documentation]
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2853
### How should this be tested?
1. Read the doc
### Screenshots (if appropriate)
Before
<img width="726" alt="screen shot 2017-08-15 at 3 08 14 pm" src="https://user-images.githubusercontent.com/19610222/29304040-d9f9adac-81cb-11e7-9aa3-0a325511e267.png">
After
<img width="615" alt="screen shot 2017-08-15 at 3 04 44 pm" src="https://user-images.githubusercontent.com/19610222/29304001-a26fca6a-81cb-11e7-8cf2-be10eef781d9.png">
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Loun Lee <beneficial02@gmail.com>
Closes#2530 from beneficial02/feature/change_doc_order and squashes the following commits:
10a7172 [Loun Lee] feature: change the order of doc content
### What is this PR for?
At compile time Zeppelin is downloading several external resources.
I want to be able to provide alternative URLs to compile when internet is not available(i.e. behind corporate proxy).
### What type of PR is it?
[Improvement]
### What is the Jira issue?
[ZEPPELIN-2766]
### How should this be tested?
```
mvn -DskipTests -Dplugin.frontend.downloadRoot=<alernative-url> -Dplugin.frontend.nodeDownloadRoot=<alernative-url> -Dplugin.frontend.yarnDownloadRoot=<alernative-url> -Dplugin.frontend.npmDownloadRoot=<alernative-url> -Dpypi.repo.url=<alernative-url> clean package
```
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: andrea <andrea.peruffo1982@gmail.com>
Closes#2483 from andreaTP/softCodeRepos and squashes the following commits:
6894bcd5 [andrea] added docs for additional configurations
647417c3 [andrea] fix typo
1e5fd85f [andrea] remove deprecated downloadRoot
0aea864a [andrea] Keep online resources configurable
### What is this PR for?
Complementing Zeppelin-1501 (support for Cassandra with SSL in interpreter) with documentation updates.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2835
* [Zeppelin-2835]
### How should this be tested?
Check that documentation for cassandra interpreter includes cassandra.ssl options.
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
No
Author: Alex Lourie <alex@instaclustr.com>
Author: Alex Lourie <djay.il@gmail.com>
Closes#2517 from alourie/Zeppelin-2835-addSSL-doco and squashes the following commits:
16719766 [Alex Lourie] Updates
9058ee45 [Alex Lourie] Fix the highlight
02625c94 [Alex Lourie] Adding documentation for Cassandra interpreter SSL
### What is this PR for?
This PR adds support for such a configuration which can give access to user who belongs to "any of" the roles defined in Shiro configuration. By default, as per Shiro implementation, user is allowed only when he/she belongs to "all" the roles defined.
This PR fixes the problem for static users/roles in Shiro as well as Active Directory and/or LDAP based user-group-roles mapping.
### What type of PR is it?
Improvement
### TODO
* [x] - Add documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2825
### How should this be tested?
Add the following in shiro.ini:
[main]
...
anyofroles = org.apache.zeppelin.utils.AnyOfRolesAuthorizationFilter
[urls]
...
/api/interpreter/** = authc, **anyofroles**[admin, role1]
/api/configurations/** = authc, roles[admin]
/api/credential/** = authc, roles[admin]
### Screenshots (if appropriate)
Not applicable
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Vipin Rathor <v.rathor@gmail.com>
Closes#2515 from VipinRathor/ZEPPELIN-2825 and squashes the following commits:
01deb25b0 [Vipin Rathor] ZEPPELIN-2825 - Added license header
2105810f7 [Vipin Rathor] ZEPPELIN-2825 - Fix formatting for doc changes
95a9b4f41 [Vipin Rathor] ZEPPELIN-2825 - Fix formatting in doc changes
3f49d8413 [Vipin Rathor] ZEPPELIN-2825 - Add documentation for supporting any of the Shiro roles
c5fc9deac [Vipin Rathor] ZEPPELIN-2825 - Fix Zeppelin to support any of the Shiro roles
### What is this PR for?
The HTTP Strict-Transport-Security response header (often abbreviated as HSTS) is a security feature that lets a web site tell browsers that it should only be communicated with using HTTPS, instead of using HTTP.
Note: The Strict-Transport-Security header is ignored by the browser when your site is accessed using HTTP; this is because an attacker may intercept HTTP connections and inject the header or remove it. When your site is accessed over HTTPS with no certificate errors, the browser knows your site is HTTPS capable and will honor the Strict-Transport-Security header.
The HTTP X-XSS-Protection response header is a feature of Internet Explorer, Chrome and Safari that stops pages from loading when they detect reflected cross-site scripting (XSS) attacks.
### What type of PR is it?
[Bug Fix | Improvement ]
### What is the Jira issue?
* [ZEPPELIN-2775](https://issues.apache.org/jira/browse/ZEPPELIN-2775)
### How should this be tested?
Make a curl call to Zeppelin? Go to Chrome Browser and select "More Tools" -> "Developer Tools" from the right-side menu. Under Network Section, select any request and check for "Response Headers". You should see below headers along with existing ones.
> strict-transport-security:max-age=631138519
> x-xss-protection:1; mode=block
<img width="1436" alt="screen shot 2017-07-14 at 8 19 14 pm" src="https://user-images.githubusercontent.com/6433184/28217231-16ce6cee-68d2-11e7-91aa-77ad083612c7.png">
### Questions:
* Does this needs documentation?
Author: krishna-pandey <krish.pandey21@gmail.com>
Closes#2492 from krishna-pandey/ZEPPELIN-2775 and squashes the following commits:
7d9978e49 [krishna-pandey] Modified Documentation as per review.
6733289ed [krishna-pandey] Adding documentation for HTTP Security Headers
754d2d71e [krishna-pandey] Supplying String instead of Int (required for Response Header)
468231cc6 [krishna-pandey] Added configurable Strict-Transport-Security and X-XSS-Protection Headers
The github preview is fine but the html on https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/security/shiroauthentication.html#ldap is not displaying the code fragment correctly.
### What is this PR for?
Fixes the documentation orientation for shiro LDAPRealm
### What type of PR is it?
[Documentation]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2804
### How should this be tested?
Build the documentation and verify that the problem as stated in image is resolved.
### Screenshots (if appropriate)
<img width="856" alt="screen shot 2017-07-21 at 10 08 04 am" src="https://user-images.githubusercontent.com/5103613/28454775-c91fbe1e-6dfc-11e7-9601-6279be44f124.png">
### Questions:
* Does the licenses files need update? N
* Is there breaking changes for older versions? N
* Does this needs documentation? N
Author: Sohaib Iftikhar <sohaib1692@gmail.com>
Closes#2500 from sohaibiftikhar/patch-1 and squashes the following commits:
94e18433 [Sohaib Iftikhar] Fixed documentation for shiro LDAPRealm
### What is this PR for?
Fixing bug on enabling Helium plugins (artifact is empty, not null) when sending curl command.
After noticing out-of-date documentation, I updated it
Also noticed issues on missing try/catch expressions and missing info messages on enable/disable packages.
### What type of PR is it?
[Bug Fix | Documentation | Refactoring]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2699
### How should this be tested?
Run "curl -X POST http://localhost:8080/api/helium/enable/{packageName}"
Check that the package gets enabled
### Questions:
* Does the licenses files need update? N
* Is there breaking changes for older versions? N
* Does this needs documentation? Y (Done)
Author: Nelson Costa <nelson.costa85@gmail.com>
Closes#2446 from necosta/zeppelin2699 and squashes the following commits:
473f9b43 [Nelson Costa] [ZEPPELIN-2699] Fixed initialization issue
45b77513 [Nelson Costa] [ZEPPELIN-2699] More corrections
c8230218 [Nelson Costa] [ZEPPELIN-2699] Bug-fix on gathering package info
2b1ff8ae [Nelson Costa] [ZEPPELIN-2699] Changes after code review
a71b9fb9 [Nelson Costa] [ZEPPELIN-2699] Changes after code-review
5d179218 [Nelson Costa] [ZEPPELIN-2699] Helium REST API bug-fixes. Also updated documentation
I was unaware of this really nice feature, which led me to writing my own "zeppelin_show" function.
### What is this PR for?
(tiny) improvement of the docs
Author: TwUxTLi51Nus <TwUxTLi51Nus@users.noreply.github.com>
Closes#2488 from TwUxTLi51Nus/patch-1 and squashes the following commits:
7ab6f41 [TwUxTLi51Nus] Add z.show() feature to docs
### What is this PR for?
Since it is quite useful to expose the application info for user to monitor and debug, so here propose to enable "zeppelin.livy.displayAppInfo" to true as default.
### What type of PR is it?
Improvement
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2716
### How should this be tested?
Manual verification.
Author: jerryshao <sshao@hortonworks.com>
Closes#2459 from jerryshao/ZEPPELIN-2716 and squashes the following commits:
c9506ab [jerryshao] Address UT failure
32d3f72 [jerryshao] Change the default value of zeppelin.livy.displayAppInfo to true
### What is this PR for?
When "spark.ui.enabled" property is set to "false" we should not show the Spark UI button.
We keep the same behaviour when this property does not exist or when it exists and it's set to true.
### What type of PR is it?
[ Improvement]
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2714
### How should this be tested?
1. Open Zeppelin, go to Interpreter > Spark
1. Click spark ui > msg: "No spark application running"
1. Go to a paragraph, run "sc.parallelize(1 to 100).count()", check "SPARK JOB" button
1. Go to Interpreter > Spark > Set "spark.ui.enabled" to "false"
1. Rerun paragraph > "SPARK JOB" button not visible
1. Go to Interpreter > Spark > "spark ui" button not visible
Also, test with "spark.ui.enabled"="true" and other workflow combinations
### Questions:
* Does the licenses files need update? N
* Is there breaking changes for older versions? N
* Does this needs documentation? N
Author: Nelson Costa <nelson.costa85@gmail.com>
Closes#2456 from necosta/zeppelin2714 and squashes the following commits:
278a23e [Nelson Costa] [ZEPPELIN-2714] Fixed broken unit-test
edf29cc [Nelson Costa] [ZEPPELIN-2714] Minor final fix
294dea8 [Nelson Costa] [ZEPPELIN-2714] Improvements to change request
a85864c [Nelson Costa] [ZEPPELIN-2714] Soft-code Spark UI button visualization
### What is this PR for?
I spoiled the previous PR #2251
Added widgets (string, text, url, password, url, checkbox) to properties of interpreters. Those are widgets for properties customization. Properties must have the ability to customize the display (for example password).
### What type of PR is it?
Feature
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2403
### How should this be tested?
- remove conf/interpreter.json
- Try new form (create, edit) of interpreter settings
### Screenshots (if appropriate)
edit
view
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Tinkoff DWH <tinkoff.dwh@gmail.com>
Author: isys.mreshetov <m.reshetov@i-sys.ru>
Closes#2268 from tinkoff-dwh/ZEPPELIN-2403 and squashes the following commits:
75a10464 [isys.mreshetov] ZEPPELIN-2403 imports fix
7be8ddff [isys.mreshetov] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
585fc364 [isys.mreshetov] ZEPPELIN-2403 documentation fix
4b633993 [isys.mreshetov] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
726c1f31 [isys.mreshetov] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
b17dfb59 [isys.mreshetov] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
098fbd14 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
a5f13272 [Tinkoff DWH] [ZEPPELIN-2403] checkstyle fix
fd25c467 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
e35ff58f [Tinkoff DWH] [ZEPPELIN-2403] fix checkstyle
7c25b6db [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
10ce996a [Tinkoff DWH] [ZEPPELIN-2403] merge widget and type
ca1e2bf7 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
99daca6d [Tinkoff DWH] [ZEPPELIN-2403] fix rest api test
f735c0a9 [Tinkoff DWH] [ZEPPELIN-2403] fix test
c6d24c4c [Tinkoff DWH] [ZEPPELIN-2403] converter for old settings to new (with widgets)
76a98083 [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2403
b41e7a3f [Tinkoff DWH] ZEPPELIN-2403 checkstyle
637cb0a1 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
e92713c7 [Tinkoff DWH] [ZEPPELIN-2403] generalized types, added new types
07160e00 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
a495137f [Tinkoff DWH] ZEPPELIN-2403 eslint fix
fd8d2781 [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-2403_backup
4f271d9b [Tinkoff DWH] ZEPPELIN-2403 rename to widget added new widgets string, number, url
dd5d6c80 [Tinkoff DWH] ZEPPELIN-2403 did properties immutable, added new type 'checkbox'
14353b12 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
12499ae1 [Tinkoff DWH] Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
45f5f627 [Tinkoff DWH] ZEPPELIN-2403 added interpreter property types
### What is this PR for?
fix: broken image URLs in 0.8.0-SNAPSHOT doc
using the path `/asset` (the absolute path) for image URLs is actually invalid. That's because each version has its own image directory. So they should use the relative path. `{{BASE_PATH}}`
```
➜ asf-zeppelin tree site | grep asset
├── assets # root asset, we shouldn't use it in versioned doc.
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ ├── assets
```
### What type of PR is it?
[Bug Fix]
### Todos
DONE
### What is the Jira issue?
[ZEPPELIN-2707](https://issues.apache.org/jira/browse/ZEPPELIN-2707)
### How should this be tested?
1. cd `docs/`
2. build: `bundle exec jekyll build --safe`
3. check whether links in `_site` include `/docs/0.8.0-SNAPSHOT` as prefix or not
### Screenshots (if appropriate)
#### Current
http://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/usage/interpreter/overview.html
#### After
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Closes#2450 from 1ambda/ZEPPELIN-2707/should-use-its-own-asset-directory and squashes the following commits:
fb70214a [1ambda] fix: Use its own asset dir
### What is this PR for?
Pig 0.17.0 has just been released. This PR is to upgrade pig to 0.17.0 and support spark engine which is a big milestone of pig 0.17.0
Main Changes:
* Upgrade pig to 0.17.0
* Remove some code using java reflection in `PigUtils.java`, as pig 0.17.0 has some improvement and expose new apis which could be used pig interpreter.
* Support spark engine
### What type of PR is it?
[Improvement | Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://github.com/zjffdu/zeppelin/tree/ZEPPELIN-2615
### How should this be tested?
Unit test is added and also manually test spark yarn-client mode in pig tutorial note.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2431 from zjffdu/ZEPPELIN-2615 and squashes the following commits:
d4e9a6d [Jeff Zhang] Address comments
4b4e3db [Jeff Zhang] ZEPPELIN-2615. Upgrade pig to 0.17.0 to support spark engine
