mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
13 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Jeff Zhang
|
4c8f20ae33 |
ZEPPELIN-3051. Support Interpreter Process Recovery
### What is this PR for? This PR is for the purpose of recover running interpreter process when zeppelin server is restarted. This would be useful when restarting zeppelin without interrupt current running interpreter processes, should be useful when admin do maintenance or upgrading. Interface `RecoveryStorage` is used for storing the information of running interpreter process. Currently it only has one implementation `FileSystemRecoveryStorage`, other implementation could be done later (such as zookeeper based). `InterpreterLauncher` is the component where to recover the running interpreter process. Test: * RecoveryTest.java * FileSystemRecoveryStorageTest.java Design Doc: https://docs.google.com/document/d/1Plm3Hd40aGdNaXmjdsoY4ek3f-gTijTMGMkNjAZN39Y/edit?usp=sharing ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3051 ### How should this be tested? Unit test & Integration Test is added. Also manually verified. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2668 from zjffdu/ZEPPELIN-3051 and squashes the following commits: |
||
|
Jeff Zhang
|
13f8e6cc65 |
ZEPPELIN-3085 Introduce generic ConfInterpreter for more fine-grained control of interpreter setting
### What is this PR for?
Zeppelin's interpreter setting is shared by all the users and notes, if you want to have different setting you have to create new interpreter, e.g. you can create `spark_jar1` for running spark with dependency jar1 and `spark_jar2` for running spark with dependency jar2.
This approach works, but not so convenient. `ConfInterpreter` can provide more fine-grained control on interpreter setting and more flexibility.
`ConfInterpreter` is a generic interpreter that could be used by any interpreters. The input format should be property file format.
In the first paragraph, we use ConfInterpreter to make custom configuration of spark interpreter (set app name, yarn-client mode & add spark-csv dependencies). Then you can run the second paragraph which use spark-csv.
It can be used to make custom setting for any interpreter. `ConfInterpreter` would run in the zeppelin server side, it would update the interpreter properties before you launch the interpreter process, so it needs to run before interpreter process launched. And when interpreter process is launched is determined by interpreter mode setting. So users needs to understand the interpreter mode setting of zeppelin and be aware when interpreter process is launched. E.g. If we set spark interpreter setting as isolated per note. Under this setting, each note will launch one interpreter process. In this scenario, user need to put `ConfInterpreter` as the first paragraph as the above example. Otherwise the customized setting can not be applied (The paragraph using ConfInterpreter will report ERROR).
### What type of PR is it?
[Feature | Documentation]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3085
### How should this be tested?
* Unit test, System test is added, also manually verified it.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes #2692 from zjffdu/ZEPPELIN-3085 and squashes the following commits:
|
||
|
tinkoff-dwh
|
50cfabdf9b |
[ZEPPELIN-1363] Note level dynamic form
### What is this PR for? Added dynamic forms for Note. All paragraphs has access to note forms. ### What type of PR is it? [Feature] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1363 ### How should this be tested? Create global interpreter JDBC and Spark. Create note with 2 paragraphs ``` %pyspark print("Textbox paragraph " + z.textbox('input', 'default')) print("Textbox note " + z.noteTextbox('note_input', 'default_note')) print("Select paragraph " + z.select("sel", [("1","opt1"), ("2","opt2"), ("3","opt3")])) print("Select note " + z.noteSelect("sel_note", [("1","noteOpt1"), ("2","noteOpt2"), ("3","noteOpt3")])) options = [("key1","Name1"), ("key2","Name2")] print("Checkbox paragraph "+ " and ".join(z.checkbox("chk", options, ["key1"]))) print("Checkbox note "+ " and ".join(z.noteCheckbox("chk_note", options, ["key1","key2"]))) ``` ``` %jdbc select '$${checkbox:chk_note=key1|key2,key1|key2} $${note_input} ${note_input=sameName}' ``` ### Screenshots (if appropriate) 1) native forms  2) remove  3) simple forms  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: tinkoff-dwh <tinkoff.dwh@gmail.com> Closes #2641 from tinkoff-dwh/ZEPPELIN-1363 and squashes the following commits: |
||
|
tinkoff-dwh
|
7241348cff |
[ZEPPELIN-2813] revisions comparator
### What is this PR for? Sometimes need to see the difference between versions and to switch to another version and look for changes are not convenient (the page reloaded). This feature allows you to compare any two versions of the notebook. ### What type of PR is it? Feature ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2813 ### How should this be tested? 1 make some commits. Сchange the contents of paragraphs (delete, add, edit) 2 open Revisions comparator 3 compare revisions and check diff ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? yes (updated) * Is there breaking changes for older versions? no * Does this needs documentation? no Author: tinkoff-dwh <tinkoff.dwh@gmail.com> Author: Tinkoff DWH <tinkoff.dwh@gmail.com> Closes #2506 from tinkoff-dwh/ZEPPELIN-2813 and squashes the following commits: |
||
|
Jeff Zhang
|
d925e052ad |
[Doc] Doc for LifecycleManager
### What is this PR for?
Documentation for LifecycleManager
### What type of PR is it?
[ Documentation ]
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes #2644 from zjffdu/Doc_Lifecycle and squashes the following commits:
|
||
|
chilang
|
3eeb2282b9 |
ZEPPELIN-1897 REST API - Update paragraph text/title via API
### What is this PR for?
Support updating paragraph text/title via REST API
https://issues.apache.org/jira/browse/ZEPPELIN-1897
### What type of PR is it?
Improvement
### Todos
### What is the Jira issue?
[ZEPPELIN-1897]
### How should this be tested?
1. Create a note and paragraph
2. Hit REST API to update the paragraph, e.g. `curl -X PUT http://localhost:8080/api/notebook/NOTE_ID/paragraph/PARAGRAPH_ID -d '{"text": "some updated text"}'`
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, doc update included in the change.
Author: chilang <chilang@gmail.com>
Closes #2604 from chilang/master and squashes the following commits:
|
||
|
Paolo Genissel
|
e47b30a88f |
[ZEPPELIN-2848] Added new type of user to only run notebook
### What is this PR for? The idea of this PR is to provide a new kind of user : Runner. Basically, what it does is that it just removes write authorization and allow user to read and run note. ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? [ZEPPELIN-2848] https://issues.apache.org/jira/browse/ZEPPELIN-2848 ### How should this be tested? - Log in as admin - Create new notebook and create a paragraph with the interpreter you want - Assign runner right to user1 - Log in as user1 - Try to run the paragraph (should work) - Try to modify the paragraph (should fail) - Log in as user2 - Try to run the paragraph (should fail) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? Yes * Does this needs documentation? No Author: Paolo Genissel <paolo.genissel-monsallier@1000mercis.com> Author: gfalcone <paologenissel@gmail.com> Author: Paolo Genissel <paologenissel@gmail.com> Closes #2526 from gfalcone/new_type_runner and squashes the following commits: |
||
|
Nelson Costa
|
e1f20ddda2 |
[ZEPPELIN-2699] Helium REST API bug-fixes. Also updated documentation
### What is this PR for? Fixing bug on enabling Helium plugins (artifact is empty, not null) when sending curl command. After noticing out-of-date documentation, I updated it Also noticed issues on missing try/catch expressions and missing info messages on enable/disable packages. ### What type of PR is it? [Bug Fix | Documentation | Refactoring] ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2699 ### How should this be tested? Run "curl -X POST http://localhost:8080/api/helium/enable/{packageName}" Check that the package gets enabled ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? Y (Done) Author: Nelson Costa <nelson.costa85@gmail.com> Closes #2446 from necosta/zeppelin2699 and squashes the following commits: |
||
|
Nelson Costa
|
96d78ee57b |
[ZEPPELIN-2714] Soft-code Spark UI button visualization
### What is this PR for? When "spark.ui.enabled" property is set to "false" we should not show the Spark UI button. We keep the same behaviour when this property does not exist or when it exists and it's set to true. ### What type of PR is it? [ Improvement] ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2714 ### How should this be tested? 1. Open Zeppelin, go to Interpreter > Spark 1. Click spark ui > msg: "No spark application running" 1. Go to a paragraph, run "sc.parallelize(1 to 100).count()", check "SPARK JOB" button 1. Go to Interpreter > Spark > Set "spark.ui.enabled" to "false" 1. Rerun paragraph > "SPARK JOB" button not visible 1. Go to Interpreter > Spark > "spark ui" button not visible Also, test with "spark.ui.enabled"="true" and other workflow combinations ### Questions: * Does the licenses files need update? N * Is there breaking changes for older versions? N * Does this needs documentation? N Author: Nelson Costa <nelson.costa85@gmail.com> Closes #2456 from necosta/zeppelin2714 and squashes the following commits: |
||
|
1ambda
|
86bc9332e0 |
[ZEPPELIN-2582][DOCS] docs for interpreter binding modes
### What is this PR for? Updated `interpreter_binding_mode.md` since users are sometimes confused what this mode means and there is already opened JIRA issue. This documentation will be helpful to Zeppelin users. **disclaimer: content was copied from [here](https://medium.com/leemoonsoo/apache-zeppelin-interpreter-mode-explained-bae0525d0555) with author's consent.** ### What type of PR is it? [Documentation] ### Todos DONE ### What is the Jira issue? [ZEPPELIN-2582](https://issues.apache.org/jira/browse/ZEPPELIN-2582) ### How should this be tested? 1. setup rvm 2.1.0+ and install required bundles for `docs/` 2. bundle exec jekyll serve --watch 3. http://localhost:4000/usage/interpreter/interpreter_binding_mode.html ### Screenshots (if appropriate)       ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - This PR is about docs. Author: 1ambda <1amb4a@gmail.com> Closes #2437 from 1ambda/ZEPPELIN-2582/docs-for-interpreter-modes and squashes the following commits: |
||
|
Tinkoff DWH
|
155a55b560 |
[ZEPPELIN-2403] interpreter property widgets
### What is this PR for? I spoiled the previous PR #2251 Added widgets (string, text, url, password, url, checkbox) to properties of interpreters. Those are widgets for properties customization. Properties must have the ability to customize the display (for example password). ### What type of PR is it? Feature ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-2403 ### How should this be tested? - remove conf/interpreter.json - Try new form (create, edit) of interpreter settings ### Screenshots (if appropriate) edit  view  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Tinkoff DWH <tinkoff.dwh@gmail.com> Author: isys.mreshetov <m.reshetov@i-sys.ru> Closes #2268 from tinkoff-dwh/ZEPPELIN-2403 and squashes the following commits: |
||
|
1ambda
|
1c23f21388 |
[ZEPPELIN-2707][DOCS][HOTFIX] fix: broken image URLs in 0.8.0-SNAPSHOT doc
### What is this PR for?
fix: broken image URLs in 0.8.0-SNAPSHOT doc
using the path `/asset` (the absolute path) for image URLs is actually invalid. That's because each version has its own image directory. So they should use the relative path. `{{BASE_PATH}}`
```
➜ asf-zeppelin tree site | grep asset
├── assets # root asset, we shouldn't use it in versioned doc.
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ │ ├── assets
│ ├── assets
```
### What type of PR is it?
[Bug Fix]
### Todos
DONE
### What is the Jira issue?
[ZEPPELIN-2707](https://issues.apache.org/jira/browse/ZEPPELIN-2707)
### How should this be tested?
1. cd `docs/`
2. build: `bundle exec jekyll build --safe`
3. check whether links in `_site` include `/docs/0.8.0-SNAPSHOT` as prefix or not
### Screenshots (if appropriate)
#### Current
http://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/usage/interpreter/overview.html
#### After
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Closes #2450 from 1ambda/ZEPPELIN-2707/should-use-its-own-asset-directory and squashes the following commits:
|
||
|
1ambda
|
4b6d3e5574 |
[ZEPPELIN-2596] Improving documentation page
### What is this PR for? Improving documentation page. Please check *TODO* and *Screenshots* sections for detail. The motivation is described in [the JIRA ticket](https://issues.apache.org/jira/browse/ZEPPELIN-2583) and discussion is ongoing on the mailing list. ### What type of PR is it? [Improvement | Documentation] ### Todos * [x] - improved the navbar style * [x] - improved the main page * [x] - re-organized content structure * [x] - added tutorial pages: `spark_with_zeppelin.md`, `python_with_zeppelin.md`, `sql_with_zeppelin.md` for overview * [x] - added `multi_user_support.md` page to provide overview * [x] - added the empty `interpreter_binding_mode` page. This will be handed in the different issue: [ZEPPELIN-2582](https://issues.apache.org/jira/browse/ZEPPELIN-2582) * [x] - added the empty `trouble_shooting` page. This can be filled in the following PRs. * [x] - added the empty `useful_developer_tools` page. This can be filled in the following PRs. ### What is the Jira issue? [ZEPPELIN-2596](https://issues.apache.org/jira/browse/ZEPPELIN-2596) ### How should this be tested? 1. checkout 2. `cd docs` 3. `bundle install` (make sure that you have ruby 2.1.0+ and bundle) 4. `bundle exec jekyll serve --watch` 5. open `localhost:4000` ### Screenshots (if appropriate) #### better navbar: before  #### better navbar: after  #### improved main page: before  #### improved main page: after  #### organized content structure: before  #### organized content structure: after  ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - related with docs Author: 1ambda <1amb4a@gmail.com> Closes #2371 from 1ambda/updating-version-doc and squashes the following commits: |
