mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
38 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Lee moon soo
|
850fd81a51 |
[ZEPPELIN-212] Multiple paragraph results
### What is this PR for?
Currently a paragraph can display only a single type of result.
For example
```
print("""
%text textout
%html htmlout
""")
```
will display only last display system detected, "htmlout".
This pr implements multiple results supports, so not only the last, but also all the display systems in a paragraph outputs can be displayed.
To do that, Note.json format need to be changed. from
```
paragraph : [
{
result : {
code: "",
type: "",
msg: ""
},
config : {
graph : {},
...
},
...
},
...
],
...
```
to
```
paragraph : [
{
results : {
{
code: "",
msg: [
{
type: "",
data: ""
},
{
type: "",
data: ""
},
...
]
},
config : {
results : [
{
graph : {},
},
{
graph : {},
},
...
]
},
...
},
...
],
...
```
### What type of PR is it?
Improvement
### Todos
* [x] - Make InterpreterResult and InterpreterOutput support multiple display system
* [x] - Automatic migration old format to new format
* [x] - Render multiple results in front-end
* [x] - Take care Importing old notebook format
* [x] - Make helium framework support multiple results
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-212
### How should this be tested?
run following code in a single spark paragraph.
```
%spark
// default output is text
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// print something in html
println(s"""%html <h3><font style="color:blue">Some HTML</font></h3>""")
// create table
println(s"""%table key\tvalue
sun\t100
moon\t200
""")
// display text again
println("""%text""")
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// another table
Thread.sleep(1000)
println(s"""%table key\tvalue
apple\t100
banana\t200
""")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? yes
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes #1658 from Leemoonsoo/ZEPPELIN-212 and squashes the following commits:
|
||
|
1ambda
|
e33be25256 |
[ZEPPELIN-1684] Add GET /interpreter/setting/id REST API
### What is this PR for? Due to the lack of this missing API, some front-end code retrieve all settings. This is inefficient. We can modify those codes by adding this API. Also, i refactored the whole `InterpreterRestApiTest` ### What type of PR is it? [Feature | Documentation | Refactoring] ### What is the Jira issue? [ZEPPELIN-1684](https://issues.apache.org/jira/browse/ZEPPELIN-1684) ### How should this be tested? You can run unit test `InterpreterRestApiTest` ### Screenshots (if appropriate) Updated doc screenshots <img width="900" alt="new_api_doc" src="https://cloud.githubusercontent.com/assets/4968473/20423419/74eeb2a0-adb3-11e6-9a69-58e33457d514.png"> ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - YES, but done. Author: 1ambda <1amb4a@gmail.com> Closes #1655 from 1ambda/feat/create-interpreter-get-api and squashes the following commits: |
||
|
Mina Lee
|
24187b0a75 |
[ZEPPELIN-1029] Add rest api for paragraph config update
### What is this PR for?
This PR adds new end point for updating paragraph configuration
### What type of PR is it?
Improvement | Documentation
### What is the Jira issue?
[ZEPPELIN-1029](https://issues.apache.org/jira/browse/ZEPPELIN-1029)
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Mina Lee <minalee@apache.org>
Closes #1592 from minahlee/ZEPPELIN-1029 and squashes the following commits:
|
||
|
Mina Lee
|
dd20e7bf8b |
[ZEPPELIN-1564] Enable note deletion and paragraph output clear from main page
### What is this PR for? - Enables removing note and clear all paragraph's output from Zeppelin main page. - Add rest api for clearing all paragraph output Next possible improvement can be removing notes in folder level and rename folder. ### What type of PR is it? Improvement ### Todos * [x] - Merge #1567 and apply security to `clearAllParagraphOutput` rest api method ### What is the Jira issue? [ZEPPELIN-1564](https://issues.apache.org/jira/browse/ZEPPELIN-1564) ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? yes Author: Mina Lee <minalee@apache.org> Closes #1565 from minahlee/ZEPPELIN-1564 and squashes the following commits: |
||
|
astroshim
|
5991a35774 |
[ZEPPELIN-1306] Interpreter restarts on a note.
### What is this PR for? This PR is for usability of restarting interpreter. ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1306 ### How should this be tested? You can restart interpreter on the interpreter binding page on a note. and can only restart `scoped` and `isolated` interpreter. Please refer to screen shot. ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Closes #1302 from astroshim/ZEPPELIN-1306 and squashes the following commits: |
||
|
Naveen Subramanian
|
4812719738 |
ZEPPELIN-1562 updated api doc
What is this PR for?
The URL for running a paragraph synchronously using REST api is mistakenly given as "http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId] "
Changed the doc as per code
What type of PR is it?
[Documentation]
Todos
What is the Jira issue?
ZEPPELIN-1562
How should this be tested?
Hit the url in below format for asynchrnous
http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]
the same in synchronous
http://[zeppelin-server]:[zeppelin-port]/api/notebook/run/[notebookId]/[paragraphId]
Screenshots (if appropriate)
Questions:
Does the licenses files need update? NO
Is there breaking changes for older versions? NO
Does this needs documentation? NO
Author: Naveen Subramanian <nsubramanian-consultant@quixey.com>
Closes #1560 from snaveenp/ZEPPELIN-1562-api-doc and squashes the following commits:
|
||
|
hyonzin
|
4f6a0e34ff |
[ZEPPELIN-1549] Change NotebookID variable name to NoteID
### What is this PR for? This PR fixes wrong written NotebookID to NoteID. ### What type of PR is it? [Improvement] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1549 ### Questions: * Does the licenses files need update? No. * Is there breaking changes for older versions? No. * Does this needs documentation? No Author: hyonzin <hyeonjin507@gmail.com> Author: 정현진 <hyeonjin507@gmail.com> Author: Mina Lee <minalee@apache.org> Closes #1518 from hyonzin/ZEPPELIN-1549 and squashes the following commits: |
||
|
DuyHai DOAN
|
7e2a1b5d4f |
[ZEPPELIN-699] Add new synchronous paragraph run REST API
### What is this PR for? Right now, when calling the REST API `http://<ip>:<port>/api/notebook/job/<note_id>/<paragraph_id>` Zeppelin always returns **OK** as shown by this source code: https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java#L477 This ticket will update the behavior so that Zeppelin also return the result of the paragraph execution ### What type of PR is it? [Improvement] ### Todos * [ ] - Code Review * [ ] - Simple Test ### Is there a relevant Jira issue? **[ZEPPELIN-699]** ### How should this be tested? * `git fetch origin pull/746/head:ParagraphExecutionRESTAPI` * `git checkout ParagraphExecutionRESTAPI` * `mvn clean package -DskipTests` * `bin/zeppelin-daemon.sh restart` * Create a new note * In the first paragraph, put the following code ```scala %sh echo "Current time = "`date +"%T" ``` * Retrieve the current note id in the URL * Retrieve the current paragraph id * Use a REST Client like **[POSTman]** to create a HTTP POST query `http://<ip>:<port>/api/notebook/run/<note_id>/<paragraph_id>` * You should receive something similar as follow for answer ``` { "status": "OK", "body": { "code": "SUCCESS", "type": "TEXT", "msg": "Current time = 16:14:18\n" } } ``` ### Screenshots (if appropriate)  API Documentation update **Existing asynchronous API**  **New synchronous API**  ### Questions: * Does the licenses files need update? --> **No** * Is there breaking changes for older versions? --> **No** * Does this needs documentation? --> **Yes** [ZEPPELIN-699]: https://issues.apache.org/jira/browse/ZEPPELIN-699 [POSTman]: https://www.getpostman.com/ Author: DuyHai DOAN <doanduyhai@gmail.com> Closes #746 from doanduyhai/ZEPPELIN-699 and squashes the following commits: |
||
|
CloverHearts
|
42e3a141de |
[ZEPPELIN-960] When there is no interpreter, paragraph runJobapi modified.
### What is this PR for?
Among Zeppelin API Fixed runNote.
According to whether the interpreter specified,
was added to an exception processing for a result and operation.
run success (HTTP STATUS : 200)
```
{
"status": "OK"
}
```
can't not found note id (HTTP STATUS : 404)
```
{
"status": "NOT_FOUND",
"message": "note not found."
}
```
interpter not found or not bind (HTTP STATUS : 412 - PRECONDITION FAILED)
```
{
"status": "PRECONDITION_FAILED",
"message": "paragraph_1469771130099_-278315611 Not selected or Invalid Interpreter bind"
}
```
### What type of PR is it?
Bug Fix
### Todos
- [x] runNote api can apply by jdbc alias naming.
- [x] The results are recorded in the paragraph.
- [x] Binding fails when the interpreter 'HTTP STATUS CODE (412: PRECONDITION FAILED) and returns a message.
- [x] modification docs.
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-960
### How should this be tested?
execute to curl command line.
1. curl -XPOST http://(your zeppelin ip):(port)/api/notebook/job/2A94M5J1Z -H'Content-Type:application/json'
```
curl -XPOST http://127.0.0.1:8080/api/notebook/job/2A94M5J1Z -H'Content-Type:application/json'
```
2.
if correct for execution then
```
{"status":"OK"}
```
if invalid notebook id then
```
{
"status": "NOT_FOUND",
"message": "note not found."
}
```
if not binding interpreters
```
{
"status": "PRECONDITION_FAILED",
"message": "paragraph_(paragraphid) Not selected or Invalid Interpreter bind"
}
```
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: CloverHearts <cloverheartsdev@gmail.com>
Author: CloverHearts <cloverheartsdev+github@gmail.com>
Closes #1242 from cloverhearts/ZEPPELIN-960 and squashes the following commits:
|
||
|
Kavin
|
47ac1d41e7 |
[ZEPPELIN-728] Can't POST interpreter setting (CorsFilter?)
### What is this PR for? This handles the NPE when the input json is empty for the interpreter setting POST request. ### What type of PR is it? Bug Fix ### Todos NA ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-728 ### How should this be tested? When empty json is sent for interpreter setting POST request, 400 status code should be returned. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Kavin <kavin.kumar@imaginea.com> Author: Kavin Kumar <junokavin@gmail.com> Closes #1345 from kavinkumarks/zeppelin-728-fix-NPE-intepreter-setting-post and squashes the following commits: |
||
|
Kavin Kumar
|
bba6ddd249 |
[ZEPPELIN-1268] As an enduser, I would like to embed paragraph and refresh the content of it in my custom UI
### What is this PR for? This pull request is to add the below options so when someone develops a custom UI it would ease them. 1. Add a new option "Get paragraph id" in the notebook UI 2. Add a new REST API to get the status of a single paragraph ### What type of PR is it? Improvement ### Todos * Need to update the REST APIs documentation http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/rest-api/rest-notebook.html#get-the-status-of-all-paragraphs ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1268 ### How should this be tested? * Navigate to the notebook UI and a new option "Get paragraph id" will be shown for each paragraph and on clicking on it, the paragraph id would be alerted * The new REST API http://%5Bzeppelin-server%5D:%5Bzeppelin-port%5D/api/notebook/job/%5BnotebookId%5D/%5BparagraphId%5D could be accessed to get the status of a single paragraph. ### Screenshots (if appropriate) ### Questions: Author: Kavin Kumar <junokavin@gmail.com> Author: Kavin <kavin.kumar@imaginea.com> Closes #1262 from kavinkumarks/zeppelin-1268-custom-UI and squashes the following commits: |
||
|
AhyoungRyu
|
85d4df4f0c |
[ZEPPELIN-1219] Add searching feature to Zeppelin docs site
### What is this PR for? As more and more document pages are added, it's really hard to find specific pages. So I added searching feature to Zeppelin documentation site([jekyll](https://jekyllrb.com/) based site) using [lunr.js](http://lunrjs.com/). - **How does it work?** I created [`search_data.json`]( |
||
|
Kousuke Saruta
|
045b2d24d8 |
[ZEPPELIN-1274]Write "Spark SQL" in docs rather than "SparkSQL"
### What is this PR for? Some of the doc files say "SparkSQL" but the collect spelling is "Spark SQL" (need a white space between "Spark" and "SQL"). Lets's replace it with the collect one. ### What type of PR is it? Improvement ### Todos * [x] - Replace all of "SparkSQL" in some files into "Spark SQL". ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1274 ### How should this be tested? Check the changes by some people. Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Closes #1271 from sarutak/ZEPPELIN-1274 and squashes the following commits: |
||
|
Kousuke Saruta
|
8a179f4b66 |
[ZEPPELIN-1289] Update the default value of 'spark.executor.memory' property
### What is this PR for?
As of Spark 1.5.0, the default value of 'spark.executor.memory' is 1g, not 512m so let's follow the change.
### What type of PR is it?
Improvement
### Todos
* [x] - Replaced all of the occurrences of the default value of spark.executor.memory with '1g'
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1289
### How should this be tested?
Should be checked by some people's eyes.
### Questions:
* Does the licenses files need update?: no
* Is there breaking changes for older versions?: no
* Does this needs documentation?: no
Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>
Closes #1280 from sarutak/ZEPPELIN-1289 and squashes the following commits:
|
||
|
AhyoungRyu
|
6bd4ede7e5 |
[DOC][MINOR] Add shell interpreter docs to _navigation.html
### What is this PR for? After #1087 merged, a new docs `shell.md` was added. But in the docs website, still Shell interpreter link points to `pleasecontribute.html`. So I changed this link, applied TOC and added more descriptions. ### What type of PR is it? Documentation ### Todos * [x] - Change `pleasecontribute.html` -> `shell.html` * [x] - Apply TOC(table of contents) * [x] - Add more description to `shell.md` ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: AhyoungRyu <fbdkdud93@hanmail.net> Closes #1138 from AhyoungRyu/improve/shell-docs and squashes the following commits: |
||
|
astroshim
|
c348161df0 |
[ZEPPELIN-1023] Add more credential apis.
### What is this PR for? This PR is for supporting various Credential APIs for users. ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1023 ### How should this be tested? - to create credential information. ``` curl -XPUT -H "Content-Type: application/json" "http://localhost:8080/api/credential" -d '{"entity" : "e1", "username" : "user1", "password" : "testpass"}' ``` - to get credential information. ``` curl -XGET -H "Content-Type: application/json" "http://localhost:8080/api/credential" ``` - to remove credential entity information. ``` curl -XDELETE -H "Content-Type: application/json" "http://localhost:8080/api/credential/e1" ``` - to remove all credential information. ``` curl -XDELETE -H "Content-Type: application/json" "http://localhost:8080/api/credential" ``` ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Author: AhyoungRyu <fbdkdud93@hanmail.net> Author: HyungSung <hsshim@nflabs.com> Closes #1030 from astroshim/ZEPPELIN-1023 and squashes the following commits: |
||
|
AhyoungRyu
|
5975125f18 |
[ZEPPELIN-1018] Apply auto "Table of Contents" generator to Zeppelin docs website
### What is this PR for? I added auto TOC(Table of Contents) generator for Zeppelin documentation website. TOC can help people looking through whole contents at a glance and finding what they want quickly. I just added `<div id="toc"></div>` to the each documentation header. [`toc`](https://github.com/apache/zeppelin/compare/master...AhyoungRyu:ZEPPELIN-1018?expand=1#diff-85af09fb498a5667ea455391533f945dR3) recognize `<h2>` & `<h3>` as a title in the docs and it automatically generate TOC. So I set a rule for this work. (I'll write this rule on `docs/CONTRIBUTING.md` or [docs/howtocontributewebsite](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/howtocontributewebsite.html)). ``` # Level-1 Heading <- Use only for the main title of the page ## Level-2 Heading <- Start with this one ### Level-3 heading <- Only use this one for child of Level-2 toc only recognize Level-2 & Level-3 ``` Please see the below attached screenshot image. ### What type of PR is it? Improvement & Documentation ### Todos * [x] - Add TOC generator * [x] - Apply TOC(`<div id="toc"></div>`) to every documentation and reorganize each headers(apply the above rule) * [x] - Fix some broken code block in several docs * [x] - Apply TOC to `r.md` (Currently R docs has some duplicated info since [this one]( |
||
|
Jongyoul Lee
|
1c3373937a |
ZEPPELIN-934 Merge Phoenix interpreter into JDBC interpreter
### What is this PR for?
Removing Phoenix interpreter and adding example for using it through JDBC
### What type of PR is it?
[Feature]
### Todos
* [x] - Merge it into JDBC
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-934
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes #982 from jongyoul/ZEPPELIN-934 and squashes the following commits:
|
||
|
Jongyoul Lee
|
1587b8686b |
ZEPPELIN-925 Part 2. Merge Hive Interpreter into JDBC
### What is this PR for? Fixed some wrong documentation and settings. This is minor issue. ### What type of PR is it? [Feature | Documentation] ### Todos * [x] - Remove settings about Hive and documentation ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-925 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jongyoul Lee <jongyoul@gmail.com> Closes #981 from jongyoul/ZEPPELIN-925-1 and squashes the following commits: |
||
|
Jongyoul Lee
|
7e1ad5b14a |
ZEPPELIN-974 Merge TajoInterpreter into JdbcInterpreter
### What is this PR for? Merging TajoInterpreter into JdbcInterpreter, and removing TajoInterprete ### What type of PR is it? [Feature] ### Todos * [x] - Remove TajoInterpreter and Document jdbc.md ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-974 ### How should this be tested? ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jongyoul Lee <jongyoul@gmail.com> Closes #985 from jongyoul/ZEPPELIN-974 and squashes the following commits: |
||
|
Mina Lee
|
70ab1a376d |
[ZEPPELIN-952] Refine website style
### What is this PR for? - update document style (font, line-spacing) - apply same formats for documents - fix broke document styles ### What type of PR is it? Documentation ### What is the Jira issue? [ZEPPELIN-952](https://issues.apache.org/jira/browse/ZEPPELIN-952) ### Screenshots (if appropriate) **Before** <img width="1184" alt="screen shot 2016-06-04 at 9 51 38 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803667/d0dd5ac2-2a9f-11e6-9ed0-ddc369a97612.png"> **After** <img width="1184" alt="screen shot 2016-06-04 at 9 15 08 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803666/cd9212ea-2a9f-11e6-986e-17992a495ab6.png"> **Before** <img width="1183" alt="screen shot 2016-06-04 at 10 08 53 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803695/03e73126-2aa1-11e6-8675-3ca437aeb833.png"> **After** <img width="1184" alt="screen shot 2016-06-04 at 10 08 18 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803696/078ce866-2aa1-11e6-9044-4f5e16649eb4.png"> **Before** <img width="1184" alt="screen shot 2016-06-04 at 10 10 47 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803704/5787e9ba-2aa1-11e6-804c-076a8f3aa852.png"> **After** <img width="1184" alt="screen shot 2016-06-04 at 10 11 22 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803707/5afb5d0c-2aa1-11e6-98c7-7440db35bd2f.png"> **Before** <img width="188" alt="screen shot 2016-06-04 at 10 12 36 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803719/92e5cc3e-2aa1-11e6-9a9f-e12150e78733.png"> **After** <img width="199" alt="screen shot 2016-06-04 at 10 12 55 pm" src="https://cloud.githubusercontent.com/assets/8503346/15803721/958e8c00-2aa1-11e6-8768-8350db6e7173.png"> ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Mina Lee <minalee@nflabs.com> Closes #962 from minahlee/ZEPPELIN-952 and squashes the following commits: |
||
|
Luciano Resende
|
803ebc2539 |
[ZEPPELIN-898] Update resources to use Zeppelin TLP links
### What is this PR for?
Update resources to use Zeppelin TLP links
### What type of PR is it?
[Bug Fix]
### What is the Jira issue?
[ZEPPELIN-898](https://issues.apache.org/jira/browse/ZEPPELIN-898)
Author: Luciano Resende <lresende@apache.org>
Closes #927 from lresende/incubator-links and squashes the following commits:
|
||
|
Cheng-Yu Hsu
|
a313e492c4 |
Fix typos in docs
### What is this PR for?
1. Fix some typos in docs.
2. Remove trailing white spaces for each line.
3. Remove leading white spaces if a line contains no content.
4. Add trailing new line for each file.
### What type of PR is it?
Improvement | Documentation
### Todos
None
### What is the Jira issue?
N/A
### How should this be tested?
Build the doc site and check.
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? *no*
* Is there breaking changes for older versions? *no*
* Does this needs documentation? *no*
Author: Cheng-Yu Hsu <m@cyhsu.me>
Closes #852 from cyhsutw/fix-typos-in-docs and squashes the following commits:
|
||
|
Eric Charles
|
7d6cc7e991 |
R and SparkR Support [WIP]
### What is this PR for? Implement R and SpakR Intepreter as part of the Spark Interpreter Group. It also implements R and Scala binding (in both directions). ### What type of PR is it? [Feature] ### Todos * [ ] - Documentation * [ ] - Unit test (if relevant, as we depend on R being available on the host) * [ ] - Assess licensing (a priori ok as we don't delive anything out of ASL2, but we should corectly phrase the NOTICE as we depend on non-ASL2 comptabile licenses (R) to run the interpreter). ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-156 SparkR support ### How should this be tested? You need R available on the host running the notebook. For Centos: yum install R R-devel For Ubuntu: apt-get install r-base r-cran-rserve Install additional R packages: ``` curl https://cran.r-project.org/src/contrib/Archive/rscala/rscala_1.0.6.tar.gz -o /tmp/rscala_1.0.6.tar.gz R CMD INSTALL /tmp/rscala_1.0.6.tar.gz R -e "install.packages('ggplot2', repos = 'http://cran.us.r-project.org')" R -e install.packages('knitr', repos = 'http://cran.us.r-project.org') ``` - Build + launch Zeppelin and test the R note. !!! you need rscala_1.0.6 (if not, you need to build with -Drscala.version=...) ### Screenshots (if appropriate) #### Simple R [](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/simple-r.png) #### Plot [](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/plot.png) #### Scala R Binding [](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/scala-r.png) #### R Scala Binding [](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/r-scala.png) #### SparkR [](https://raw.githubusercontent.com/datalayer/zeppelin-R/rscala/_Rimg/sparkr.png) ### Questions: * Does the licenses files need update? to be checked... (cfr R needs to be available to make this interpreter operational). * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Eric Charles <eric@datalayer.io> Author: Lee moon soo <moon@apache.org> This patch had conflicts when merged, resolved by Committer: Lee moon soo <moon@apache.org> Closes #702 from echarles/rscala-z and squashes the following commits: |
||
|
Mina Lee
|
218a3b5bca |
[Zeppelin-630] Introduce new way of dependency loading to intepreter
### What is this PR for?
With this PR user will be able to set external libraries to be loaded to specific interpreter.
Note that the scope of this PR is downloading libraries to local repository, not distributing them to other nodes. Only spark interpreter distributes loaded dependencies to worker nodes at the moment.
Here is a brief explanation how the code works.
1. get rest api request for interpreter dependency setting from front-end
2. download the libraries in `ZEPPELIN_HOME/local-repo` and copy them to `ZEPPELIN_HOME/local-repo/{interpreterId}`
3. `ZEPPELIN_HOME/local-repo/{interpreterId}/*.jar` are added to interpreter classpath when interpreter process starts
### What type of PR is it?
Improvement
### Todos
* [x] Add tests
* [x] Update docs
### Is there a relevant Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-630
And this PR will resolve [ZEPPELIN-194](https://issues.apache.org/jira/browse/ZEPPELIN-194) [ZEPPELIN-381](https://issues.apache.org/jira/browse/ZEPPELIN-381) [ZEPPELIN-609](https://issues.apache.org/jira/browse/ZEPPELIN-609)
### How should this be tested?
1. Add repository(in interpreter menu, click gear button placed top right side)
```
id: spark-packages
url: http://dl.bintray.com/spark-packages/maven
snapshot: false
```
2. Set dependency in spark interpreter(click edit button of spark interpreter setting)
```
artifact: com.databricks:spark-csv_2.10:1.3.0
```
3. Download example csv file
```
$ wget https://github.com/databricks/spark-csv/raw/master/src/test/resources/cars.csv
```
4. run below code in paragraph
```
val df = sqlContext.read
.format("com.databricks.spark.csv")
.option("header", "true") // Use first line of all files as header
.option("inferSchema", "true") // Automatically infer data types
.load("file:///your/download/path/cars.csv")
df.registerTempTable("cars")
```
```
%sql select * from cars
```
### Screenshots (if appropriate)
* Toggle repository list
<img width="1146" alt="screen shot 2016-01-25 at 12 24 44 pm" src="https://cloud.githubusercontent.com/assets/8503346/12563475/52f060ac-c35f-11e5-8621-d8eb97b4d6a1.png">
* Add new repository
<img width="1146" alt="screen shot 2016-01-25 at 12 25 23 pm" src="https://cloud.githubusercontent.com/assets/8503346/12563472/52eb545e-c35f-11e5-9050-a5306d2765f1.png">
* Show repository info
<img width="1146" alt="screen shot 2016-01-25 at 12 25 28 pm" src="https://cloud.githubusercontent.com/assets/8503346/12563473/52ebab84-c35f-11e5-9acb-3a356c855dc7.png">
* Interpreter dependency
<img width="1146" alt="screen shot 2016-01-25 at 12 27 27 pm" src="https://cloud.githubusercontent.com/assets/8503346/12563471/52eadd9e-c35f-11e5-8e1a-f583ea8800aa.png">
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions?
- For the users who use rest api for creat/update interpreter setting, `dependencies` object should be added to request payload.
- %dep interpreter is deprecated. The functionality is still there, but recommend to load third party dependency via interpreter menu.
* Does this needs documentation? Yes
Author: Mina Lee <minalee@nflabs.com>
Closes #673 from minahlee/ZEPPELIN-630 and squashes the following commits:
|
||
|
Jungtaek Lim
|
9308ee151c |
ZEPPELIN-612 Introduce 'Configuration' page with WebSocket API / REST API
### What is this PR for? To introduce 'Configuration' page to Zeppelin GUI. Also introduce REST API / WebSocket operation. ### What type of PR is it? Feature ### Todos ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-612 ### How should this be tested? - apply this PR - build and start Zeppelin - open UI and confirm Configuration tab is shown - open Configuration page and confirm every configurations (properties) except password are shown ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes, I've address it regarding REST API. Author: Jungtaek Lim <kabhwan@gmail.com> Closes #645 from HeartSaVioR/ZEPPELIN-612 and squashes the following commits: |
||
|
swakrish
|
2714d28b5a |
Rest Apis to export/import with common code base
1) Rest apis to export notebook as JSON 2) Rest api to import notebook from JSON https://issues.apache.org/jira/browse/ZEPPELIN-590 Author: swakrish <swakrish@gmail.com> Author: Ramaswamy Devarajan <ram.devarajan@us.ibm.com> Closes #614 from swakrish/master and squashes the following commits: |
||
|
Ryu Ah young
|
8c848f0de9 |
Fix some typos, grammars and Increase readability documentation
### What is this PR for? It would be better if Zeppelin provides looking good documentation. ### What type of PR is it? Improvement ### Todos * [x] - Fix typos, grammars and Increase readability writingzeppelininterpreter.md * [x] - Also spark.md * [x] - tutorial.md * [x] - cassandra.md * [x] - elasticsearch.md * [x] - flink.md * [x] - hive.md * [x] - lens.md * [x] - markdown.md * [x] - rest-interpreter.md ### Is there a relevant Jira issue? No : ) ### How should this be tested? These documentations are in [here](http://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/). ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Ryu Ah young <fbdkdud93@hanmail.net> Closes #578 from AhyoungRyu/Fix-Typo and squashes the following commits: |
||
|
Jungtaek Lim
|
3bfd97e236 |
ZEPPELIN-511 REST API: Insert / Retrieve / Move / Delete operation for paragraph
### What is this PR for? This issue is intended to fill gap between REST API and WebSocket operations. For now we can only access notebook and paragraph to READONLY (not writing) via REST API but after this PR, we can insert / retrieve / move / delete paragraph via REST API. ### What type of PR is it? Feature ### Todos ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-511 ### How should this be tested? Please follow the explanation of added REST APIs from rest-notebook.md. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? (No) * Is there breaking changes for older versions? (No) * Does this needs documentation? (Yes, I've addressed it.) Author: Jungtaek Lim <kabhwan@gmail.com> Closes #550 from HeartSaVioR/ZEPPELIN-511 and squashes the following commits: |
||
|
Jungtaek Lim
|
f78d3096e0 |
ZEPPELIN-528 REST API: add "progress" to 'get notebook job' API
### What is this PR for? To add 'progress' to 'get notebook job' REST API. All paragraphs which status is 'running' will include 'progress' to their job status. ### What type of PR is it? Improvement ### Todos ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-528 ### How should this be tested? 1. Run a notebook or paragraph which takes some times. Paragraph which could show progress (%) would be more appreciated. Confirm that its status is 'RUNNING'. 2. Call ```http://<zeppelin host>:<zeppelin port>/api/notebook/job/<notebook id>``` to see if it has 'progress' field in JSON. ### Screenshots (if appropriate)   ### Questions: * Does the licenses files need update? (No) * Is there breaking changes for older versions? (No) * Does this needs documentation? (Yes or no. Output of the JSON format could be updated, but it could be treated as not mandatory.) Author: Jungtaek Lim <kabhwan@gmail.com> Closes #563 from HeartSaVioR/ZEPPELIN-528 and squashes the following commits: |
||
|
Alexander Bezzubov
|
82de508d72 |
ZEPPELIN-501 Notebook search
### What is this PR for? This PR has for goal to allow the user to search through the code in all the paragraphs *and notebook names* in all the notebooks It add a simple 'search bar' to the nav-bar of Zeppelin WebApp, and an in-memory fulltext search index of paragraphs to the backend. The search is pretty basic now, fine-tuning it for better search over all types of source code will be a subject of further work. ### What type of PR is it? Feature ### Todos * [x] - Fix typos 💃 |
||
|
Jungtaek Lim
|
16e921b6ad |
ZEPPELIN-509 Introduce a new REST API: get notebook information
### What is this PR for? To add a new feature - "get notebook information" via REST API. ### What type of PR is it? Feature ### Todos ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-509 ### How should this be tested? Just request ```http://<zeppelin host>:<zeppelin port>/api/notebook/<notebookId>```. If it exists, response would be http status 200 with json body including paragraphs information. If it doesn't exist, response would be http status 404. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? (No) * Is there breaking changes for older versions? (No, it's a new feature.) * Does this needs documentation? (Yes, I've addressed it.) Author: Jungtaek Lim <kabhwan@gmail.com> Closes #542 from HeartSaVioR/ZEPPELIN-509 and squashes the following commits: |
||
|
Jungtaek Lim
|
9fec2598ca |
ZEPPELIN-507 Expand "run paragraph" of REST API to update dynamic form's value
### What is this PR for? To update dynamic form's value (via params) when running paragraph via REST API ### What type of PR is it? Improvement ### Todos ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-507 ### How should this be tested? * Create a new notebook with one paragraph which contains ``` %spark val form1 = z.input("form1").toString ``` * Get paragraph ID via websocket (I think we should add a new REST API to get it easily.) * Request ```http://<zeppelin host>:<zeppelin port>/api/notebook/job/<notebook id>/<paragraph id>``` with POST method including JSON payload, ``` { "params": { "form1": "value1" } } ``` * Confirm that paragraph has run and input text of form1 is filled with "value1" (from the UI) ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? (No) * Is there breaking changes for older versions? (No, additonal json payload is optional.) * Does this needs documentation? (Yes, I've addressed it.) Author: Jungtaek Lim <kabhwan@gmail.com> Closes #541 from HeartSaVioR/ZEPPELIN-507 and squashes the following commits: |
||
|
Jungtaek Lim
|
1ba6e2a596 |
Fix typo / grammar from rest-notebook.md
Just modifying doc so I don't file a new issue to JIRA. Author: Jungtaek Lim <kabhwan@gmail.com> Closes #545 from HeartSaVioR/fix-typo-from-rest-notebook and squashes the following commits: |
||
|
astroshim
|
be8a747290 |
[Improve] Add interpreter restart REST API doc
### What is this PR for? Add restarting interpreter REST API information to the document. There is no information in the document even though REST API exist. ### What type of PR is it? Documentation ### Todos * [x] - add restart REST API document. ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-521 ### How should this be tested? You can get the test information from https://github.com/apache/incubator-zeppelin/tree/master/docs. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Closes #555 from astroshim/ZEPPELIN-521 and squashes the following commits: |
||
|
Jungtaek Lim
|
d69a30eec7 |
ZEPPELIN-506 Expand "create notebook" of REST API to have initial paragraphs
### What is this PR for? To provide initial paragraphs when creating new notebook via REST API ### What type of PR is it? Improvement ### Todos * [ ] - ### Is there a relevant Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-506 ### How should this be tested? 1. Request ```http://<zeppelin host>:<zeppelin port>/api/notebook``` with POST method including JSON payload, ``` { "name": "Test for initial paragraphs2", "paragraphs": [ { "text": "%spark\nvar hello: Int=1" }, { "text": "%spark\nhello = hello + 1" }, { "text": "%spark\nimport java.util.Date\n\nprintln(hello + \", current time is <\" + new Date + \">\")" } ] } ``` 2. Check notebook is made with initial paragraphs via UI or REST API or Web socket. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? (No) * Is there breaking changes for older versions? (No, added parameter is optional.) * Does this needs documentation? (Yes, I've updated related doc.) Author: Jungtaek Lim <kabhwan@gmail.com> Closes #536 from HeartSaVioR/ZEPPELIN-506 and squashes the following commits: |
||
|
astroshim
|
664a13a2f6 |
Add job operation API
This PR is related to https://issues.apache.org/jira/browse/ZEPPELIN-137. I added some notebook job operations. Here is the examples. * Get notebook job status. ``` ]# curl -XGET http://121.156.59.2:8080/api/notebook/job/2B3Z5BXJA {"status":"OK","body":[{"id":"20151121-212654_766735423","status":"FINISHED","finished":"Tue Nov 24 14:21:40 KST 2015","started":"Tue Nov 24 14:21:39 KST 2015"},{"id":"20151121-212657_730976687","status":"FINISHED","finished":"Tue Nov 24 14:21:40 KST 2015","started":"Tue Nov 24 14:21:40 KST 2015"},{"id":"20151121-235508_752057578","status":"FINISHED","finished":"Tue Nov 24 14:21:40 KST 2015","started":"Tue Nov 24 14:21:40 KST 2015"},{"id":"20151121-235900_1491052248","status":"FINISHED","finished":"Tue Nov 24 14:21:41 KST 2015","started":"Tue Nov 24 14:21:40 KST 2015"},{"id":"20151121-235909_1520022794","status":"RUNNING","finished":"Tue Nov 24 02:53:51 KST 2015","started":"Tue Nov 24 14:21:41 KST 2015"}]} ``` * Run notebook job. ``` ]# curl -XDELETE http://121.156.59.2:8080/api/notebook/job/2B3Z5BXJA {"status":"ACCEPTED"} ``` * Stop(Delete) notebook job. ``` ]# curl -XDELETE http://121.156.59.2:8080/api/notebook/job/2B3Z5BXJA {"status":"ACCEPTED"} ``` * Run requested paragraph job. ``` ]# curl -XPOST http://121.156.59.2:8080/api/notebook/job/2B3Z5BXJA/20151121-212654_766735423 {"status":"ACCEPTED"} ``` * Stop(Delete) requested paragraph job. ``` ]# curl -XDELETE http://121.156.59.2:8080/api/notebook/job/2B3Z5BXJA/20151121-212654_766735423 {"status":"ACCEPTED"} ``` Author: astroshim <hsshim@nflabs.com> Author: root <root@ktadm002.(none)> Closes #465 from astroshim/improve/addApis and squashes the following commits: |
||
|
Lee moon soo
|
c2cbafd1d8 |
ZEPPELIN-412 Documentation based on Zeppelin version
https://issues.apache.org/jira/browse/ZEPPELIN-412 To provide documentation based on Zeppelin version, like Spark, Flink project does, it need to separate documentations from website. * docs will be kept in Zeppelin main source tree and being built and published under 'docs' menu on website with specific version number. * website will be kept in gh-pages branch and provides menu for multiple version of docs. This PR removes unnecessary pages, which is provided by website. (for example download page) This is the screenshot after applying this PR   Author: Lee moon soo <moon@apache.org> Closes #430 from Leemoonsoo/ZEPPELIN-412 and squashes the following commits: |
