### What is this PR for?
ZeppelinContext can be used to exchange DataFrames but there are some nasty tricks and typecasts.
It's good to provide some examples.
### What type of PR is it?
Documentation
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mohammad Amin Khashkhashi Moghaddam <amin.moghaddamV@gmail.com>
Closes#1677 from m30m/patch-3 and squashes the following commits:
a039d5c [Mohammad Amin Khashkhashi Moghaddam] Add doc for exchanging data frames
### What is this PR for?
Currently a paragraph can display only a single type of result.
For example
```
print("""
%text textout
%html htmlout
""")
```
will display only last display system detected, "htmlout".
This pr implements multiple results supports, so not only the last, but also all the display systems in a paragraph outputs can be displayed.
To do that, Note.json format need to be changed. from
```
paragraph : [
{
result : {
code: "",
type: "",
msg: ""
},
config : {
graph : {},
...
},
...
},
...
],
...
```
to
```
paragraph : [
{
results : {
{
code: "",
msg: [
{
type: "",
data: ""
},
{
type: "",
data: ""
},
...
]
},
config : {
results : [
{
graph : {},
},
{
graph : {},
},
...
]
},
...
},
...
],
...
```
### What type of PR is it?
Improvement
### Todos
* [x] - Make InterpreterResult and InterpreterOutput support multiple display system
* [x] - Automatic migration old format to new format
* [x] - Render multiple results in front-end
* [x] - Take care Importing old notebook format
* [x] - Make helium framework support multiple results
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-212
### How should this be tested?
run following code in a single spark paragraph.
```
%spark
// default output is text
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// print something in html
println(s"""%html <h3><font style="color:blue">Some HTML</font></h3>""")
// create table
println(s"""%table key\tvalue
sun\t100
moon\t200
""")
// display text again
println("""%text""")
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// another table
Thread.sleep(1000)
println(s"""%table key\tvalue
apple\t100
banana\t200
""")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? yes
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1658 from Leemoonsoo/ZEPPELIN-212 and squashes the following commits:
0c6a13c [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
519cf23 [Lee moon soo] Update PythonInterpreterPandasSqlTest
e6268ac [Lee moon soo] Update PythonInterpreterMatplotlibTest.java
f5034b8 [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
d5981d5 [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
a1fe729 [Lee moon soo] document note format change
282504e [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
7ab6679 [Lee moon soo] update selenium test
6a897a5 [Lee moon soo] Update rest-api doc
cbbd58a [Lee moon soo] update unittest
e89c9b8 [Lee moon soo] restore tutoral note
3ba37b7 [Lee moon soo] Let old version import note without error
d09e03f [Lee moon soo] enable helium only in the last result
6682908 [Lee moon soo] Remove unnecessary listener method
0f1d28f [Lee moon soo] Merge branch 'master' into ZEPPELIN-212
73b3a81 [Lee moon soo] update selenium test
e69f1a1 [Lee moon soo] update test
f12230f [Lee moon soo] Remove unnecessary newline in test
26e201a [Lee moon soo] Update testcase
b01d70f [Lee moon soo] Make helium app work
0a5b4d1 [Lee moon soo] Update HeliumApplicationFactoryTest
d8ca07f [Lee moon soo] update NotebookTest
aaf9778 [Lee moon soo] fix r test
3e43df4 [Lee moon soo] make zeppelin-web test pass
1d6fd3e [Lee moon soo] fix compile errors
0156ffc [Lee moon soo] take care angular object
804768d [Lee moon soo] Take care output streaming
57d6b2f [Lee moon soo] Render multiple results
95b6037 [Lee moon soo] Multiple results
### What is this PR for?
In ZEPPELIN-1607, I'd like refactor livy interpreter to scoped mode by default, this require username when open this interpreter. So I propose to pass username when creating interpreter through thrift.
What I did in this PR.
* update `RemoteInterpreterService.thrift` and regenerate the java thrift code.
* update `genthrift.sh`, otherwise hashCode method won't be generated correctly.
* This is one compilation issue (`PythonDockerInterpreterTest.java`) in the existing master branch, I also fix it here.
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1707
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1679 from zjffdu/ZEPPELIN-1707 and squashes the following commits:
763455f [Jeff Zhang] regenerate it using thrift 0.9.2
a247552 [Jeff Zhang] ZEPPELIN-1707. Pass userName when creating interpreter through thrift
### What is this PR for?
https://github.com/apache/zeppelin/pull/1554 added a `processenduser.md`, but the navigation menu different name on it's link `userimpersonation.html`
This PR changes filename from `processenduser.md` to `userimpersonation.md`.
### What type of PR is it?
Hot Fix
### Todos
* [x] - Change file name
### What is the Jira issue?
https://github.com/apache/zeppelin/pull/1554
### How should this be tested?
build docs and navigate "Interpreter -> Interpreter User Impersonation"
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes#1704 from Leemoonsoo/fix_link and squashes the following commits:
8ede96e [Lee moon soo] Change filename
### What is this PR for?
Currently, the z.run command is restricted.
Only paragraphs in a single note can be executed.
I have modified this to allow you to freely execute paragraphs of other notes.
This PR provides the basis for the freeful use of Zeppelin's resources at each Interpreter implementation.
### What type of PR is it?
Improvement, Feature
### Todos
- [x] extends z.run
- [x] run all paragraph in external note
- [x] run paragraph for external note.
- [x] get resource for zeppelin in each interpreter.
- [x] improve test case.
- [x] how to use docuement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1665
### How should this be tested?
Currently under development.
run paragraph in same note
```
%spark
z.run("paragraphID")
```
run paragraph with external note
```
z.run("noteid", "paragraphid");
```
all note run
```
z.runNote("noteid");
```
### Screenshots (if appropriate)
- paragraph run
- noterun
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: CloverHearts <cloverheartsdev@gmail.com>
Closes#1637 from cloverhearts/extends-zrun-remote-transaction and squashes the following commits:
41fa9d7 [CloverHearts] restore unless changed and import
113b475 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
03a3a2b [CloverHearts] testcase change z.run(2, context) to z.run(2)
2a2c173 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
f2e3bcf [CloverHearts] fix TestCase
5a80a5a [CloverHearts] last test case time check to print string
e6cd82c [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
3862166 [CloverHearts] regenerate thrfit class
5ec4640 [CloverHearts] change defined protocol for thrift
7562535 [CloverHearts] remove unused import and asterisk import
8a54917 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
342752d [CloverHearts] add document for extends z.run and z.runNote
292319a [CloverHearts] add test case for extends z.run and z.runNote
10c2a47 [CloverHearts] Implement runNote and re implement run method
f9661c8 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
9ab05af [CloverHearts] Change structure and remove remoteWorksManager
8cbe46c [CloverHearts] remote remoteworksController in interpreter.java
8d42c16 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
f11fed4 [CloverHearts] Merge branch 'workflow' into extends-zrun-remote-transaction
c074f07 [CloverHearts] fix sio support
4b1ef08 [CloverHearts] fix thrift interface
2628a20 [CloverHearts] fix thrift
6fbe08a [CloverHearts] Merge branch 'master' into workflow
3f75bd5 [CloverHearts] support scald
55e8704 [CloverHearts] support spark r
5a7886f [CloverHearts] fix sio support
afb9db7 [CloverHearts] Merge branch 'master' into workflow
3ed556c [CloverHearts] remove debug console message.
3d34f9e [CloverHearts] Implement getParagraphRunner transaction.
2523238 [CloverHearts] Implement eventForWait class
0570ae8 [CloverHearts] add remote works controller class and include interpreter factory
6e1f219 [CloverHearts] code base workflow for remote zeppelin server control default thrift transaction.
### What is this PR for?
Allow users to issue insert/update/upsert statements from Zeppelin notes
### What type of PR is it?
Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
[ZEPPELIN-1645](https://issues.apache.org/jira/browse/ZEPPELIN-1645)
### How should this be tested?
For Phoenix Interpreter:
Paragraph 1:
%jdbc
UPSERT INTO CX_TEST (ACCT_NBR, HSE_ID) VALUES ('a', 'b')
Paragraph 2:
%jdbc
select count(*) from cx_test
Result: 1
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
Yes - JDBC README updated
Author: Randy Gelhausen <rgelhau@gmail.com>
Closes#1617 from randerzander/master and squashes the following commits:
1cf1ce3 [Randy Gelhausen] Added check for connection autocommit status
### What is this PR for?
This PR is for the multi-tenant of JDBC Interpreter.
User can create a user/password for JDBC account at the [Credential page](http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html).
The `Entity` of `Credential` is match with JDBC interpreter group name.
If the account for JDBC is not setted in the `Interpreter property` then use `Credential`'s.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1567
### How should this be tested?
Please refer to testMultiTenant() of JDBCInterpreterTest/
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1539 from astroshim/jdbc-impersonation and squashes the following commits:
46fce31 [astroshim] add explanation of InterpreterGroup
7a92236 [astroshim] fix doc and remove persist value.
63f5ea7 [astroshim] Merge branch 'master' into jdbc-impersonation
267277a [astroshim] rebase
649ff6e [astroshim] rebase
872fb49 [astroshim] fix ScioInterpreterTestCase
4387a5b [astroshim] Merge branch 'master' into jdbc-impersonation
47c463f [astroshim] update doc and html
d4eb178 [astroshim] fix docs
59aa9ff [astroshim] Merge branch 'master' into jdbc-impersonation
bf61afd [astroshim] fix testcase
5c0f5d7 [astroshim] rebase
79ba25b [astroshim] Merge branch 'master' into jdbc-impersonation
1f9c2c0 [astroshim] clean redundant code
a2f5687 [astroshim] fix impersonation
9962181 [astroshim] fix InterpreterOutput of PySparkInterpreterTest case
b55aceb [astroshim] Merge branch 'master' into jdbc-impersonation
24a8226 [astroshim] fix doc
086dfda [astroshim] fix testcase
34fe0a6 [astroshim] fix code for more simple.
fee7086 [astroshim] fix build error.
a305eca [astroshim] Merge branch 'master' into jdbc-impersonation
df80741 [astroshim] documentation for credential.
df1b1dc [astroshim] rebase and entity name convention.
63d6a1c [astroshim] change thrift version to 0.9.2
6573c1c [astroshim] change variable name
f311f34 [astroshim] fix typo
722e333 [astroshim] change testcase name
9161937 [astroshim] clean code
3dafdf0 [astroshim] add testcase
373d5f1 [astroshim] pass replName to Interpreter and use credential info for jdbc auth.
### What is this PR for?
Inspired by ZEPPELIN-1671 conda interpreter.
Docker can provides kind of virtual environment for python like conda does.
This PR implements %python.docker interpreter that helps run python process in docker container.
This PR implements feature on top of https://github.com/apache/zeppelin/pull/1645
### What type of PR is it?
Feature
### Todos
* [x] - basic feature
* [x] - unittest
* [x] - documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1683
### How should this be tested?
see screenshot
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1654 from Leemoonsoo/pydocker and squashes the following commits:
22507e6 [Lee moon soo] Add new line at the end of the file
41c09d9 [Lee moon soo] Run python process in docker container
### What is this PR for?
Conda interpreter that manages conda environment for PythonInterpreter
### What type of PR is it?
Feature
### Todos
* [x] - Basic impl
* [x] - update doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1671
### How should this be tested?
Recreate(or create new) your python interpreter setting in gui.
List all conda env
```
%python.conda
```
Activate env
```
%python.conda activate [name]
```
Deactivate env
```
%python.conda deactivate
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1645 from Leemoonsoo/conda and squashes the following commits:
4842b0a [Lee moon soo] Add usage in doc
d979c6a [Lee moon soo] Add unittest
b889443 [Lee moon soo] add usage template
9ae553b [Lee moon soo] Format output and add usage command
171cbeb [Lee moon soo] make sure single char interpreter name can be parsed
6b9525f [Lee moon soo] Fix unittest
1223796 [Lee moon soo] Remove unnecessary log
394cf8c [Lee moon soo] Conda interpreter implementation
### What is this PR for?
%dep interpreter was going to be deprecated, but we had feedback from many users that this feature provide different advantage over dependency loading via GUI so we want to keep supporting it. This PR remove deprecated message when users use %dep interpreter.
### Todos
- [x] Fix test
### What type of PR is it?
Documentation
### What is the Jira issue?
[ZEPPELIN-1622](https://issues.apache.org/jira/browse/ZEPPELIN-1622)
### How should this be tested?
Run %dep interpreter such as `z.load('/your/library.jar)` and see if deprecated message is gone.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mina Lee <minalee@apache.org>
Closes#1605 from minahlee/ZEPPELIN-1622 and squashes the following commits:
b604c4c [Mina Lee] Add %dep z.load integration test
e3ae4c8 [Mina Lee] Remove %dep interpreter deprecated message
### What is this PR for?
Due to the lack of this missing API, some front-end code retrieve all settings. This is inefficient.
We can modify those codes by adding this API.
Also, i refactored the whole `InterpreterRestApiTest`
### What type of PR is it?
[Feature | Documentation | Refactoring]
### What is the Jira issue?
[ZEPPELIN-1684](https://issues.apache.org/jira/browse/ZEPPELIN-1684)
### How should this be tested?
You can run unit test `InterpreterRestApiTest`
### Screenshots (if appropriate)
Updated doc screenshots
<img width="900" alt="new_api_doc" src="https://cloud.githubusercontent.com/assets/4968473/20423419/74eeb2a0-adb3-11e6-9a69-58e33457d514.png">
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - YES, but done.
Author: 1ambda <1amb4a@gmail.com>
Closes#1655 from 1ambda/feat/create-interpreter-get-api and squashes the following commits:
89ff7fd [1ambda] style: Apply zeppelin checkstyle xml
c901287 [1ambda] docs: Add new api to rest-interpreter.html
2641a73 [1ambda] refactor: InterpreterRestApiTest
e63c80e [1ambda] refactor: testSettingsCRUD
9bc24fd [1ambda] feat: Add /interpreter/setting/id API and test
### What is this PR for?
In multi-user environment when users create a notebook normally they would expect that notebook be listed only in their list/workbench. Currently Zeppelin creates/imports notes as public by default. There should be at least a way to pass through configuration to make this behaviour private be default. Should discuss whether it's public or private by default.
### What type of PR is it?
Improvement
### Todos
* [x] - set permissions on create/import
* [x] - test
* [x] - review, feedback, decide whether public/private by default
* [x] - pass through env. config
### What is the Jira issue?
[ZEPPELIN-1657](https://issues.apache.org/jira/browse/ZEPPELIN-1657)
### How should this be tested?
1. set `zeppelin.notebook.public` property as false in `zeppelin-site.xml`
2. login as user1 and create noteA, note should appear in your list of notes
3. logout and login as user2, shouldn't be able to see noteA
### Screenshots (if appropriate)
TBD
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? maybe
Author: Khalid Huseynov <khalidhnv@gmail.com>
Closes#1625 from khalidhuseynov/feat/public-note-create and squashes the following commits:
010c262 [Khalid Huseynov] fix test: set state back
fc91b2f [Khalid Huseynov] add description to install config
f862ae1 [Khalid Huseynov] fix InterpreterFactory test
876b798 [Khalid Huseynov] fix vfs testSave npe
1a945e8 [Khalid Huseynov] add test
496b80c [Khalid Huseynov] read env var from conf file directly
28abffa [Khalid Huseynov] set permissions on note create and import
44297a4 [Khalid Huseynov] set new note permissions
9732409 [Khalid Huseynov] add variable to ZeppelinConfiguration
4062af0 [Khalid Huseynov] add var to env.sh
b7f28b3 [Khalid Huseynov] add property to site.xml
### What is this PR for?
Currently `{{note.id}}>{{noteName(note)}}` is not shown in `notebookashomepage.md`. So I added markdown escaping for this.
### What type of PR is it?
Documentation
### What is the Jira issue?
no Jira issue for this
### How should this be tested?
Please see the below screenshot imgs :)
### Screenshots (if appropriate)
- Before
<img width="797" alt="screen shot 2016-11-15 at 9 22 15 am" src="https://cloud.githubusercontent.com/assets/10060731/20298331/9b2e8828-ab15-11e6-9010-50df6100a961.png">
- After
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Closes#1634 from AhyoungRyu/add-markdown-escaping and squashes the following commits:
34a4bc7 [AhyoungRyu] Add markdown escaping to find some missing parts
Have recreated this from https://github.com/apache/zeppelin/pull/1322
### What is this PR for?
While running a Notebook using shell, spark, python uses same user as which zeppelin server is running. Which means these interprets have same permission on file system as zeppelin server.
IMO users should be able to impersonate themselves as a complete security system.
### What type of PR is it?
[Improvement]
### Todos
- [x] - Update doc
- [x] - FIX NPEs
- [x] - FIX CI
### What is the Jira issue?
- [ZEPPELIN-1320](https://issues.apache.org/jira/browse/ZEPPELIN-1320)
### How should this be tested?
- Enable shiro auth in shiro.ini
- Add ssh key for the same user you want to try and impersonate (say user1).
```
adduser user1
ssh-keygen
ssh user1localhost mkdir -p .ssh
cat ~/.ssh/id_rsa.pub | ssh user1localhost 'cat >> .ssh/authorized_keys'
```
- Start zeppelin server, try and run following in paragraph in a notebook
- Go to interpreter setting page, and enable "User Impersonate" in any of the interpreter (in my example its shell interpreter)
```
%sh
whoami
```
Check that it should run as new user, i.e. "user1"
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? yes
Author: Prabhjyot Singh <prabhjyotsingh@gmail.org>
Closes#1554 from prabhjyotsingh/ZEPPELIN-1320-2 and squashes the following commits:
dc69c9d [Prabhjyot Singh] @Leemoonsoo review comment: making ZEPPELIN_SSH_COMMAND configurable
1b26cc0 [Prabhjyot Singh] add doc
5a76839 [Prabhjyot Singh] show User Impersonate only when interpreter setting is "per user" and "isolated"
02c3084 [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1320-2
03b2f20 [Prabhjyot Singh] use user instead of ""
0ff80ec [Prabhjyot Singh] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1320-2
dd0731d [Prabhjyot Singh] fix missing test cases
aff1bf0 [Prabhjyot Singh] user should have option to run these interpreters as different user.
### What is this PR for?
This adds note to configuration to make clear about `sparkmaster` hostname defined in `/etc/hosts`
### What type of PR is it?
Improvement | Documentation
### Todos
* [x] - add note
### What is the Jira issue?
n/a
### How should this be tested?
see documentation
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Khalid Huseynov <khalidhnv@gmail.com>
Closes#1636 from khalidhuseynov/docs/spark-yarn-add-note-hostname and squashes the following commits:
9648e08 [Khalid Huseynov] address feedback, fix path
78e7907 [Khalid Huseynov] add note about sparkmaster to each section
887b499 [Khalid Huseynov] update docker file path yarn mode
9e4ad68 [Khalid Huseynov] add sparkmaster hostname note
### What is this PR for?
Fix formatting/typos/style in scio doc
### What type of PR is it?
Documentation
Author: Rafal Wojdyla <rav@spotify.com>
Closes#1633 from ravwojdyla/fix_scio_doc and squashes the following commits:
888ec68 [Rafal Wojdyla] Fix formatting/typos/style in scio doc
### What is this PR for?
Remove unmanaged, old library markdown4j dep which exists just for Websequence and YUML plugins.
(related to https://github.com/apache/zeppelin/pull/1384)
By adding Websequence and YUML plugins to pegdown
- Removing markdown4j dependency which is unmanaged library currently.
- Addtionally, we can remove `markdown.parser.type` options in **markdown interpreter**
- Fixing some bugs in Websequence and YUML plugins
- Enable others to add more plugins using pegdown sytnax.
### What type of PR is it?
Improvement
### Todos
Nothing
### What is the Jira issue?
[JIRA - ZEPPELIN-1614](https://issues.apache.org/jira/browse/ZEPPELIN-1614)
### How should this be tested?
Some functional tests are included.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? - YES
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - YES
Author: 1ambda <1amb4a@gmail.com>
Closes#1594 from 1ambda/feat/remove-markdown4j-dep and squashes the following commits:
5af1607 [1ambda] fix: Failed tests in InterpreterRestApiTest
c57fdcb [1ambda] docs: Update markdown.md
5c62236 [1ambda] docs: Update upgrade.md for '%md'
a1e779d [1ambda] style: Use zeppelin checkstyle.xml
13e0dc4 [1ambda] Update: interpreter setting and docs
de3549d [1ambda] chore: Cleanup duplicated markdown4j license
7c5d41e [1ambda] fix: Parse style param optionally in webseq
8831ca1 [1ambda] fix: Wrap exceptions in catch stmt
9268695 [1ambda] Revert "fix: Cleanup unused Markdown4j Parser"
33fb800 [1ambda] Revert "docs: Remove markdown.parser.type option"
fddc459 [1ambda] Revert "chore: Remove markdown4j dep and update license"
a59ebbd [1ambda] Revert "fix: Set {} to avoid 503"
4e48933 [1ambda] Revert "fix: Parse style param optionally in webseq"
8cfb2c8 [1ambda] Revert "fix: style and misspell in docs"
73956e0 [1ambda] Revert "fix: Propagate exception in YUML plugin"
1b7787f [1ambda] fix: Propagate exception in YUML plugin
c656d08 [1ambda] fix: style and misspell in docs
dc4f110 [1ambda] fix: Parse style param optionally in webseq
b43e14e [1ambda] fix: Set {} to avoid 503
c48cc53 [1ambda] chore: Remove markdown4j dep and update license
81fdfcc [1ambda] docs: Remove markdown.parser.type option
cf19f0b [1ambda] fix: Cleanup unused Markdown4j Parser
98b2809 [1ambda] fix: Add missing docs
3e9716d [1ambda] feat: Yuml markdown plugin
3247c67 [1ambda] feat: Support webseq markdown plugin
### What is this PR for?
Add 2 JDBC configs and useful links to JDBC doc so that users can connect those backends easily :)
### What type of PR is it?
Documentation
### Todos
* [ ] - Task
### What is the Jira issue?
[ZEPPELIN-1650](https://issues.apache.org/jira/browse/ZEPPELIN-1650)
### How should this be tested?
See screenshots.
### Screenshots (if appropriate)
<img width="706" alt="example_redshift" src="https://cloud.githubusercontent.com/assets/4968473/20208653/01b1e468-a833-11e6-9166-b05d15e9a5fe.png">
<img width="577" alt="example_mariadb" src="https://cloud.githubusercontent.com/assets/4968473/20208654/02b49a22-a833-11e6-9b29-5a61f2b6a4bb.png">
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Closes#1621 from 1ambda/jdbc-doc-improve and squashes the following commits:
0aab878 [1ambda] docs: Update postgres img ;)
6009eb9 [1ambda] docs: Add postgres section, hive2 doc link
ce24af1 [AhyoungRyu] Make mariadb and redshift screenshots same with others
2fb5db1 [1ambda] docs: Add mariadb, redshift configs
### What is this PR for?
This is document for Apache Kylin Interpreter for Apache Zeppelin
### What type of PR is it?
Documentation
### Todos
Done
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-415
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
This is document
Author: zhongjian <jiatuer@163.com>
Author: Jason <jiatuer@163.com>
Closes#1576 from janzhongi/master and squashes the following commits:
5464006 [zhongjian] add default value for kylin query api config
c26fa77 [Jason] ZEPPELIN-415 document for Apache Kylin Interpreter
### What is this PR for?
Doc should deliver key features and recommended usage more simple and easy way.
- docs/install/install.md has lots of duplicated section with README.md.
- docs/install/install.md includes install from binary as well as build from source. I've seen that makes some beginners try download binary and then source build it again.
- recommended and key usage need to be highlighted.
- Be less verbose in key instructions. Move optional, additional info from in the middle of key instruction to end of the each page.
### What type of PR is it?
Improvement
### Todos
* [x] - improve doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1644
### How should this be tested?
Run doc locally
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes#1615 from Leemoonsoo/ZEPPELIN-1644 and squashes the following commits:
e554216 [Lee moon soo] Add build item in index.md
c6b25f8 [Lee moon soo] Update suggested build command.
c7a19eb [Lee moon soo] Add build.md
e59fe3f [Lee moon soo] make document easier to follow key instructions
### What is this PR for?
This PR fixes Apache Ignite version mentions in interpreter documentation.
### What type of PR is it?
[Improvement]
Author: agura <agura@gridgain.com>
Author: agura <andreyn.gura@gmail.com>
Closes#1581 from agura/ignite-docs and squashes the following commits:
59fae20 [agura] Interpreter binding information link is fixed
33c9cac [agura] Minor fixes
d8710b3 [agura] Ignite versions fixed
### What is this PR for?
This PR adds support for formatting math formula formatting in %html display system using MathJax library.
### What type of PR is it?
Feature
### Todos
* [x] - Format math formula with MathJax library
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-777
### How should this be tested?
try run following codes.
```
%md
When \\(a \\ne 0\\), there are two solutions to \\(ax^2 + bx + c = 0\\) and they are
$$x = {-b \pm \sqrt{b^2-4ac} \over 2a}.$$
```
Note MathJax works better with `markdown.parser.type` property set `pegdown`, in markdown interpreter. With default markdown4j parser, some formula is not well displayed. (for example, ax^2) I think this will not be a big problem because of we'll remove markdown4j https://github.com/apache/zeppelin/pull/1594
```
%sh echo -e "%html \$\$a = b\$\$"
```
```
%spark println("%html $$b = c$$")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? yes
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1606 from Leemoonsoo/ZEPPELIN-777 and squashes the following commits:
af8e079 [Lee moon soo] Package MathJax resources
2afedde [Lee moon soo] Fix typo
dd02bec [Lee moon soo] Add doc for mathmetical expression
174d7ad [Lee moon soo] Add license
bb762c3 [Lee moon soo] Format formula using MathJax
### What is this PR for?
This PR adds new end point for updating paragraph configuration
### What type of PR is it?
Improvement | Documentation
### What is the Jira issue?
[ZEPPELIN-1029](https://issues.apache.org/jira/browse/ZEPPELIN-1029)
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Mina Lee <minalee@apache.org>
Closes#1592 from minahlee/ZEPPELIN-1029 and squashes the following commits:
c8357ca [Mina Lee] Add rest api for paragraph config update
### What is this PR for?
This PR adds Mahout functionality for the Spark Interpreter.
### What type of PR is it?
Improvement
### Todos
- [x] Implement Mahout Interpreter in Spark
- [x] Add Unit Tests
- [x] Add Documentation
- [x] Add Example Notebook
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-116
### How should this be tested?
Open a Spark Notebook with Mahout enabled and run a few simple commands using the R-Like DSL and Spark Distributed Context (Mahout Specific)
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update?
No
- Is there breaking changes for older versions?
No
- Does this needs documentation?
Yes
Author: rawkintrevo <trevor.d.grant@gmail.com>
Closes#928 from rawkintrevo/mahout-terp and squashes the following commits:
ed6eff0 [rawkintrevo] [ZEPPELIN-116] renamed add_mahout_interpreters.py and overwrite_existing feature
e7d4e12 [rawkintrevo] [ZEPPELIN-116] Made add_mahout.py script more resilient
7e83832 [rawkintrevo] [ZEPPELIN-116] Add Mahout Interpreters
### What is this PR for?
This PR adds [PAM](https://en.wikipedia.org/wiki/Pluggable_authentication_module) authentication support based on the introduced Shiro security implementation. With PAM support system users have immediate access to a secured Zeppelin instance.
### What type of PR is it?
Feature
### Todos
* [x] - Create PAM realm
* [x] - Create test for PAM authentication
* [x] - Test with running Zeppelin instance
### What is the Jira issue?
[ZEPPELIN-1611](https://issues.apache.org/jira/browse/ZEPPELIN-1611])
### How should this be tested?
`PamRealmTest` executes an automated test if the environment variables `PAM_USER` and `PAM_PASS` are set. This should be set to system username and password.
The test also includes a main function to manually execute the test. Setting the environment variables for example on MacOS for your IDE use `launchctl setenv PAM_USER user` and `launchctl setenv PAM_PASS xxxxx`, the test can then be run from your IDE.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: hkropp <hkropp@hortonworks.com>
Closes#1589 from hkropp/ZEPPELIN-1611 and squashes the following commits:
bc01862 [hkropp] [Zeppelin-1611] - Support PAM (System User) Authentication
### What is this PR for?
This PR is the first of two major steps needed to improve matplotlib integration in Zeppelin (ZEPPELIN-1344). The latter, which is a plotting backend with fully interactive tools enabled, will be done afterwards in a separate PR. This PR specifically for automatically displaying output from calls to matplotlib plotting functions inline with each paragraph. Thanks to the addition of post-execute hooks (ZEPPELIN-1423), there is no need to call any `show()` function to display an inline plot, just like in Jupyter.
### What type of PR is it?
Improvement
### Todos
The main code has been written and anyone who reads this is encouraged to test it, but there are a few minor todos:
- [x] - Add unit tests
- [x] - Add documentation
- [x] - Add screenshot showing iterative plotting with angular mode
### What is the Jira issue?
[ZEPPELIN-1345](https://issues.apache.org/jira/browse/ZEPPELIN-1345)
### How should this be tested?
In a pyspark or python paragraph, enter and run
``` python
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
```
The plot should be displayed automatically without calling any `show()` function whatsoever. A special method called `configure_mpl()` can also be used to modify the inline plotting behavior. For example,
``` python
z.configure_mpl(close=False, angular=True)
plt.plot([1, 2, 3])
```
allows for iterative updates to the plot provided you have PY4J installed for your python installation (which of course is always the case if you use pypsark). To clarify, this feature only currently works with pyspark (not python as there are no `angularBind()` and `angularUnbind()` methods yet). Doing something like:
```
plt.plot([3, 2, 1])
```
will update the plot that was generated by the previous paragraph by leveraging Zeppelin's Angular Display System. However, by setting `close=False`, matplotlib will no longer automatically close figures so it is now up to the user to explicitly close each figure instance they create. There's quite a bit more options for `z.configure_mpl()`, but I will save that discussion for the documentation.
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? No
- Is there breaking changes for older versions? No
- Does this needs documentation? Yes
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1534 from agoodm/ZEPPELIN-1345 and squashes the following commits:
9ef6ff7 [Alex Goodman] Move mpl backend files to /interpreter
24f89c6 [Alex Goodman] Catch potential NullPointerExceptions from hook registry
bdb584e [Alex Goodman] Make sure expressions are printed when no plots are shown
22b6fe4 [Alex Goodman] Remove unused variable
d3d1aa0 [Alex Goodman] Fix CI test failure
c90d204 [Alex Goodman] Update spark.md
bcf0bf3 [Alex Goodman] Update python.md for new matplotlib integration
c9b65a5 [Alex Goodman] Add iterative plotting example image
8029a05 [Alex Goodman] Update python/README.md
f2d9e86 [Alex Goodman] Exclude tests are excluded in python/pom.xml
86b1c90 [Alex Goodman] Fix tutorial notebook not loading
c37b00f [Alex Goodman] Fix legend in tutorial notebook
a321d79 [Alex Goodman] Update python.md
82350e3 [Alex Goodman] Update matplotlib tutorial notebook
9792f97 [Alex Goodman] Add unit tests
8b9b973 [Alex Goodman] Fix NullPointerExceptions in unit tests
82135ad [Alex Goodman] Removed unused variable
f9c9498 [Alex Goodman] Added support for Angular Display System
edf750a [Alex Goodman] Add new matplotlib backend for python/pyspark interpreters
### What is this PR for?
- Enables removing note and clear all paragraph's output from Zeppelin main page.
- Add rest api for clearing all paragraph output
Next possible improvement can be removing notes in folder level and rename folder.
### What type of PR is it?
Improvement
### Todos
* [x] - Merge #1567 and apply security to `clearAllParagraphOutput` rest api method
### What is the Jira issue?
[ZEPPELIN-1564](https://issues.apache.org/jira/browse/ZEPPELIN-1564)
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? yes
Author: Mina Lee <minalee@apache.org>
Closes#1565 from minahlee/ZEPPELIN-1564 and squashes the following commits:
749aebe [Mina Lee] Merge branch 'master' of https://github.com/apache/zeppelin into ZEPPELIN-1564
1393ee9 [Mina Lee] Rename class name from UnauthorizedException to ForbiddenException Update clear output rest api doc response code
2ee452e [Mina Lee] Add auth check before clearing all paragraph
fb7e6ae [Mina Lee] Merge branch 'master' of https://github.com/apache/zeppelin into ZEPPELIN-1564
f349dbf [Mina Lee] Change post to put
7eb3521 [Mina Lee] Give writer permission to clear output
dea3ef6 [Mina Lee] Remove unused import
d66600c [Mina Lee] Add rest api endpoint for clear paragraph result to document
3d19141 [Mina Lee] Add rest api for clear all paragraph result and add test
98d7604 [Mina Lee] Add clearAllParagraphOutput unit test
4adddb4 [Mina Lee] Clear all paragraphs and remove note from main page
### What is this PR for?
removing support on old versions of Spark including testing and building them.
### What type of PR is it?
[Feature]
### Todos
* [x] - Remove .travis.yml
* [x] - Remove pom.xml
* [x] - Remove some docs
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1599
### How should this be tested?
No test. Check travis simplified
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? You cannot use spark from 1.1 to 1.3 any longer
* Does this needs documentation? Yes, should remove some docs
Removed some profiles concerning old versions of Spark
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#1578 from jongyoul/ZEPPELIN-1599 and squashes the following commits:
acf514f [Jongyoul Lee] Fixed the script not for recognizing old versions
4bc11d6 [Jongyoul Lee] Added some docs for the deprecation on support for old versions of Spark
207502d [Jongyoul Lee] Removed some tests for old versions of Spark Removed some profiles concerning old versions of Spark
### What is this PR for?
It's a bit misleading that both regular webserver location and websocket location use the exact same address. It's better to specify that websocket requests should be proxy-passed to `/ws` subdirectory.
### What type of PR is it?
Documentation
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mohammad Amin Khashkhashi Moghaddam <amin.moghaddamV@gmail.com>
Closes#1573 from m30m/patch-2 and squashes the following commits:
aa60fd5 [Mohammad Amin Khashkhashi Moghaddam] Add `/ws` for nginx websocket configuration
### What is this PR for?
A simple update of the documentation to let user know how that they need to copy `shiro.ini.template` to `shiro.ini`
### What type of PR is it?
[Documentation]
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: Anthony Corbacho <corbacho.anthony@gmail.com>
Closes#1571 from anthonycorbacho/doc/ShiroTemplate and squashes the following commits:
2f73f56 [Anthony Corbacho] Fix typo :: copy -> cp ^^
bcba1db [Anthony Corbacho] Fix typo number 2 Yo
f933a6a [Anthony Corbacho] Fix typo
3f11260 [Anthony Corbacho] Update shiro part about coping shiro.ini.template to shiro.ini in conf
### What is this PR for?
This PR is for usability of restarting interpreter.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1306
### How should this be tested?
You can restart interpreter on the interpreter binding page on a note.
and can only restart `scoped` and `isolated` interpreter.
Please refer to screen shot.
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1302 from astroshim/ZEPPELIN-1306 and squashes the following commits:
186a361 [astroshim] rollback noteId check null
f38231c [astroshim] noteId Preconditions
af8a2d8 [astroshim] modify null check to Preconditions
72f3e4f [astroshim] fix testcase.
4dda64b [astroshim] rebase
a626188 [astroshim] rebase and fixing testcase
fc1c819 [astroshim] Merge branch 'ZEPPELIN-1306' of https://github.com/astroshim/zeppelin into ZEPPELIN-1306
499aa6b [astroshim] add PySparkInterpreter testcase
e8e0c17 [astroshim] rebase
36f5642 [astroshim] add InterpreterForNote testcase
fa45bb5 [astroshim] remove debug log
5c4b32a [astroshim] Merge branch 'master' into ZEPPELIN-1306
4c4339f [astroshim] Merge branch 'master' into ZEPPELIN-1306
72a2259 [astroshim] add testcase and fix the doc.
3127154 [astroshim] Merge branch 'master' into ZEPPELIN-1306
0453402 [astroshim] Merge branch 'master' into ZEPPELIN-1306
ecfa7fb [astroshim] remove dud code.
78fdd74 [astroshim] update alert message.
aa30b39 [astroshim] inactive link when interpreter is not selected.
123882d [astroshim] change button to icon
a980500 [astroshim] update html
f9ea386 [astroshim] fix code style
2cea65b [astroshim] change method name
50c6acf [astroshim] delete unnecessary code
29966a0 [astroshim] change UI and restart interpreter process
322d427 [astroshim] Merge branch 'master' into ZEPPELIN-1306
e698e6f [astroshim] change method name
03bdd05 [astroshim] Merge branch 'master' into ZEPPELIN-1306
4dbe05a [astroshim] change argument string to boolean
439b361 [astroshim] fix js code style.
5ad4503 [astroshim] Interpreter restarts on a note.
### What is this PR for?
This PR enables edit on double click for markdown/angular paragraph. Users can change `editOnDblClick` field to be `false` by editting `interpreter/md/interpreter-setting.json` or `conf/interpreter.json`. In the same context, users can set other type paragraphs to be editable on double click by setting `editOnDblClick` to be true.
This PR also fixes bug that syntax highlight doesn't work on pasted code.
### What type of PR is it?
Feature
### Todos
* [x] Create test
* [x] Update docs
### What is the Jira issue?
[ZEPPELIN-1566](https://issues.apache.org/jira/browse/ZEPPELIN-1566)
### How should this be tested?
1. Create new markdown interpreter
2. Create notebook and run markdown paragraph
3. Double click markdown paragraph to edit
### Screenshots (if appropriate)
**Edit on double click and hide editor on paragraph run**
**Syntax highlight on paste**
Before
After
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Mina Lee <minalee@apache.org>
Closes#1540 from minahlee/ZEPPELIN-1566 and squashes the following commits:
89ea628 [Mina Lee] Remove redundant code
4ffc446 [Mina Lee] Enable focus first paragraph on note reload
c566706 [Mina Lee] Check null condition of editor setting and return default setting
38d5e35 [Mina Lee] Update document about how to make paragraph editable on double click
b42039e [Mina Lee] Add integration test for editOnDblClick
0a26207 [Mina Lee] Add editOnDblClick field in interpreter-setting.json
93abe6a [Mina Lee] Make paragraph editable on doubleclick if editOnDblClick set true in interpreter-setting.json
359dc0b [Mina Lee] Split getAndSetEditorSetting method into getEditorSetting and setEditorLanguage method
### What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html
### What type of PR is it?
[ Documentation ]
### Todos
* [x] - Task
### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/
* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533]
### How should this be tested?
Outline the steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
spelling error
responce => response
seprated => separated
Author: terrylee <hunylee@users.noreply.github.com>
Author: Mina Lee <minalee@apache.org>
Closes#1544 from hunylee/patch-1 and squashes the following commits:
1095a19 [terrylee] Merge pull request #1 from minahlee/fix_typo
9c26998 [terrylee] correct separated
1a3e187 [Mina Lee] fix typo
0490435 [terrylee] spelling error
### What is this PR for?
This PR fixes wrong written NotebookID to NoteID.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1549
### Questions:
* Does the licenses files need update? No.
* Is there breaking changes for older versions? No.
* Does this needs documentation? No
Author: hyonzin <hyeonjin507@gmail.com>
Author: 정현진 <hyeonjin507@gmail.com>
Author: Mina Lee <minalee@apache.org>
Closes#1518 from hyonzin/ZEPPELIN-1549 and squashes the following commits:
2c5d461 [hyonzin] fix pullNoteID to pullNoteId
f843abd [hyonzin] Fix missed line
22aecb3 [hyonzin] Merge branch 'master' of https://github.com/apache/zeppelin into ZEPPELIN-1549
ac03666 [정현진] Merge pull request #1 from minahlee/ZEPPELIN-1549
8b3fffd [Mina Lee] Change notebook to note and fix indentation
000605f [hyonzin] Change clonedNotebookId to clonedNoteId
496695c [hyonzin] Change noteID to noteId
1e87463 [hyonzin] Remove tab indent
5647d37 [hyonzin] Rebase and solve conflicts
09bacd8 [hyonzin] Fix more lines unchanged
070bc2d [hyonzin] fix more in ZeppelinRestApiTest.java
24822a3 [hyonzin] Fix more code not changed (notebookIndex to noteSearchService)
4b4e1e8 [hyonzin] Fix detail (function's name) & Change some placeholder
429203d [hyonzin] Fix details & convention to camel
5fa270d [hyonzin] pull upstream master & fix some details
294bea5 [hyonzin] Fix some wrong written term: Notebook -> Note
cc0d315 [hyonzin] Change NotebookID variable name to NoteID
### What is this PR for?
Add missing TOC(Table of Contents) in install_with_flink_and_spark_cluster.md
### What type of PR is it?
Documentation
### What is the Jira issue?
no Jira issue for this
Author: oganizang <jangow77@gmail.com>
Closes#1543 from oganizang/master and squashes the following commits:
832410a [oganizang] First open source commit
### What is this PR for?
Fix minor typos in docs
### What type of PR is it?
Documentation
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: k <K@kui-MacBook-Pro.local>
Author: Mina Lee <minalee@apache.org>
Author: laesunk <laesunk@gmail.com>
Closes#1542 from laesunk/fix_typo and squashes the following commits:
7b703a7 [laesunk] Merge pull request #1 from minahlee/fix_typo
28fc9ce [Mina Lee] Fix more typos
a55ae4a [Mina Lee] Restore wrong changes during rebase
190ac05 [k] fix words
01ce71b [k] fix likly
### What is this PR for?
See #1439. This is a second attempt based on prior feedback, particularly from Leemoonsoo who advised that it would be better to contain all of the code inside of the Interpreter Process. Aside from this, the major changes since the previous PR are:
- The terminology "hooks" is used instead of "callbacks". Since the information being passed to the interpreter isn't actually a "callback", strictly speaking, the more general term "hook" is probably a better choice. All variable / class names were updated correspondingly.
- The registry is now keyed by class name rather than the `replName` specified by the magic on top of the paragraph. The interface provided by `ZeppelinContext` allows for a `replName` to be converted to `className` through an internal mapping.
- Two new event codes, `PRE_EXEC_DEV` and `POST_EXEC_DEV` have been added. This is primarily intended to separate use cases for the interpreter maintainers (via subclasses of `Interpreter`) and the notebook users (via the `ZeppelinContext` class), as otherwise user attempts at registering a hook would overwrite that set by the interpreter maintainer.
- Global scope for hook registration is supported for the developer use cases.
### What type of PR is it?
New Feature
### What is the Jira issue?
[ZEPPELIN-1423](https://issues.apache.org/jira/browse/ZEPPELIN-1423)
### How should this be tested?
In a new note, add the following lines of code to a paragraph:
```python
%pyspark
z.registerHook("post_exec", "print 'This code should be executed before the paragraph code!'")
z.registerHook("pre_exec", "print 'This code should be executed after the paragraph code!'")
```
Then run any other paragraph in the note containing some other code, eg
```python
%pyspark
print "This code should be entered into the paragraph by the user!"
```
The output should be:
```
This code should be executed before the paragraph code!
This code should be entered into the paragraph by the user!
This code should be executed after the paragraph code!
```
You should also test out the other two methods (`getCallback()` and `unregisterCallback()`) specified in `ZeppelinContext.java`.
One final caveat that should be mentioned: If there are errors in the code you specify for a pre-execute event, it will render the interpreter useless since the current implementation prepends the the code specified in `pre_exec` directly to the paragraph entered code before calling `interpret()`. The current workaround for this would be to either restart the interpreter group or call `unregisterHook()` via a different REPL within the interpreter group (eg, `z.unregisterHook("pre_exec", "pyspark")` from the spark interpreter). I would appreciate if anyone here would be willing to share any better approaches here.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1470 from agoodm/ZEPPELIN-1423v2 and squashes the following commits:
56ede60 [Alex Goodman] Automatically detect default interpreter for registerHook()
044a99d [Alex Goodman] Ensure that registered hooks are applied after call to open()
1331fe1 [Alex Goodman] Update interpreters.md
07cac65 [Alex Goodman] Implemented user-defined hook registry system for spark/pyspark interpreters
8fad936 [Alex Goodman] Added Interpreter Hooks to Interpreter Process
### What is this PR for?
In document page(http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/), the search button goes to next line when display's width shortens.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1552
### Screenshots (if appropriate)
<img width="927" alt="2016-10-15 2 04 00" src="https://cloud.githubusercontent.com/assets/6567102/19398321/0d76a1c4-9287-11e6-86e0-9f120c00b143.png">
when (768px ≤ width < 992px)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: hyonzin <hyeonjin507@gmail.com>
Author: AhyoungRyu <ahyoungryu@apache.org>
Author: 정현진 <hyeonjin507@gmail.com>
Closes#1525 from hyonzin/ZEPPELIN-1552 and squashes the following commits:
f1d97ef [AhyoungRyu] Fix search icon's placement in small browser
244992d [정현진] Straighten wrong term
97d1bdf [hyonzin] Fix search link button to not goes next line when display's width shortens
### What is this PR for?
Based on #338 , I refactor most of pig interpreter. As I don't think the approach in #338 is the best approach. In #338, we use script `bin/pig` to launch pig script, it is different to control that job (hard to kill and get progress and stats info). In this PR, I use pig api to launch pig script. Besides that I implement another interpreter type `%pig.query` to leverage the display system of zeppelin. For the details you can check `pig.md`
### What type of PR is it?
[Feature]
### Todos
* Syntax Highlight
* new interpreter type `%pig.udf`, so that user can write pig udf in zeppelin directly and don't need to build udf jar manually.
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-335
### How should this be tested?
Unit test is added and also manual test is done
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Author: Ali Bajwa <abajwa@hortonworks.com>
Author: AhyoungRyu <ahyoungryu@apache.org>
Author: Jeff Zhang <zjffdu@gmail.com>
Closes#1476 from zjffdu/ZEPPELIN-335 and squashes the following commits:
73a07f0 [Jeff Zhang] minor update
a1b742b [Jeff Zhang] minor update on doc
e858301 [Jeff Zhang] address comments
c85a090 [Jeff Zhang] add license
58b4b2f [Jeff Zhang] minor update of docs
1ae7db2 [Jeff Zhang] Merge pull request #2 from AhyoungRyu/ZEPPELIN-335/docs
fe014a7 [AhyoungRyu] Fix docs title in front matter
df7a6db [AhyoungRyu] Add pig.md to dropdown menu
5e2e222 [AhyoungRyu] Minor update for pig.md
39f161a [Jeff Zhang] address comments
05a3b9b [Jeff Zhang] add pig.md
a09a7f7 [Jeff Zhang] refactor pig Interpreter
c28beb5 [Ali Bajwa] Updated based on comments: 1. Documentation: added pig.md with interpreter documentation and added pig entry to index.md 2. Added test junit test based on passwd file parsing example here https://pig.apache.org/docs/r0.10.0/start.html#run 3. Removed author tag from comment (this was copied from shell interpreter https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L42) 4. Implemented cancel functionality 5. Display output stream in case of error
2586336 [Ali Bajwa] exposed timeout and pig executable via interpreter and added comments
7abad20 [Ali Bajwa] initial commit of pig interpreter
### What is this PR for?
For now, it is hard to figure out what the yarn application of the livy session represent, it would be better to display the appId and webui link in the output of LivyInterpreter for diagnosing purpose. It can also be applied to the native SparkInterpreter, but could be done in another ticket.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1430
### How should this be tested?
Tested manually
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1426 from zjffdu/ZEPPELIN-1430 and squashes the following commits:
88009cb [Jeff Zhang] update doc
459f75e [Jeff Zhang] fix typo
eb7ec27 [Jeff Zhang] add flag to enable display app info in frontend
a087a1d [Jeff Zhang] ZEPPELIN-1430. Display appId and webui link in LivyInterpreter's output
### What is this PR for?
This PR sets default value for ZEPPELIN_INTP_MEM to avoid OOM Exception in SparkInterpreter when Zeppelin has zero configuration. This PR should be merged to both branch-0.6 and master.
### What type of PR is it?
Bug Fix
### How should this be tested?
1. Build with:
```
mvn clean package -DskipTests -pl '!zeppelin-distribution,!file,!alluxio,!livy,!hbase,!bigquery,!python,!jdbc,!ignite,!lens,!postgresql,!cassandra,!kylin,!elasticsearch,!flink,!markdown,!shell,!angular'
```
2. Unset SPARK_HOME in conf/zeppelin-env.sh if you have.
3. Run Zeppelin with java 1.7.
4. Run tutorial and see if it doesn't hang.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mina Lee <minalee@apache.org>
Closes#1505 from minahlee/hotfix/default_intp_jvm and squashes the following commits:
0dfda4f [Mina Lee] Set default ZEPPELIN_INTP_MEM
### What is this PR for?
When using Zeppelin behind corporate firewall, sometimes the dependencies download just fails silently. This PR has 2 objectives:
* add proxy credentials information for dependencies repo
* raise clear error message in case of dependencies download failure
There are 3 commits.
The first one add extra inputs in the form for adding new repository
The second commit fixes some issues and display a clear and explicit error message when download of dependencies fail.
Before that, when the download fails, we can see the below behaviour
* the error message is displayed twice because the call twice the method `checkDownloadingDependencies();`. One in the success callback of:
```javascript
$scope.updateInterpreterSetting = function(form, settingId) {
...
$http.put(baseUrlSrv.getRestApiBase() + '/interpreter/setting/' + settingId, request)
.success(function(data, status, headers, config) {
$scope.interpreterSettings[index] = data.body;
removeTMPSettings(index);
thisConfirm.close();
checkDownloadingDependencies();
$route.reload();
})
.error(function(data, status, headers, config) {
...
};
```
Another call is inside success callback of `getInterpreterSettings()`
```javascript
var getInterpreterSettings = function() {
$http.get(baseUrlSrv.getRestApiBase() + '/interpreter/setting')
.success(function(data, status, headers, config) {
$scope.interpreterSettings = data.body;
checkDownloadingDependencies();
}).error(function(data, status, headers, config) {
....
```
The problem is that `$route.reload();` in the success callback of `updateInterpreterSetting()` will trigger `init()` then `getInterpreterSettings()` so `checkDownloadingDependencies()` is called twice.
I remove the call to `checkDownloadingDependencies()` from success callback of `updateInterpreterSetting()`
The second modification is on class `DependencyResolver`. In the screen capture above, we get a **cryptic** NullPointerException coming from `DefaultRepositorySystem`. I now catch this NPE to wrap it into a more sensible and clearer exception:
```java
public List<ArtifactResult> getArtifactsWithDep(String dependency,
Collection<String> excludes) throws RepositoryException {
Artifact artifact = new DefaultArtifact(dependency);
DependencyFilter classpathFilter = DependencyFilterUtils.classpathFilter(JavaScopes.COMPILE);
PatternExclusionsDependencyFilter exclusionFilter =
new PatternExclusionsDependencyFilter(excludes);
CollectRequest collectRequest = new CollectRequest();
collectRequest.setRoot(new Dependency(artifact, JavaScopes.COMPILE));
synchronized (repos) {
for (RemoteRepository repo : repos) {
collectRequest.addRepository(repo);
}
}
DependencyRequest dependencyRequest = new DependencyRequest(collectRequest,
DependencyFilterUtils.andFilter(exclusionFilter, classpathFilter));
//Catch NPE thrown by aether and give a proper error message
try {
return system.resolveDependencies(session, dependencyRequest).getArtifactResults();
} catch (NullPointerException ex) {
throw new RepositoryException(String.format("Cannot fetch dependencies for %s", dependency));
}
}
```
The result is much more cleaner
The last commit is just doc update
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Code Review
* [ ] - Simple test with no Internet connection
* [ ] - Test within a corporate firewall env with a third-party dependency, requiring download
### What is the Jira issue?
**[ZEPPELIN-1376]**
### How should this be tested?
##### Simple test
* `git fetch origin pull/1369/head:WebProxy`
* `git checkout WebProxy`
* `mvn clean package -DskipTests`
* `bin/zeppelin-daemon.sh restart`
* disconnect from the Internet (pull out the cable, shutdown wifi ...)
* add a random dependency to the Spark interpreter (take `info.archinnov:achilles-core:4.2.2` for example)
* validate the change, you should see an error popup on the top-right corner saying that Zeppelin cannot download the dependency
##### Corporate firewall test
* follow the steps above for simple test
* create a new repository (see how to **[here]**) and set the proxy information
* retry the steps above to ensure that the download is successful
### Screenshots (if appropriate)
See above
### Questions:
* Does the licenses files need update? --> **NO**
* Is there breaking changes for older versions? --> **NO**
* Does this needs documentation? --> **YES, DONE**
[ZEPPELIN-1376]: https://issues.apache.org/jira/browse/ZEPPELIN-1376
[here]: http://localhost:4000/manual/dependencymanagement.html
Author: DuyHai DOAN <doanduyhai@gmail.com>
Author: doanduyhai <doanduyhai@apache.org>
Closes#1369 from doanduyhai/ZEPPELIN-1376 and squashes the following commits:
b8d44e7 [doanduyhai] [ZEPPELIN-1376] Improve error popup display
177fbd3 [DuyHai DOAN] [ZEPPELIN-1376] Fixes JS bug to display error popup for other interpreters
9f76ef4 [DuyHai DOAN] [ZEPPELIN-1376] Do not repeat the same error popup multiple times
b264193 [DuyHai DOAN] [ZEPPELIN-1376] Add unit test and fix impl for DependencyResolver to catch NPE
1913a0a [DuyHai DOAN] [ZEPPELIN-1376] Update documentation
f01be9b [DuyHai DOAN] [ZEPPELIN-1376] Raise clear error message in case of dependencies download failure
6f2b6f8 [DuyHai DOAN] [ZEPPELIN-1376] Add proxy credentials information for dependencies repo
### What is this PR for?
A link for `the community interpreters` in [here](http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html#please-note) is pointing to `../manual/interpreterinstallation.md#available-community-managed-interpreters` now. It should be `../manual/interpreterinstallation.html#available-community-managed-interpreters`.
### What type of PR is it?
Bug Fix & Documenation
### What is the Jira issue?
no Jira issue
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: AhyoungRyu <ahyoungryu@apache.org>
Closes#1498 from AhyoungRyu/fix/deadlink and squashes the following commits:
f8e8835 [AhyoungRyu] community interpreters -> community managed interpreters
0260d30 [AhyoungRyu] Fix deadlink in datasource_authorization.md
### What is this PR for?
Use a different variable for ssl port number
- add new property for ssl port
### What type of PR is it?
Improvement
### Todos
* [x] - Add documentation in upgrade.md
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1321
### How should this be tested?
Add ZEPPELIN_SSL_PORT or zeppelin.server.ssl.port in zeppelin env or site xml and configure the rest of ssl properties and start zeppelin.
### Screenshots (if appropriate)
n/a
### Questions:
* Does the licenses files need update? n/a
* Is there breaking changes for older versions? n/a
* Does this needs documentation? Updated install doc
Author: Renjith Kamath <renjith.kamath@gmail.com>
Closes#1489 from r-kamath/ZEPPELIN-1321 and squashes the following commits:
8f9006a [Renjith Kamath] ZEPPELIN-1321 update zeppelin upgrade guide
41899b1 [Renjith Kamath] ZEPPELIN-1321 Zeppelin HTTP and HTTPS port should be managed seperately
### What is this PR for?
Fixing the image path of the tutorial made in #418
### What type of PR is it?
Hot Fix
### What is the Jira issue?
None, just a Hotfix
### How should this be tested?
Build, move into full website, you should see the images
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Damien CORNEAU <corneadoug@gmail.com>
Closes#1474 from corneadoug/fix/FlinkTutorialImg and squashes the following commits:
cabd8df [Damien CORNEAU] Change path to tutorial images
