mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
576 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Lee moon soo
|
850fd81a51 |
[ZEPPELIN-212] Multiple paragraph results
### What is this PR for?
Currently a paragraph can display only a single type of result.
For example
```
print("""
%text textout
%html htmlout
""")
```
will display only last display system detected, "htmlout".
This pr implements multiple results supports, so not only the last, but also all the display systems in a paragraph outputs can be displayed.
To do that, Note.json format need to be changed. from
```
paragraph : [
{
result : {
code: "",
type: "",
msg: ""
},
config : {
graph : {},
...
},
...
},
...
],
...
```
to
```
paragraph : [
{
results : {
{
code: "",
msg: [
{
type: "",
data: ""
},
{
type: "",
data: ""
},
...
]
},
config : {
results : [
{
graph : {},
},
{
graph : {},
},
...
]
},
...
},
...
],
...
```
### What type of PR is it?
Improvement
### Todos
* [x] - Make InterpreterResult and InterpreterOutput support multiple display system
* [x] - Automatic migration old format to new format
* [x] - Render multiple results in front-end
* [x] - Take care Importing old notebook format
* [x] - Make helium framework support multiple results
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-212
### How should this be tested?
run following code in a single spark paragraph.
```
%spark
// default output is text
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// print something in html
println(s"""%html <h3><font style="color:blue">Some HTML</font></h3>""")
// create table
println(s"""%table key\tvalue
sun\t100
moon\t200
""")
// display text again
println("""%text""")
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// another table
Thread.sleep(1000)
println(s"""%table key\tvalue
apple\t100
banana\t200
""")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? yes
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes #1658 from Leemoonsoo/ZEPPELIN-212 and squashes the following commits:
|
||
|
CloverHearts
|
9db840c655 |
[ZEPPELIN-1665] Z.run with external note executable and access resource for zeppelin in each interpreter
### What is this PR for? Currently, the z.run command is restricted. Only paragraphs in a single note can be executed. I have modified this to allow you to freely execute paragraphs of other notes. This PR provides the basis for the freeful use of Zeppelin's resources at each Interpreter implementation. ### What type of PR is it? Improvement, Feature ### Todos - [x] extends z.run - [x] run all paragraph in external note - [x] run paragraph for external note. - [x] get resource for zeppelin in each interpreter. - [x] improve test case. - [x] how to use docuement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1665 ### How should this be tested? Currently under development. run paragraph in same note ``` %spark z.run("paragraphID") ``` run paragraph with external note ``` z.run("noteid", "paragraphid"); ``` all note run ``` z.runNote("noteid"); ``` ### Screenshots (if appropriate) - paragraph run  - noterun  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: CloverHearts <cloverheartsdev@gmail.com> Closes #1637 from cloverhearts/extends-zrun-remote-transaction and squashes the following commits: |
||
|
Anthony Corbacho
|
33e2dab37e |
[ZEPPELIN-1690] - ZeppelinHubNotebookRepo multy user handling
### What is this PR for? This PR bring multi user handling to ZeppelinHubNotebookRepo. ### What type of PR is it? [Improvement ] ### What is the Jira issue? * [ZEPPELIN-1690](https://issues.apache.org/jira/browse/ZEPPELIN-1690) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Anthony Corbacho <corbacho.anthony@gmail.com> Closes #1635 from anthonycorbacho/feat/ZeppelinHubRepoMultiUser and squashes the following commits: |
||
|
1ambda
|
3389e8cfcb |
[ZEPPELIN-1666] DON'T share mutable deps, options between interpreters in each group (bug)
### What is this PR for? Every interpreter shares their `List<Dependency>` and `InterpreterOption` object with other interpreters in the same group since these objects are mutable and just returned from `InterpreterSettingRef` in InterpreterFactory. I attached GIF ### What type of PR is it? [Bug Fix] ### Todos Nothing ### What is the Jira issue? [ZEPPELIN-1666](https://issues.apache.org/jira/browse/ZEPPELIN-1666) ### How should this be tested? I included unit test for it in `InterpreterRestApiTest.testCreatedInterpreterDependencies`. You can reproduce and debug this by checking out commit |
||
|
1ambda
|
abe03a866f |
[ZEPPELIN-1667] Fix maven options while updating node, npm packages
### What is this PR for? Update maven plugin, node vesion, npm package and fix tests to make #1639 pass CI ### What is the Jira issue? Sub issue of #1639 ### Questions: Does the licenses files need update? NO Is there breaking changes for older versions? YES (front-end build tool usage changed) Does this needs documentation? YES (front-end build tool usage changed) Author: 1ambda <1amb4a@gmail.com> Closes #1659 from 1ambda/pr/1639 and squashes the following commits: |
||
|
Mina Lee
|
4dd36bfa31 |
[ZEPPELIN-1623] Fix flaky test - testEditOnDoubleClick
### What is this PR for?
#1657 is merged, but CI failure still exists on master. Check [this link](https://issues.apache.org/jira/browse/ZEPPELIN-1623) to see the test failure.
I will trigger this PR 5 times and attach CI link on comments if all of them pass to be sure that the issue is gone.
### What type of PR is it?
Hot Fix
### What is the Jira issue?
[ZEPPELIN-1623](https://issues.apache.org/jira/browse/ZEPPELIN-1623)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mina Lee <minalee@apache.org>
Closes #1667 from minahlee/ZEPPELIN-1623 and squashes the following commits:
|
||
|
karuppayya
|
1375379a28 |
ZEPPELIN-1653: Make UI responsive
### What is this PR for? When we do the following operations, the entire notebook is transmitted back to client. 1) Insert paragraph 2) Remove paragraph 3) Update paragraph 4) Paragraph status update 5) commit paragraph When the json becomes larger(which happens when large output is stored in the json), the time to transfer the notebook to client is directly proportional.(we can check this in chrome browser-> dev console-> select ws request-> on the right pane that opens, select frames tab and observe the length column) And since the UI update is based on this new transmitted json, the UI seems unresponsive/laggy. In this PR, Making the updates selective.(Not sending the complete json back to client) ### What type of PR is it? Improvement ### Todos NA ### What is the Jira issue? ZEPPELIN-1653 ### How should this be tested? -In chrome browser-> dev console-> select ws request-> on the right pane that opens, select frames tab and observe *NOTE* message. We should not see the message for the above mentioned operations except when the notebook is loaded is for first time - We should not see *GET_NOTE* log message for above operations. ### Screenshots NA ### Questions: * Does the licenses files need update?No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: karuppayya <karuppayya1990@gmail.com> Author: Karup <karuppayya@outlook.com> Closes #1624 from karup1990/ZEPPELIN-1653 and squashes the following commits: |
||
|
Lee moon soo
|
f6995738af |
[ZEPPELIN-1623] Fix flaky test - ParagraphActionsIT.testEditOnDoubleClick
### What is this PR for?
Fix flaky test - ParagraphActionsIT.testEditOnDoubleClick
Before send the keyevent, make sure textarea is displayed.
### What type of PR is it?
Bug Fix
### Todos
* [x] - wait for textarea ready
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1623
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes #1657 from Leemoonsoo/ZEPPELIN-1623 and squashes the following commits:
|
||
|
gss2002
|
8aad9ea392 |
ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox
### What is this PR for?
ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.
In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.
The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.
I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.
Example - SecurityUtils
String name = realm.getClass().getName();
if (name.equals("org.apache.shiro.realm.text.IniRealm")) {
allRoles = ((IniRealm) realm).getIni().get("roles");
break;
} else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) {
allRoles = ((LdapRealm) realm).getListRoles();
break;
}
Example - SecurityRestApi:
String name = realm.getClass().getName();
if (LOG.isDebugEnabled()) {
LOG.debug("RealmClass.getName: " + name);
}
if (name.equals("org.apache.shiro.realm.text.IniRealm")) {
usersList.addAll(getUserListObj.getUserList((IniRealm) realm));
rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm));
} else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) {
usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText));
} else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) {
usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText));
rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm));
} else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) {
usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm,
searchText));
} else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) {
usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm));
}
Please see feedback from previous PRs related to this JIRA:
https://github.com/apache/zeppelin/pull/1513
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1472
### How should this be tested?
Update shiro.ini to use configuration similar to below:
# Sample LDAP configuration, for user Authentication, currently tested for single Realm
[main]
ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
ldapADGCRealm.authorizationEnabled = true
ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
ldapADGCRealm.userSearchAttributeName = sAMAccountName
ldapADGCRealm.contextFactory.authenticationMechanism = simple
ldapADGCRealm.groupObjectClass = group
ldapADGCRealm.memberAttribute = member
ldapADGCRealm.rolesByGroup = hdpeng: admin, \
hadoopusers: user
securityManager.realms = $ldapADGCRealm
sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager
### If caching of user is required then uncomment below lines
#cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
#securityManager.cacheManager = $cacheManager
securityManager.sessionManager = $sessionManager
# 86,400,000 milliseconds = 24 hour
securityManager.sessionManager.globalSessionTimeout = 86400000
shiro.loginUrl = /api/login
[roles]
# 'admin' role has all permissions, indicated by the wildcard '*'
admin = *
user = *
[urls]
# anon means the access is anonymous.
# authcBasic means Basic Auth Security
# authc means Form based Auth Security
# To enfore security, comment the line below and uncomment the next one
#/api/version = anon
#/** = anon
/api/interpreter/** = authc, roles[admin]
/api/configurations/** = authc, roles[admin]
/api/credential/** = authc, roles[admin]
/api/login = authc
/api/login/logout = authc
/api/security/ticket = authc
/** = authc, roles[admin, user]
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? n
* Is there breaking changes for older versions? n
* Does this needs documentation? y
merge latest commits
Author: gss2002 <greg@senia.org>
Author: gss2002 <gsenia@apache.org>
Closes #1614 from gss2002/ZEPPELIN-1472 and squashes the following commits:
|
||
|
Mina Lee
|
85c50a8aba |
[ZEPPELIN-1622] Remove %dep interpreter deprecated message
### What is this PR for? %dep interpreter was going to be deprecated, but we had feedback from many users that this feature provide different advantage over dependency loading via GUI so we want to keep supporting it. This PR remove deprecated message when users use %dep interpreter. ### Todos - [x] Fix test ### What type of PR is it? Documentation ### What is the Jira issue? [ZEPPELIN-1622](https://issues.apache.org/jira/browse/ZEPPELIN-1622) ### How should this be tested? Run %dep interpreter such as `z.load('/your/library.jar)` and see if deprecated message is gone. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Mina Lee <minalee@apache.org> Closes #1605 from minahlee/ZEPPELIN-1622 and squashes the following commits: |
||
|
1ambda
|
e33be25256 |
[ZEPPELIN-1684] Add GET /interpreter/setting/id REST API
### What is this PR for? Due to the lack of this missing API, some front-end code retrieve all settings. This is inefficient. We can modify those codes by adding this API. Also, i refactored the whole `InterpreterRestApiTest` ### What type of PR is it? [Feature | Documentation | Refactoring] ### What is the Jira issue? [ZEPPELIN-1684](https://issues.apache.org/jira/browse/ZEPPELIN-1684) ### How should this be tested? You can run unit test `InterpreterRestApiTest` ### Screenshots (if appropriate) Updated doc screenshots <img width="900" alt="new_api_doc" src="https://cloud.githubusercontent.com/assets/4968473/20423419/74eeb2a0-adb3-11e6-9a69-58e33457d514.png"> ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - YES, but done. Author: 1ambda <1amb4a@gmail.com> Closes #1655 from 1ambda/feat/create-interpreter-get-api and squashes the following commits: |
||
|
Anthony Corbacho
|
5e85e6ea6f |
[ZEPPELIN-1610] - Add notebook watcher
### What is this PR for? Add a Simple way to switch a websocket connection to a new state; watcher. A websocket watcher is a special connection that will watch most of the web socket even in Zeppelin, this cam be used to monitor zeppelin server activity. ### What type of PR is it? [Feature] ### Todos * [x] - Add watcher Queue * [x] - Add endpoint to switch from regular client to watcher * [x] - Add a way to generate a uniq key when zeppelin server restart * [x] - Add example on how to use watcher. ### What is the Jira issue? * [ZEPPELIN-1610](https://issues.apache.org/jira/browse/ZEPPELIN-1610) ### How should this be tested? You will have to create your own websocket client and provide a valid http header (`X-Watcher-Key`) when you connect to zeppelin ws something like ``` private Session openWatcherSession() { ClientUpgradeRequest request = new ClientUpgradeRequest(); request.setHeader(WatcherSecurityKey.HTTP_HEADER, WatcherSecurityKey.getKey()); WatcherWebsocket socket = WatcherWebsocket.createInstace(); Future<Session> future = null; Session session = null; try { future = wsClient.connect(socket, zeppelinWebsocketUrl, request); session = future.get(); } catch (IOException | InterruptedException | ExecutionException e) { LOG.error("Couldn't establish websocket connection to Zeppelin ", e); return session; } return session; } ``` ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Anthony Corbacho <corbacho.anthony@gmail.com> Closes #1588 from anthonycorbacho/feat/updateWebsocketInZeppelinHubRepo and squashes the following commits: |
||
|
1ambda
|
ab2cdfd384 |
[ZEPPELIN-1614] Remove markdown4j dep
### What is this PR for? Remove unmanaged, old library markdown4j dep which exists just for Websequence and YUML plugins. (related to https://github.com/apache/zeppelin/pull/1384) By adding Websequence and YUML plugins to pegdown - Removing markdown4j dependency which is unmanaged library currently. - Addtionally, we can remove `markdown.parser.type` options in **markdown interpreter** - Fixing some bugs in Websequence and YUML plugins - Enable others to add more plugins using pegdown sytnax. ### What type of PR is it? Improvement ### Todos Nothing ### What is the Jira issue? [JIRA - ZEPPELIN-1614](https://issues.apache.org/jira/browse/ZEPPELIN-1614) ### How should this be tested? Some functional tests are included. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? - YES * Is there breaking changes for older versions? - NO * Does this needs documentation? - YES Author: 1ambda <1amb4a@gmail.com> Closes #1594 from 1ambda/feat/remove-markdown4j-dep and squashes the following commits: |
||
|
karuppayya
|
96ca84da15 |
ZEPPELIN-1643:Make spark web UI accesible from interpreters page
### What is this PR for? Make spark web UI accesible from interpreters page ### What type of PR is it? Improvement ### Todos NA ### What is the Jira issue? ZEPPELIN-1643 ### How should this be tested? Start sparkcontext. Goto interpreters page. Corresponding to the spark interpreter used in notebook, there will be *spark ui* button. Clicking the button should open a new tab with the application's spark web UI. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? NA * Is there breaking changes for older versions? No * Does this needs documentation? No Author: karuppayya <karuppayya1990@gmail.com> Author: Karup <karuppayya@outlook.com> Closes #1613 from karup1990/ZEPPELIN-1643 and squashes the following commits: |
||
|
Khalid Huseynov
|
6069a083af |
[ZEPPELIN-1647] Save roles and use for broadcasting note list per user
### What is this PR for? So far roles have been accessible only from SecurityUtils for Rest api or from websocket message field. However sometimes it's required to access roles in websocket server even without receiving message, say for broadcasting note list per user. More details in issue. ### What type of PR is it? Bug Fix | Improvement ### Todos * [x] - add roles in NotebookAuthorization * [x] - assign roles on login * [x] - use roles on broadcast * [x] - test ### What is the Jira issue? [ZEPPELIN-1647](https://issues.apache.org/jira/browse/ZEPPELIN-1647) ### How should this be tested? login as user1, and user2 at same time and each user should have own workbench (based on notebook permissions) ### Screenshots (if appropriate) TBD ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Khalid Huseynov <khalidhnv@gmail.com> Closes #1619 from khalidhuseynov/fix/apply-correct-roles-broadcast and squashes the following commits: |
||
|
Jun
|
c1254f7b0d |
[ZEPPELIN-1628] Enable renaming note from the main page
### What is this PR for? Now users can rename a note from the main page! This new feature will improve UX. I divided [ZEPPELIN-1598](https://issues.apache.org/jira/browse/ZEPPELIN-1598) into sub-tasks since renaming folder gonna be huge. I will open PR for [ZEPPELIN-1629](https://issues.apache.org/jira/browse/ZEPPELIN-1629) after merging this PR. By the way, I have a question! Does a `writer` can rename a note? Currently, only an owner can rename a note. ### What type of PR is it? [Feature] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1628 ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jun <i2r.jun@gmail.com> Closes #1609 from tae-jun/ZEPPELIN-1628 and squashes the following commits: |
||
|
Mina Lee
|
24187b0a75 |
[ZEPPELIN-1029] Add rest api for paragraph config update
### What is this PR for?
This PR adds new end point for updating paragraph configuration
### What type of PR is it?
Improvement | Documentation
### What is the Jira issue?
[ZEPPELIN-1029](https://issues.apache.org/jira/browse/ZEPPELIN-1029)
### How should this be tested?
Outline the steps to test the PR here.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Mina Lee <minalee@apache.org>
Closes #1592 from minahlee/ZEPPELIN-1029 and squashes the following commits:
|
||
|
hkropp
|
f866d234a1 |
[Zeppelin-1611] - Support PAM (System User) Authentication
### What is this PR for?
This PR adds [PAM](https://en.wikipedia.org/wiki/Pluggable_authentication_module) authentication support based on the introduced Shiro security implementation. With PAM support system users have immediate access to a secured Zeppelin instance.
### What type of PR is it?
Feature
### Todos
* [x] - Create PAM realm
* [x] - Create test for PAM authentication
* [x] - Test with running Zeppelin instance
### What is the Jira issue?
[ZEPPELIN-1611](https://issues.apache.org/jira/browse/ZEPPELIN-1611])
### How should this be tested?
`PamRealmTest` executes an automated test if the environment variables `PAM_USER` and `PAM_PASS` are set. This should be set to system username and password.
The test also includes a main function to manually execute the test. Setting the environment variables for example on MacOS for your IDE use `launchctl setenv PAM_USER user` and `launchctl setenv PAM_PASS xxxxx`, the test can then be run from your IDE.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: hkropp <hkropp@hortonworks.com>
Closes #1589 from hkropp/ZEPPELIN-1611 and squashes the following commits:
|
||
|
Anthony Corbacho
|
cb175032a2 |
[HOT-FIX] - Prevent double injection of cross-origin headers from rest api
### What is this PR for?
Prevent error `XMLHttpRequest cannot load http://localhost:8080/api/version. The 'Access-Control-Allow-Origin' header contains multiple values 'http://localhost:9000, http://localhost:9000', but only one is allowed. Origin 'http://localhost:9000' is therefore not allowed access`
### What type of PR is it?
[Hot Fix]
Author: Anthony Corbacho <corbacho.anthony@gmail.com>
Closes #1604 from anthonycorbacho/fix/DoubleOriginInjectionInResponseHeader and squashes the following commits:
|
||
|
Mina Lee
|
dd20e7bf8b |
[ZEPPELIN-1564] Enable note deletion and paragraph output clear from main page
### What is this PR for? - Enables removing note and clear all paragraph's output from Zeppelin main page. - Add rest api for clearing all paragraph output Next possible improvement can be removing notes in folder level and rename folder. ### What type of PR is it? Improvement ### Todos * [x] - Merge #1567 and apply security to `clearAllParagraphOutput` rest api method ### What is the Jira issue? [ZEPPELIN-1564](https://issues.apache.org/jira/browse/ZEPPELIN-1564) ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? yes Author: Mina Lee <minalee@apache.org> Closes #1565 from minahlee/ZEPPELIN-1564 and squashes the following commits: |
||
|
Anthony Corbacho
|
0e087455af |
[ZEPPELIN-1615] - Zeppelin should be able to run without Shiro
### What is this PR for? Right now, Zeppelin use Shiro by default even if you dont need it. (It will use shiro.ini.template file if it doenst find shiro.ini), this behaviors is a little flacky and we should start zeppelin without shiro context if user doenst want to use it. ### What type of PR is it? [Bug Fix | Improvement ] ### Todos * [x] - Update configuration - Return empty if shiro file not found * [x] - refactor Rest Api handler, if shiro.ini not found start a handler without shiro context * [x] - refactor SecurityUtils to handle the case of shiro is disabled. ### What is the Jira issue? * [ZEPPELIN-1615](https://issues.apache.org/jira/browse/ZEPPELIN-1615) ### How should this be tested? Start zeppelin without shiro.ini ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Anthony Corbacho <corbacho.anthony@gmail.com> Closes #1595 from anthonycorbacho/fix/ShiroTempleteLoginRequired and squashes the following commits: |
||
|
Kavin
|
f1e9c52087 |
[Zeppelin-53] Broken newly created Notebook if there is no permissions to mkdir on disk
### What is this PR for?
Error message should be shown to the user when there is permission issue with the 'notebook' storage folder.
### What type of PR is it?
Bug Fix
### Todos
NA
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-53
### How should this be tested?
- Create the notebook storage folder (i.e zeppelin/notebook) with only read and execute permission
- Try to create a notebook from UI
- The below error message as in the screenshot should be rendered
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? No
- Is there breaking changes for older versions? No
- Does this needs documentation? No
Author: Kavin <kavin.kumar@imaginea.com>
Closes #1535 from kavinkumarks/zeppelin-53-create-notebook-mkdir-issue and squashes the following commits:
|
||
|
CloverHearts
|
55de3fba71 |
[hotfix] does not showing notebooklist on navbar
### What is this PR for? Currently, if you enabled for shiro then not showing notebook list on navbar. (If you refresh, it's well viewwing.) i resolve this problem. ### What type of PR is it? Bug Fix ### Todos - [x] fixed a bug ( navbar.controller.js) - [x] added testcase ### How should this be tested? 1. endabled shiro (per user login) 2. login. 3. on click to notebook menu on navbar. ### Screenshots (if appropriate) #### before  #### resolve after (on this pr)  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: CloverHearts <cloverheartsdev@gmail.com> Closes #1586 from cloverhearts/ZEPPELIN-NOTEBOOKLIST-SHIRO and squashes the following commits: |
||
|
Anthony Corbacho
|
32fe23f2e2 |
[ZEPPELIN-1586] Add security check in NotebookRestApi
### What is this PR for? Bring some security check in `NotebookRestApi`. ### What type of PR is it? [Bug Fix | Improvement | Refactoring] ### Todos - [x] - Create a proper way to throw webapp error - [x] - Add in `NotebookAuthorization` some method to check if user is owner, reader or writer - [x] - Add Authorization check in `NotebookRestapi` - [x] - Add New test for security in notebook rest api ### What is the Jira issue? - [ZEPPELIN-1586](https://issues.apache.org/jira/browse/ZEPPELIN-1586) ### How should this be tested? First, force Zeppelin to use auth. - In `conf/zeppelin-site.xml` change `zeppelin.anonymous.allowed` to **false** ``` <property> <name>zeppelin.anonymous.allowed</name> <value>false</value> <description>Anonymous user allowed by default</description> </property> ``` - In `conf/shiro.ini` set Shiro to use `Auth` at the end of the file ``` #/** = anon /** = authc ``` - Start Zeppelin, login and set some permission to a note - try to get a note from Zeppelin Rest Api `http://localhost:8080/api/notebook/{noteId}` (you can use your browser or curl (if you use curl please add shiro token to curl cookie)) ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? No - Is there breaking changes for older versions? No - Does this needs documentation? Maybe Author: Anthony Corbacho <corbacho.anthony@gmail.com> Closes #1567 from anthonycorbacho/fix/ZEPPELIN-1586 and squashes the following commits: |
||
|
Mina Lee
|
28adacb9c8 |
[ZEPPELIN-1461] Doesn't display "description" value in interpreter creation page
### What is this PR for? #1522 tried to fix same issue by making as least change as possible, but fixing it in backend side looks like more proper approach as https://github.com/apache/zeppelin/pull/1522#issuecomment-255109922. This PR fixes ZEPPELIN-1461 by changing `properties` field of `InterpreterSetting` class from `Properties` -> `Object`. ### What type of PR is it? Bug Fix ### What is the Jira issue? [ZEPPELIN-1461](https://issues.apache.org/jira/browse/ZEPPELIN-1461) ### Screenshots (if appropriate) Before  After  ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? no Author: Mina Lee <minalee@apache.org> Closes #1559 from minahlee/ZEPPELIN-1461 and squashes the following commits: |
||
|
Kavin
|
0f56337ce3 |
[ZEPPELIN-1061] Select default interpreter while creating note
### What is this PR for? This handles setting the default interpreter on creating a note through the zeppelin UI. ### What type of PR is it? Feature ### Todos NA ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1061 ### How should this be tested? - Go to the 'Create Note' dialog and provide the name with the default interpreter selected.After clicking on the Create button, the selected interpreter should be shown properly in the interpreter binding section. - If there is no interpreter selected, then the system default will be used. ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? No - Is there breaking changes for older versions? No - Does this needs documentation? No Author: Kavin <kavin.kumar@imaginea.com> Closes #1520 from kavinkumarks/zeppelin-1061-create-note-default-interpreter and squashes the following commits: |
||
|
Jan Hentschel
|
99b975f5c5 |
[ZEPPELIN-429] Replaced explicit types with diamonds
### What is this PR for? Changed all occurrences of explicit types to diamonds in the different modules. ### What type of PR is it? Improvement ### Todos ## ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-429 ### How should this be tested? Run the normal Maven build process. ### Screenshots (if appropriate) ### Questions: - Does the licenses files need update? No - Is there breaking changes for older versions? No - Does this needs documentation? No Author: Jan Hentschel <jan.hentschel@ultratendency.com> Closes #1551 from HorizonNet/ZEPPELIN-429 and squashes the following commits: |
||
|
astroshim
|
5991a35774 |
[ZEPPELIN-1306] Interpreter restarts on a note.
### What is this PR for? This PR is for usability of restarting interpreter. ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1306 ### How should this be tested? You can restart interpreter on the interpreter binding page on a note. and can only restart `scoped` and `isolated` interpreter. Please refer to screen shot. ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Closes #1302 from astroshim/ZEPPELIN-1306 and squashes the following commits: |
||
|
Mina Lee
|
60089f0f58 |
[ZEPPELIN-1566] Make paragraph editable with double click
### What is this PR for? This PR enables edit on double click for markdown/angular paragraph. Users can change `editOnDblClick` field to be `false` by editting `interpreter/md/interpreter-setting.json` or `conf/interpreter.json`. In the same context, users can set other type paragraphs to be editable on double click by setting `editOnDblClick` to be true. This PR also fixes bug that syntax highlight doesn't work on pasted code. ### What type of PR is it? Feature ### Todos * [x] Create test * [x] Update docs ### What is the Jira issue? [ZEPPELIN-1566](https://issues.apache.org/jira/browse/ZEPPELIN-1566) ### How should this be tested? 1. Create new markdown interpreter 2. Create notebook and run markdown paragraph 3. Double click markdown paragraph to edit ### Screenshots (if appropriate) **Edit on double click and hide editor on paragraph run**  **Syntax highlight on paste** Before  After  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Mina Lee <minalee@apache.org> Closes #1540 from minahlee/ZEPPELIN-1566 and squashes the following commits: |
||
|
Anthony Corbacho
|
824fd955fb |
[ZEPPELIN-1575] Notebook Repo settings UI
### What is this PR for? The idea behind this feature is to bring flexibility to the user to let him configure his repo via UI without restarting Apache Zeppelin. This is flexible enough to handle most of the basic needs, but more complex case can be added in the future. I will not implement all NotebookRepo, this should be done in different PR since it will depend on special needs. I will just provide a simple example with VFS notebook repo. ***NB***: this scope of the PR doesn't include save change in the configuration file of zeppelin as well as implementation of every notebook repo, git, s3 etcetc, this will be added later if needed. ### What type of PR is it? * **Improvement** ### Todos * [x] - Implement backend. * [x] - Implement frontend. * [x] - Implement LocalVFS directory change as exemple. ### What is the Jira issue? * [ZEPPELIN-1575](https://issues.apache.org/jira/browse/ZEPPELIN-1575) ### How should this be tested? Go to notebook repo setting page, edit the notebook path and save. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? NO * Is there breaking changes for older versions? NO * Does this needs documentation? YES Author: Anthony Corbacho <corbacho.anthony@gmail.com> Author: Damien CORNEAU <corneadoug@gmail.com> Closes #1553 from anthonycorbacho/feat/NotebookRepoSettings and squashes the following commits: |
||
|
Kai Jiang
|
0dd04d5284 |
[MINOR] remove comments in NotebookRestApi.java
### What is this PR for?
Since ZEPPELIN-1162 (PR #1341) has been solved, we should remove TODO comments correspondingly.
### What type of PR is it?
Minor
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Kai Jiang <jiangkai@gmail.com>
Closes #1556 from vectorijk/minor-remove-comments and squashes the following commits:
|
||
|
hyonzin
|
4f6a0e34ff |
[ZEPPELIN-1549] Change NotebookID variable name to NoteID
### What is this PR for? This PR fixes wrong written NotebookID to NoteID. ### What type of PR is it? [Improvement] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1549 ### Questions: * Does the licenses files need update? No. * Is there breaking changes for older versions? No. * Does this needs documentation? No Author: hyonzin <hyeonjin507@gmail.com> Author: 정현진 <hyeonjin507@gmail.com> Author: Mina Lee <minalee@apache.org> Closes #1518 from hyonzin/ZEPPELIN-1549 and squashes the following commits: |
||
|
Khalid Huseynov
|
dfbea2eb98 |
[Zeppelin-1561] Improve sync for multiuser environment
### What is this PR for? apply multi-tenancy for storage sync mechanism ### What type of PR is it? Bug Fix | Improvement ### Todos * [x] - broadcast on sync * [x] - set permissions for pulled notes * [x] - add test ### What is the Jira issue? [ZEPPELIN-1561](https://issues.apache.org/jira/browse/ZEPPELIN-1561) ### How should this be tested? Outline the steps to test the PR here. ### Screenshots (if appropriate) green CI ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Khalid Huseynov <khalidhnv@gmail.com> Closes #1537 from khalidhuseynov/improve/sync-multiuser and squashes the following commits: |
||
|
Prabhjyot Singh
|
9e9ea3aea0 |
[ZEPPELIN-1483] Zeppelin home page list notebooks doesn't show notebook with group permission
### What is this PR for? Zeppelin home page list notebooks doesn't show notebook with group permission ### What type of PR is it? [Bug Fix] ### Todos * [x] - consume userAndRole instead of AuthenticationInfo ### What is the Jira issue? * [ZEPPELIN-1483](https://issues.apache.org/jira/browse/ZEPPELIN-1483) ### How should this be tested? In current scenario only those notebook lists that have direct user permission, those with group does not list up, but if user have link to those notebook, it can still be accessed. IMO the notebook with group permission should also be listed in the home screen. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? n/a * Is there breaking changes for older versions? n/a * Does this needs documentation? n/a Author: Prabhjyot Singh <prabhjyotsingh@gmail.com> Author: Prabhjyot Singh <prabhjyotsingh@gmail.org> Closes #1454 from prabhjyotsingh/ZEPPELIN-1483 and squashes the following commits: |
||
|
CloverHearts
|
908b2a74ff |
[ZEPPELIN-1210] Run interpreter per user
### What is this PR for? Enabling each user to run same interpreter. ### What type of PR is it? [Improvement] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1210 ### How should this be tested? 1. Enable shiro to use authentication mode 1. Check `per user` in your interpreter tab 1. Run different paragraphs with different users 1. Run `%spark sc.version`, you will see the two `res0: ...` in your paragraphs ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: CloverHearts <cloverheartsdev@gmail.com> Author: Jongyoul Lee <jongyoul@gmail.com> Closes #1265 from jongyoul/ZEPPELIN-1210 and squashes the following commits: |
||
|
rajarajan-g
|
1214463c66 |
[ZEPPELIN-1542] Cleanup of Note created during jUnit test in zeppelin-server, zeppelin-zengine
### What is this PR for?
While running jUnit tests in zeppelin-server & zeppelin-zengine projects, several note are created during the process and those notes are never purged after the execution of the tests.
### What type of PR is it?
Bug Fix
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1542
### How should this be tested?
Prerequisites:
1. Get the list of note that already exist in server.
Now build zeppelin project by running maven commands(mvn clean package)(mvn clean package -pl 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server').
1. Now no extra note should get created after running the maven command
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: rajarajan-g <rajarajan.ganesan@imaginea.com>
Closes #1524 from rajarajan-g/ZEPPELIN-1542 and squashes the following commits:
|
||
|
Khalid Huseynov
|
b77f9ea8d7 |
[ZEPPELIN-1437, 1438] Multi-user note management - user aware reload broadcast
### What is this PR for? This PR addresses part of multi-user note management in Zeppelin. One of the tasks namely listing notes per user on Zeppelin start was addressed in #1330. However that PR didn't solve all problems, and reloading notes was incomplete as well as socket broadcast was not user aware [ZEPPELIN-1437](https://issues.apache.org/jira/browse/ZEPPELIN-1437), [ZEPPELIN-1438](https://issues.apache.org/jira/browse/ZEPPELIN-1438). This PR addresses those issue. ### What type of PR is it? Improvement ### Todos * [x] - list notes per user on reload * [x] - broadcast per user (multicast) * [x] - tests * [x] - use authorization module to filter notes on sync * [x] - broadcast on permissions change * [ ] - discussion and review ### What is the Jira issue? [Zeppelin-1437](https://issues.apache.org/jira/browse/ZEPPELIN-1437), [ZEPPELIN-1438](https://issues.apache.org/jira/browse/ZEPPELIN-1438) ### How should this be tested? 1. Start Zeppelin 2. Login as user1, and user2 on different windows 3. Each user should be able to see their own note workbench 4. If note changed to private (readers, writers not empty), that note should disappear from others note workbench. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Khalid Huseynov <khalidhnv@gmail.com> Closes #1392 from khalidhuseynov/feat/multi-user-notes and squashes the following commits: |
||
|
Jongyoul Lee
|
d2f63f6f96 |
[ZEPPELIN-1520] Change test environment to use the local mode of Apache Spark
### What is this PR for? Simplify travis test to reduce usage of resources ### What type of PR is it? [Improvement] ### Todos * [x] - Remove start-up/stop SparkCluster ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1520 ### How should this be tested? Travis will pass without any error ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Jongyoul Lee <jongyoul@gmail.com> Closes #1487 from jongyoul/ZEPPELIN-1520 and squashes the following commits: |
||
|
DuyHai DOAN
|
79fd3a0242 |
[ZEPPELIN-1376] Add proxy credentials for dependency repo for corporate firewall use-cases
### What is this PR for? When using Zeppelin behind corporate firewall, sometimes the dependencies download just fails silently. This PR has 2 objectives: * add proxy credentials information for dependencies repo * raise clear error message in case of dependencies download failure There are 3 commits. The first one add extra inputs in the form for adding new repository  The second commit fixes some issues and display a clear and explicit error message when download of dependencies fail. Before that, when the download fails, we can see the below behaviour  * the error message is displayed twice because the call twice the method `checkDownloadingDependencies();`. One in the success callback of: ```javascript $scope.updateInterpreterSetting = function(form, settingId) { ... $http.put(baseUrlSrv.getRestApiBase() + '/interpreter/setting/' + settingId, request) .success(function(data, status, headers, config) { $scope.interpreterSettings[index] = data.body; removeTMPSettings(index); thisConfirm.close(); checkDownloadingDependencies(); $route.reload(); }) .error(function(data, status, headers, config) { ... }; ``` Another call is inside success callback of `getInterpreterSettings()` ```javascript var getInterpreterSettings = function() { $http.get(baseUrlSrv.getRestApiBase() + '/interpreter/setting') .success(function(data, status, headers, config) { $scope.interpreterSettings = data.body; checkDownloadingDependencies(); }).error(function(data, status, headers, config) { .... ``` The problem is that `$route.reload();` in the success callback of `updateInterpreterSetting()` will trigger `init()` then `getInterpreterSettings()` so `checkDownloadingDependencies()` is called twice. I remove the call to `checkDownloadingDependencies()` from success callback of `updateInterpreterSetting()` The second modification is on class `DependencyResolver`. In the screen capture above, we get a **cryptic** NullPointerException coming from `DefaultRepositorySystem`. I now catch this NPE to wrap it into a more sensible and clearer exception: ```java public List<ArtifactResult> getArtifactsWithDep(String dependency, Collection<String> excludes) throws RepositoryException { Artifact artifact = new DefaultArtifact(dependency); DependencyFilter classpathFilter = DependencyFilterUtils.classpathFilter(JavaScopes.COMPILE); PatternExclusionsDependencyFilter exclusionFilter = new PatternExclusionsDependencyFilter(excludes); CollectRequest collectRequest = new CollectRequest(); collectRequest.setRoot(new Dependency(artifact, JavaScopes.COMPILE)); synchronized (repos) { for (RemoteRepository repo : repos) { collectRequest.addRepository(repo); } } DependencyRequest dependencyRequest = new DependencyRequest(collectRequest, DependencyFilterUtils.andFilter(exclusionFilter, classpathFilter)); //Catch NPE thrown by aether and give a proper error message try { return system.resolveDependencies(session, dependencyRequest).getArtifactResults(); } catch (NullPointerException ex) { throw new RepositoryException(String.format("Cannot fetch dependencies for %s", dependency)); } } ``` The result is much more cleaner  The last commit is just doc update  ### What type of PR is it? [Improvement] ### Todos * [ ] - Code Review * [ ] - Simple test with no Internet connection * [ ] - Test within a corporate firewall env with a third-party dependency, requiring download ### What is the Jira issue? **[ZEPPELIN-1376]** ### How should this be tested? ##### Simple test * `git fetch origin pull/1369/head:WebProxy` * `git checkout WebProxy` * `mvn clean package -DskipTests` * `bin/zeppelin-daemon.sh restart` * disconnect from the Internet (pull out the cable, shutdown wifi ...) * add a random dependency to the Spark interpreter (take `info.archinnov:achilles-core:4.2.2` for example) * validate the change, you should see an error popup on the top-right corner saying that Zeppelin cannot download the dependency ##### Corporate firewall test * follow the steps above for simple test * create a new repository (see how to **[here]**) and set the proxy information * retry the steps above to ensure that the download is successful ### Screenshots (if appropriate) See above ### Questions: * Does the licenses files need update? --> **NO** * Is there breaking changes for older versions? --> **NO** * Does this needs documentation? --> **YES, DONE** [ZEPPELIN-1376]: https://issues.apache.org/jira/browse/ZEPPELIN-1376 [here]: http://localhost:4000/manual/dependencymanagement.html Author: DuyHai DOAN <doanduyhai@gmail.com> Author: doanduyhai <doanduyhai@apache.org> Closes #1369 from doanduyhai/ZEPPELIN-1376 and squashes the following commits: |
||
|
Lee moon soo
|
1f6ff6875e |
[ZEPPELIN-1480] Blocking message pending 10000 for BLOCKING
### What is this PR for?
This patch try to address problem described in ZEPPELIN-1480
### What type of PR is it?
Bug Fix
### Todos
* [x] - Make websocket send thread safe
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1480
### How should this be tested?
Create multiple paragraphs (for example 10 ```%sh date``` paragraphs) and schedule it every 10sec `0/10 * * * * ?`
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes #1490 from Leemoonsoo/ZEPPELIN-1480 and squashes the following commits:
|
||
|
Renjith Kamath
|
a66b019989 |
ZEPPELIN-1321 Zeppelin HTTP and HTTPS port should be managed seperately
### What is this PR for? Use a different variable for ssl port number - add new property for ssl port ### What type of PR is it? Improvement ### Todos * [x] - Add documentation in upgrade.md ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1321 ### How should this be tested? Add ZEPPELIN_SSL_PORT or zeppelin.server.ssl.port in zeppelin env or site xml and configure the rest of ssl properties and start zeppelin. ### Screenshots (if appropriate) n/a ### Questions: * Does the licenses files need update? n/a * Is there breaking changes for older versions? n/a * Does this needs documentation? Updated install doc Author: Renjith Kamath <renjith.kamath@gmail.com> Closes #1489 from r-kamath/ZEPPELIN-1321 and squashes the following commits: |
||
|
CloverHearts
|
23a7e1b45c |
[ZEPPELIN-1502] Highlights initialization code editor bugs
### What is this PR for? When there are a variety of para graph interpreter present, When you refresh the page, the code highlights are incorrectly applied. ### What type of PR is it? Bug Fix ### Todos - [x] - Create orderId key in getEditorMode function. ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1502 ### How should this be tested? 1. Create a para-graph form below. ``` %spark println("spark syn 01"); ``` ``` %spark println("spark syn 02"); ``` ``` %spark println("spark syn 03"); ``` ``` %spark println("spark syn 04"); ``` ``` %spark println("spark syn 05"); ``` ``` %pyspark print ("pyspark syn); ``` ``` %sql SELECT * FROM SQL_HIGH WHERE ONMYCOM ``` 2. Check the highlights of each of the para graphs. 3. Refresh the page and check the highlights again. ### Screenshots (if appropriate) #### bug (focus on pyspark)  #### this pr (focus on pyspark)  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: CloverHearts <cloverheartsdev@gmail.com> Closes #1468 from cloverhearts/dev/codeeditor-bugfixed and squashes the following commits: |
||
|
Kavin
|
74ef094f6b |
[ZEPPELIN-808] Changed Permissions on Notebooks do not provide helpful error to user
### What is this PR for? This is about showing information to the user when there are errors on running paragraphs eg. there could be permission related issue with notebook. ### What type of PR is it? Improvement ### Todos NA ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-808 ### How should this be tested? * Create a notebook and change the file system permission for the notebook folder to have system level write permission to a different user.For eg. if you are running your local zeppelin server with id [USERNAME], then change the file system permission for one of your notebooks created with the former username to different one eg. ROOT user who will only have the write permission * Try to run all the paragraphs or any individual paragraph for the notebook * The information as shown in the screenshot should be displayed and the dialog could be closed by the 'Close' button. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Kavin <kavin.kumar@imaginea.com> Closes #1408 from kavinkumarks/zeppelin-808-handle-notebook-permission-error and squashes the following commits: |
||
|
Jeff Zhang
|
89cf8262e6 |
ZEPPELIN-1442. UDF can not be found due to 2 instances of SparkSession is created
### What is this PR for? The issue is that we create 2 SparkSession in zeppelin_pyspark.py (Because we create SQLContext first which will create SparkSession underlying). This cause 2 instances of SparkSession in JVM side and this means we have 2 instances of Catalog as well. So udf registered in SQLContext can not be used in SparkSession. This PR will create SparkSession first and then assign its internal SQLContext to sqlContext in pyspark. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1442 ### How should this be tested? Integration test is added. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No … Author: Jeff Zhang <zjffdu@apache.org> Closes #1452 from zjffdu/ZEPPELIN-1442 and squashes the following commits: |
||
|
Jeff Zhang
|
c717daf655 |
ZEPPELIN-1427. Scala z.show() doesn't work on v.0.6.1
### What is this PR for?
`ZeppelinContext.show` doesn't work for spark 1.6. The root cause is that `Dataset` is also available in spark 1.6, so the following line will be false when cls is `Dataset` while o is `Dataframe` in spark 1.6
```
if (cls.isInstance(o)) {
```
This PR create a list of supported class and make it a member of `ZeppelinContext `so that we don't need to create it every time.
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1427
### How should this be tested?
Tested it manually on spark 1.6 using the following sample code
```
z.show(sqlContext.sql("show tables"))
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes #1440 from zjffdu/ZEPPELIN-1427 and squashes the following commits:
|
||
|
Mina Lee
|
a3ca800311 |
[ZEPPELIN-1026] set syntax highlight based on default bound interpreter
### What is this PR for? This is complete work of #1148. Comments and tasks on #1148 has been handled in this PR. - Add syntax language information in `interpreter-setting.json` - When user type `%replName` in paragraph, back-end check if the interpreter name with `replName` exists, and return language information to front-end if it does - If user doesn't specify `%replName`, default interpreter's language will be used - Using alias name for paragraph syntax highlight ### What type of PR is it? [Bug Fix | Improvement] ### What is the Jira issue? [ZEPPELIN-1026](https://issues.apache.org/jira/browse/ZEPPELIN-1026) ### How should this be tested? 1. Create new note and make markdown interpreter to be default. 2. See if markdown syntax is applied. ### Screenshots (if appropriate) #### Case 1. When the default interpreter set to python interpreter. **Before** Has `scala` as syntax highlight language when %python is not set. <img width="665" alt="screen shot 2016-07-07 at 10 46 20 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png"> **After** Has `python` as syntax highlight language even when %python is not set. <img width="666" alt="screen shot 2016-07-07 at 10 44 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png"> #### Case 2. When use alias name as repl name. **Before** <img width="742" alt="screen shot 2016-09-08 at 4 22 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png"> **After** <img width="741" alt="screen shot 2016-09-08 at 4 34 57 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png"> ### Further possible improvements There are still several cases that Zeppelin doesn't handle syntax highlight well. These can be handled with another jira ticket/PR. 1. When default bound interpreter changes, syntax highlight is not changed accordingly 2. When copy/paste code, syntax highlight won't be applied properly since Zeppelin only checks changes when cursor is in first line. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes(for creating new interpreter) Author: Mina Lee <minalee@apache.org> Closes #1415 from minahlee/ZEPPELIN-1026 and squashes the following commits: |
||
|
Jeff Zhang
|
c61f1fbced |
ZEPPELIN-1411. UDF with pyspark not working - object has no attribute 'parseDataType'
### What is this PR for? The root cause is that SQLContext's signature changes in spark 2.0. Spark 1.6 ``` def __init__(self, sparkContext, sqlContext=None): ``` Spark 2.0 ``` def __init__(self, sparkContext, sparkSession=None, jsqlContext=None): ``` So we need to create SQLContext using named parameters, otherwise it would take intp.getSQLContext() as sparkSession which cause the issue. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1411 ### How should this be tested? Tested using the example code in ZEPPELIN-1411. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #1404 from zjffdu/ZEPPELIN-1411 and squashes the following commits: |
||
|
Prabhjyot Singh
|
1e8559e651 |
ZEPPELIN-1456: Flaky Test: AuthenticationIT
### What is this PR for?
This started happening after ZEPPELIN-1144, https://github.com/apache/zeppelin/pull/1330.
This test (testGroupPermission) is to validate the group related permission on a notebook.
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Fix CI
### What is the Jira issue?
* [ZEPPELIN-1456](https://issues.apache.org/jira/browse/ZEPPELIN-1456)
### How should this be tested?
CI xxx.10 (selenium) should be green
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? n/a
* Is there breaking changes for older versions? n/a
* Does this needs documentation? n/a
Author: Prabhjyot Singh <prabhjyotsingh@gmail.com>
Closes #1444 from prabhjyotsingh/ZEPPELIN-1456 and squashes the following commits:
|
||
|
Minwoo Kang
|
29021896bf |
[ZEPPELIN-1144]Zeppelin home page should only list notebooks with read or write permission
### What is this PR for?
If logged in user does not have Read and Write permission for a notebook, user should not see the notebook in the zeppelin home page.
### What type of PR is it?
Bug Fix
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1144
### How should this be tested?
* unit test
* online test
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Minwoo Kang <minwoo.kang@outlook.com>
Closes #1330 from mwkang/ZEPPELIN-1144 and squashes the following commits:
|
||
|
Kavin
|
294eef1327 |
[ZEPPELIN-1142] Zeppelin allows two users to simultaneously edit the notebook permissions
### What is this PR for? This is about storing the owner information on creating a note so when the same user tries to edit the permissions of the note he could do it successfully. ### What type of PR is it? Improvement ### Todos NA ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1142 ### How should this be tested? Check the below cases via the web application or REST API calls and the owner information should be persisted properly. * Create note * Clone note * Import note ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Kavin <kavin.kumar@imaginea.com> Closes #1393 from kavinkumarks/zeppelin-1142-simultaneous-note-permission-error and squashes the following commits: |
