### What is this PR for?
ZeppelinContext can be used to exchange DataFrames but there are some nasty tricks and typecasts.
It's good to provide some examples.
### What type of PR is it?
Documentation
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mohammad Amin Khashkhashi Moghaddam <amin.moghaddamV@gmail.com>
Closes#1677 from m30m/patch-3 and squashes the following commits:
a039d5c [Mohammad Amin Khashkhashi Moghaddam] Add doc for exchanging data frames
### What is this PR for?
It is introduced in ZEPPELIN-1399, we call `InterpreterFactory.getInterpreter` before calling `Paragraph.jobRun`. This PR would ignore the `InterpreterException` in `Paragraph.isValidIntepreter`
### What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1716
### How should this be tested?
Tested manually, see the following screenshot
### Screenshots (if appropriate)
Before this PR, only see the following error in log, but not in frontend
```
ERROR [2016-11-28 15:16:47,378] ({qtp1471868639-17} NotebookServer.java[onMessage]:303) - Can't handle message
org.apache.zeppelin.interpreter.InterpreterException: spark.sparkr interpreter not found
at org.apache.zeppelin.interpreter.InterpreterFactory.getInterpreter(InterpreterFactory.java:1302)
at org.apache.zeppelin.notebook.Paragraph.isValidInterpreter(Paragraph.java:596)
at org.apache.zeppelin.notebook.Paragraph.getMagic(Paragraph.java:586)
at org.apache.zeppelin.socket.NotebookServer.runParagraph(NotebookServer.java:1337)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:226)
at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)
```
After this PR
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1691 from zjffdu/ZEPPELIN-1716 and squashes the following commits:
ad7ec63 [Jeff Zhang] ZEPPELIN-1716. Error of interpreter not found is not propagated to frontend.
### What is this PR for?
Because `conda run` command removed since version `4.0.9`, PythonCondaInterpreter not working after the `conda-4.0.9`.
This PR fixes this issue.
I tested conda-4.2.12 and conda-4.0.9 .
### What type of PR is it?
Bug Fix | Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1724
### How should this be tested?
Please refer to https://github.com/apache/zeppelin/pull/1645
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1699 from astroshim/ZEPPELIN-1724 and squashes the following commits:
294b6f9 [astroshim] refactoring and fix testcase
8c3fbd3 [astroshim] fix conda version
### What is this PR for?
Currently a paragraph can display only a single type of result.
For example
```
print("""
%text textout
%html htmlout
""")
```
will display only last display system detected, "htmlout".
This pr implements multiple results supports, so not only the last, but also all the display systems in a paragraph outputs can be displayed.
To do that, Note.json format need to be changed. from
```
paragraph : [
{
result : {
code: "",
type: "",
msg: ""
},
config : {
graph : {},
...
},
...
},
...
],
...
```
to
```
paragraph : [
{
results : {
{
code: "",
msg: [
{
type: "",
data: ""
},
{
type: "",
data: ""
},
...
]
},
config : {
results : [
{
graph : {},
},
{
graph : {},
},
...
]
},
...
},
...
],
...
```
### What type of PR is it?
Improvement
### Todos
* [x] - Make InterpreterResult and InterpreterOutput support multiple display system
* [x] - Automatic migration old format to new format
* [x] - Render multiple results in front-end
* [x] - Take care Importing old notebook format
* [x] - Make helium framework support multiple results
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-212
### How should this be tested?
run following code in a single spark paragraph.
```
%spark
// default output is text
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// print something in html
println(s"""%html <h3><font style="color:blue">Some HTML</font></h3>""")
// create table
println(s"""%table key\tvalue
sun\t100
moon\t200
""")
// display text again
println("""%text""")
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// another table
Thread.sleep(1000)
println(s"""%table key\tvalue
apple\t100
banana\t200
""")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? yes
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1658 from Leemoonsoo/ZEPPELIN-212 and squashes the following commits:
0c6a13c [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
519cf23 [Lee moon soo] Update PythonInterpreterPandasSqlTest
e6268ac [Lee moon soo] Update PythonInterpreterMatplotlibTest.java
f5034b8 [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
d5981d5 [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
a1fe729 [Lee moon soo] document note format change
282504e [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
7ab6679 [Lee moon soo] update selenium test
6a897a5 [Lee moon soo] Update rest-api doc
cbbd58a [Lee moon soo] update unittest
e89c9b8 [Lee moon soo] restore tutoral note
3ba37b7 [Lee moon soo] Let old version import note without error
d09e03f [Lee moon soo] enable helium only in the last result
6682908 [Lee moon soo] Remove unnecessary listener method
0f1d28f [Lee moon soo] Merge branch 'master' into ZEPPELIN-212
73b3a81 [Lee moon soo] update selenium test
e69f1a1 [Lee moon soo] update test
f12230f [Lee moon soo] Remove unnecessary newline in test
26e201a [Lee moon soo] Update testcase
b01d70f [Lee moon soo] Make helium app work
0a5b4d1 [Lee moon soo] Update HeliumApplicationFactoryTest
d8ca07f [Lee moon soo] update NotebookTest
aaf9778 [Lee moon soo] fix r test
3e43df4 [Lee moon soo] make zeppelin-web test pass
1d6fd3e [Lee moon soo] fix compile errors
0156ffc [Lee moon soo] take care angular object
804768d [Lee moon soo] Take care output streaming
57d6b2f [Lee moon soo] Render multiple results
95b6037 [Lee moon soo] Multiple results
### What is this PR for?
There has been issues with downloading\caching Spark, esp in #1689#1696
This is hotfix for Spark download on CI.
### What type of PR is it?
Hot Fix
### Todos
- [x] do not use distrs.apache.org
- [x] levirage `download-maven-plugin` cache for Spark download
- [x] set timeout 1min and 5 re-tries on download
- [x] un-pack them under `/target/` so `mvn clean` works as expected
- [x] mute logs for `./testing/install_external_dependencies.sh`
### How should this be tested?
In CI logs, Spark should be downloaded by `spark-dependencies` and cached under `${HOME}/.m2/repository/.cache/maven-download-plugin`
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1709 from bzz/make-ci-stabel and squashes the following commits:
06c031c [Alexander Bezzubov] Move logging config to MAVEN_OPTS
702dcdd [Alexander Bezzubov] Spark download\cached, using download-maven-plugin
7040b09 [Alexander Bezzubov] Switch Spark download dir
1d85b5c [Alexander Bezzubov] Mute dependency install logs
78109af [Alexander Bezzubov] Set readTimeOut for download-maven-plugin
7a64690 [Alexander Bezzubov] Bump download-maven-plugin version to lastes 1.3.0
605dea9 [Alexander Bezzubov] Spark 2.0.1 on CI, same as in pom.xml
9ee9c04 [Alexander Bezzubov] Direct Spark download url for CI as INFRA-12996
### What is this PR for?
This PR Added "Clear output" of paragraph to the keyboard shortcut.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1686
### How should this be tested?
Click in paragraph and press Ctrl+Shift+c
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: soralee <sora0728@nflabs.com>
Closes#1686 from soralee/ZEPPELIN-1686 and squashes the following commits:
e6e8d42 [soralee] changed short key from ctrl+shift+c to ctrl+alt+l
bbd96f9 [soralee] changed short key from ctrl+shift+c to ctrl+alt+l
225d18d [soralee] changed short key from ctrl+shift+c to ctrl+alt+l
d89afa1 [soralee] Added clear output to a keyboard shortcut
### What is this PR for?
Implement notebookRepo setting in ZeppelinHub notebook repo.
This change will allow users to directly switch from instance without switching token in `zeppelin-env.sh`.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* [ZEPPELIN-1691](https://issues.apache.org/jira/browse/ZEPPELIN-1691)
### How should this be tested?
You need to have a valid [Zeppelinhub account](https://www.zeppelinhub.com/register).
You need to enable zeppelinhub notebook repo (only) in zeppelin-env.sh
```
export ZEPPELIN_NOTEBOOK_STORAGE="org.apache.zeppelin.notebook.repo.zeppelinhub.ZeppelinHubRepo"
export ZEPPELINHUB_API_ADDRESS="https://www.zeppelinhub.com"
```
[Enable shiro](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html#1-enable-shiro) and set it to use [Zeppelinhub realm](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html#zeppelinhub).
Start zeppelin, login, you will see your list of notebooks, if you want to switch instance, go to NotebookRepos, and edit.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: Anthony Corbacho <corbacho.anthony@gmail.com>
Closes#1681 from anthonycorbacho/ZEPPELIN-1691 and squashes the following commits:
fe86558 [Anthony Corbacho] Add logger in getUserInstances try-catch
4ed62af [Anthony Corbacho] Implement NotebookRepo settings option in ZeppelinHub notebook Repository. This change will allow users to directly switch from instance without switching token in zeppelin-env.sh.
### What is this PR for?
Previously on #1676, I excluded path **twice**. It's my mistake! khalidhuseynov pointed that out (Thanks!).
### What type of PR is it?
Hot Fix
### How should this be tested?
Build with command `mvn clean package install -DskipTests -DskipRat`
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: Jun Kim <i2r.jun@gmail.com>
Closes#1706 from tae-jun/patch-2 and squashes the following commits:
ae6f00e [Jun Kim] Exclude org/apache/zeppelin/scio/avro/* once
### What is this PR for?
At times on downloading dependency/changing version for an interpreter results in error
```
INFO [2016-11-29 11:16:52,998] ({qtp60559178-18} InterpreterRestApi.java[updateSetting]:137) - Update interpreterSetting 2C2ENJBEV
ERROR [2016-11-29 11:16:53,047] ({Thread-61} InterpreterFactory.java[run]:494) - Error while downloading repos for interpreter group : jdbc, go to interpreter setting page click on edit and save it again to make this interpreter work properly. : File does not exist: /Users/xxx/local-repo/2C2ENJBEV/twill-zookeeper-0.6.0-incubating.jar
java.io.FileNotFoundException: File does not exist: /Users/xxx/local-repo/2C2ENJBEV/twill-zookeeper-0.6.0-incubating.jar
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2275)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653)
at org.apache.zeppelin.interpreter.InterpreterFactory$3.run(InterpreterFactory.java:470)
ERROR [2016-11-29 11:16:53,047] ({Thread-64} InterpreterFactory.java[run]:494) - Error while downloading repos for interpreter group : jdbc, go to interpreter setting page click on edit and save it again to make this interpreter work properly. : File does not exist: /Users/xxx/local-repo/2C2ENJBEV/zookeeper-3.4.6.jar
java.io.FileNotFoundException: File does not exist: /Users/xxx/local-repo/2C2ENJBEV/zookeeper-3.4.6.jar
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2275)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653)
at org.apache.zeppelin.interpreter.InterpreterFactory$3.run(InterpreterFactory.java:470)
```
but on re-saving it works fine.
### What type of PR is it?
[Bug Fix]
### Todos
### What is the Jira issue?
* [ZEPPELIN-1725](https://issues.apache.org/jira/browse/ZEPPELIN-1725)
### How should this be tested?
Configure following as dependencies in JDBC
```
org.apache.hive:hive-jdbc:2.0.1
org.apache.hadoop:hadoop-common:2.7.1
org.apache.hive.shims:hive-shims-0.23:2.1.0
org.apache.phoenix:phoenix-core:4.7.0-HBase-1.1
```
now try changing one of it, say "org.apache.hadoop:hadoop-common:2.7.1" to "org.apache.hadoop:hadoop-common:2.6.0"
Save should happen all the time.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? n/a
* Is there breaking changes for older versions? n/a
* Does this needs documentation? n/a
Author: Prabhjyot Singh <prabhjyotsingh@gmail.org>
Closes#1698 from prabhjyotsingh/ZEPPELIN-1725 and squashes the following commits:
b535bb9 [Prabhjyot Singh] do a check for FileNotFoundException, which would mean it was already deleted/not present.
### What is this PR for?
In ZEPPELIN-1607, I'd like refactor livy interpreter to scoped mode by default, this require username when open this interpreter. So I propose to pass username when creating interpreter through thrift.
What I did in this PR.
* update `RemoteInterpreterService.thrift` and regenerate the java thrift code.
* update `genthrift.sh`, otherwise hashCode method won't be generated correctly.
* This is one compilation issue (`PythonDockerInterpreterTest.java`) in the existing master branch, I also fix it here.
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1707
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1679 from zjffdu/ZEPPELIN-1707 and squashes the following commits:
763455f [Jeff Zhang] regenerate it using thrift 0.9.2
a247552 [Jeff Zhang] ZEPPELIN-1707. Pass userName when creating interpreter through thrift
### What is this PR for?
This PR removes a non-standard string prototype method `contains` that can cause potential bugs in the future maintenance.
### What type of PR is it?
Bug Fix
### Todos
* [x] - remove usage of non-standard string method `contains` in favor of standard `indexOf`
### How should this be tested?
Download as `csv / tsv` (graph view) should work as expected
Author: felizbear <ilya@nflabs.com>
Closes#1701 from felizbear/front-end-do-not-modify-protoypes-please and squashes the following commits:
ae9820e [felizbear] remove usage of non-standard string method
### What is this PR for?
https://github.com/apache/zeppelin/pull/1554 added a `processenduser.md`, but the navigation menu different name on it's link `userimpersonation.html`
This PR changes filename from `processenduser.md` to `userimpersonation.md`.
### What type of PR is it?
Hot Fix
### Todos
* [x] - Change file name
### What is the Jira issue?
https://github.com/apache/zeppelin/pull/1554
### How should this be tested?
build docs and navigate "Interpreter -> Interpreter User Impersonation"
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes#1704 from Leemoonsoo/fix_link and squashes the following commits:
8ede96e [Lee moon soo] Change filename
### What is this PR for?
This PR fixes build error of scio Intepreter on CentOS.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1714
### How should this be tested?
Run `mvn clean package -DskipTest' on CentOS
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1685 from astroshim/fix/scio-builderror and squashes the following commits:
330f38d [astroshim] add scala prefix
### What is this PR for?
Currently, the z.run command is restricted.
Only paragraphs in a single note can be executed.
I have modified this to allow you to freely execute paragraphs of other notes.
This PR provides the basis for the freeful use of Zeppelin's resources at each Interpreter implementation.
### What type of PR is it?
Improvement, Feature
### Todos
- [x] extends z.run
- [x] run all paragraph in external note
- [x] run paragraph for external note.
- [x] get resource for zeppelin in each interpreter.
- [x] improve test case.
- [x] how to use docuement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1665
### How should this be tested?
Currently under development.
run paragraph in same note
```
%spark
z.run("paragraphID")
```
run paragraph with external note
```
z.run("noteid", "paragraphid");
```
all note run
```
z.runNote("noteid");
```
### Screenshots (if appropriate)
- paragraph run
- noterun
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: CloverHearts <cloverheartsdev@gmail.com>
Closes#1637 from cloverhearts/extends-zrun-remote-transaction and squashes the following commits:
41fa9d7 [CloverHearts] restore unless changed and import
113b475 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
03a3a2b [CloverHearts] testcase change z.run(2, context) to z.run(2)
2a2c173 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
f2e3bcf [CloverHearts] fix TestCase
5a80a5a [CloverHearts] last test case time check to print string
e6cd82c [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
3862166 [CloverHearts] regenerate thrfit class
5ec4640 [CloverHearts] change defined protocol for thrift
7562535 [CloverHearts] remove unused import and asterisk import
8a54917 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
342752d [CloverHearts] add document for extends z.run and z.runNote
292319a [CloverHearts] add test case for extends z.run and z.runNote
10c2a47 [CloverHearts] Implement runNote and re implement run method
f9661c8 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
9ab05af [CloverHearts] Change structure and remove remoteWorksManager
8cbe46c [CloverHearts] remote remoteworksController in interpreter.java
8d42c16 [CloverHearts] Merge branch 'master' into extends-zrun-remote-transaction
f11fed4 [CloverHearts] Merge branch 'workflow' into extends-zrun-remote-transaction
c074f07 [CloverHearts] fix sio support
4b1ef08 [CloverHearts] fix thrift interface
2628a20 [CloverHearts] fix thrift
6fbe08a [CloverHearts] Merge branch 'master' into workflow
3f75bd5 [CloverHearts] support scald
55e8704 [CloverHearts] support spark r
5a7886f [CloverHearts] fix sio support
afb9db7 [CloverHearts] Merge branch 'master' into workflow
3ed556c [CloverHearts] remove debug console message.
3d34f9e [CloverHearts] Implement getParagraphRunner transaction.
2523238 [CloverHearts] Implement eventForWait class
0570ae8 [CloverHearts] add remote works controller class and include interpreter factory
6e1f219 [CloverHearts] code base workflow for remote zeppelin server control default thrift transaction.
### What is this PR for?
Enabling to change values of multiple selectboxes witout running the paragraph
### What type of PR is it?
[Bug Fix]
### Todos
No
### What is the Jira issue?
[ZEPPELIN-1013]
### How should this be tested?
1. Create a paragraph with a selectbox via z.select(...)
2. Select any value other than currently selected
Actual result: value changes and paragraph gets executed
Expected result: value changes
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
No
Author: Mleekko <mleekko@gmail.com>
Closes#1647 from Mleekko/master and squashes the following commits:
13b65a2 [Mleekko] [ZEPPELIN-1013] Don't run paragraph on selectbox change
### What is this PR for?
Allow users to issue insert/update/upsert statements from Zeppelin notes
### What type of PR is it?
Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
[ZEPPELIN-1645](https://issues.apache.org/jira/browse/ZEPPELIN-1645)
### How should this be tested?
For Phoenix Interpreter:
Paragraph 1:
%jdbc
UPSERT INTO CX_TEST (ACCT_NBR, HSE_ID) VALUES ('a', 'b')
Paragraph 2:
%jdbc
select count(*) from cx_test
Result: 1
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
Yes - JDBC README updated
Author: Randy Gelhausen <rgelhau@gmail.com>
Closes#1617 from randerzander/master and squashes the following commits:
1cf1ce3 [Randy Gelhausen] Added check for connection autocommit status
### What is this PR for?
Changed the phase of xml validation plugin from validate to verify so that developers can use `mvn test` in submodule dirctories
(**this PR doesn't affect on runtime application behaivor.**)
```
[INFO] --- xml-maven-plugin:1.0.1:validate (default) zeppelin-markdown ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.643 s
[INFO] Finished at: 2016-11-28T17:53:47+09:00
[INFO] Final Memory: 17M/307M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:xml-maven-plugin:1.0.1:validate (default) on project zeppelin-markdown: Failed to load schema with public ID null, system ID _tools/maven-4.0.0.xsd: /Users/lambda/github/apache-zeppelin/zeppelin-feature/markdown/_tools/maven-4.0.0.xsd (No such file or directory) -> [Help 1]
[ERROR]
```
### What type of PR is it?
[Improvement]
### Todos
### What is the Jira issue?
[ZEPPELIN-1695](https://issues.apache.org/jira/browse/ZEPPELIN-1695?filter=-3)
### How should this be tested?
```
$ cd markdown
$ mvn test
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Closes#1693 from 1ambda/chore/modify-xml-plugin-phase-to-verify and squashes the following commits:
cc3858b [1ambda] chore: Set xml plugin phase to verify
### What is this PR for?
Since most of authentication related realm classes has moved from `org.apache.zeppelin.server` package into `org.apache.zeppelin.realm`, we need to update corresponding `shiro.ini.template` file
### What type of PR is it?
Improvement | Documentation
### Todos
* [x] - update class packages
### What is the Jira issue?
N/A
### How should this be tested?
same as for testing `LdapGroupRealm` or `ActiveDirectoryGroupRealm`
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Khalid Huseynov <khalidhnv@gmail.com>
Closes#1700 from khalidhuseynov/update/shiro-config and squashes the following commits:
2eabc24 [Khalid Huseynov] update package names
### What is this PR for?
Take 2 of #1618 because I had some earlier problems with rebasing. Since, then I have added some new features, namely:
- Matplotlib integration tests for pyspark
- `install_external_dependencies.sh` which conditionally installs the R or python dependencies based on the specified build profile in `.travis.yml`. This saves several minutes of time for a few of the build profiles since the R dependencies are compiled from source and therefore take quite a bit of time to install.
- The extra python unit tests which require external dependencies (`matplotlib` and `pandas`) are now relegated to two separate build profiles. This is done primarily to efficiently test both Python 2 and 3.
- Some minor bugs in the python and pyspark interpreters (mostly with respect to python 3 compatibility) were caught as a result of these tests, and are also fixed in this PR.
### What type of PR is it?
Improvement and Bugfix
### What is the Jira issue?
[ZEPPELIN-1639](https://issues.apache.org/jira/browse/ZEPPELIN-1639)
### How should this be tested?
CI tests should be green!
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1632 from agoodm/ZEPPELIN-1639 and squashes the following commits:
01380c2 [Alex Goodman] Make sure python 3 profile uses scala 2.11
363019e [Alex Goodman] Use spark 2.0 with python 3
a4f43af [Alex Goodman] Update comments in .travis.yml
5a60181 [Alex Goodman] Isolate python tests
73663f6 [Alex Goodman] Update tests for new InterpreterContext constructor
5709c5d [Alex Goodman] Re-add pyspark to build profile
ee95d67 [Alex Goodman] Move python 3 tests to all modules with spark 2.0
3a76958 [Alex Goodman] Travis
42da31c [Alex Goodman] Shorten python version
b6b88be [Alex Goodman] Add python dependencies to .travis.yml
### What is this PR for?
Fix error with
```
$ ls -la .spark-dist ${HOME}/.m2/repository/.cache/maven-download-plugin
ls: cannot access /home/travis/.m2/repository/.cache/maven-download-plugin: No such file or directory
.spark-dist:
total 4
drwxr-xr-x 2 travis travis 6 Nov 25 19:27 .
drwxr-xr-x 47 travis travis 4096 Nov 25 19:27 ..
The command "ls -la .spark-dist ${HOME}/.m2/repository/.cache/maven-download-plugin" failed and exited with 2 during .
Your build has been stopped.
```
https://travis-ci.org/apache/zeppelin/jobs/178918031
### What type of PR is it?
Bug Fix
### How should this be tested?
CI
### Questions:
* Does the licenses files need update? None
* Is there breaking changes for older versions? None
* Does this needs documentation? None
Author: Felix Cheung <felixcheung_m@hotmail.com>
Closes#1683 from felixcheung/fixls and squashes the following commits:
e399ce0 [Felix Cheung] this passes http://lint.travis-ci.org/0548861 [Felix Cheung] fix exit code
### What is this PR for?
When building Zeppelin with `mvn clean package install -DskipTests -DskipRat`, it fails on `scio` with an error log below:
```sh
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.13:check (checkstyle-fail-build) on project zeppelin-scio_2.10: You have 51 Checkstyle violations. -> [Help 1]
```
This is because of **style check on a generated source**, which is exactly `$ZEPPELIN_HOME/scio/target/generated-sources/avro/org/apache/zeppelin/scio/avro/Account.java`
This PR will make style check excludes the generated source.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1705](https://issues.apache.org/jira/browse/ZEPPELIN-1705)
### How should this be tested?
Run `mvn clean package install -DskipTests -DskipRat`. It should print `BUILD SUCCESS`
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: Jun <i2r.jun@gmail.com>
Closes#1676 from tae-jun/ZEPPELIN-1705 and squashes the following commits:
2c1c56b [Jun] Exclude unnecessary source file when check style on scio
