…e to disable proxy behavior
### What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html
### What type of PR is it?
[Improvement]
* add a configuration item zeppelin.jdbc.auth.kerberos.proxy.enable to disable kerberos behaviour
as we know， in current version of zeppelin， if we have kerberos auth configured，and using zeppelin with a user login，the jdbc interpreter will do a proxy behavior with the login user automatically，but in many cases， we do not want do this，and we do not want bind the zeppelin user system with kerberos auth system. I think it's make senses to add a configuration item to disable this behavior.
### Todos
* [ ] - Task
### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/
* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533]
[ZEPPELIN-2353] (https://issues.apache.org/jira/browse/ZEPPELIN-2353)
### How should this be tested?
Outline the steps to test the PR here.
when we have kerberos auth configured, and using zeppelin with a login user
add configuration:
"zeppelin.jdbc.auth.kerberos.proxy.enable=false"
for jdbc iterpreter configuration, the jdbc iterpreter will not do the proxy behavior with the login user
### Screenshots (if appropriate)
<img width="1405" alt="2017-04-04 9 12 03" src="https://cloud.githubusercontent.com/assets/869480/24658501/7739e680-197c-11e7-90ab-c1938e31efc7.png">
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
Author: LeiWang <wanglei6744@163.com>
Author: lei wang <lei1989@outlook.com>
Closes#2222 from wary/main-master and squashes the following commits:
1f26bc3 [LeiWang] Update jdbc.md
ba34a41 [LeiWang] Update JDBCInterpreter.java
444b6ab [LeiWang] Update JDBCInterpreter.java
ea11ce4 [LeiWang] Update jdbc.md
ee9cd18 [LeiWang] Update jdbc.md
e04fb38 [LeiWang] Update jdbc.md
75e4b9f [LeiWang] Update jdbc.md
b850a42 [LeiWang] Update JDBCInterpreter.java
067696a [lei wang] add configure zeppelin.jdbc.auth.kerberos.proxy for kerberos auth type to disable proxy behavior
### What is this PR for?
Groovy Interpreter
### What type of PR is it?
Feature
### Todos
* [ Tests ] - Task
* [ Documentation ] - Task
### What is the Jira issue?
[ZEPPELIN-2176]
### How should this be tested?
Follow the groovy interpreter documentation samples
### Questions:
* Does the licenses files need update? YES
* Is there breaking changes for older versions? NO
* Does this needs documentation? YES
Author: dlukyanov <dlukyanov@ukr.net>
Author: dm <dm>
Closes#2135 from dlukyanov/master and squashes the following commits:
faf213f [dlukyanov] ZEPPELIN-2176 comments from @AhyoungRyu - remove @author - remove commented code
89c3ed5 [dlukyanov] retry
ca65947 [dlukyanov] deprecated
3dd53e2 [dlukyanov] ZEPPELIN-2176 comments from @AhyoungRyu - Zeppelin follows Google Java code - interpreter alphabetical order in _navigation.html - direct link to MarkupBuilder in groovy help
fe08159 [dlukyanov] retry
ca8bea6 [dlukyanov] Update groovy.md
a5b37a1 [dlukyanov] ZEPPELIN-2176 https://github.com/apache/zeppelin/pull/2135#issuecomment-289308850 - Inside of docs directory, groovy.md will need some header to be compiled with Jekyll - Menu in docs also need link to groovy - .travis.yml we need add !groovy
4abf649 [dm] Merge branch 'master' of https://github.com/apache/zeppelin41a1702 [dlukyanov] ZEPPELIN-2176 https://github.com/apache/zeppelin/pull/2135#issuecomment-288829494 - implement shared script variables - move docs - implement run methods
dd388b3 [dlukyanov] retry
b34b42a [dlukyanov] retry
0d7732a [dlukyanov] retry
2646fa8 [dlukyanov] ZEPPELIN-2176 groovy interpreter, fix unchecked, add to configs, move HTTP.groovy to resources to simplify build, add default z-properties
5fa26e0 [dlukyanov] ZEPPELIN-2176 groovy interpreter, fix unchecked, add to configs, move HTTP.groovy to resources to simplify build, add default z-properties
aa427cd [dlukyanov] retry
addf167 [dlukyanov] retry
db4c35b [dlukyanov] Update README.md
fa779ea [dlukyanov] groovy interpreter
### What is this PR for?
For now, every time when I want to try new spark version, I have to change file `SparkVersion.java` and rebuild it. It is not so convenient, so I'd like to add property `zeppelin.spark. enableSupportedVersionCheck` for spark interpreter. So that I can try new spark version by setting this property as false, of course it is only for zeppelin developer.
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2324
### How should this be tested?
Verify it in spark master
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2197 from zjffdu/ZEPPELIN-2324 and squashes the following commits:
f821839 [Jeff Zhang] address comments
4193ee3 [Jeff Zhang] ZEPPELIN-2324. Add property zeppelin.spark.unSupportedVersionCheck for trying new spark version
… to display results
### What is this PR for?
Livy SQL interpreter truncate result strings of size greater than 20. In some cases, we like to see the full string. We are adding a interpreter property **zeppelin.livy.spark.sql.field.truncate** to control whether to truncate strings or not. By default, **zeppelin.livy.spark.sql.field.truncate** is set to **true**.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1965
### How should this be tested?
Set zeppelin.livy.spark.sql.field.truncate to true or false
Run a SQL query which produces string values of length greater than 20.
Depending on the value of zeppelin.livy.spark.sql.field.truncate, the strings will either get truncated or not.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Benoy Antony <benoy@apache.org>
Closes#2201 from benoyantony/master and squashes the following commits:
bb006c0 [Benoy Antony] changed field name and description
9eae68b [Benoy Antony] added a null check to avoid testcase failures, another nullcheck for backward compatibility and added two new testcases
ab1ead2 [Benoy Antony] documented zeppelin.livy.spark.sql.truncate
b6252be [Benoy Antony] [ZEPPELIN-1965] Livy SQL Interpreter: Should use df.show(1000, false) to display results
### What is this PR for?
Livy server support https, but the currently livy interpreter doesn't support it. This PR is for for the supporting to connect with livy through https
### What type of PR is it?
[Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2261
### How should this be tested?
Tested manually on livy server with ssl enabled.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2139 from zjffdu/ZEPPELIN-2261 and squashes the following commits:
52fc204 [Jeff Zhang] address comment
53230c3 [Jeff Zhang] [ZEPPELIN-2261]. Support to connect with livy through https
### What is this PR for?
Separate precode by prefix. Added the ability to set different precode for different data sources
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2245
### How should this be tested?
1. Set properties
```
default.password 1
default.precode set search_path='test_path'
default.url jdbc:postgresql://localhost:5432/
default.user postgres
mysql.driver com.mysql.jdbc.Driver
mysql.password 1
mysql.precode set v=12
mysql.url jdbc:mysql://localhost:3306/
mysql.user root
```
2. Run
`show search_path`
3. Run
```
%jdbc(mysql)
select v
```
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Tinkoff DWH <tinkoff.dwh@gmail.com>
Closes#2121 from tinkoff-dwh/ZEPPELIN-2245 and squashes the following commits:
970c064 [Tinkoff DWH] [ZEPPELIN-2245] editing documentation
a136a0e [Tinkoff DWH] [ZEPPELIN-2245] documentation for usage of precode
f896ea8 [Tinkoff DWH] [ZEPPELIN-2245] separate precode into JDBCInterpreter
### What is this PR for?
Added new property "hive.proxy.user" to disable hive impersonation (on some clusters, this option is disabled) in order to make Hive Interpreter even without this
### What type of PR is it?
Feature
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1968
### How should this be tested?
Set "hive.proxy.user" to true in the jdbc interpreter setttings, and you should see "Using hive proxy user" in the jdbc logs.
If "hive.proxy.user" has another value, this is not mentionned in the logs
You can also test with the appropriate hive configuration, but this could take longer :)
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Paolo Genissel <paolo.genissel-monsallier@1000mercis.com>
Closes#2051 from gfalcone/hive_impersonation and squashes the following commits:
a39d11c [Paolo Genissel] Fixed last NPE
1f7f685 [Paolo Genissel] Fixed NPE when getting hive.proxy.user property
433eefb [Paolo Genissel] Added documentation for feature
d6f0c62 [Paolo Genissel] Added property to disable hive user impersonation
### What is this PR for?
Adds property "precode". Value of property contains SQL which executes while opening connection.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1988
### How should this be tested?
1) Set property zeppelin.interpreter.precode =` set search_path='test, public' `
2) Execute `%jdbc show search_path`
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Tinkoff DWH <tinkoff.dwh@gmail.com>
Closes#2078 from tinkoff-dwh/ZEPPELIN-1988 and squashes the following commits:
cd46cce [Tinkoff DWH] [ZEPPELIN-1988] trim precode
42ffcb7 [Tinkoff DWH] [ZEPPELIN-1988] fix condition
7636b3f [Tinkoff DWH] Merge remote-tracking branch 'origin/master' into ZEPPELIN-1988
66d6ae4 [Tinkoff DWH] [ZEPPELIN-1988] fixes of review items
9d37bc4 [Tinkoff DWH] [ZEPPELIN-1988] fix
ba3477a [Tinkoff DWH] [ZEPPELIN-1988] add property "precode" to JDBCInterpreter
### What is this PR for?
`zeppelin.pyspark.python` is zeppelin configuration for the python exec on driver side, it won't affect executor side. It would be better to use `PYSPARK_PYTHON` and `PYSPARK_DRIVER_PYTHON` which is what spark use officially. So that user can define their own python exec in interpreter setting for different version of python rather than defining them `zeppelin-env.sh` which is shared globally.
### What type of PR is it?
[ Improvement ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2195
### How should this be tested?
Tested it manually.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2079 from zjffdu/ZEPPELIN-2195 and squashes the following commits:
fa71cb2 [Jeff Zhang] address comments
fd89a1e [Jeff Zhang] ZEPPELIN-2195. Use PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON over zeppelin.pyspark.python
### What is this PR for?
Add pig wiki page pig doc
### What type of PR is it?
[Documentation |]
### Todos
* [ ] - Task
### What is the Jira issue?
No jira created
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#2004 from zjffdu/pig_doc and squashes the following commits:
e5a564a [Jeff Zhang] rename zeppelin to Zeppelin and pig to Pig
65458ff [Jeff Zhang] address comments and minor update on pig tutorial
c6cb5ff [Jeff Zhang] update pig tutorial
b8542de [Jeff Zhang] [MINOR] add pig wiki page to pig doc
### What is this PR for?
We don't have to maintain PostgresqlInterpreter because JDBCInterpreter covers all functions of PostgresqlInterpreter. It reduces maintenance costs.
### What type of PR is it?
[Feature]
### Todos
* [x] - Remove files and lines related to PostgresqlInterpreter
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2003
### How should this be tested?
N/A
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? Yes
* Does this needs documentation? Yes, but already documented in jdbc docs
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#1945 from jongyoul/ZEPPELIN-2003 and squashes the following commits:
5dc58f2 [Jongyoul Lee] Fixed doc
ec31f78 [Jongyoul Lee] Change docs to make a link to jdbc.md
bfd70ef [Jongyoul Lee] Removed Postgresql from docs and configurations
98a106e [Jongyoul Lee] Removed whole packages about Postgresql
### What is this PR for?
Add HTTP client to elasticsearch interpreter.
### What type of PR is it?
Feature
### Todos
* [X] - Source code
* [X] - Tests
* [X] - License
* [X] - Docs
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1821
### How should this be tested?
* Start an Elasticsearch node
* Configure the elasticsearch interpreter to use http
* Create queries in a note using elasticsearch
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? Yes
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Bruno Bonnin <bbonnin@gmail.com>
Author: Bruno Bonnin <bruno.bonnin@myscript.com>
Closes#1902 from bbonnin/master and squashes the following commits:
f5a539e [Bruno Bonnin] Remove commented code lines
86153a8 [Bruno Bonnin] Merge remote-tracking branch 'upstream/master'
2e1bbbd [Bruno Bonnin] Merge remote-tracking branch 'upstream/master'
19e888e [Bruno Bonnin] Remove bad code in test
523d155 [Bruno Bonnin] Replace Java 8 methods
6bcf369 [Bruno Bonnin] Fix issue with id containing special chars (/, #)
4e9812e [Bruno Bonnin] Merge elasticsearch/pom.xml
5a96ae0 [Bruno Bonnin] Merge branch 'master' into master
e2365fb [Bruno Bonnin] Update elasticsearch/pom.xml
28b9805 [Bruno Bonnin] Update img
549db39 [Bruno Bonnin] Add HTTP client to elasticsearch interpreter
f4c5ac3 [Bruno Bonnin] HTTP-based Elasticsearch client
### What is this PR for?
Refactor the livy interpreter to use multiple `InterpreterResult` for displaying appInfo.
### What type of PR is it?
[Refactoring]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1852
### How should this be tested?
Add integration test and also test it manually
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1796 from zjffdu/ZEPPELIN-1852 and squashes the following commits:
724f22e [Jeff Zhang] update doc
e1c2eb9 [Jeff Zhang] ZEPPELIN-1852. Use multiple InterpreterResult for displaying appInfo
### What is this PR for?
Livy 0.3 support cancel operation, this PR is to support cancel in livy interpreter. First we would check the livy version, then based on the livy version, we would call the livy rest api to cancel the statement.
### What type of PR is it?
Improvement | Feature ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1432
### How should this be tested?
Tested manually, because cancel is only avaible in livy 0.3 which is not released yet.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1859 from zjffdu/ZEPPELIN-1432 and squashes the following commits:
83eaf83 [Jeff Zhang] minor update
200ca71 [Jeff Zhang] address comments
1cbeb26 [Jeff Zhang] add zeppelin.livy.pull_status.interval.millis
070fea0 [Jeff Zhang] ZEPPELIN-1432. Support cancellation of paragraph execution
### What is this PR for?
Add support for jceks stored password instead of using password in clear text. This security enhancement prevents any user from reading clear passwords from interpreter json using shell/py/spark etc. #1315 is the parent PR which fixed similar a issue in Active Directory (shiro.ini config).
### What type of PR is it?
Improvement
### Todos
### What is the Jira issue?
ZEPPELIN-1935
### How should this be tested?
Create a keystore file using the hadoop credential commandline, for this the hadoop commons should be in the classpath.
`hadoop credential create jdbc.password -provider jceks://file/user/zeppelin/conf/zeppelin.jceks`
Use the jceks file and the key to configure jdbc interpreter. Example interpreter setting:
#### Following are the supported settings for backward compatibility
- Settings with username and jceks
```
default.driver org.postgresql.Driver
default.jceks.credentialKey jdbc.password
default.jceks.file jceks://file/tmp/zeppelin.jceks
default.url jdbc:postgresql://rkamath-local-1:5432/
default.user rk-user
```
- Settings with user name and clear text password
```
default.driver org.postgresql.Driver
default.url jdbc:postgresql://rkamath-local-1:5432/
default.user rk-user
default.password password1
```
### Screenshots (if appropriate)
<img width="1392" alt="screen shot 2017-01-10 at 7 02 12 pm" src="https://cloud.githubusercontent.com/assets/2031306/21808016/5e602982-d767-11e6-88f9-3d15b9a7f0b8.png">
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Renjith Kamath <renjith.kamath@gmail.com>
Closes#1881 from r-kamath/ZEPPELIN-1935 and squashes the following commits:
5cc4db1 [Renjith Kamath] ZEPPELIN-1935 revert wildcard imports
dfabe3a [Renjith Kamath] Merge branch 'master' of https://github.com/apache/zeppelin into ZEPPELIN-1935
e62088a [Renjith Kamath] ZEPPELIN-1935 fix log message
d41d56c [Renjith Kamath] ZEPPELIN-1935 Add jceks stored password support for jdbc interpreter
### What is this PR for?
Two improvements for pig interpreter.
* Set job name via paragraph title if it exists, otherwise use the last line of pig script
* Allow to set any pig property in interpreter setting
### What type of PR is it?
[ Improvement]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1933
### How should this be tested?
Unit tested and manually tested.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1885 from zjffdu/ZEPPELIN-1933 and squashes the following commits:
d2e1cd4 [Jeff Zhang] address comments
9cee380 [Jeff Zhang] ZEPPELIN-1933. Set pig job name and allow to set pig property in pig interpreter setting
### What is this PR for?
* Minor update for pig interpreter
* Add one sample pig tutorial note which do the same thing as the spark tutorial note.
### What type of PR is it?
[Improvement | Documentation ]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1867
### How should this be tested?
Tested manually
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1830 from zjffdu/ZEPPELIN-1867 and squashes the following commits:
1c0d819 [Jeff Zhang] rename note name
50198a1 [Jeff Zhang] add more description of tutorial note
88385f2 [Jeff Zhang] Add pig tutorial note
25216f8 [Jeff Zhang] ZEPPELIN-1867. Update document for pig interpreter and add one sample note
### What is this PR for?
This PR will add an example of batch processing with Flink to Zeppelin tutorial notebooks. There are no any Flink notebooks in the tutorial at the moment.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1787](https://issues.apache.org/jira/browse/ZEPPELIN-1787)
### How should this be tested?
You should open `Using Flink for batch processing` notebook from the `Zeppelin Tutorial` folder and run all paragraphs one by one
### Questions:
* Does the licenses files need update? - **no**
* Is there breaking changes for older versions? - **no**
* Does this needs documentation? - **no**
Author: Alexander Shoshin <Alexander_Shoshin@epam.com>
Closes#1758 from AlexanderShoshin/ZEPPELIN-1787 and squashes the following commits:
83cbffb [Alexander Shoshin] remove localhost url
5255e17 [Alexander Shoshin] Merge branch 'master' into ZEPPELIN-1787
0b9df56 [Alexander Shoshin] add a link for this notebook to Zeppelin documentation
593c47d [Alexander Shoshin] convert notebook to 0.7.0 format
9013620 [Alexander Shoshin] convert notebook to 0.6.2 format
fe2a39e [Alexander Shoshin] add download instruction, change "wget" to "curl"
f64b60a [Alexander Shoshin] [ZEPPELIN-1787] Add an example of Flink Notebook
### What is this PR for?
Add `README` file for developer, besides update the `livy.md` for the recent changes on livy interpreter.
### What type of PR is it?
[Documentation]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1853
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1798 from zjffdu/ZEPPELIN-1853 and squashes the following commits:
940ebae [Jeff Zhang] remove upgrade
7233a4d [Jeff Zhang] address comments
72f40c7 [Jeff Zhang] ZEPPELIN-1853. Add README for livy interpreter
### What is this PR for?
Update scio doc - fix type $ should be %
### What type of PR is it?
Documentation
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1800
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Rafal Wojdyla <rav@spotify.com>
Closes#1754 from ravwojdyla/fix_1800 and squashes the following commits:
d4287a9 [Rafal Wojdyla] Update scio doc - fix type $ should be %
### What is this PR for?
Currently, the parameter "project" is defined with "kylin.query.project" in properties. It's not convenience when query Kylin among different projects. May I propose introducing %kylin(project_name) at the interpreter runtime? If not set, the default project will work, otherwise, will use explicit project_name for the query request.
### What type of PR is it?
Improvement
### Todos
DONE
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1512
### How should this be tested?
%kylin select ...
%kylin(new_project) select ....
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update?
No.
- Is there breaking changes for older versions?
No.
- Does this needs documentation?
Will do later.
Author: Yiming Liu <liuyiming.vip@gmail.com>
Closes#1477 from yiming187/ZEPPELIN-1512 and squashes the following commits:
b58ee7f [Yiming Liu] [ZEPPELIN-1512] Support Kylin project name in interpreter runtime
### What is this PR for?
hive-site.xml is required to configure HiveContext in SparkInterpreter.
So it'll be helpful if document provide at least simple example that can help user get some idea.
### What type of PR is it?
Improvement
### Todos
* [x] - Add simple example
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1728
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes#1708 from Leemoonsoo/ZEPPELIN-1728 and squashes the following commits:
98860ba [Lee moon soo] an example configure hive-site.xml into the classpath
### What is this PR for?
ZeppelinContext can be used to exchange DataFrames but there are some nasty tricks and typecasts.
It's good to provide some examples.
### What type of PR is it?
Documentation
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mohammad Amin Khashkhashi Moghaddam <amin.moghaddamV@gmail.com>
Closes#1677 from m30m/patch-3 and squashes the following commits:
a039d5c [Mohammad Amin Khashkhashi Moghaddam] Add doc for exchanging data frames
### What is this PR for?
Allow users to issue insert/update/upsert statements from Zeppelin notes
### What type of PR is it?
Improvement
### Todos
* [ ] - Task
### What is the Jira issue?
[ZEPPELIN-1645](https://issues.apache.org/jira/browse/ZEPPELIN-1645)
### How should this be tested?
For Phoenix Interpreter:
Paragraph 1:
%jdbc
UPSERT INTO CX_TEST (ACCT_NBR, HSE_ID) VALUES ('a', 'b')
Paragraph 2:
%jdbc
select count(*) from cx_test
Result: 1
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
Yes - JDBC README updated
Author: Randy Gelhausen <rgelhau@gmail.com>
Closes#1617 from randerzander/master and squashes the following commits:
1cf1ce3 [Randy Gelhausen] Added check for connection autocommit status
### What is this PR for?
This PR is for the multi-tenant of JDBC Interpreter.
User can create a user/password for JDBC account at the [Credential page](http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html).
The `Entity` of `Credential` is match with JDBC interpreter group name.
If the account for JDBC is not setted in the `Interpreter property` then use `Credential`'s.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1567
### How should this be tested?
Please refer to testMultiTenant() of JDBCInterpreterTest/
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1539 from astroshim/jdbc-impersonation and squashes the following commits:
46fce31 [astroshim] add explanation of InterpreterGroup
7a92236 [astroshim] fix doc and remove persist value.
63f5ea7 [astroshim] Merge branch 'master' into jdbc-impersonation
267277a [astroshim] rebase
649ff6e [astroshim] rebase
872fb49 [astroshim] fix ScioInterpreterTestCase
4387a5b [astroshim] Merge branch 'master' into jdbc-impersonation
47c463f [astroshim] update doc and html
d4eb178 [astroshim] fix docs
59aa9ff [astroshim] Merge branch 'master' into jdbc-impersonation
bf61afd [astroshim] fix testcase
5c0f5d7 [astroshim] rebase
79ba25b [astroshim] Merge branch 'master' into jdbc-impersonation
1f9c2c0 [astroshim] clean redundant code
a2f5687 [astroshim] fix impersonation
9962181 [astroshim] fix InterpreterOutput of PySparkInterpreterTest case
b55aceb [astroshim] Merge branch 'master' into jdbc-impersonation
24a8226 [astroshim] fix doc
086dfda [astroshim] fix testcase
34fe0a6 [astroshim] fix code for more simple.
fee7086 [astroshim] fix build error.
a305eca [astroshim] Merge branch 'master' into jdbc-impersonation
df80741 [astroshim] documentation for credential.
df1b1dc [astroshim] rebase and entity name convention.
63d6a1c [astroshim] change thrift version to 0.9.2
6573c1c [astroshim] change variable name
f311f34 [astroshim] fix typo
722e333 [astroshim] change testcase name
9161937 [astroshim] clean code
3dafdf0 [astroshim] add testcase
373d5f1 [astroshim] pass replName to Interpreter and use credential info for jdbc auth.
### What is this PR for?
Inspired by ZEPPELIN-1671 conda interpreter.
Docker can provides kind of virtual environment for python like conda does.
This PR implements %python.docker interpreter that helps run python process in docker container.
This PR implements feature on top of https://github.com/apache/zeppelin/pull/1645
### What type of PR is it?
Feature
### Todos
* [x] - basic feature
* [x] - unittest
* [x] - documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1683
### How should this be tested?
see screenshot
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1654 from Leemoonsoo/pydocker and squashes the following commits:
22507e6 [Lee moon soo] Add new line at the end of the file
41c09d9 [Lee moon soo] Run python process in docker container
### What is this PR for?
Conda interpreter that manages conda environment for PythonInterpreter
### What type of PR is it?
Feature
### Todos
* [x] - Basic impl
* [x] - update doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1671
### How should this be tested?
Recreate(or create new) your python interpreter setting in gui.
List all conda env
```
%python.conda
```
Activate env
```
%python.conda activate [name]
```
Deactivate env
```
%python.conda deactivate
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1645 from Leemoonsoo/conda and squashes the following commits:
4842b0a [Lee moon soo] Add usage in doc
d979c6a [Lee moon soo] Add unittest
b889443 [Lee moon soo] add usage template
9ae553b [Lee moon soo] Format output and add usage command
171cbeb [Lee moon soo] make sure single char interpreter name can be parsed
6b9525f [Lee moon soo] Fix unittest
1223796 [Lee moon soo] Remove unnecessary log
394cf8c [Lee moon soo] Conda interpreter implementation
### What is this PR for?
%dep interpreter was going to be deprecated, but we had feedback from many users that this feature provide different advantage over dependency loading via GUI so we want to keep supporting it. This PR remove deprecated message when users use %dep interpreter.
### Todos
- [x] Fix test
### What type of PR is it?
Documentation
### What is the Jira issue?
[ZEPPELIN-1622](https://issues.apache.org/jira/browse/ZEPPELIN-1622)
### How should this be tested?
Run %dep interpreter such as `z.load('/your/library.jar)` and see if deprecated message is gone.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mina Lee <minalee@apache.org>
Closes#1605 from minahlee/ZEPPELIN-1622 and squashes the following commits:
b604c4c [Mina Lee] Add %dep z.load integration test
e3ae4c8 [Mina Lee] Remove %dep interpreter deprecated message
### What is this PR for?
Fix formatting/typos/style in scio doc
### What type of PR is it?
Documentation
Author: Rafal Wojdyla <rav@spotify.com>
Closes#1633 from ravwojdyla/fix_scio_doc and squashes the following commits:
888ec68 [Rafal Wojdyla] Fix formatting/typos/style in scio doc
### What is this PR for?
Remove unmanaged, old library markdown4j dep which exists just for Websequence and YUML plugins.
(related to https://github.com/apache/zeppelin/pull/1384)
By adding Websequence and YUML plugins to pegdown
- Removing markdown4j dependency which is unmanaged library currently.
- Addtionally, we can remove `markdown.parser.type` options in **markdown interpreter**
- Fixing some bugs in Websequence and YUML plugins
- Enable others to add more plugins using pegdown sytnax.
### What type of PR is it?
Improvement
### Todos
Nothing
### What is the Jira issue?
[JIRA - ZEPPELIN-1614](https://issues.apache.org/jira/browse/ZEPPELIN-1614)
### How should this be tested?
Some functional tests are included.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? - YES
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - YES
Author: 1ambda <1amb4a@gmail.com>
Closes#1594 from 1ambda/feat/remove-markdown4j-dep and squashes the following commits:
5af1607 [1ambda] fix: Failed tests in InterpreterRestApiTest
c57fdcb [1ambda] docs: Update markdown.md
5c62236 [1ambda] docs: Update upgrade.md for '%md'
a1e779d [1ambda] style: Use zeppelin checkstyle.xml
13e0dc4 [1ambda] Update: interpreter setting and docs
de3549d [1ambda] chore: Cleanup duplicated markdown4j license
7c5d41e [1ambda] fix: Parse style param optionally in webseq
8831ca1 [1ambda] fix: Wrap exceptions in catch stmt
9268695 [1ambda] Revert "fix: Cleanup unused Markdown4j Parser"
33fb800 [1ambda] Revert "docs: Remove markdown.parser.type option"
fddc459 [1ambda] Revert "chore: Remove markdown4j dep and update license"
a59ebbd [1ambda] Revert "fix: Set {} to avoid 503"
4e48933 [1ambda] Revert "fix: Parse style param optionally in webseq"
8cfb2c8 [1ambda] Revert "fix: style and misspell in docs"
73956e0 [1ambda] Revert "fix: Propagate exception in YUML plugin"
1b7787f [1ambda] fix: Propagate exception in YUML plugin
c656d08 [1ambda] fix: style and misspell in docs
dc4f110 [1ambda] fix: Parse style param optionally in webseq
b43e14e [1ambda] fix: Set {} to avoid 503
c48cc53 [1ambda] chore: Remove markdown4j dep and update license
81fdfcc [1ambda] docs: Remove markdown.parser.type option
cf19f0b [1ambda] fix: Cleanup unused Markdown4j Parser
98b2809 [1ambda] fix: Add missing docs
3e9716d [1ambda] feat: Yuml markdown plugin
3247c67 [1ambda] feat: Support webseq markdown plugin
### What is this PR for?
Add 2 JDBC configs and useful links to JDBC doc so that users can connect those backends easily :)
### What type of PR is it?
Documentation
### Todos
* [ ] - Task
### What is the Jira issue?
[ZEPPELIN-1650](https://issues.apache.org/jira/browse/ZEPPELIN-1650)
### How should this be tested?
See screenshots.
### Screenshots (if appropriate)
<img width="706" alt="example_redshift" src="https://cloud.githubusercontent.com/assets/4968473/20208653/01b1e468-a833-11e6-9166-b05d15e9a5fe.png">
<img width="577" alt="example_mariadb" src="https://cloud.githubusercontent.com/assets/4968473/20208654/02b49a22-a833-11e6-9b29-5a61f2b6a4bb.png">
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Closes#1621 from 1ambda/jdbc-doc-improve and squashes the following commits:
0aab878 [1ambda] docs: Update postgres img ;)
6009eb9 [1ambda] docs: Add postgres section, hive2 doc link
ce24af1 [AhyoungRyu] Make mariadb and redshift screenshots same with others
2fb5db1 [1ambda] docs: Add mariadb, redshift configs
### What is this PR for?
This is document for Apache Kylin Interpreter for Apache Zeppelin
### What type of PR is it?
Documentation
### Todos
Done
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-415
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
This is document
Author: zhongjian <jiatuer@163.com>
Author: Jason <jiatuer@163.com>
Closes#1576 from janzhongi/master and squashes the following commits:
5464006 [zhongjian] add default value for kylin query api config
c26fa77 [Jason] ZEPPELIN-415 document for Apache Kylin Interpreter
### What is this PR for?
Doc should deliver key features and recommended usage more simple and easy way.
- docs/install/install.md has lots of duplicated section with README.md.
- docs/install/install.md includes install from binary as well as build from source. I've seen that makes some beginners try download binary and then source build it again.
- recommended and key usage need to be highlighted.
- Be less verbose in key instructions. Move optional, additional info from in the middle of key instruction to end of the each page.
### What type of PR is it?
Improvement
### Todos
* [x] - improve doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1644
### How should this be tested?
Run doc locally
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Lee moon soo <moon@apache.org>
Closes#1615 from Leemoonsoo/ZEPPELIN-1644 and squashes the following commits:
e554216 [Lee moon soo] Add build item in index.md
c6b25f8 [Lee moon soo] Update suggested build command.
c7a19eb [Lee moon soo] Add build.md
e59fe3f [Lee moon soo] make document easier to follow key instructions
### What is this PR for?
This PR fixes Apache Ignite version mentions in interpreter documentation.
### What type of PR is it?
[Improvement]
Author: agura <agura@gridgain.com>
Author: agura <andreyn.gura@gmail.com>
Closes#1581 from agura/ignite-docs and squashes the following commits:
59fae20 [agura] Interpreter binding information link is fixed
33c9cac [agura] Minor fixes
d8710b3 [agura] Ignite versions fixed
### What is this PR for?
This PR adds support for formatting math formula formatting in %html display system using MathJax library.
### What type of PR is it?
Feature
### Todos
* [x] - Format math formula with MathJax library
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-777
### How should this be tested?
try run following codes.
```
%md
When \\(a \\ne 0\\), there are two solutions to \\(ax^2 + bx + c = 0\\) and they are
$$x = {-b \pm \sqrt{b^2-4ac} \over 2a}.$$
```
Note MathJax works better with `markdown.parser.type` property set `pegdown`, in markdown interpreter. With default markdown4j parser, some formula is not well displayed. (for example, ax^2) I think this will not be a big problem because of we'll remove markdown4j https://github.com/apache/zeppelin/pull/1594
```
%sh echo -e "%html \$\$a = b\$\$"
```
```
%spark println("%html $$b = c$$")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? yes
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1606 from Leemoonsoo/ZEPPELIN-777 and squashes the following commits:
af8e079 [Lee moon soo] Package MathJax resources
2afedde [Lee moon soo] Fix typo
dd02bec [Lee moon soo] Add doc for mathmetical expression
174d7ad [Lee moon soo] Add license
bb762c3 [Lee moon soo] Format formula using MathJax
### What is this PR for?
This PR adds Mahout functionality for the Spark Interpreter.
### What type of PR is it?
Improvement
### Todos
- [x] Implement Mahout Interpreter in Spark
- [x] Add Unit Tests
- [x] Add Documentation
- [x] Add Example Notebook
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-116
### How should this be tested?
Open a Spark Notebook with Mahout enabled and run a few simple commands using the R-Like DSL and Spark Distributed Context (Mahout Specific)
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update?
No
- Is there breaking changes for older versions?
No
- Does this needs documentation?
Yes
Author: rawkintrevo <trevor.d.grant@gmail.com>
Closes#928 from rawkintrevo/mahout-terp and squashes the following commits:
ed6eff0 [rawkintrevo] [ZEPPELIN-116] renamed add_mahout_interpreters.py and overwrite_existing feature
e7d4e12 [rawkintrevo] [ZEPPELIN-116] Made add_mahout.py script more resilient
7e83832 [rawkintrevo] [ZEPPELIN-116] Add Mahout Interpreters
### What is this PR for?
This PR is the first of two major steps needed to improve matplotlib integration in Zeppelin (ZEPPELIN-1344). The latter, which is a plotting backend with fully interactive tools enabled, will be done afterwards in a separate PR. This PR specifically for automatically displaying output from calls to matplotlib plotting functions inline with each paragraph. Thanks to the addition of post-execute hooks (ZEPPELIN-1423), there is no need to call any `show()` function to display an inline plot, just like in Jupyter.
### What type of PR is it?
Improvement
### Todos
The main code has been written and anyone who reads this is encouraged to test it, but there are a few minor todos:
- [x] - Add unit tests
- [x] - Add documentation
- [x] - Add screenshot showing iterative plotting with angular mode
### What is the Jira issue?
[ZEPPELIN-1345](https://issues.apache.org/jira/browse/ZEPPELIN-1345)
### How should this be tested?
In a pyspark or python paragraph, enter and run
``` python
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
```
The plot should be displayed automatically without calling any `show()` function whatsoever. A special method called `configure_mpl()` can also be used to modify the inline plotting behavior. For example,
``` python
z.configure_mpl(close=False, angular=True)
plt.plot([1, 2, 3])
```
allows for iterative updates to the plot provided you have PY4J installed for your python installation (which of course is always the case if you use pypsark). To clarify, this feature only currently works with pyspark (not python as there are no `angularBind()` and `angularUnbind()` methods yet). Doing something like:
```
plt.plot([3, 2, 1])
```
will update the plot that was generated by the previous paragraph by leveraging Zeppelin's Angular Display System. However, by setting `close=False`, matplotlib will no longer automatically close figures so it is now up to the user to explicitly close each figure instance they create. There's quite a bit more options for `z.configure_mpl()`, but I will save that discussion for the documentation.
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? No
- Is there breaking changes for older versions? No
- Does this needs documentation? Yes
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1534 from agoodm/ZEPPELIN-1345 and squashes the following commits:
9ef6ff7 [Alex Goodman] Move mpl backend files to /interpreter
24f89c6 [Alex Goodman] Catch potential NullPointerExceptions from hook registry
bdb584e [Alex Goodman] Make sure expressions are printed when no plots are shown
22b6fe4 [Alex Goodman] Remove unused variable
d3d1aa0 [Alex Goodman] Fix CI test failure
c90d204 [Alex Goodman] Update spark.md
bcf0bf3 [Alex Goodman] Update python.md for new matplotlib integration
c9b65a5 [Alex Goodman] Add iterative plotting example image
8029a05 [Alex Goodman] Update python/README.md
f2d9e86 [Alex Goodman] Exclude tests are excluded in python/pom.xml
86b1c90 [Alex Goodman] Fix tutorial notebook not loading
c37b00f [Alex Goodman] Fix legend in tutorial notebook
a321d79 [Alex Goodman] Update python.md
82350e3 [Alex Goodman] Update matplotlib tutorial notebook
9792f97 [Alex Goodman] Add unit tests
8b9b973 [Alex Goodman] Fix NullPointerExceptions in unit tests
82135ad [Alex Goodman] Removed unused variable
f9c9498 [Alex Goodman] Added support for Angular Display System
edf750a [Alex Goodman] Add new matplotlib backend for python/pyspark interpreters
### What is this PR for?
Based on #338 , I refactor most of pig interpreter. As I don't think the approach in #338 is the best approach. In #338, we use script `bin/pig` to launch pig script, it is different to control that job (hard to kill and get progress and stats info). In this PR, I use pig api to launch pig script. Besides that I implement another interpreter type `%pig.query` to leverage the display system of zeppelin. For the details you can check `pig.md`
### What type of PR is it?
[Feature]
### Todos
* Syntax Highlight
* new interpreter type `%pig.udf`, so that user can write pig udf in zeppelin directly and don't need to build udf jar manually.
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-335
### How should this be tested?
Unit test is added and also manual test is done
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Author: Ali Bajwa <abajwa@hortonworks.com>
Author: AhyoungRyu <ahyoungryu@apache.org>
Author: Jeff Zhang <zjffdu@gmail.com>
Closes#1476 from zjffdu/ZEPPELIN-335 and squashes the following commits:
73a07f0 [Jeff Zhang] minor update
a1b742b [Jeff Zhang] minor update on doc
e858301 [Jeff Zhang] address comments
c85a090 [Jeff Zhang] add license
58b4b2f [Jeff Zhang] minor update of docs
1ae7db2 [Jeff Zhang] Merge pull request #2 from AhyoungRyu/ZEPPELIN-335/docs
fe014a7 [AhyoungRyu] Fix docs title in front matter
df7a6db [AhyoungRyu] Add pig.md to dropdown menu
5e2e222 [AhyoungRyu] Minor update for pig.md
39f161a [Jeff Zhang] address comments
05a3b9b [Jeff Zhang] add pig.md
a09a7f7 [Jeff Zhang] refactor pig Interpreter
c28beb5 [Ali Bajwa] Updated based on comments: 1. Documentation: added pig.md with interpreter documentation and added pig entry to index.md 2. Added test junit test based on passwd file parsing example here https://pig.apache.org/docs/r0.10.0/start.html#run 3. Removed author tag from comment (this was copied from shell interpreter https://github.com/apache/incubator-zeppelin/blob/master/shell/src/main/java/org/apache/zeppelin/shell/ShellInterpreter.java#L42) 4. Implemented cancel functionality 5. Display output stream in case of error
2586336 [Ali Bajwa] exposed timeout and pig executable via interpreter and added comments
7abad20 [Ali Bajwa] initial commit of pig interpreter
### What is this PR for?
For now, it is hard to figure out what the yarn application of the livy session represent, it would be better to display the appId and webui link in the output of LivyInterpreter for diagnosing purpose. It can also be applied to the native SparkInterpreter, but could be done in another ticket.
### What type of PR is it?
[Feature]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1430
### How should this be tested?
Tested manually
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1426 from zjffdu/ZEPPELIN-1430 and squashes the following commits:
88009cb [Jeff Zhang] update doc
459f75e [Jeff Zhang] fix typo
eb7ec27 [Jeff Zhang] add flag to enable display app info in frontend
a087a1d [Jeff Zhang] ZEPPELIN-1430. Display appId and webui link in LivyInterpreter's output
### What is this PR for?
Phoenix has two different connection types: thick and thin.
This PR is about describing the difference between the two and including properties for both in docs/interpreter/jdbc.md
### What type of PR is it?
Documentation
### What is the Jira issue?
[ZEPPELIN-1452](https://issues.apache.org/jira/browse/ZEPPELIN-1452)
### How should this be tested?
No tests necessary
### Questions:
* Does the licenses files need update?
No
* Is there breaking changes for older versions?
No
* Does this needs documentation?
No
Author: Randy Gelhausen <rgelhau@gmail.com>
Closes#1436 from randerzander/master and squashes the following commits:
3ed029e [Randy Gelhausen] Update jdbc.md
6e2dff0 [Randy Gelhausen] Update jdbc.md
2964ba8 [Randy Gelhausen] Update jdbc.md
d2720c3 [Randy Gelhausen] tweaked jdbc readme
38b1692 [Randy Gelhausen] Updated JDBC docs to include Phoenix thin client
## What is this PR for?
The PR is a interpreter for [Apache Beam](http://beam.incubator.apache.org) which is an open source unified platform for data processing pipelines. A pipeline can be build using one of the Beam SDKs.
The execution of the pipeline is done by different Runners . Currently, Beam supports Apache Flink Runner, Apache Spark Runner, and Google Dataflow Runner.
### What type of PR is it?
- Feature
### Todos
* Test case
* Review Comments
* Documentation
### What is the Jira issue?
* [ZEPPELIN-682]
### How should this be tested?
- Start the Zeppelin server
- The prefix of interpreter is `%beam` and then write your code with required imports and the runner
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
Author: mfelgamal <mahmoudf.elgamal@gmail.com>
Author: Fouad <fuad.assayadi@gmail.com>
Closes#1334 from mfelgamal/beam-interpreter-static-repl-7 and squashes the following commits:
da66c27 [mahmoudelgamal] Modify condition of checking static modifier
55c1322 [mahmoudelgamal] set spark version to 1.6.2 and throw original exception
27d7690 [mahmoudelgamal] set spark version to 1.6.1 and some modifications
750041c [mahmoudelgamal] Add readme file and modify pom file and travis.yml
ca88f94 [mahmoudelgamal] edit pom file and .travis.yml
3d65427 [mahmoudelgamal] update .travis.yml file
f19f98d [mahmoudelgamal] Make easy example with imports ands some modifications
74c14ca [mahmoudelgamal] Update the licenses
acc7afb [mahmoudelgamal] Change beam to version 0.2.0
e821614 [mahmoudelgamal] Removing hadoop-core and print stack trace to failure
5cb7c7b [mahmoudelgamal] Add some changes to doc and pom file
75fc4f7 [mahmoudelgamal] add interpreter to navigation.html and remove extra spaces and lines
9b1b385 [mahmoudelgamal] put beam in alphabetical order
9c1e25d [mahmoudelgamal] Adding changes like logging and conventions and license
2aa6d65 [mahmoudelgamal] changing class name to StaticRepl and adding some modifications
7cf25fb [mahmoudelgamal] Adding some tests
3c5038f [mahmoudelgamal] Modifying the documentation
5695077 [mahmoudelgamal] Modifying pom file and Making documentation
26fc59b [mahmoudelgamal] Refactoring of the code
3a2bd85 [mahmoudelgamal] Adding the beam to zeppelin 7
ab7ee2d [mahmoudelgamal] beam interpreter
85957ff [mfelgamal] Merge pull request #10 from apache/master
852c3d3 [mfelgamal] Merge pull request #9 from apache/master
a4bcc0d [mfelgamal] Merge pull request #8 from apache/master
858f1e1 [mfelgamal] Merge pull request #7 from apache/master
03a1e80 [mfelgamal] Merge pull request #4 from apache/master
2586651 [Fouad] Merge pull request #2 from apache/master
### What is this PR for?
Support table markdown syntax issued by [ZEPPELIN-1387](https://issues.apache.org/jira/browse/ZEPPELIN-1387?jql=project%20%3D%20ZEPPELIN)
### What type of PR is it?
[Bug Fix | Improvement]
This PR can be categorized as bug fix and improvement since it not only resolves the above issue but also support other markdown syntaxes.
### Todos
* [ ] - Check the license issue of the [pegdown](https://github.com/sirthias/pegdown) library introduced by this PR
### What is the Jira issue?
[ZEPPELIN-1387](https://issues.apache.org/jira/browse/ZEPPELIN-1387?jql=project%20%3D%20ZEPPELIN)
### How should this be tested?
Write markdown texts and compare them with expected html DOMs. I'v also included some tests for this PR.
### Screenshots (if appropriate)
<img width="708" alt="markdown" src="https://cloud.githubusercontent.com/assets/4968473/18061274/1f2be526-6e5d-11e6-9f1a-3528f3958d2c.png">
### Questions:
* Does the licenses files need update?
* Does coding style is appropriate?
### Additional Comments
We might solve this issue by implementing custom table plugin for markdown4j by referring [the existing work of txtmark](178486805e).
But I think it is not good idea in regard to coverage, maintainability and efficiency since markdown4j is currently not developed actively and it costs to implement all markdown plugins which is not supported by markdown4j.
Author: 1ambda <1amb4a@gmail.com>
Closes#1384 from 1ambda/fix-zeppelin-1387 and squashes the following commits:
16cda72 [1ambda] fix: Merge with 3c8158 to resolve CI failure
e6d41c8 [1ambda] fix: Resolve merge conflict with 8f344dbe08929a [1ambda] fix: Handle more specific exception in catch block
8b1e017 [1ambda] chore: Move github-markdown-css license to bin_licenses
4d1cb3c [1ambda] fix: Typo in docs/interpreter/markdown.md
85a5e3a [1ambda] fix: Use bower to install github-markdown-css
297733f [1ambda] fix: Modify github-markdown-css license
947a92a [1ambda] chore: Add license to newly created java files
d228423 [1ambda] docs: Update markdown docs config, examples
2b6516c [1ambda] feat: Support markdown.parser.type attr in md
d2d4455 [1ambda] style: Reformat using intellij-java-google-style
bf9100d [1ambda] chore: Restore markdown4j dependency
55a2f10 [1ambda] fix: Add MarkdownParser interface to support mulitple parsers
c33c715 [1ambda] fix: Remove the ANCHORLINKS option
9cf31d0 [1ambda] fix: Use markdown-body class (default)
f741949 [1ambda] fix: Add styles for markdown
603d3db [1ambda] fix: Add missing transitive deps for pegdown
7aecdcb [1ambda] chore: Add pegdown to the binary license list
fa14b3e [1ambda] style: Apply google java code style guide
029f550 [1ambda] [ZEPPELIN-1387] Support table in markdown interpreter
### What is this PR for?
Several changes on doc of spark interpreter.
* %spark, %sql, %pyspark only works when spark is the default interpreter group of note. So I update the doc to use the full interpreter name.
* Add SparkSession for 2.0
* Also add comments inline with other changes to explain the reason.
### What type of PR is it?
[Documentation]
### Todos
* [ ] - Task
### What is the Jira issue?
* No jira created.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Author: Jeff Zhang <zjffdu@apache.org>
Author: Jeff Zhang <zjffdu@gmail.com>
Closes#1398 from zjffdu/spark_doc_fix and squashes the following commits:
ac01f2b [Jeff Zhang] Merge pull request #1 from AhyoungRyu/spark_doc_fix/ahyoung
5fa523f [AhyoungRyu] Fix typos
3c0f678 [AhyoungRyu] Add 'R' and refine a sentence
2336900 [AhyoungRyu] Improve spark.md
40d4b11 [Jeff Zhang] [MINOR] Doc fix for spark interpreter
### What is this PR for?
Adding extra libraries to livy interpreter which isn't exist by default.
### What type of PR is it?
[ Improvement ]
### Todos
* [Test case ] - Task
### What is the Jira issue?
* [ZEPPELIN-1258]
### How should this be tested?
- Create new livy interpreter or modify the default.
- Set `livy.spark.jars.packages` to list of maven coordinates of jars. The format for the coordinates should be groupId:artifactId:version.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: mahmoudelgamal <mahmoudf.elgamal@gmail.com>
Author: mfelgamal <mahmoudf.elgamal@gmail.com>
Author: Fouad <fuad.assayadi@gmail.com>
Closes#1255 from mfelgamal/feature/modifying-livy-doc and squashes the following commits:
5411a2f [mahmoudelgamal] changeing default to example
d7f6818 [mahmoudelgamal] Adding livy.spark.jars.packages property
98090ab [mahmoudelgamal] Adding external libraries to livy
03a1e80 [mfelgamal] Merge pull request #4 from apache/master
2586651 [Fouad] Merge pull request #2 from apache/master
### What is this PR for?
This PR adds support for plotting png images using the matplotlib helper function within a python interpreter (eg `z.show()`). The primary motivation for this is due to the overhead incurred from svg images, which can lag the notebooks if multiple, complicated images are generated (for example, multiple filled contour plots). png images are more lightweight, but of course come at a cost of image quality due to them being raster rather than vector like svg. The support for png images is incorporated through the use of a new optional argument to `z.show` called `fmt` which can be one of `'svg'` or `'png'`. The same code that is currently used in show is used for svg images while the code for png images relies on converting the image directly to a byte array and then entering the decoded byte string directly into an HTML image tag. Currently `fmt` defaults to `'png'` but I think we should consider discussing the pros and cons of each option in this PR.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1318](https://issues.apache.org/jira/browse/ZEPPELIN-1318)
### How should this be tested?
In a notebook cell, enter:
```python
%python
import matplotlib.pyplot as plt
import numpy as np
plt.figure()
plt.plot(np.arange(10))
z.show(plt, fmt=fmt)
```
Where `fmt` may be one of `'svg'` or `'png'`, and any other input should result in a `ValueError`. I would also recommend testing the example in the screenshot below.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes (if the changes to the `help()` docstring are not sufficient)
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1329 from agoodm/ZEPPELIN-1318 and squashes the following commits:
2e9ce4c [Alex Goodman] Update python.md
1efa0c9 [Alex Goodman] ZEPPELIN-1318 - Add support for png images in z.show()
