### What is this PR for?
Use shared test library versions in maven config so that lib versions do mot be fragmented.
Previously we used multiple versions of
- Junit (4.11, 4.12)
- mockito (1.9.0, 1.10.8, ...)
- powermock (...)
### What type of PR is it?
[Improvement]
### What is the Jira issue?
[ZEPPELIN-1695](https://issues.apache.org/jira/browse/ZEPPELIN-1695)
### How should this be tested?
Use this command to see test libraries share versions or not
```
$ mvn org.apache.maven.plugins:maven-help-plugin:2.2:effective-pom | vim -
```
### Questions:
* Does the licenses files need update? - YES, I updated JUnit version to 4.12 from 4.11
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Closes#1727 from 1ambda/feat/centralise-testing-libraries and squashes the following commits:
b6fd336 [1ambda] chore: Shared mockito, powermock version
941f1ba [1ambda] chore: Update junit to 4.12
### What is this PR for?
Because `conda run` command removed since version `4.0.9`, PythonCondaInterpreter not working after the `conda-4.0.9`.
This PR fixes this issue.
I tested conda-4.2.12 and conda-4.0.9 .
### What type of PR is it?
Bug Fix | Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1724
### How should this be tested?
Please refer to https://github.com/apache/zeppelin/pull/1645
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1699 from astroshim/ZEPPELIN-1724 and squashes the following commits:
294b6f9 [astroshim] refactoring and fix testcase
8c3fbd3 [astroshim] fix conda version
### What is this PR for?
Currently a paragraph can display only a single type of result.
For example
```
print("""
%text textout
%html htmlout
""")
```
will display only last display system detected, "htmlout".
This pr implements multiple results supports, so not only the last, but also all the display systems in a paragraph outputs can be displayed.
To do that, Note.json format need to be changed. from
```
paragraph : [
{
result : {
code: "",
type: "",
msg: ""
},
config : {
graph : {},
...
},
...
},
...
],
...
```
to
```
paragraph : [
{
results : {
{
code: "",
msg: [
{
type: "",
data: ""
},
{
type: "",
data: ""
},
...
]
},
config : {
results : [
{
graph : {},
},
{
graph : {},
},
...
]
},
...
},
...
],
...
```
### What type of PR is it?
Improvement
### Todos
* [x] - Make InterpreterResult and InterpreterOutput support multiple display system
* [x] - Automatic migration old format to new format
* [x] - Render multiple results in front-end
* [x] - Take care Importing old notebook format
* [x] - Make helium framework support multiple results
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-212
### How should this be tested?
run following code in a single spark paragraph.
```
%spark
// default output is text
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// print something in html
println(s"""%html <h3><font style="color:blue">Some HTML</font></h3>""")
// create table
println(s"""%table key\tvalue
sun\t100
moon\t200
""")
// display text again
println("""%text""")
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// another table
Thread.sleep(1000)
println(s"""%table key\tvalue
apple\t100
banana\t200
""")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? yes
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1658 from Leemoonsoo/ZEPPELIN-212 and squashes the following commits:
0c6a13c [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
519cf23 [Lee moon soo] Update PythonInterpreterPandasSqlTest
e6268ac [Lee moon soo] Update PythonInterpreterMatplotlibTest.java
f5034b8 [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
d5981d5 [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
a1fe729 [Lee moon soo] document note format change
282504e [Lee moon soo] Merge remote-tracking branch 'origin/master' into ZEPPELIN-212
7ab6679 [Lee moon soo] update selenium test
6a897a5 [Lee moon soo] Update rest-api doc
cbbd58a [Lee moon soo] update unittest
e89c9b8 [Lee moon soo] restore tutoral note
3ba37b7 [Lee moon soo] Let old version import note without error
d09e03f [Lee moon soo] enable helium only in the last result
6682908 [Lee moon soo] Remove unnecessary listener method
0f1d28f [Lee moon soo] Merge branch 'master' into ZEPPELIN-212
73b3a81 [Lee moon soo] update selenium test
e69f1a1 [Lee moon soo] update test
f12230f [Lee moon soo] Remove unnecessary newline in test
26e201a [Lee moon soo] Update testcase
b01d70f [Lee moon soo] Make helium app work
0a5b4d1 [Lee moon soo] Update HeliumApplicationFactoryTest
d8ca07f [Lee moon soo] update NotebookTest
aaf9778 [Lee moon soo] fix r test
3e43df4 [Lee moon soo] make zeppelin-web test pass
1d6fd3e [Lee moon soo] fix compile errors
0156ffc [Lee moon soo] take care angular object
804768d [Lee moon soo] Take care output streaming
57d6b2f [Lee moon soo] Render multiple results
95b6037 [Lee moon soo] Multiple results
### What is this PR for?
Take 2 of #1618 because I had some earlier problems with rebasing. Since, then I have added some new features, namely:
- Matplotlib integration tests for pyspark
- `install_external_dependencies.sh` which conditionally installs the R or python dependencies based on the specified build profile in `.travis.yml`. This saves several minutes of time for a few of the build profiles since the R dependencies are compiled from source and therefore take quite a bit of time to install.
- The extra python unit tests which require external dependencies (`matplotlib` and `pandas`) are now relegated to two separate build profiles. This is done primarily to efficiently test both Python 2 and 3.
- Some minor bugs in the python and pyspark interpreters (mostly with respect to python 3 compatibility) were caught as a result of these tests, and are also fixed in this PR.
### What type of PR is it?
Improvement and Bugfix
### What is the Jira issue?
[ZEPPELIN-1639](https://issues.apache.org/jira/browse/ZEPPELIN-1639)
### How should this be tested?
CI tests should be green!
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1632 from agoodm/ZEPPELIN-1639 and squashes the following commits:
01380c2 [Alex Goodman] Make sure python 3 profile uses scala 2.11
363019e [Alex Goodman] Use spark 2.0 with python 3
a4f43af [Alex Goodman] Update comments in .travis.yml
5a60181 [Alex Goodman] Isolate python tests
73663f6 [Alex Goodman] Update tests for new InterpreterContext constructor
5709c5d [Alex Goodman] Re-add pyspark to build profile
ee95d67 [Alex Goodman] Move python 3 tests to all modules with spark 2.0
3a76958 [Alex Goodman] Travis
42da31c [Alex Goodman] Shorten python version
b6b88be [Alex Goodman] Add python dependencies to .travis.yml
### What is this PR for?
This PR is for the multi-tenant of JDBC Interpreter.
User can create a user/password for JDBC account at the [Credential page](http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html).
The `Entity` of `Credential` is match with JDBC interpreter group name.
If the account for JDBC is not setted in the `Interpreter property` then use `Credential`'s.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1567
### How should this be tested?
Please refer to testMultiTenant() of JDBCInterpreterTest/
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
Author: astroshim <hsshim@nflabs.com>
Closes#1539 from astroshim/jdbc-impersonation and squashes the following commits:
46fce31 [astroshim] add explanation of InterpreterGroup
7a92236 [astroshim] fix doc and remove persist value.
63f5ea7 [astroshim] Merge branch 'master' into jdbc-impersonation
267277a [astroshim] rebase
649ff6e [astroshim] rebase
872fb49 [astroshim] fix ScioInterpreterTestCase
4387a5b [astroshim] Merge branch 'master' into jdbc-impersonation
47c463f [astroshim] update doc and html
d4eb178 [astroshim] fix docs
59aa9ff [astroshim] Merge branch 'master' into jdbc-impersonation
bf61afd [astroshim] fix testcase
5c0f5d7 [astroshim] rebase
79ba25b [astroshim] Merge branch 'master' into jdbc-impersonation
1f9c2c0 [astroshim] clean redundant code
a2f5687 [astroshim] fix impersonation
9962181 [astroshim] fix InterpreterOutput of PySparkInterpreterTest case
b55aceb [astroshim] Merge branch 'master' into jdbc-impersonation
24a8226 [astroshim] fix doc
086dfda [astroshim] fix testcase
34fe0a6 [astroshim] fix code for more simple.
fee7086 [astroshim] fix build error.
a305eca [astroshim] Merge branch 'master' into jdbc-impersonation
df80741 [astroshim] documentation for credential.
df1b1dc [astroshim] rebase and entity name convention.
63d6a1c [astroshim] change thrift version to 0.9.2
6573c1c [astroshim] change variable name
f311f34 [astroshim] fix typo
722e333 [astroshim] change testcase name
9161937 [astroshim] clean code
3dafdf0 [astroshim] add testcase
373d5f1 [astroshim] pass replName to Interpreter and use credential info for jdbc auth.
### What is this PR for?
Inspired by ZEPPELIN-1671 conda interpreter.
Docker can provides kind of virtual environment for python like conda does.
This PR implements %python.docker interpreter that helps run python process in docker container.
This PR implements feature on top of https://github.com/apache/zeppelin/pull/1645
### What type of PR is it?
Feature
### Todos
* [x] - basic feature
* [x] - unittest
* [x] - documentation
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1683
### How should this be tested?
see screenshot
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1654 from Leemoonsoo/pydocker and squashes the following commits:
22507e6 [Lee moon soo] Add new line at the end of the file
41c09d9 [Lee moon soo] Run python process in docker container
### What is this PR for?
Conda interpreter that manages conda environment for PythonInterpreter
### What type of PR is it?
Feature
### Todos
* [x] - Basic impl
* [x] - update doc
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1671
### How should this be tested?
Recreate(or create new) your python interpreter setting in gui.
List all conda env
```
%python.conda
```
Activate env
```
%python.conda activate [name]
```
Deactivate env
```
%python.conda deactivate
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes#1645 from Leemoonsoo/conda and squashes the following commits:
4842b0a [Lee moon soo] Add usage in doc
d979c6a [Lee moon soo] Add unittest
b889443 [Lee moon soo] add usage template
9ae553b [Lee moon soo] Format output and add usage command
171cbeb [Lee moon soo] make sure single char interpreter name can be parsed
6b9525f [Lee moon soo] Fix unittest
1223796 [Lee moon soo] Remove unnecessary log
394cf8c [Lee moon soo] Conda interpreter implementation
### What is this PR for?
There have been reports of #1534 causing the python interpreter to always show an error because `z` is not being set. As it turns out this is a result of improperly handling the case when matplotlib isn't found when initializing the interpreter.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1656](https://issues.apache.org/jira/browse/ZEPPELIN-1656)
### How should this be tested?
Run any simple python paragraph.
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1628 from agoodm/patch-1 and squashes the following commits:
67f2ad5 [Alex Goodman] python interpeter should work when matplotlib is not installed
0a7a9d7 [Alex Goodman] Fix indent in bootstrap.py
### What is this PR for?
After #1534 , Dynamic Forms were no longer working in the python interpreter. This is because the `Py4jZeppelinContext` constructor did not initialize the `_displayhook` which is always called on post-execute.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1655](https://issues.apache.org/jira/browse/ZEPPELIN-1655)
### How should this be tested?
Run the following `%python` paragraph, being sure that Py4j is installed:
```python
%python
a, b, c = (1, 2, 3)
z.select("Choose a letter", ([a,"a"], [b,"b"], [c,"c"] ))
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1626 from agoodm/ZEPPELIN-1655 and squashes the following commits:
2e4ee2d [Alex Goodman] Make sure _displayhook is initialized in Py4jZeppelinContext
### What is this PR for?
Add support to display optionally Pandas DataFrame index using z.show(show_index=True) in python interpreter. By default, DataFrame index will not be displayed.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1358](https://issues.apache.org/jira/browse/ZEPPELIN-1358)
### How should this be tested?
```
mvn -Dpython.test.exclude='' test -pl python -am
```
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? no
- Is there breaking changes for older versions? no
- Does this needs documentation? no
Author: Paul Bustios <pbustios@gmail.com>
Closes#1378 from bustios/ZEPPELIN-1358 and squashes the following commits:
7842f71 [Paul Bustios] Add param to make the index to be shown optinally
6767b40 [Paul Bustios] Add support to display Pandas DataFrame index using z.show() and modifies test.
### What is this PR for?
This PR is the first of two major steps needed to improve matplotlib integration in Zeppelin (ZEPPELIN-1344). The latter, which is a plotting backend with fully interactive tools enabled, will be done afterwards in a separate PR. This PR specifically for automatically displaying output from calls to matplotlib plotting functions inline with each paragraph. Thanks to the addition of post-execute hooks (ZEPPELIN-1423), there is no need to call any `show()` function to display an inline plot, just like in Jupyter.
### What type of PR is it?
Improvement
### Todos
The main code has been written and anyone who reads this is encouraged to test it, but there are a few minor todos:
- [x] - Add unit tests
- [x] - Add documentation
- [x] - Add screenshot showing iterative plotting with angular mode
### What is the Jira issue?
[ZEPPELIN-1345](https://issues.apache.org/jira/browse/ZEPPELIN-1345)
### How should this be tested?
In a pyspark or python paragraph, enter and run
``` python
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
```
The plot should be displayed automatically without calling any `show()` function whatsoever. A special method called `configure_mpl()` can also be used to modify the inline plotting behavior. For example,
``` python
z.configure_mpl(close=False, angular=True)
plt.plot([1, 2, 3])
```
allows for iterative updates to the plot provided you have PY4J installed for your python installation (which of course is always the case if you use pypsark). To clarify, this feature only currently works with pyspark (not python as there are no `angularBind()` and `angularUnbind()` methods yet). Doing something like:
```
plt.plot([3, 2, 1])
```
will update the plot that was generated by the previous paragraph by leveraging Zeppelin's Angular Display System. However, by setting `close=False`, matplotlib will no longer automatically close figures so it is now up to the user to explicitly close each figure instance they create. There's quite a bit more options for `z.configure_mpl()`, but I will save that discussion for the documentation.
### Screenshots (if appropriate)
### Questions:
- Does the licenses files need update? No
- Is there breaking changes for older versions? No
- Does this needs documentation? Yes
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1534 from agoodm/ZEPPELIN-1345 and squashes the following commits:
9ef6ff7 [Alex Goodman] Move mpl backend files to /interpreter
24f89c6 [Alex Goodman] Catch potential NullPointerExceptions from hook registry
bdb584e [Alex Goodman] Make sure expressions are printed when no plots are shown
22b6fe4 [Alex Goodman] Remove unused variable
d3d1aa0 [Alex Goodman] Fix CI test failure
c90d204 [Alex Goodman] Update spark.md
bcf0bf3 [Alex Goodman] Update python.md for new matplotlib integration
c9b65a5 [Alex Goodman] Add iterative plotting example image
8029a05 [Alex Goodman] Update python/README.md
f2d9e86 [Alex Goodman] Exclude tests are excluded in python/pom.xml
86b1c90 [Alex Goodman] Fix tutorial notebook not loading
c37b00f [Alex Goodman] Fix legend in tutorial notebook
a321d79 [Alex Goodman] Update python.md
82350e3 [Alex Goodman] Update matplotlib tutorial notebook
9792f97 [Alex Goodman] Add unit tests
8b9b973 [Alex Goodman] Fix NullPointerExceptions in unit tests
82135ad [Alex Goodman] Removed unused variable
f9c9498 [Alex Goodman] Added support for Angular Display System
edf750a [Alex Goodman] Add new matplotlib backend for python/pyspark interpreters
### What is this PR for?
This PR enables edit on double click for markdown/angular paragraph. Users can change `editOnDblClick` field to be `false` by editting `interpreter/md/interpreter-setting.json` or `conf/interpreter.json`. In the same context, users can set other type paragraphs to be editable on double click by setting `editOnDblClick` to be true.
This PR also fixes bug that syntax highlight doesn't work on pasted code.
### What type of PR is it?
Feature
### Todos
* [x] Create test
* [x] Update docs
### What is the Jira issue?
[ZEPPELIN-1566](https://issues.apache.org/jira/browse/ZEPPELIN-1566)
### How should this be tested?
1. Create new markdown interpreter
2. Create notebook and run markdown paragraph
3. Double click markdown paragraph to edit
### Screenshots (if appropriate)
**Edit on double click and hide editor on paragraph run**
**Syntax highlight on paste**
Before
After
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes
Author: Mina Lee <minalee@apache.org>
Closes#1540 from minahlee/ZEPPELIN-1566 and squashes the following commits:
89ea628 [Mina Lee] Remove redundant code
4ffc446 [Mina Lee] Enable focus first paragraph on note reload
c566706 [Mina Lee] Check null condition of editor setting and return default setting
38d5e35 [Mina Lee] Update document about how to make paragraph editable on double click
b42039e [Mina Lee] Add integration test for editOnDblClick
0a26207 [Mina Lee] Add editOnDblClick field in interpreter-setting.json
93abe6a [Mina Lee] Make paragraph editable on doubleclick if editOnDblClick set true in interpreter-setting.json
359dc0b [Mina Lee] Split getAndSetEditorSetting method into getEditorSetting and setEditorLanguage method
### What is this PR for?
This is complete work of #1148. Comments and tasks on #1148 has been handled in this PR.
- Add syntax language information in `interpreter-setting.json`
- When user type `%replName` in paragraph, back-end check if the interpreter name with `replName` exists, and return language information to front-end if it does
- If user doesn't specify `%replName`, default interpreter's language will be used
- Using alias name for paragraph syntax highlight
### What type of PR is it?
[Bug Fix | Improvement]
### What is the Jira issue?
[ZEPPELIN-1026](https://issues.apache.org/jira/browse/ZEPPELIN-1026)
### How should this be tested?
1. Create new note and make markdown interpreter to be default.
2. See if markdown syntax is applied.
### Screenshots (if appropriate)
#### Case 1. When the default interpreter set to python interpreter.
**Before**
Has `scala` as syntax highlight language when %python is not set.
<img width="665" alt="screen shot 2016-07-07 at 10 46 20 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png">
**After**
Has `python` as syntax highlight language even when %python is not set.
<img width="666" alt="screen shot 2016-07-07 at 10 44 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png">
#### Case 2. When use alias name as repl name.
**Before**
<img width="742" alt="screen shot 2016-09-08 at 4 22 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png">
**After**
<img width="741" alt="screen shot 2016-09-08 at 4 34 57 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png">
### Further possible improvements
There are still several cases that Zeppelin doesn't handle syntax highlight well. These can be handled with another jira ticket/PR.
1. When default bound interpreter changes, syntax highlight is not changed accordingly
2. When copy/paste code, syntax highlight won't be applied properly since Zeppelin only checks changes when cursor is in first line.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes(for creating new interpreter)
Author: Mina Lee <minalee@apache.org>
Closes#1415 from minahlee/ZEPPELIN-1026 and squashes the following commits:
c66fb0e [Mina Lee] Move getEditorSetting to InterpreterFactory class
2d56222 [Mina Lee] Add description about default syntax highlight in doc
08ccad9 [Mina Lee] Fix test
0874522 [Mina Lee] Change condition for triggering 'getAndSetEditorSetting' to reduce front-end <-> back-end communication
9e4f2e9 [Mina Lee] Change the way to read interpreter language from interpreter-setting.json after #114575543b3 [Mina Lee] Add test
565d9d0 [Mina Lee] [DOC] Setting syntax highlight when writing new interpreter
20132ca [Mina Lee] Get paragraph editor mode from backend
52f4207 [Mina Lee] Align comments for readability
26cbbb8 [Mina Lee] Add editor field
### What is this PR for?
currently, has not support multiline exception text on python interpreter.
for example:
```
Exception: blabla
```
is error.
but
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception: test exception
```
is sucess (now)
to resolve this issue.
### What type of PR is it?
Bug Fix
### Todos
- [x] modification pythonErrorIn method
- [x] add test case
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1412?jql=project%20%3D%20ZEPPELIN%20AND%20status%20%3D%20Open
### How should this be tested?
added test case.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: CloverHearts <cloverheartsdev@gmail.com>
Closes#1407 from cloverhearts/dev/ZEPPELIN-1412 and squashes the following commits:
e674134 [CloverHearts] add multiline support pythonErrorIn method
### What is this PR for?
Currently, height parameter for z.show implementation to display PNG images in Python interpreter is not working. This PR fix that bug.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1327](https://issues.apache.org/jira/browse/ZEPPELIN-1327)
### How should this be tested?
```python
import matplotlib.pyplot as plt
x = [1,2,3,4,5]
y = [6,7,8,9,0]
plt.plot(x, y, marker="o")
z.show(plt, height="200px")
plt.close()
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Paul Bustios <pbustios@gmail.com>
Closes#1352 from bustios/ZEPPELIN-1327 and squashes the following commits:
8eff11a [Paul Bustios] Change default values of width and height and add img tag for PNG images
b3c74a8 [Paul Bustios] Add comment explaining the need for decoding bytes to string
1a78a37 [Paul Bustios] Fix bug in z.show for Python interpreter
### What is this PR for?
Support for plotting PNG images via matplotlib inline for the python interpreter was recently added (#1329). However, these changes did not work for python3 since it handles strings differently. This PR aims to make the inline plotting compatible with both python 2 and 3.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
* [ZEPPELIN-1328](https://issues.apache.org/jira/browse/ZEPPELIN-1328)
### How should this be tested?
In a python interpreteter cell, make sure the following produce an image:
```python
%python
import matplotlib.pyplot as plt
import numpy as np
x = np.arange(5)
plt.plot(x)
z.show(plt, fmt='png') # Repeat for fmt='svg'
```
This should be tested for both python2 and 3 interpreters (via the interpreter settings page).
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1343 from agoodm/ZEPPELIN-1328 and squashes the following commits:
772313f [Alex Goodman] Redo io import structure to make z.show() work for both matplotlib plots and pandas dataframes in python2/3
6a8f3ab [Alex Goodman] Add python3 support for matplotlib inline plotting in python interpreter
### What is this PR for?
This PR adds support for plotting png images using the matplotlib helper function within a python interpreter (eg `z.show()`). The primary motivation for this is due to the overhead incurred from svg images, which can lag the notebooks if multiple, complicated images are generated (for example, multiple filled contour plots). png images are more lightweight, but of course come at a cost of image quality due to them being raster rather than vector like svg. The support for png images is incorporated through the use of a new optional argument to `z.show` called `fmt` which can be one of `'svg'` or `'png'`. The same code that is currently used in show is used for svg images while the code for png images relies on converting the image directly to a byte array and then entering the decoded byte string directly into an HTML image tag. Currently `fmt` defaults to `'png'` but I think we should consider discussing the pros and cons of each option in this PR.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1318](https://issues.apache.org/jira/browse/ZEPPELIN-1318)
### How should this be tested?
In a notebook cell, enter:
```python
%python
import matplotlib.pyplot as plt
import numpy as np
plt.figure()
plt.plot(np.arange(10))
z.show(plt, fmt=fmt)
```
Where `fmt` may be one of `'svg'` or `'png'`, and any other input should result in a `ValueError`. I would also recommend testing the example in the screenshot below.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes (if the changes to the `help()` docstring are not sufficient)
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes#1329 from agoodm/ZEPPELIN-1318 and squashes the following commits:
2e9ce4c [Alex Goodman] Update python.md
1efa0c9 [Alex Goodman] ZEPPELIN-1318 - Add support for png images in z.show()
### What is this PR for?
It is not necessary to call print to display output in PythonInterpreter. 2 main changes:
* the root cause is the displayhook in bootstrap.py
* also did some code refactoring on PythonInterpreter
### What type of PR is it?
[Bug Fix]
### Todos
* [ ] - Task
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1287
### How should this be tested?
Verify it manually
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jeff Zhang <zjffdu@apache.org>
Closes#1278 from zjffdu/ZEPPELIN-1287 and squashes the following commits:
b48b56f [Jeff Zhang] fix unit test fail
3e9f169 [Jeff Zhang] address comments
0eade71 [Jeff Zhang] ZEPPELIN-1287. No need to call print to display output in PythonInterpreter
### What is this PR for?
Small refactoring of Python interpreter, that is what it is.
### What type of PR is it?
Refactoring
### Todos
* [x] refactor `help()`
* [x] impl `maxResult` fetch from JVM
### How should this be tested?
`cd python && mvn -Dpython.test.exclude='' test ` pass (given that `pip install pandasql` and `pip install py4j`)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1275 from bzz/python/refactoring and squashes the following commits:
15a35c8 [Alexander Bezzubov] Make .help() method a single string literal
e800fd7 [Alexander Bezzubov] Make Python fetch maxResults from JVM
### What is this PR for?
Bug fix in z.show() for matplotlib graphs and code refactoring
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1261](https://issues.apache.org/jira/browse/ZEPPELIN-1261)
### How should this be tested?
```
%python
import matplotlib.pyplot as plt
x = [1,2,3,4,5]
y = [6,7,8,9,0]
plt.plot(x, y, marker="o")
z.show(plt, height="20em")
plt.close()
```
```
%python
import matplotlib.pyplot as plt
x = [1,2,3,4,5]
y = [6,7,8,9,0]
plt.plot(x, y, marker="o")
z.show(plt, height="300px")
plt.close()
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Paul Bustios <pbustios@gmail.com>
Closes#1267 from bustios/ZEPPELIN-1261 and squashes the following commits:
82c631a [Paul Bustios] Add 100% as a default value for width and height z.show() parameters
3e0ef22 [Paul Bustios] Bug fix in show_matplotlib
### What is this PR for?
Casting data types in Pandas DataFrame to string in z.show()
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1255](https://issues.apache.org/jira/browse/ZEPPELIN-1255)
### How should this be tested?
```
%python
import pandas as pd
df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None)
df.columns=[1, 2, 3, 'PetalWidth', 'Name']
z.show(df)
%python.sql
SELECT * FROM df LIMIT 10
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: paulbustios <pbustios@gmail.com>
Closes#1249 from bustios/ZEPPELIN-1255 and squashes the following commits:
82c1412 [paulbustios] Add test case for z.show() Pandas DataFrame
4a8c0a9 [paulbustios] [ZEPPELIN-1255] Add cast to string in z.show() for Pandas DataFrame
### What is this PR for?
Set project url to `http://zeppelin.apache.org` in pom.xml files
### What type of PR is it?
Refactoring
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Mina Lee <minalee@apache.org>
Closes#1221 from minahlee/pom_url and squashes the following commits:
10de8cb [Mina Lee] Remove child url
ef0ef04 [Mina Lee] Change main class package name
ead4064 [Mina Lee] Use consistent url in pom.xml
### What is this PR for?
Hotfix for [ZEPPELIN-1244](https://issues.apache.org/jira/browse/ZEPPELIN-1244)
### What type of PR is it?
Hot Fix
### What is the Jira issue?
[ZEPPELIN-1244](https://issues.apache.org/jira/browse/ZEPPELIN-1244)
### How should this be tested?
Remove interpreter configuration and try `%python.sql` i.e over [bank.csv](http://mlr.cs.umass.edu/ml/datasets/Bank+Marketing) read as `pandas.dataframe`
```
#stop zeppelin
rm conf/interpreter-setting.json
#start zeppelin
%python
import pandas as pd
rates = pd.read_csv("http://www3.dsi.uminho.pt/pcortez/data/bank.csv", sep=";")
%python.sql
SELECT * FROM rates LIMIT 10
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1238 from bzz/python/fix/ZEPPELIN-1244 and squashes the following commits:
befeebe [Alexander Bezzubov] add correct %python.sql interpreter name to configuration
### What is this PR for?
PythonInterpreter can not use dynamic form with python3.
Fix this problem.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1206
### How should this be tested?
```
%python
print(z.input("test"))
```
Make a note above, and run it.
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: Sangmin Yoon <sangmin.yoon@croquis.com>
Closes#1213 from sixmen/fix_python3 and squashes the following commits:
be6f68b [Sangmin Yoon] fix "name 'z' is not defined" with python3
### What is this PR for?
Add new interpreter to Python group: `%python.sql` for SQL over DataFrame support
### What type of PR is it?
Improvement
### TODOs
* [x] add new interpreter `%python.sql`
* [x] add test
* [x] make Python-dependant tests, excluded from CI
* PythonInterpreterWithPythonInstalledTest
* PythonPandasSqlInterpreterTest
* run manually by `mvn -Dpython.test.exclude='' test -pl python -am`
* [x] add docs `%python.sql`
* [x] make `%python.sql` fail gracefully in case there is no Pandas or PandaSQL installed
* [x] after #747 is merged - rebase and remove `-Dpython.test.exclude=''` from both profiles
### What is the Jira issue?
[ZEPPELIN-1115](https://issues.apache.org/jira/browse/ZEPPELIN-1115)
### How should this be tested?
`mvn -Dpython.test.exclude='' test -pl python -am` should pass or manually run
- Given the DataFrame i.e
```
%python
import pandas as pd
rates = pd.read_csv("bank.csv", sep=";")
```
- SQL query it like
```
%python.sql
SELECT * FROM rates LIMIT 10
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No, no dependencies were included in source or binary release
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1164 from bzz/ZEPPELIN-1115/python/add-sql-for-dataframes and squashes the following commits:
0f2f852 [Alexander Bezzubov] Fail SQL gracefully if no python dependencies installed
aca2bdf [Alexander Bezzubov] Fix typos in docs ⚡158ba6a [Alexander Bezzubov] Remove third-party dependant test from CI
5fe46fc [Alexander Bezzubov] Update Python Matplotlib notebook example
72884c8 [Alexander Bezzubov] Add docs for %python.sql feature
e931dc4 [Alexander Bezzubov] Make test for PythonPandasSqlInterpreter usable
76bbb44 [Alexander Bezzubov] Complete implementation of the PythonPandasSqlInterpreter
f6ca1eb [Alexander Bezzubov] Add %python.sql to interpreter menue
11ba490 [Alexander Bezzubov] Add draft implementation of %python.sql for DataFrames
### What is this PR for?
fix flaky `PythonInterpreter.testClose()` test
### What type of PR is it?
Hot Fix
### What is the Jira issue?
[ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063)
### How should this be tested?
CI should pass
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1134 from bzz/ZEPPELIN-1063/python/add-retrys and squashes the following commits:
65698d2 [Alexander Bezzubov] Reduce 😴 time in python interpreter unit-test
1be7732 [Alexander Bezzubov] Adding timeout for py4j gateway to have time to shutdown
3529fe5 [Alexander Bezzubov] fix PythonInterpreterTest.testClose() test
### What is this PR for?
Bump up version to 0.7.0-SNAPSHOT
Author: Mina Lee <minalee@apache.org>
Closes#1016 from minahlee/0.7.0-SNAPSHOT and squashes the following commits:
541e1b3 [Mina Lee] Bump up zeppelin-examples version to 0.7.0-SNAPSHOT
ea8c0ad [Mina Lee] Bump up version to 0.7.0-SNAPSHOT
### What is this PR for?
Implement paragraph ERROR status for Python interpreter in case of Error or Exception in the output.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1105](https://issues.apache.org/jira/browse/ZEPPELIN-1105)
### How should this be tested?
CI should pass, or
```
mvn "-Dtest=org.apache.zeppelin.python.PythonInterpreterWithPythonInstalledTest" test -pl python
```
should pass, or paragraph status should be ERROR for something like
```
import some-thing
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1124 from bzz/ZEPPELIN-1105/python/add-paragraph-error-statu and squashes the following commits:
a7bf8f3 [Alexander Bezzubov] Python: add missing license header
b585982 [Alexander Bezzubov] Python: include Python-dependant tests to 1 CI profile
e7d5371 [Alexander Bezzubov] Python: add ERROR paragraph status on Error and Exception in output
4c1107b [Alexander Bezzubov] Refactoring: rename and extract var assignment
### What is this PR for?
fix flaky python interpreter test
### What type of PR is it?
Bug Fix
### Todos
* [x] cleanup test code
* [x] fix flaky open port check
### What is the Jira issue?
[ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063)
### How should this be tested?
`mvn "-Dtest=org.apache.zeppelin.python.PythonInterpreterTest" test -pl python`
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1094 from bzz/fix/python-tests/ZEPPELIN-1063 and squashes the following commits:
b9de5a2 [Alexander Bezzubov] Python: refactoring open port checking
09f3018 [Alexander Bezzubov] Python: refactoring - arrange imports
46e42e8 [Alexander Bezzubov] Python: refactoring mock() structure and JavaDocs
651d8e8 [Alexander Bezzubov] Python: refactoring loggers
d7f8cdd [Alexander Bezzubov] Python: normalize newlines in tests
### What is this PR for?
Update `testPy4jIsNotInstalled `, `testPy4jIsInstalled` test
- `z.show` -> `def show` to check `show` function is defined
- check if `bootstrap_input.py` excuted by checking `z = Py4jZeppelinContext` instead of `z = PyZeppelinContext`
- add license header in `__init__.py` file
### What type of PR is it?
Hot Fix
Author: Mina Lee <minalee@apache.org>
Closes#1075 from minahlee/adjustPythonTest and squashes the following commits:
d46c5e1 [Mina Lee] Update api name in docs
6d82e9f [Mina Lee] Add license to __init__.py
f66e9dc [Mina Lee] Fix python test case
### What is this PR for?
Returning back to pass the CI
### What type of PR is it?
[Hot Fix]
### Todos
* [x] - Fix test cases
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1048
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#1073 from jongyoul/hotfix-fix-pythoninterpretertest and squashes the following commits:
32be5d1 [Jongyoul Lee] Fixed test failed
### What is this PR for?
Display Pandas DataFrame using Zeppelin's Table Display system.
### What type of PR is it?
Feature
### Todos
* [x] fix NPE in logs on empty paragraph execution
* [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)`
* [x] pandas: support `z.show(df)`
* [x] update docs
### What is the Jira issue?
[ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048)
### How should this be tested?
"Zeppelin Tutorial: Python - matplotlib basic" should work, and
```python
import pandas as pd
rates = pd.read_csv("bank.csv", sep=";")
z.show(rates)
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1067 from bzz/python/pandas-support and squashes the following commits:
3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API
ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration
71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame
52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system
bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs
a7248cd [Alexander Bezzubov] Python: draft of pandas support
15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
### What is this PR for?
This PR is applying new interpreter register mechanism to python interpreter.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1045](https://issues.apache.org/jira/browse/ZEPPELIN-1045)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Mina Lee <minalee@apache.org>
Closes#1063 from minahlee/ZEPPELIN-1045 and squashes the following commits:
66b8f73 [Mina Lee] Add zeppelin.python.maxResult property to python interpreter
5013890 [Mina Lee] Apply new mechanism to PythonInterpreter
### What is this PR for?
This PR is to automate release publish to maven repository.
We used to use maven-deploy-plugin and maven-release-plugin for release but somehow it didn't work well with Zeppelin so 0.5.5 and 0.5.6 haven't been published to maven repository.
Publishing release to maven repository will eventually help zeppelin to reduce binary package size by leading users to use Dynamic interpreter loading(#908).
Originally below modules were skipped for maven release
- all interpreters(except spark)
- zeppelin-display
- zeppelin-server
- zeppelin-distribution
on the other hand this pr will skip only:
- zeppelin-distribution
### What type of PR is it?
Infra
### Todos
- [x] Include SparkR/R interpreter in release
- [x] Create common_release.sh to remove build configuration duplication
- [x] Check curl networking failure
### What is the Jira issue?
[ZEPPELIN-986](https://issues.apache.org/jira/browse/ZEPPELIN-986)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, https://cwiki.apache.org/confluence/display/ZEPPELIN/Preparing+Zeppelin+Release will be updated accordingly once this pr is merged.
Author: Mina Lee <minalee@apache.org>
Closes#994 from minahlee/ZEPPELIN-986 and squashes the following commits:
b0e8e67 [Mina Lee] Revert "Add geode, scalding profile in maven artifact build"
cd4cbcd [Mina Lee] curl failure check
c0ea07c [Mina Lee] Fix wrong indentation
a88bc1d [Mina Lee] Add geode, scalding profile in maven artifact build
2cced61 [Mina Lee] Add r to binary package and maven build
903bc12 [Mina Lee] Move duplicate code to common_release.sh
a3eb676 [Mina Lee] Include zeppelin-server module in publish artifiact
48d338f [Mina Lee] Rollback mistakenly removed plugin
aafaf42 [Mina Lee] Follow google shell style guide
30dcc65 [Mina Lee] remove deploy plugin from pom since custom script will be used instead for deploy
cd1f08c [Mina Lee] Refactor create release script
e764f5f [Mina Lee] Add maven publish release script
### What is this PR for?
This is first step improving current Python interpreter implementation.
It has just a cleanup, style and docs improvements.
### What type of PR is it?
Improvement
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1021 from bzz/improve/python and squashes the following commits:
3cafa22 [Alexander Bezzubov] Python: make interpreter logs less verbose
744f7d2 [Alexander Bezzubov] Python: move technical details from doc to README.md
0f74e5d [Alexander Bezzubov] Python: return first running job
08c2bb4 [Alexander Bezzubov] Python: upd Java code to conform project conventions
e84cd5c [Alexander Bezzubov] Python: normalize newlines in python
58efcc9 [Alexander Bezzubov] Python: normalize newlines in Java
### What is this PR for?
When people implement a new interpreter, they extend [interpreter.java](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java) as described in [here](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/writingzeppelininterpreter.html). Among the several methods in [interpreter.java](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java), [completion API](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java#L109) enables auto-completion.
However this API is too simple compared to other project's auto-completion and hard to add more at the moment. So for the aspect of further expansion, it would be better to separate and restructure this API before the this release( 0.6.0 ).
### What type of PR is it?
Improvement
### Todos
* [x] - Create new structure : `InterpreterCompletion` in `RemoteInterpreterService.thrift` and regenerate `zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/thrift/*` files
* [x] - Change all existing `List<String> completion` -> `List<InterpreterCompletion> completion`
* [x] - Change `paragraph.controller.js` to point real `name` and `value`
### What is the Jira issue?
[ZEPPELIN-982](https://issues.apache.org/jira/browse/ZEPPELIN-982)
### How should this be tested?
Since this improvement is just API change, it should work same as before. So after applying this patch, and check whether auto-completion works well or not.
Use `. + ctrl` for auto-completion. For example,
```
%spark
sc.version
```
When after typing `sc.` and pushing `. + ctrl` down, `version` should be shown in the auto-completion list.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Closes#984 from AhyoungRyu/ZEPPELIN-982 and squashes the following commits:
311dc29 [AhyoungRyu] Fix travis
9d384ec [AhyoungRyu] Address @minalee feedback
fdfae8f [AhyoungRyu] Address @jongyoul review
bd4f8c0 [AhyoungRyu] Remove abstract and make it return null by default
f8352c7 [AhyoungRyu] Fix travis error
43d81f6 [AhyoungRyu] Remove console.log
24912fa [AhyoungRyu] Fix type casting error in SparkInterpreter
80e295b [AhyoungRyu] Change return type
bd04c22 [AhyoungRyu] Apply new InterpreterCompletion class to all interpreter class files
c283043 [AhyoungRyu] Apply new InterpreterCompletion class under zeppelin-zengine/
dbecc51 [AhyoungRyu] Apply new InterpreterCompletion class under zeppelin-server/
6449455 [AhyoungRyu] Apply new InterpreterCompletion class under zeppelin-interpreter/
919b159 [AhyoungRyu] Add automatically generated thrift class
9e69e11 [AhyoungRyu] Change v -> v.name & v.value in front
73e374e [AhyoungRyu] Define InterpreterCompletion structure to thrift file
### What is this PR for?
Avoiding and preventing undesired NullPointerException during re-loading python interpreter
Since py4j is optional when PythonInterpreter starts, it should be optional too while it close.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* N/A
### How should this be tested?
Reload PythongInterpreter on Zeppelin web "Interpreter" menu and check the log message if it complains NullPointerException
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Kwon, Yeong-Eon <happylogin@gmail.com>
Author: Y.E. Kwon <OutOfBedlam@users.noreply.github.com>
Closes#1009 from OutOfBedlam/python and squashes the following commits:
5ed2413 [Y.E. Kwon] Fix code style
ccbf7ef [Kwon, Yeong-Eon] Prevent NullPointerException if "gatewayServer" does not exist
### What is this PR for?
Remove `incubating` term from pom files
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Mina Lee <minalee@nflabs.com>
Closes#942 from minahlee/tlp/removeIncubating and squashes the following commits:
e605b54 [Mina Lee] Remove incubating from pom files
### What is this PR for?
Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL.
The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change.
I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG.
### What type of PR is it?
[Feature]
### Todos
* [x] - Code review
* [x] - Improve bootstrap.py : choose available helper functions and their names
* [x] - Unit / IT tests ?
* [x] documentation updates needed, that AhyoungRyu pointed out
* [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license
* [x] double-check that code formatting conforms project style guide
* [x] the branch need to be rebased on latest master.
### What is the Jira issue?
[ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22)
### How should this be tested?
1. In interpreter screen, in Python section, specify in python.path the python binary you want to use
2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities.
3. Install py4j (pip install py4j) if you want to use input form
### Screenshots
### Questions:
* Does the licenses files need update? Yes, only bin-license (py4j)
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Hervé RIVIERE <hriviere@users.noreply.github.com>
Closes#869 from hriviere/PR_interpreter_python and squashes the following commits:
80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license
a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review
3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention
54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review
6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license
11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md
e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON
c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md
5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description
7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing
fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python
02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax)
60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention
9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html
7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error
1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
