### What is this PR for?
fix flaky `PythonInterpreter.testClose()` test
### What type of PR is it?
Hot Fix
### What is the Jira issue?
[ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063)
### How should this be tested?
CI should pass
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1134 from bzz/ZEPPELIN-1063/python/add-retrys and squashes the following commits:
65698d2 [Alexander Bezzubov] Reduce 😴 time in python interpreter unit-test
1be7732 [Alexander Bezzubov] Adding timeout for py4j gateway to have time to shutdown
3529fe5 [Alexander Bezzubov] fix PythonInterpreterTest.testClose() test
### What is this PR for?
Bump up version to 0.7.0-SNAPSHOT
Author: Mina Lee <minalee@apache.org>
Closes#1016 from minahlee/0.7.0-SNAPSHOT and squashes the following commits:
541e1b3 [Mina Lee] Bump up zeppelin-examples version to 0.7.0-SNAPSHOT
ea8c0ad [Mina Lee] Bump up version to 0.7.0-SNAPSHOT
### What is this PR for?
Implement paragraph ERROR status for Python interpreter in case of Error or Exception in the output.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1105](https://issues.apache.org/jira/browse/ZEPPELIN-1105)
### How should this be tested?
CI should pass, or
```
mvn "-Dtest=org.apache.zeppelin.python.PythonInterpreterWithPythonInstalledTest" test -pl python
```
should pass, or paragraph status should be ERROR for something like
```
import some-thing
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1124 from bzz/ZEPPELIN-1105/python/add-paragraph-error-statu and squashes the following commits:
a7bf8f3 [Alexander Bezzubov] Python: add missing license header
b585982 [Alexander Bezzubov] Python: include Python-dependant tests to 1 CI profile
e7d5371 [Alexander Bezzubov] Python: add ERROR paragraph status on Error and Exception in output
4c1107b [Alexander Bezzubov] Refactoring: rename and extract var assignment
### What is this PR for?
fix flaky python interpreter test
### What type of PR is it?
Bug Fix
### Todos
* [x] cleanup test code
* [x] fix flaky open port check
### What is the Jira issue?
[ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063)
### How should this be tested?
`mvn "-Dtest=org.apache.zeppelin.python.PythonInterpreterTest" test -pl python`
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1094 from bzz/fix/python-tests/ZEPPELIN-1063 and squashes the following commits:
b9de5a2 [Alexander Bezzubov] Python: refactoring open port checking
09f3018 [Alexander Bezzubov] Python: refactoring - arrange imports
46e42e8 [Alexander Bezzubov] Python: refactoring mock() structure and JavaDocs
651d8e8 [Alexander Bezzubov] Python: refactoring loggers
d7f8cdd [Alexander Bezzubov] Python: normalize newlines in tests
### What is this PR for?
Update `testPy4jIsNotInstalled `, `testPy4jIsInstalled` test
- `z.show` -> `def show` to check `show` function is defined
- check if `bootstrap_input.py` excuted by checking `z = Py4jZeppelinContext` instead of `z = PyZeppelinContext`
- add license header in `__init__.py` file
### What type of PR is it?
Hot Fix
Author: Mina Lee <minalee@apache.org>
Closes#1075 from minahlee/adjustPythonTest and squashes the following commits:
d46c5e1 [Mina Lee] Update api name in docs
6d82e9f [Mina Lee] Add license to __init__.py
f66e9dc [Mina Lee] Fix python test case
### What is this PR for?
Returning back to pass the CI
### What type of PR is it?
[Hot Fix]
### Todos
* [x] - Fix test cases
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1048
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes#1073 from jongyoul/hotfix-fix-pythoninterpretertest and squashes the following commits:
32be5d1 [Jongyoul Lee] Fixed test failed
### What is this PR for?
Display Pandas DataFrame using Zeppelin's Table Display system.
### What type of PR is it?
Feature
### Todos
* [x] fix NPE in logs on empty paragraph execution
* [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)`
* [x] pandas: support `z.show(df)`
* [x] update docs
### What is the Jira issue?
[ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048)
### How should this be tested?
"Zeppelin Tutorial: Python - matplotlib basic" should work, and
```python
import pandas as pd
rates = pd.read_csv("bank.csv", sep=";")
z.show(rates)
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1067 from bzz/python/pandas-support and squashes the following commits:
3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API
ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration
71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame
52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system
bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs
a7248cd [Alexander Bezzubov] Python: draft of pandas support
15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
### What is this PR for?
This PR is applying new interpreter register mechanism to python interpreter.
### What type of PR is it?
Improvement
### What is the Jira issue?
[ZEPPELIN-1045](https://issues.apache.org/jira/browse/ZEPPELIN-1045)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Mina Lee <minalee@apache.org>
Closes#1063 from minahlee/ZEPPELIN-1045 and squashes the following commits:
66b8f73 [Mina Lee] Add zeppelin.python.maxResult property to python interpreter
5013890 [Mina Lee] Apply new mechanism to PythonInterpreter
### What is this PR for?
This PR is to automate release publish to maven repository.
We used to use maven-deploy-plugin and maven-release-plugin for release but somehow it didn't work well with Zeppelin so 0.5.5 and 0.5.6 haven't been published to maven repository.
Publishing release to maven repository will eventually help zeppelin to reduce binary package size by leading users to use Dynamic interpreter loading(#908).
Originally below modules were skipped for maven release
- all interpreters(except spark)
- zeppelin-display
- zeppelin-server
- zeppelin-distribution
on the other hand this pr will skip only:
- zeppelin-distribution
### What type of PR is it?
Infra
### Todos
- [x] Include SparkR/R interpreter in release
- [x] Create common_release.sh to remove build configuration duplication
- [x] Check curl networking failure
### What is the Jira issue?
[ZEPPELIN-986](https://issues.apache.org/jira/browse/ZEPPELIN-986)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, https://cwiki.apache.org/confluence/display/ZEPPELIN/Preparing+Zeppelin+Release will be updated accordingly once this pr is merged.
Author: Mina Lee <minalee@apache.org>
Closes#994 from minahlee/ZEPPELIN-986 and squashes the following commits:
b0e8e67 [Mina Lee] Revert "Add geode, scalding profile in maven artifact build"
cd4cbcd [Mina Lee] curl failure check
c0ea07c [Mina Lee] Fix wrong indentation
a88bc1d [Mina Lee] Add geode, scalding profile in maven artifact build
2cced61 [Mina Lee] Add r to binary package and maven build
903bc12 [Mina Lee] Move duplicate code to common_release.sh
a3eb676 [Mina Lee] Include zeppelin-server module in publish artifiact
48d338f [Mina Lee] Rollback mistakenly removed plugin
aafaf42 [Mina Lee] Follow google shell style guide
30dcc65 [Mina Lee] remove deploy plugin from pom since custom script will be used instead for deploy
cd1f08c [Mina Lee] Refactor create release script
e764f5f [Mina Lee] Add maven publish release script
### What is this PR for?
This is first step improving current Python interpreter implementation.
It has just a cleanup, style and docs improvements.
### What type of PR is it?
Improvement
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes#1021 from bzz/improve/python and squashes the following commits:
3cafa22 [Alexander Bezzubov] Python: make interpreter logs less verbose
744f7d2 [Alexander Bezzubov] Python: move technical details from doc to README.md
0f74e5d [Alexander Bezzubov] Python: return first running job
08c2bb4 [Alexander Bezzubov] Python: upd Java code to conform project conventions
e84cd5c [Alexander Bezzubov] Python: normalize newlines in python
58efcc9 [Alexander Bezzubov] Python: normalize newlines in Java
### What is this PR for?
When people implement a new interpreter, they extend [interpreter.java](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java) as described in [here](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/writingzeppelininterpreter.html). Among the several methods in [interpreter.java](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java), [completion API](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java#L109) enables auto-completion.
However this API is too simple compared to other project's auto-completion and hard to add more at the moment. So for the aspect of further expansion, it would be better to separate and restructure this API before the this release( 0.6.0 ).
### What type of PR is it?
Improvement
### Todos
* [x] - Create new structure : `InterpreterCompletion` in `RemoteInterpreterService.thrift` and regenerate `zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/thrift/*` files
* [x] - Change all existing `List<String> completion` -> `List<InterpreterCompletion> completion`
* [x] - Change `paragraph.controller.js` to point real `name` and `value`
### What is the Jira issue?
[ZEPPELIN-982](https://issues.apache.org/jira/browse/ZEPPELIN-982)
### How should this be tested?
Since this improvement is just API change, it should work same as before. So after applying this patch, and check whether auto-completion works well or not.
Use `. + ctrl` for auto-completion. For example,
```
%spark
sc.version
```
When after typing `sc.` and pushing `. + ctrl` down, `version` should be shown in the auto-completion list.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: AhyoungRyu <fbdkdud93@hanmail.net>
Closes#984 from AhyoungRyu/ZEPPELIN-982 and squashes the following commits:
311dc29 [AhyoungRyu] Fix travis
9d384ec [AhyoungRyu] Address @minalee feedback
fdfae8f [AhyoungRyu] Address @jongyoul review
bd4f8c0 [AhyoungRyu] Remove abstract and make it return null by default
f8352c7 [AhyoungRyu] Fix travis error
43d81f6 [AhyoungRyu] Remove console.log
24912fa [AhyoungRyu] Fix type casting error in SparkInterpreter
80e295b [AhyoungRyu] Change return type
bd04c22 [AhyoungRyu] Apply new InterpreterCompletion class to all interpreter class files
c283043 [AhyoungRyu] Apply new InterpreterCompletion class under zeppelin-zengine/
dbecc51 [AhyoungRyu] Apply new InterpreterCompletion class under zeppelin-server/
6449455 [AhyoungRyu] Apply new InterpreterCompletion class under zeppelin-interpreter/
919b159 [AhyoungRyu] Add automatically generated thrift class
9e69e11 [AhyoungRyu] Change v -> v.name & v.value in front
73e374e [AhyoungRyu] Define InterpreterCompletion structure to thrift file
### What is this PR for?
Avoiding and preventing undesired NullPointerException during re-loading python interpreter
Since py4j is optional when PythonInterpreter starts, it should be optional too while it close.
### What type of PR is it?
[Improvement]
### What is the Jira issue?
* N/A
### How should this be tested?
Reload PythongInterpreter on Zeppelin web "Interpreter" menu and check the log message if it complains NullPointerException
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: Kwon, Yeong-Eon <happylogin@gmail.com>
Author: Y.E. Kwon <OutOfBedlam@users.noreply.github.com>
Closes#1009 from OutOfBedlam/python and squashes the following commits:
5ed2413 [Y.E. Kwon] Fix code style
ccbf7ef [Kwon, Yeong-Eon] Prevent NullPointerException if "gatewayServer" does not exist
### What is this PR for?
Remove `incubating` term from pom files
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Mina Lee <minalee@nflabs.com>
Closes#942 from minahlee/tlp/removeIncubating and squashes the following commits:
e605b54 [Mina Lee] Remove incubating from pom files
### What is this PR for?
Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL.
The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change.
I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG.
### What type of PR is it?
[Feature]
### Todos
* [x] - Code review
* [x] - Improve bootstrap.py : choose available helper functions and their names
* [x] - Unit / IT tests ?
* [x] documentation updates needed, that AhyoungRyu pointed out
* [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license
* [x] double-check that code formatting conforms project style guide
* [x] the branch need to be rebased on latest master.
### What is the Jira issue?
[ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22)
### How should this be tested?
1. In interpreter screen, in Python section, specify in python.path the python binary you want to use
2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities.
3. Install py4j (pip install py4j) if you want to use input form
### Screenshots
### Questions:
* Does the licenses files need update? Yes, only bin-license (py4j)
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes
Author: Hervé RIVIERE <hriviere@users.noreply.github.com>
Closes#869 from hriviere/PR_interpreter_python and squashes the following commits:
80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license
a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review
3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention
54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review
6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license
11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md
e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON
c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md
5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description
7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing
fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python
02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax)
60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention
9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html
7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error
1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
