mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
42 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Lee moon soo
|
1972a58620 |
[HOTFIX] Dynamic form in python interpreter
### What is this PR for? https://github.com/apache/zeppelin/pull/2106 rewrote python interpreter. But dynamic form feature is not rewritten correctly. ### What type of PR is it? Hot Fix ### Todos * [x] - Bring dynamic form back ### What is the Jira issue? https://github.com/apache/zeppelin/pull/2106 ### How should this be tested? run ``` %python print("Hello "+z.input("name", "sun")) ``` ``` %python print("Hello "+z.select("day", [("1","mon"), ("2","tue"), ("3","wed"), ("4","thurs"), ("5","fri"), ("6","sat"), ("7","sun")])) ``` ``` %python options = [("apple","Apple"), ("banana","Banana"), ("orange","Orange")] print("Hello "+ " and ".join(z.checkbox("fruit", options, ["apple"]))) ``` ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo <moon@apache.org> Closes #2155 from Leemoonsoo/python_get_interpreter_context and squashes the following commits: |
||
|
astroshim
|
287ffd50e2 |
Rewrite PythonInterpreter.
### What is this PR for? I've been testing the python interpreter and I found at least 4 major issues in the current python interpreter. 1. not working streaming output. - https://issues.apache.org/jira/browse/ZEPPELIN-2225 2. printed "..." when there is indent in the python code. - https://issues.apache.org/jira/browse/ZEPPELIN-1929 3. very slow output of matplotlib - https://issues.apache.org/jira/browse/ZEPPELIN-1894 - https://issues.apache.org/jira/browse/ZEPPELIN-1360 4. Unexpected output of matplotlib. - https://issues.apache.org/jira/browse/ZEPPELIN-2107 so I changed python interpreter to use py4j based on pyspark interpreter and would be fixed above issues. and I am going to recreate conda, docker for python interpreter ASAP. ### What type of PR is it? Bug Fix | Hot Fix | Refactoring ### How should this be tested? 1. not working streaming output. ``` import time for x in range(0, 5): print x time.sleep(1) ``` 2. printed "..." when there is indent in the python code. ``` def fn(): print("hi") fn() ``` 3. very slow output of matplotlib. ``` import matplotlib import sys import matplotlib.pyplot as plt plt.plot([1,2,3]) ``` 4. Unexpected output of matplotlib. ``` import matplotlib.pyplot as plt import matplotlib as mpl # Make a figure and axes with dimensions as desired. fig = plt.figure(figsize=(8, 3)) ax1 = fig.add_axes([0.05, 0.80, 0.9, 0.15]) ax2 = fig.add_axes([0.05, 0.475, 0.9, 0.15]) ax3 = fig.add_axes([0.05, 0.15, 0.9, 0.15]) # Set the colormap and norm to correspond to the data for which # the colorbar will be used. cmap = mpl.cm.cool norm = mpl.colors.Normalize(vmin=5, vmax=10) # ColorbarBase derives from ScalarMappable and puts a colorbar # in a specified axes, so it has everything needed for a # standalone colorbar. There are many more kwargs, but the # following gives a basic continuous colorbar with ticks # and labels. cb1 = mpl.colorbar.ColorbarBase(ax1, cmap=cmap, norm=norm, orientation='horizontal') cb1.set_label('Some Units') # The second example illustrates the use of a ListedColormap, a # BoundaryNorm, and extended ends to show the "over" and "under" # value colors. cmap = mpl.colors.ListedColormap(['r', 'g', 'b', 'c']) cmap.set_over('0.25') cmap.set_under('0.75') # If a ListedColormap is used, the length of the bounds array must be # one greater than the length of the color list. The bounds must be # monotonically increasing. bounds = [1, 2, 4, 7, 8] norm = mpl.colors.BoundaryNorm(bounds, cmap.N) cb2 = mpl.colorbar.ColorbarBase(ax2, cmap=cmap, norm=norm, # to use 'extend', you must # specify two extra boundaries: boundaries=[0] + bounds + [13], extend='both', ticks=bounds, # optional spacing='proportional', orientation='horizontal') cb2.set_label('Discrete intervals, some other units') # The third example illustrates the use of custom length colorbar # extensions, used on a colorbar with discrete intervals. cmap = mpl.colors.ListedColormap([[0., .4, 1.], [0., .8, 1.], [1., .8, 0.], [1., .4, 0.]]) cmap.set_over((1., 0., 0.)) cmap.set_under((0., 0., 1.)) bounds = [-1., -.5, 0., .5, 1.] norm = mpl.colors.BoundaryNorm(bounds, cmap.N) cb3 = mpl.colorbar.ColorbarBase(ax3, cmap=cmap, norm=norm, boundaries=[-10] + bounds + [10], extend='both', # Make the length of each extension # the same as the length of the # interior colors: extendfrac='auto', ticks=bounds, spacing='uniform', orientation='horizontal') cb3.set_label('Custom extension lengths, some other units') plt.show() ``` ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <hsshim@zepl.com> Author: Lee moon soo <moon@apache.org> Author: HyungSung <hsshim@nflabs.com> Closes #2106 from astroshim/py4jPythonInterpreter and squashes the following commits: |
||
|
1ambda
|
034fdc6735 |
[ZEPPELIN-1917] Improve python.conda interpreter
### What is this PR for? Add missing commands to the `python.conda` interpreter - `conda info` - `conda list` - `conda create` - `conda install` - `conda uninstall (alias of remove)` - `conda env *` #### Implementation Detail The reason I modified `PythonProcess` is due to NPE ```java // https://github.com/apache/zeppelin/blob/master/python/src/main/java/org/apache/zeppelin/python/PythonProcess.java#L107-L118 public String sendAndGetResult(String cmd) throws IOException { writer.println(cmd); writer.println(); writer.println("\"" + STATEMENT_END + "\""); StringBuilder output = new StringBuilder(); String line = null; // NPE when line is null while (!(line = reader.readLine()).contains(STATEMENT_END)) { logger.debug("Read line from python shell : " + line); output.append(line + "\n"); } return output.toString(); } ``` ``` java.lang.NullPointerException at org.apache.zeppelin.python.PythonProcess.sendAndGetResult(PythonProcess.java:113) at org.apache.zeppelin.python.PythonInterpreter.sendCommandToPython(PythonInterpreter.java:250) at org.apache.zeppelin.python.PythonInterpreter.bootStrapInterpreter(PythonInterpreter.java:272) at org.apache.zeppelin.python.PythonInterpreter.open(PythonInterpreter.java:100) at org.apache.zeppelin.python.PythonCondaInterpreter.restartPythonProcess(PythonCondaInterpreter.java:139) at org.apache.zeppelin.python.PythonCondaInterpreter.interpret(PythonCondaInterpreter.java:88) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:494) at org.apache.zeppelin.scheduler.Job.run(Job.java:175) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) ``` ### What type of PR is it? [Improvement | Refactoring] ### Todos * [x] - info * [x] - list * [x] - create * [x] - install * [x] - uninstall (= remove) * [x] - env * ### What is the Jira issue? [ZEPPELIN-1917](https://issues.apache.org/jira/browse/ZEPPELIN-1917) ### How should this be tested? 1. Install [miniconda](http://conda.pydata.org/miniconda.html) 2. Make sure that your python interpreter can use `conda` (check the Interpreter Binding page) 3. Remove `test` conda env since we will create in the following section ```sh $ conda env remove --yes --name test ``` 4. Run these commands with `%python.conda` ``` %python.conda info %python.conda env list %python.conda create --name test # you should be able to see `test` in the list %python.conda env list %python.conda activate pymysql %python.conda install pymysql # you should be able to import %python import pymysql.cursors %python.conda uninstall pymysql %python.conda deactivate pymysql # you should be able to see `No module named pymysql.cursor` since we deactivated %python import pymysql.cursors ``` ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1amb4a@gmail.com> Closes #1868 from 1ambda/ZEPPELIN-1917/improve-conda-interpreter and squashes the following commits: |
||
|
astroshim
|
8e6bfb45ea |
[ZEPPELIN-1930](HotFix) PythonInterpreter syntax error.
### What is this PR for? This PR fixes syntax error of PythonInterpreter. ### What type of PR is it? Bug Fix ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1930 ### How should this be tested? Please run following code on the paragraph of PythonInterpreter. ``` for x in range(0,3): print ("hi") ``` ### Screenshots (if appropriate) - before  - after  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <hsshim@zepl.com> Closes #1877 from astroshim/ZEPPELIN-1930 and squashes the following commits: |
||
|
astroshim
|
8e6f333a69 |
Add PythonDockerInterpreter to interpreter group
### What is this PR for? Recently PythonDockerInterpreter added to PythonInterpreter in https://github.com/apache/zeppelin/pull/1654. ### What type of PR is it? Bug Fix | Improvement ### Screenshots (if appropriate) - before  - after  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Closes #1707 from astroshim/add/python-docker-interpreter and squashes the following commits: |
||
|
astroshim
|
01c45d7ae6 |
[ZEPPELIN-1724] conda run command removed in 4.1.0
### What is this PR for? Because `conda run` command removed since version `4.0.9`, PythonCondaInterpreter not working after the `conda-4.0.9`. This PR fixes this issue. I tested conda-4.2.12 and conda-4.0.9 . ### What type of PR is it? Bug Fix | Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1724 ### How should this be tested? Please refer to https://github.com/apache/zeppelin/pull/1645 ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Closes #1699 from astroshim/ZEPPELIN-1724 and squashes the following commits: |
||
|
Lee moon soo
|
850fd81a51 |
[ZEPPELIN-212] Multiple paragraph results
### What is this PR for?
Currently a paragraph can display only a single type of result.
For example
```
print("""
%text textout
%html htmlout
""")
```
will display only last display system detected, "htmlout".
This pr implements multiple results supports, so not only the last, but also all the display systems in a paragraph outputs can be displayed.
To do that, Note.json format need to be changed. from
```
paragraph : [
{
result : {
code: "",
type: "",
msg: ""
},
config : {
graph : {},
...
},
...
},
...
],
...
```
to
```
paragraph : [
{
results : {
{
code: "",
msg: [
{
type: "",
data: ""
},
{
type: "",
data: ""
},
...
]
},
config : {
results : [
{
graph : {},
},
{
graph : {},
},
...
]
},
...
},
...
],
...
```
### What type of PR is it?
Improvement
### Todos
* [x] - Make InterpreterResult and InterpreterOutput support multiple display system
* [x] - Automatic migration old format to new format
* [x] - Render multiple results in front-end
* [x] - Take care Importing old notebook format
* [x] - Make helium framework support multiple results
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-212
### How should this be tested?
run following code in a single spark paragraph.
```
%spark
// default output is text
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// print something in html
println(s"""%html <h3><font style="color:blue">Some HTML</font></h3>""")
// create table
println(s"""%table key\tvalue
sun\t100
moon\t200
""")
// display text again
println("""%text""")
(1 to 3).foreach{i=>
println(new java.util.Date())
Thread.sleep(1000)
}
// another table
Thread.sleep(1000)
println(s"""%table key\tvalue
apple\t100
banana\t200
""")
```
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? yes
* Does this needs documentation? yes
Author: Lee moon soo <moon@apache.org>
Closes #1658 from Leemoonsoo/ZEPPELIN-212 and squashes the following commits:
|
||
|
Alex Goodman
|
4ac577f711 |
[ZEPPELIN-1639] Add tests with external python dependencies to CI build
### What is this PR for? Take 2 of #1618 because I had some earlier problems with rebasing. Since, then I have added some new features, namely: - Matplotlib integration tests for pyspark - `install_external_dependencies.sh` which conditionally installs the R or python dependencies based on the specified build profile in `.travis.yml`. This saves several minutes of time for a few of the build profiles since the R dependencies are compiled from source and therefore take quite a bit of time to install. - The extra python unit tests which require external dependencies (`matplotlib` and `pandas`) are now relegated to two separate build profiles. This is done primarily to efficiently test both Python 2 and 3. - Some minor bugs in the python and pyspark interpreters (mostly with respect to python 3 compatibility) were caught as a result of these tests, and are also fixed in this PR. ### What type of PR is it? Improvement and Bugfix ### What is the Jira issue? [ZEPPELIN-1639](https://issues.apache.org/jira/browse/ZEPPELIN-1639) ### How should this be tested? CI tests should be green! ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alex Goodman <agoodm@users.noreply.github.com> Closes #1632 from agoodm/ZEPPELIN-1639 and squashes the following commits: |
||
|
1ambda
|
9780744494 |
[MINOR] Invalid constructor call in a test causes all CI failure
### What is this PR for?
A small minor fix for CI failure
- PR A added `replName` to a constructor of `InterpreterContext` as an argument.
- and PR B was merged without rebase
This results in compilation failure.
see also
- (current master) https://travis-ci.org/apache/zeppelin/builds/178654143
- (raw log) https://api.travis-ci.org/jobs/178654145/log.txt?deansi=true
```
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/travis/build/apache/zeppelin/python/src/test/java/org/apache/zeppelin/python/PythonDockerInterpreterTest.java:[73,12] no suitable constructor found for InterpreterContext(java.lang.String,java.lang.String,java.lang.String,java.lang.String,org.apache.zeppelin.user.AuthenticationInfo,java.util.HashMap<java.lang.String,java.lang.Object>,org.apache.zeppelin.display.GUI,<nulltype>,<nulltype>,<nulltype>,org.apache.zeppelin.interpreter.InterpreterOutput)
constructor org.apache.zeppelin.interpreter.InterpreterContext.InterpreterContext(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,org.apache.zeppelin.user.AuthenticationInfo,java.util.Map<java.lang.String,java.lang.Object>,org.apache.zeppelin.display.GUI,org.apache.zeppelin.display.AngularObjectRegistry,org.apache.zeppelin.resource.ResourcePool,java.util.List<org.apache.zeppelin.interpreter.InterpreterContextRunner>,org.apache.zeppelin.interpreter.InterpreterOutput,org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventClient) is not applicable
(actual and formal argument lists differ in length)
constructor org.apache.zeppelin.interpreter.InterpreterContext.InterpreterContext(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,org.apache.zeppelin.user.AuthenticationInfo,java.util.Map<java.lang.String,java.lang.Object>,org.apache.zeppelin.display.GUI,org.apache.zeppelin.display.AngularObjectRegistry,org.apache.zeppelin.resource.ResourcePool,java.util.List<org.apache.zeppelin.interpreter.InterpreterContextRunner>,org.apache.zeppelin.interpreter.InterpreterOutput) is not applicable
(actual and formal argument lists differ in length)
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ........................................... SUCCESS [ 4.191 s]
[INFO] Zeppelin: Interpreter .............................. SUCCESS [ 10.154 s]
[INFO] Zeppelin: Zengine .................................. SUCCESS [ 4.848 s]
[INFO] Zeppelin: Display system apis ...................... SUCCESS [ 16.617 s]
[INFO] Zeppelin: Spark dependencies ....................... SUCCESS [ 36.233 s]
[INFO] Zeppelin: Spark .................................... SUCCESS [ 22.821 s]
[INFO] Zeppelin: Markdown interpreter ..................... SUCCESS [ 0.633 s]
[INFO] Zeppelin: Angular interpreter ...................... SUCCESS [ 0.371 s]
[INFO] Zeppelin: Shell interpreter ........................ SUCCESS [ 0.523 s]
[INFO] Zeppelin: Livy interpreter ......................... SUCCESS [ 6.972 s]
[INFO] Zeppelin: HBase interpreter ........................ SUCCESS [ 3.623 s]
[INFO] Zeppelin: Apache Pig Interpreter ................... SUCCESS [ 3.531 s]
[INFO] Zeppelin: PostgreSQL interpreter ................... SUCCESS [ 0.591 s]
[INFO] Zeppelin: JDBC interpreter ......................... SUCCESS [ 1.166 s]
[INFO] Zeppelin: File System Interpreters ................. SUCCESS [ 1.054 s]
[INFO] Zeppelin: Flink .................................... SUCCESS [ 6.863 s]
[INFO] Zeppelin: Apache Ignite interpreter ................ SUCCESS [ 0.987 s]
[INFO] Zeppelin: Kylin interpreter ........................ SUCCESS [ 0.421 s]
[INFO] Zeppelin: Python interpreter ....................... FAILURE [ 0.481 s]
[INFO] Zeppelin: Lens interpreter ......................... SKIPPED
[INFO] Zeppelin: Apache Cassandra interpreter ............. SKIPPED
[INFO] Zeppelin: Elasticsearch interpreter ................ SKIPPED
[INFO] Zeppelin: BigQuery interpreter ..................... SKIPPED
[INFO] Zeppelin: Alluxio interpreter ...................... SKIPPED
[INFO] Zeppelin: Scio ..................................... SKIPPED
[INFO] Zeppelin: web Application .......................... SKIPPED
[INFO] Zeppelin: Server ................................... SKIPPED
[INFO] Zeppelin: Packaging distribution ................... SKIPPED
[INFO] Zeppelin: Scalding interpreter ..................... SKIPPED
[INFO] Zeppelin: Examples ................................. SKIPPED
[INFO] Zeppelin: Example application - Clock .............. SKIPPED
[INFO] Zeppelin: Example application - Horizontal Bar chart SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:02 min
[INFO] Finished at: 2016-11-24T17:21:08+00:00
[INFO] Final Memory: 181M/1246M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project zeppelin-python: Compilation failure
[ERROR] /home/travis/build/apache/zeppelin/python/src/test/java/org/apache/zeppelin/python/PythonDockerInterpreterTest.java:[73,12] no suitable constructor found for InterpreterContext(java.lang.String,java.lang.String,java.lang.String,java.lang.String,org.apache.zeppelin.user.AuthenticationInfo,java.util.HashMap<java.lang.String,java.lang.Object>,org.apache.zeppelin.display.GUI,<nulltype>,<nulltype>,<nulltype>,org.apache.zeppelin.interpreter.InterpreterOutput)
[ERROR] constructor org.apache.zeppelin.interpreter.InterpreterContext.InterpreterContext(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,org.apache.zeppelin.user.AuthenticationInfo,java.util.Map<java.lang.String,java.lang.Object>,org.apache.zeppelin.display.GUI,org.apache.zeppelin.display.AngularObjectRegistry,org.apache.zeppelin.resource.ResourcePool,java.util.List<org.apache.zeppelin.interpreter.InterpreterContextRunner>,org.apache.zeppelin.interpreter.InterpreterOutput,org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventClient) is not applicable
[ERROR] (actual and formal argument lists differ in length)
[ERROR] constructor org.apache.zeppelin.interpreter.InterpreterContext.InterpreterContext(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,org.apache.zeppelin.user.AuthenticationInfo,java.util.Map<java.lang.String,java.lang.Object>,org.apache.zeppelin.display.GUI,org.apache.zeppelin.display.AngularObjectRegistry,org.apache.zeppelin.resource.ResourcePool,java.util.List<org.apache.zeppelin.interpreter.InterpreterContextRunner>,org.apache.zeppelin.interpreter.InterpreterOutput) is not applicable
[ERROR] (actual and formal argument lists differ in length)
```
### What type of PR is it?
[Bug Fix]
### What is the Jira issue?
NO JIRA issue. (small, emergently)
### How should this be tested?
CI will do that :)
### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO
Author: 1ambda <1amb4a@gmail.com>
Closes #1680 from 1ambda/minor/fix-invalid-constructor-call-in-test and squashes the following commits:
|
||
|
astroshim
|
b7307d49de |
[ZEPPELIN-1567] Let JDBC interpreter use user credential information.
### What is this PR for? This PR is for the multi-tenant of JDBC Interpreter. User can create a user/password for JDBC account at the [Credential page](http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/datasource_authorization.html). The `Entity` of `Credential` is match with JDBC interpreter group name. If the account for JDBC is not setted in the `Interpreter property` then use `Credential`'s. ### What type of PR is it? Improvement ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1567 ### How should this be tested? Please refer to testMultiTenant() of JDBCInterpreterTest/ ### Screenshots (if appropriate) ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? no Author: astroshim <hsshim@nflabs.com> Closes #1539 from astroshim/jdbc-impersonation and squashes the following commits: |
||
|
Lee moon soo
|
caa664d6ee |
[ZEPPELIN-1683] Run python process in docker container
### What is this PR for? Inspired by ZEPPELIN-1671 conda interpreter. Docker can provides kind of virtual environment for python like conda does. This PR implements %python.docker interpreter that helps run python process in docker container. This PR implements feature on top of https://github.com/apache/zeppelin/pull/1645 ### What type of PR is it? Feature ### Todos * [x] - basic feature * [x] - unittest * [x] - documentation ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1683 ### How should this be tested? see screenshot ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #1654 from Leemoonsoo/pydocker and squashes the following commits: |
||
|
Lee moon soo
|
3665901504 |
[ZEPPELIN-1671] Conda interpreter
### What is this PR for? Conda interpreter that manages conda environment for PythonInterpreter ### What type of PR is it? Feature ### Todos * [x] - Basic impl * [x] - update doc ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1671 ### How should this be tested? Recreate(or create new) your python interpreter setting in gui. List all conda env ``` %python.conda ``` Activate env ``` %python.conda activate [name] ``` Deactivate env ``` %python.conda deactivate ``` ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #1645 from Leemoonsoo/conda and squashes the following commits: |
||
|
Alex Goodman
|
f7af21e219 |
[HOTFIX][ZEPPELIN-1656] z.show in Python interpreter does not work
### What is this PR for? There have been reports of #1534 causing the python interpreter to always show an error because `z` is not being set. As it turns out this is a result of improperly handling the case when matplotlib isn't found when initializing the interpreter. ### What type of PR is it? Bug Fix ### What is the Jira issue? [ZEPPELIN-1656](https://issues.apache.org/jira/browse/ZEPPELIN-1656) ### How should this be tested? Run any simple python paragraph. ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alex Goodman <agoodm@users.noreply.github.com> Closes #1628 from agoodm/patch-1 and squashes the following commits: |
||
|
Alex Goodman
|
aded8681bb |
[ZEPPELIN-1655] Dynamic forms in Python interpreter do not work
### What is this PR for?
After #1534 , Dynamic Forms were no longer working in the python interpreter. This is because the `Py4jZeppelinContext` constructor did not initialize the `_displayhook` which is always called on post-execute.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-1655](https://issues.apache.org/jira/browse/ZEPPELIN-1655)
### How should this be tested?
Run the following `%python` paragraph, being sure that Py4j is installed:
```python
%python
a, b, c = (1, 2, 3)
z.select("Choose a letter", ([a,"a"], [b,"b"], [c,"c"] ))
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alex Goodman <agoodm@users.noreply.github.com>
Closes #1626 from agoodm/ZEPPELIN-1655 and squashes the following commits:
|
||
|
Paul Bustios
|
dbd81bf856 |
[ZEPPELIN-1358] Add support to display Pandas DataFrame index using z.show()
### What is this PR for? Add support to display optionally Pandas DataFrame index using z.show(show_index=True) in python interpreter. By default, DataFrame index will not be displayed. ### What type of PR is it? Improvement ### What is the Jira issue? [ZEPPELIN-1358](https://issues.apache.org/jira/browse/ZEPPELIN-1358) ### How should this be tested? ``` mvn -Dpython.test.exclude='' test -pl python -am ``` ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? no - Is there breaking changes for older versions? no - Does this needs documentation? no Author: Paul Bustios <pbustios@gmail.com> Closes #1378 from bustios/ZEPPELIN-1358 and squashes the following commits: |
||
|
Alex Goodman
|
438dbca686 |
ZEPPELIN-1345 - Create a custom matplotlib backend that natively supports inline plotting in a python interpreter cell
### What is this PR for? This PR is the first of two major steps needed to improve matplotlib integration in Zeppelin (ZEPPELIN-1344). The latter, which is a plotting backend with fully interactive tools enabled, will be done afterwards in a separate PR. This PR specifically for automatically displaying output from calls to matplotlib plotting functions inline with each paragraph. Thanks to the addition of post-execute hooks (ZEPPELIN-1423), there is no need to call any `show()` function to display an inline plot, just like in Jupyter. ### What type of PR is it? Improvement ### Todos The main code has been written and anyone who reads this is encouraged to test it, but there are a few minor todos: - [x] - Add unit tests - [x] - Add documentation - [x] - Add screenshot showing iterative plotting with angular mode ### What is the Jira issue? [ZEPPELIN-1345](https://issues.apache.org/jira/browse/ZEPPELIN-1345) ### How should this be tested? In a pyspark or python paragraph, enter and run ``` python import matplotlib.pyplot as plt plt.plot([1, 2, 3]) ``` The plot should be displayed automatically without calling any `show()` function whatsoever. A special method called `configure_mpl()` can also be used to modify the inline plotting behavior. For example, ``` python z.configure_mpl(close=False, angular=True) plt.plot([1, 2, 3]) ``` allows for iterative updates to the plot provided you have PY4J installed for your python installation (which of course is always the case if you use pypsark). To clarify, this feature only currently works with pyspark (not python as there are no `angularBind()` and `angularUnbind()` methods yet). Doing something like: ``` plt.plot([3, 2, 1]) ``` will update the plot that was generated by the previous paragraph by leveraging Zeppelin's Angular Display System. However, by setting `close=False`, matplotlib will no longer automatically close figures so it is now up to the user to explicitly close each figure instance they create. There's quite a bit more options for `z.configure_mpl()`, but I will save that discussion for the documentation. ### Screenshots (if appropriate)  ### Questions: - Does the licenses files need update? No - Is there breaking changes for older versions? No - Does this needs documentation? Yes Author: Alex Goodman <agoodm@users.noreply.github.com> Closes #1534 from agoodm/ZEPPELIN-1345 and squashes the following commits: |
||
|
Mina Lee
|
60089f0f58 |
[ZEPPELIN-1566] Make paragraph editable with double click
### What is this PR for? This PR enables edit on double click for markdown/angular paragraph. Users can change `editOnDblClick` field to be `false` by editting `interpreter/md/interpreter-setting.json` or `conf/interpreter.json`. In the same context, users can set other type paragraphs to be editable on double click by setting `editOnDblClick` to be true. This PR also fixes bug that syntax highlight doesn't work on pasted code. ### What type of PR is it? Feature ### Todos * [x] Create test * [x] Update docs ### What is the Jira issue? [ZEPPELIN-1566](https://issues.apache.org/jira/browse/ZEPPELIN-1566) ### How should this be tested? 1. Create new markdown interpreter 2. Create notebook and run markdown paragraph 3. Double click markdown paragraph to edit ### Screenshots (if appropriate) **Edit on double click and hide editor on paragraph run**  **Syntax highlight on paste** Before  After  ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Mina Lee <minalee@apache.org> Closes #1540 from minahlee/ZEPPELIN-1566 and squashes the following commits: |
||
|
Kai Jiang
|
1b2635cfe6 |
[Zeppelin-1555] Eliminate prefix in PythonInterpreter exception
### What is this PR for? Solve bug metioned [here]( |
||
|
Mina Lee
|
a3ca800311 |
[ZEPPELIN-1026] set syntax highlight based on default bound interpreter
### What is this PR for? This is complete work of #1148. Comments and tasks on #1148 has been handled in this PR. - Add syntax language information in `interpreter-setting.json` - When user type `%replName` in paragraph, back-end check if the interpreter name with `replName` exists, and return language information to front-end if it does - If user doesn't specify `%replName`, default interpreter's language will be used - Using alias name for paragraph syntax highlight ### What type of PR is it? [Bug Fix | Improvement] ### What is the Jira issue? [ZEPPELIN-1026](https://issues.apache.org/jira/browse/ZEPPELIN-1026) ### How should this be tested? 1. Create new note and make markdown interpreter to be default. 2. See if markdown syntax is applied. ### Screenshots (if appropriate) #### Case 1. When the default interpreter set to python interpreter. **Before** Has `scala` as syntax highlight language when %python is not set. <img width="665" alt="screen shot 2016-07-07 at 10 46 20 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png"> **After** Has `python` as syntax highlight language even when %python is not set. <img width="666" alt="screen shot 2016-07-07 at 10 44 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png"> #### Case 2. When use alias name as repl name. **Before** <img width="742" alt="screen shot 2016-09-08 at 4 22 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png"> **After** <img width="741" alt="screen shot 2016-09-08 at 4 34 57 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png"> ### Further possible improvements There are still several cases that Zeppelin doesn't handle syntax highlight well. These can be handled with another jira ticket/PR. 1. When default bound interpreter changes, syntax highlight is not changed accordingly 2. When copy/paste code, syntax highlight won't be applied properly since Zeppelin only checks changes when cursor is in first line. ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes(for creating new interpreter) Author: Mina Lee <minalee@apache.org> Closes #1415 from minahlee/ZEPPELIN-1026 and squashes the following commits: |
||
|
CloverHearts
|
66d5811365 |
[ZEPPELIN-1412] add support multiline for pythonErrorIn method on python interpreter
### What is this PR for?
currently, has not support multiline exception text on python interpreter.
for example:
```
Exception: blabla
```
is error.
but
```
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception: test exception
```
is sucess (now)
to resolve this issue.
### What type of PR is it?
Bug Fix
### Todos
- [x] modification pythonErrorIn method
- [x] add test case
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1412?jql=project%20%3D%20ZEPPELIN%20AND%20status%20%3D%20Open
### How should this be tested?
added test case.
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
Author: CloverHearts <cloverheartsdev@gmail.com>
Closes #1407 from cloverhearts/dev/ZEPPELIN-1412 and squashes the following commits:
|
||
|
Paul Bustios
|
8064c54917 |
[ZEPPELIN-1327] Fix bug in z.show for Python interpreter
### What is this PR for? Currently, height parameter for z.show implementation to display PNG images in Python interpreter is not working. This PR fix that bug. ### What type of PR is it? Bug Fix ### What is the Jira issue? [ZEPPELIN-1327](https://issues.apache.org/jira/browse/ZEPPELIN-1327) ### How should this be tested? ```python import matplotlib.pyplot as plt x = [1,2,3,4,5] y = [6,7,8,9,0] plt.plot(x, y, marker="o") z.show(plt, height="200px") plt.close() ``` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Paul Bustios <pbustios@gmail.com> Closes #1352 from bustios/ZEPPELIN-1327 and squashes the following commits: |
||
|
Alex Goodman
|
582981677c |
ZEPPELIN-1328 - z.show in python interpreter does not display PNG images in python 3
### What is this PR for? Support for plotting PNG images via matplotlib inline for the python interpreter was recently added (#1329). However, these changes did not work for python3 since it handles strings differently. This PR aims to make the inline plotting compatible with both python 2 and 3. ### What type of PR is it? Bug Fix ### What is the Jira issue? * [ZEPPELIN-1328](https://issues.apache.org/jira/browse/ZEPPELIN-1328) ### How should this be tested? In a python interpreteter cell, make sure the following produce an image: ```python %python import matplotlib.pyplot as plt import numpy as np x = np.arange(5) plt.plot(x) z.show(plt, fmt='png') # Repeat for fmt='svg' ``` This should be tested for both python2 and 3 interpreters (via the interpreter settings page). ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alex Goodman <agoodm@users.noreply.github.com> Closes #1343 from agoodm/ZEPPELIN-1328 and squashes the following commits: |
||
|
Alex Goodman
|
8b40268d16 |
ZEPPELIN-1318 - Add support for matplotlib displaying png images in python interpreter
### What is this PR for? This PR adds support for plotting png images using the matplotlib helper function within a python interpreter (eg `z.show()`). The primary motivation for this is due to the overhead incurred from svg images, which can lag the notebooks if multiple, complicated images are generated (for example, multiple filled contour plots). png images are more lightweight, but of course come at a cost of image quality due to them being raster rather than vector like svg. The support for png images is incorporated through the use of a new optional argument to `z.show` called `fmt` which can be one of `'svg'` or `'png'`. The same code that is currently used in show is used for svg images while the code for png images relies on converting the image directly to a byte array and then entering the decoded byte string directly into an HTML image tag. Currently `fmt` defaults to `'png'` but I think we should consider discussing the pros and cons of each option in this PR. ### What type of PR is it? Improvement ### What is the Jira issue? [ZEPPELIN-1318](https://issues.apache.org/jira/browse/ZEPPELIN-1318) ### How should this be tested? In a notebook cell, enter: ```python %python import matplotlib.pyplot as plt import numpy as np plt.figure() plt.plot(np.arange(10)) z.show(plt, fmt=fmt) ``` Where `fmt` may be one of `'svg'` or `'png'`, and any other input should result in a `ValueError`. I would also recommend testing the example in the screenshot below. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes (if the changes to the `help()` docstring are not sufficient) Author: Alex Goodman <agoodm@users.noreply.github.com> Closes #1329 from agoodm/ZEPPELIN-1318 and squashes the following commits: |
||
|
Jeff Zhang
|
3dec4d7006 |
ZEPPELIN-1287. No need to call print to display output in PythonInterpreter
### What is this PR for? It is not necessary to call print to display output in PythonInterpreter. 2 main changes: * the root cause is the displayhook in bootstrap.py * also did some code refactoring on PythonInterpreter ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-1287 ### How should this be tested? Verify it manually ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #1278 from zjffdu/ZEPPELIN-1287 and squashes the following commits: |
||
|
Alexander Bezzubov
|
a922fd28c1 |
Small refactoring of Python interpreter
### What is this PR for? Small refactoring of Python interpreter, that is what it is. ### What type of PR is it? Refactoring ### Todos * [x] refactor `help()` * [x] impl `maxResult` fetch from JVM ### How should this be tested? `cd python && mvn -Dpython.test.exclude='' test ` pass (given that `pip install pandasql` and `pip install py4j`) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alexander Bezzubov <bzz@apache.org> Closes #1275 from bzz/python/refactoring and squashes the following commits: |
||
|
Paul Bustios
|
9eac20d08a |
[ZEPPELIN-1261] Bug fix in z.show() for matplotlib graphs
### What is this PR for? Bug fix in z.show() for matplotlib graphs and code refactoring ### What type of PR is it? Bug Fix ### What is the Jira issue? [ZEPPELIN-1261](https://issues.apache.org/jira/browse/ZEPPELIN-1261) ### How should this be tested? ``` %python import matplotlib.pyplot as plt x = [1,2,3,4,5] y = [6,7,8,9,0] plt.plot(x, y, marker="o") z.show(plt, height="20em") plt.close() ``` ``` %python import matplotlib.pyplot as plt x = [1,2,3,4,5] y = [6,7,8,9,0] plt.plot(x, y, marker="o") z.show(plt, height="300px") plt.close() ``` ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Paul Bustios <pbustios@gmail.com> Closes #1267 from bustios/ZEPPELIN-1261 and squashes the following commits: |
||
|
paulbustios
|
6f867ceb0c |
[ZEPPELIN-1255] Add cast to string in z.show() for Pandas DataFrame
### What is this PR for? Casting data types in Pandas DataFrame to string in z.show() ### What type of PR is it? Bug Fix ### What is the Jira issue? [ZEPPELIN-1255](https://issues.apache.org/jira/browse/ZEPPELIN-1255) ### How should this be tested? ``` %python import pandas as pd df = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data', header=None) df.columns=[1, 2, 3, 'PetalWidth', 'Name'] z.show(df) %python.sql SELECT * FROM df LIMIT 10 ``` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: paulbustios <pbustios@gmail.com> Closes #1249 from bustios/ZEPPELIN-1255 and squashes the following commits: |
||
|
Alexander Bezzubov
|
5afd7cacdb |
ZEPPELIN-1115: add correct %python.sql interpreter name to configuration
### What is this PR for?
Hotfix for [ZEPPELIN-1244](https://issues.apache.org/jira/browse/ZEPPELIN-1244)
### What type of PR is it?
Hot Fix
### What is the Jira issue?
[ZEPPELIN-1244](https://issues.apache.org/jira/browse/ZEPPELIN-1244)
### How should this be tested?
Remove interpreter configuration and try `%python.sql` i.e over [bank.csv](http://mlr.cs.umass.edu/ml/datasets/Bank+Marketing) read as `pandas.dataframe`
```
#stop zeppelin
rm conf/interpreter-setting.json
#start zeppelin
%python
import pandas as pd
rates = pd.read_csv("http://www3.dsi.uminho.pt/pcortez/data/bank.csv", sep=";")
%python.sql
SELECT * FROM rates LIMIT 10
```
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes #1238 from bzz/python/fix/ZEPPELIN-1244 and squashes the following commits:
|
||
|
Sangmin Yoon
|
80d910049b |
[ZEPPELIN-1206] fix "name 'z' is not defined" with python3
### What is this PR for?
PythonInterpreter can not use dynamic form with python3.
Fix this problem.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1206
### How should this be tested?
```
%python
print(z.input("test"))
```
Make a note above, and run it.
### Questions:
* Does the licenses files need update? NO
* Is there breaking changes for older versions? NO
* Does this needs documentation? NO
Author: Sangmin Yoon <sangmin.yoon@croquis.com>
Closes #1213 from sixmen/fix_python3 and squashes the following commits:
|
||
|
Alexander Bezzubov
|
d8b54cf76d |
ZEPPELIN-1115: Python - interpreter for SQL over DataFrame
### What is this PR for? Add new interpreter to Python group: `%python.sql` for SQL over DataFrame support ### What type of PR is it? Improvement ### TODOs * [x] add new interpreter `%python.sql` * [x] add test * [x] make Python-dependant tests, excluded from CI * PythonInterpreterWithPythonInstalledTest * PythonPandasSqlInterpreterTest * run manually by `mvn -Dpython.test.exclude='' test -pl python -am` * [x] add docs `%python.sql` * [x] make `%python.sql` fail gracefully in case there is no Pandas or PandaSQL installed * [x] after #747 is merged - rebase and remove `-Dpython.test.exclude=''` from both profiles ### What is the Jira issue? [ZEPPELIN-1115](https://issues.apache.org/jira/browse/ZEPPELIN-1115) ### How should this be tested? `mvn -Dpython.test.exclude='' test -pl python -am` should pass or manually run - Given the DataFrame i.e ``` %python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") ``` - SQL query it like ``` %python.sql SELECT * FROM rates LIMIT 10 ``` ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No, no dependencies were included in source or binary release * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1164 from bzz/ZEPPELIN-1115/python/add-sql-for-dataframes and squashes the following commits: |
||
|
Alexander Bezzubov
|
399c49b4e1 |
ZEPPELIN-1063: fix PythonInterpreter flaky test
### What is this PR for? fix flaky `PythonInterpreter.testClose()` test ### What type of PR is it? Hot Fix ### What is the Jira issue? [ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063) ### How should this be tested? CI should pass ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Alexander Bezzubov <bzz@apache.org> Closes #1134 from bzz/ZEPPELIN-1063/python/add-retrys and squashes the following commits: |
||
|
Alexander Bezzubov
|
2ee7f48cff |
ZEPPELIN-1105: Python - add paragraph ERROR status
### What is this PR for? Implement paragraph ERROR status for Python interpreter in case of Error or Exception in the output. ### What type of PR is it? Improvement ### What is the Jira issue? [ZEPPELIN-1105](https://issues.apache.org/jira/browse/ZEPPELIN-1105) ### How should this be tested? CI should pass, or ``` mvn "-Dtest=org.apache.zeppelin.python.PythonInterpreterWithPythonInstalledTest" test -pl python ``` should pass, or paragraph status should be ERROR for something like ``` import some-thing ``` ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alexander Bezzubov <bzz@apache.org> Closes #1124 from bzz/ZEPPELIN-1105/python/add-paragraph-error-statu and squashes the following commits: |
||
|
Alexander Bezzubov
|
7951e6c698 |
ZEPPELIN-1063: fix flaky python interpreter test
### What is this PR for? fix flaky python interpreter test ### What type of PR is it? Bug Fix ### Todos * [x] cleanup test code * [x] fix flaky open port check ### What is the Jira issue? [ZEPPELIN-1063](https://issues.apache.org/jira/browse/ZEPPELIN-1063) ### How should this be tested? `mvn "-Dtest=org.apache.zeppelin.python.PythonInterpreterTest" test -pl python` ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alexander Bezzubov <bzz@apache.org> Closes #1094 from bzz/fix/python-tests/ZEPPELIN-1063 and squashes the following commits: |
||
|
Mina Lee
|
df7dd5c373 |
[HOTFXI] Fix python test case and resolve rat license issue
### What is this PR for? Update `testPy4jIsNotInstalled `, `testPy4jIsInstalled` test - `z.show` -> `def show` to check `show` function is defined - check if `bootstrap_input.py` excuted by checking `z = Py4jZeppelinContext` instead of `z = PyZeppelinContext` - add license header in `__init__.py` file ### What type of PR is it? Hot Fix Author: Mina Lee <minalee@apache.org> Closes #1075 from minahlee/adjustPythonTest and squashes the following commits: |
||
|
Jongyoul Lee
|
74346a22b1 |
[HOTFIX] Fixed PythonInterpreterTest
### What is this PR for?
Returning back to pass the CI
### What type of PR is it?
[Hot Fix]
### Todos
* [x] - Fix test cases
### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1048
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Jongyoul Lee <jongyoul@gmail.com>
Closes #1073 from jongyoul/hotfix-fix-pythoninterpretertest and squashes the following commits:
|
||
|
Alexander Bezzubov
|
230d890142 |
ZEPPELIN-1048: Pandas support for python interpreter
### What is this PR for? Display Pandas DataFrame using Zeppelin's Table Display system. ### What type of PR is it? Feature ### Todos * [x] fix NPE in logs on empty paragraph execution * [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)` * [x] pandas: support `z.show(df)` * [x] update docs ### What is the Jira issue? [ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048) ### How should this be tested? "Zeppelin Tutorial: Python - matplotlib basic" should work, and ```python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") z.show(rates) ``` ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1067 from bzz/python/pandas-support and squashes the following commits: |
||
|
Mina Lee
|
ff4973d435 |
[ZEPPELIN-1045] Apply new mechanism to PythonInterpreter
### What is this PR for? This PR is applying new interpreter register mechanism to python interpreter. ### What type of PR is it? Improvement ### What is the Jira issue? [ZEPPELIN-1045](https://issues.apache.org/jira/browse/ZEPPELIN-1045) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Mina Lee <minalee@apache.org> Closes #1063 from minahlee/ZEPPELIN-1045 and squashes the following commits: |
||
|
Alexander Bezzubov
|
85ee2ddbcb |
Python: fix for 'run all' paragraphs
### What is this PR for?
Switch to FIFO scheduler as in current implementation `.interpret()` is not thread-safe and so in parallel one 'Run All' fails some paragraphs with NPE in logs
### What type of PR is it?
Bug Fix | Improvement
### How should this be tested?
'Run All' passes without NPE in logs i.e on this [Zeppelin notebook for python](https://www.zeppelinhub.com/viewer/notebooks/aHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL2J6ei9pbmN1YmF0b3ItemVwcGVsaW4vMTkyZjU3YjZjMGZkMjc4NzgwZDI3NDAzMGY1YmJlOTZlZThkNzdiYi9ub3RlYm9vay8yQlFBMzVDSlovbm90ZS5qc29u)
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
Author: Alexander Bezzubov <bzz@apache.org>
Closes #1033 from bzz/fix/python-run-all and squashes the following commits:
|
||
|
Alexander Bezzubov
|
c806d4a4c7 |
Python interpreter and doc cleanup
### What is this PR for? This is first step improving current Python interpreter implementation. It has just a cleanup, style and docs improvements. ### What type of PR is it? Improvement ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Alexander Bezzubov <bzz@apache.org> Closes #1021 from bzz/improve/python and squashes the following commits: |
||
|
AhyoungRyu
|
7b00dffd98 |
[ZEPPELIN-982] Improve interpreter completion API
### What is this PR for? When people implement a new interpreter, they extend [interpreter.java](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java) as described in [here](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/writingzeppelininterpreter.html). Among the several methods in [interpreter.java](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java), [completion API](https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java#L109) enables auto-completion. However this API is too simple compared to other project's auto-completion and hard to add more at the moment. So for the aspect of further expansion, it would be better to separate and restructure this API before the this release( 0.6.0 ). ### What type of PR is it? Improvement ### Todos * [x] - Create new structure : `InterpreterCompletion` in `RemoteInterpreterService.thrift` and regenerate `zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/thrift/*` files * [x] - Change all existing `List<String> completion` -> `List<InterpreterCompletion> completion` * [x] - Change `paragraph.controller.js` to point real `name` and `value` ### What is the Jira issue? [ZEPPELIN-982](https://issues.apache.org/jira/browse/ZEPPELIN-982) ### How should this be tested? Since this improvement is just API change, it should work same as before. So after applying this patch, and check whether auto-completion works well or not. Use `. + ctrl` for auto-completion. For example, ``` %spark sc.version ``` When after typing `sc.` and pushing `. + ctrl` down, `version` should be shown in the auto-completion list. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: AhyoungRyu <fbdkdud93@hanmail.net> Closes #984 from AhyoungRyu/ZEPPELIN-982 and squashes the following commits: |
||
|
Kwon, Yeong-Eon
|
a4f19148aa |
Prevent NullPointerException if "gatewayServer" does not exist
### What is this PR for? Avoiding and preventing undesired NullPointerException during re-loading python interpreter Since py4j is optional when PythonInterpreter starts, it should be optional too while it close. ### What type of PR is it? [Improvement] ### What is the Jira issue? * N/A ### How should this be tested? Reload PythongInterpreter on Zeppelin web "Interpreter" menu and check the log message if it complains NullPointerException ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Kwon, Yeong-Eon <happylogin@gmail.com> Author: Y.E. Kwon <OutOfBedlam@users.noreply.github.com> Closes #1009 from OutOfBedlam/python and squashes the following commits: |
||
|
Hervé RIVIERE
|
34734b9c8a |
[ZEPPELIN-502] Python interpreter group
### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots     ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: |
