zeppelin/docs/interpreter/python.md

110 lines
3.3 KiB
Markdown
Raw Normal View History

[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
---
layout: page
title: "Python Interpreter"
description: "Python Interpreter"
group: manual
---
{% include JB/setup %}
[ZEPPELIN-1018] Apply auto "Table of Contents" generator to Zeppelin docs website ### What is this PR for? I added auto TOC(Table of Contents) generator for Zeppelin documentation website. TOC can help people looking through whole contents at a glance and finding what they want quickly. I just added `<div id="toc"></div>` to the each documentation header. [`toc`](https://github.com/apache/zeppelin/compare/master...AhyoungRyu:ZEPPELIN-1018?expand=1#diff-85af09fb498a5667ea455391533f945dR3) recognize `<h2>` & `<h3>` as a title in the docs and it automatically generate TOC. So I set a rule for this work. (I'll write this rule on `docs/CONTRIBUTING.md` or [docs/howtocontributewebsite](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/howtocontributewebsite.html)). ``` # Level-1 Heading <- Use only for the main title of the page ## Level-2 Heading <- Start with this one ### Level-3 heading <- Only use this one for child of Level-2 toc only recognize Level-2 & Level-3 ``` Please see the below attached screenshot image. ### What type of PR is it? Improvement & Documentation ### Todos * [x] - Add TOC generator * [x] - Apply TOC(`<div id="toc"></div>`) to every documentation and reorganize each headers(apply the above rule) * [x] - Fix some broken code block in several docs * [x] - Apply TOC to `r.md` (Currently R docs has some duplicated info since [this one](https://github.com/apache/zeppelin/commit/d5e87fb8ba98f08db5b0a4995104ce19f182c678) and [this one](https://github.com/apache/zeppelin/commit/7d6cc7e99154e2d337c11fdf8be1a874ed3e9ada) ) * [x] - Apply TOC to `install.md` after #1010 merged * [x] - Apply TOC to `interpreterinstallation.md` after #1042 merged ### What is the Jira issue? [ZEPPELIN-1018](https://issues.apache.org/jira/browse/ZEPPELIN-1018) ### How should this be tested? 1. Apply this patch and build `docs/` with [this guide](https://github.com/apache/zeppelin/tree/master/docs#build-documentation) 2. Visit some docs page. Then you can see TOC in the header of page. ### Screenshots (if appropriate) - Automatically generated TOC in Spark interpreter docs page <img width="831" alt="screen shot 2016-06-16 at 9 37 18 pm" src="https://cloud.githubusercontent.com/assets/10060731/16140902/945b9c7a-340a-11e6-91f3-b6174738bed0.png"> ### Questions: * Does the licenses files need update? No. Actually I used [jekyll-table-of-contents#copyright](https://github.com/ghiculescu/jekyll-table-of-contents#copyright). But I don't need to add a license for this :) * Is there breaking changes for older versions? No * Does this needs documentation? Maybe Author: AhyoungRyu <fbdkdud93@hanmail.net> Closes #1031 from AhyoungRyu/ZEPPELIN-1018 and squashes the following commits: e66397b [AhyoungRyu] Apply TOC to interpreterinstallation.md 009579b [AhyoungRyu] Add more info to 'What is the next?' in install.md 04cf501 [AhyoungRyu] Revert 'where to start' section b7cbe5f [AhyoungRyu] Fix typo cf0911c [AhyoungRyu] Rename license file 388f35a [AhyoungRyu] Add jekyll-table-of-contents license info 6394c70 [AhyoungRyu] Fix image path in python.md d00e4b1 [AhyoungRyu] Move interpreter/screenshot/ -> asset/../img/docs-img/ 3ffb383 [AhyoungRyu] Remove duplicated info in r.md & apply toc a03ca99 [AhyoungRyu] Exclude toc.js from pom.xml 3fae7df [AhyoungRyu] Apply auto generated toc to install.md d114a9d [AhyoungRyu] Address @felixcheung feedback 6a788fe [AhyoungRyu] Resize TOC tab indent 6760c00 [AhyoungRyu] Apply auto TOC to all of docs under docs/storage/ fbde57f [AhyoungRyu] Apply auto TOC to all of docs under docs/quickstart/ db76eb6 [AhyoungRyu] Apply auto TOC to all of docs under docs/install/ f35db47 [AhyoungRyu] Apply auto TOC to all of docs under docs/displaysystem/ b05365f [AhyoungRyu] Apply auto TOC to all of docs under docs/rest-api/ 163691c [AhyoungRyu] Apply auto TOC to all of docs under docs/manual/ bef398e [AhyoungRyu] Apply auto TOC to all of docs under docs/development/ 9c5f76b [AhyoungRyu] Apply auto TOC to all of docs under docs/interpreter/ 587d4ba [AhyoungRyu] Apply auto TOC to all of docs under docs/security/ 1f10b97 [AhyoungRyu] Change toc configuration 78dca9e [AhyoungRyu] Add toc.js for auto generating TOC
2016-06-25 19:44:53 +00:00
# Python 2 & 3 Interpreter for Apache Zeppelin
<div id="toc"></div>
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
## Configuration
<table class="table-configuration">
<tr>
<th>Property</th>
<th>Default</th>
<th>Description</th>
</tr>
<tr>
<td>zeppelin.python</td>
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
<td>python</td>
<td>Path of the already installed Python binary (could be python2 or python3).
If python is not in your $PATH you can set the absolute directory (example : /usr/bin/python)
</td>
</tr>
<tr>
<td>zeppelin.python.maxResult</td>
<td>1000</td>
<td>Max number of dataframe rows to display.</td>
</tr>
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
</table>
## Enabling Python Interpreter
In a notebook, to enable the **Python** interpreter, click on the **Gear** icon and select **Python**
## Using the Python Interpreter
In a paragraph, use **_%python_** to select the **Python** interpreter and then input all commands.
The interpreter can only work if you already have python installed (the interpreter doesn't bring it own python binaries).
To access the help, type **help()**
## Python modules
The interpreter can use all modules already installed (with pip, easy_install...)
ZEPPELIN-1048: Pandas support for python interpreter ### What is this PR for? Display Pandas DataFrame using Zeppelin's Table Display system. ### What type of PR is it? Feature ### Todos * [x] fix NPE in logs on empty paragraph execution * [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)` * [x] pandas: support `z.show(df)` * [x] update docs ### What is the Jira issue? [ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048) ### How should this be tested? "Zeppelin Tutorial: Python - matplotlib basic" should work, and ```python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") z.show(rates) ``` ### Screenshots (if appropriate) ![screen shot 2016-06-23 at 10 29 00](https://cloud.githubusercontent.com/assets/5582506/16289133/85f0ddbc-392d-11e6-86a3-28d10e73f68d.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1067 from bzz/python/pandas-support and squashes the following commits: 3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration 71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame 52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs a7248cd [Alexander Bezzubov] Python: draft of pandas support 15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
2016-06-23 01:25:49 +00:00
## Use Zeppelin Dynamic Forms
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
You can leverage [Zeppelin Dynamic Form]({{BASE_PATH}}/manual/dynamicform.html) inside your Python code.
**Zeppelin Dynamic Form can only be used if py4j Python library is installed in your system. If not, you can install it with `pip install py4j`.**
Example :
```python
%python
### Input form
print (z.input("f1","defaultValue"))
### Select form
print (z.select("f1",[("o1","1"),("o2","2")],"2"))
### Checkbox form
print("".join(z.checkbox("f3", [("o1","1"), ("o2","2")],["1"])))
```
## Zeppelin features not fully supported by the Python Interpreter
* Interrupt a paragraph execution (`cancel()` method) is currently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter. A JIRA ticket ([ZEPPELIN-893](https://issues.apache.org/jira/browse/ZEPPELIN-893)) is opened to implement this feature in a next release of the interpreter.
* Progression bar in webUI (`getProgress()` method) is currently not implemented.
* Code-completion is currently not implemented.
## Matplotlib integration
ZEPPELIN-1048: Pandas support for python interpreter ### What is this PR for? Display Pandas DataFrame using Zeppelin's Table Display system. ### What type of PR is it? Feature ### Todos * [x] fix NPE in logs on empty paragraph execution * [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)` * [x] pandas: support `z.show(df)` * [x] update docs ### What is the Jira issue? [ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048) ### How should this be tested? "Zeppelin Tutorial: Python - matplotlib basic" should work, and ```python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") z.show(rates) ``` ### Screenshots (if appropriate) ![screen shot 2016-06-23 at 10 29 00](https://cloud.githubusercontent.com/assets/5582506/16289133/85f0ddbc-392d-11e6-86a3-28d10e73f68d.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1067 from bzz/python/pandas-support and squashes the following commits: 3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration 71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame 52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs a7248cd [Alexander Bezzubov] Python: draft of pandas support 15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
2016-06-23 01:25:49 +00:00
The python interpreter can display matplotlib graph with the function `z.show()`.
You need to have matplotlib module installed and a XServer running to use this functionality !
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
```python
%python
import matplotlib.pyplot as plt
plt.figure()
(.. ..)
ZEPPELIN-1048: Pandas support for python interpreter ### What is this PR for? Display Pandas DataFrame using Zeppelin's Table Display system. ### What type of PR is it? Feature ### Todos * [x] fix NPE in logs on empty paragraph execution * [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)` * [x] pandas: support `z.show(df)` * [x] update docs ### What is the Jira issue? [ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048) ### How should this be tested? "Zeppelin Tutorial: Python - matplotlib basic" should work, and ```python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") z.show(rates) ``` ### Screenshots (if appropriate) ![screen shot 2016-06-23 at 10 29 00](https://cloud.githubusercontent.com/assets/5582506/16289133/85f0ddbc-392d-11e6-86a3-28d10e73f68d.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1067 from bzz/python/pandas-support and squashes the following commits: 3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration 71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame 52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs a7248cd [Alexander Bezzubov] Python: draft of pandas support 15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
2016-06-23 01:25:49 +00:00
z.show(plt)
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
plt.close()
```
z.show function can take optional parameters to adapt graph width and height
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
```python
%python
ZEPPELIN-1048: Pandas support for python interpreter ### What is this PR for? Display Pandas DataFrame using Zeppelin's Table Display system. ### What type of PR is it? Feature ### Todos * [x] fix NPE in logs on empty paragraph execution * [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)` * [x] pandas: support `z.show(df)` * [x] update docs ### What is the Jira issue? [ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048) ### How should this be tested? "Zeppelin Tutorial: Python - matplotlib basic" should work, and ```python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") z.show(rates) ``` ### Screenshots (if appropriate) ![screen shot 2016-06-23 at 10 29 00](https://cloud.githubusercontent.com/assets/5582506/16289133/85f0ddbc-392d-11e6-86a3-28d10e73f68d.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1067 from bzz/python/pandas-support and squashes the following commits: 3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration 71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame 52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs a7248cd [Alexander Bezzubov] Python: draft of pandas support 15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
2016-06-23 01:25:49 +00:00
z.show(plt, width='50px')
z.show(plt, height='150px')
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
```
[ZEPPELIN-1018] Apply auto "Table of Contents" generator to Zeppelin docs website ### What is this PR for? I added auto TOC(Table of Contents) generator for Zeppelin documentation website. TOC can help people looking through whole contents at a glance and finding what they want quickly. I just added `<div id="toc"></div>` to the each documentation header. [`toc`](https://github.com/apache/zeppelin/compare/master...AhyoungRyu:ZEPPELIN-1018?expand=1#diff-85af09fb498a5667ea455391533f945dR3) recognize `<h2>` & `<h3>` as a title in the docs and it automatically generate TOC. So I set a rule for this work. (I'll write this rule on `docs/CONTRIBUTING.md` or [docs/howtocontributewebsite](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/howtocontributewebsite.html)). ``` # Level-1 Heading <- Use only for the main title of the page ## Level-2 Heading <- Start with this one ### Level-3 heading <- Only use this one for child of Level-2 toc only recognize Level-2 & Level-3 ``` Please see the below attached screenshot image. ### What type of PR is it? Improvement & Documentation ### Todos * [x] - Add TOC generator * [x] - Apply TOC(`<div id="toc"></div>`) to every documentation and reorganize each headers(apply the above rule) * [x] - Fix some broken code block in several docs * [x] - Apply TOC to `r.md` (Currently R docs has some duplicated info since [this one](https://github.com/apache/zeppelin/commit/d5e87fb8ba98f08db5b0a4995104ce19f182c678) and [this one](https://github.com/apache/zeppelin/commit/7d6cc7e99154e2d337c11fdf8be1a874ed3e9ada) ) * [x] - Apply TOC to `install.md` after #1010 merged * [x] - Apply TOC to `interpreterinstallation.md` after #1042 merged ### What is the Jira issue? [ZEPPELIN-1018](https://issues.apache.org/jira/browse/ZEPPELIN-1018) ### How should this be tested? 1. Apply this patch and build `docs/` with [this guide](https://github.com/apache/zeppelin/tree/master/docs#build-documentation) 2. Visit some docs page. Then you can see TOC in the header of page. ### Screenshots (if appropriate) - Automatically generated TOC in Spark interpreter docs page <img width="831" alt="screen shot 2016-06-16 at 9 37 18 pm" src="https://cloud.githubusercontent.com/assets/10060731/16140902/945b9c7a-340a-11e6-91f3-b6174738bed0.png"> ### Questions: * Does the licenses files need update? No. Actually I used [jekyll-table-of-contents#copyright](https://github.com/ghiculescu/jekyll-table-of-contents#copyright). But I don't need to add a license for this :) * Is there breaking changes for older versions? No * Does this needs documentation? Maybe Author: AhyoungRyu <fbdkdud93@hanmail.net> Closes #1031 from AhyoungRyu/ZEPPELIN-1018 and squashes the following commits: e66397b [AhyoungRyu] Apply TOC to interpreterinstallation.md 009579b [AhyoungRyu] Add more info to 'What is the next?' in install.md 04cf501 [AhyoungRyu] Revert 'where to start' section b7cbe5f [AhyoungRyu] Fix typo cf0911c [AhyoungRyu] Rename license file 388f35a [AhyoungRyu] Add jekyll-table-of-contents license info 6394c70 [AhyoungRyu] Fix image path in python.md d00e4b1 [AhyoungRyu] Move interpreter/screenshot/ -> asset/../img/docs-img/ 3ffb383 [AhyoungRyu] Remove duplicated info in r.md & apply toc a03ca99 [AhyoungRyu] Exclude toc.js from pom.xml 3fae7df [AhyoungRyu] Apply auto generated toc to install.md d114a9d [AhyoungRyu] Address @felixcheung feedback 6a788fe [AhyoungRyu] Resize TOC tab indent 6760c00 [AhyoungRyu] Apply auto TOC to all of docs under docs/storage/ fbde57f [AhyoungRyu] Apply auto TOC to all of docs under docs/quickstart/ db76eb6 [AhyoungRyu] Apply auto TOC to all of docs under docs/install/ f35db47 [AhyoungRyu] Apply auto TOC to all of docs under docs/displaysystem/ b05365f [AhyoungRyu] Apply auto TOC to all of docs under docs/rest-api/ 163691c [AhyoungRyu] Apply auto TOC to all of docs under docs/manual/ bef398e [AhyoungRyu] Apply auto TOC to all of docs under docs/development/ 9c5f76b [AhyoungRyu] Apply auto TOC to all of docs under docs/interpreter/ 587d4ba [AhyoungRyu] Apply auto TOC to all of docs under docs/security/ 1f10b97 [AhyoungRyu] Change toc configuration 78dca9e [AhyoungRyu] Add toc.js for auto generating TOC
2016-06-25 19:44:53 +00:00
<img class="img-responsive" src="../assets/themes/zeppelin/img/docs-img/pythonMatplotlib.png" />
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
ZEPPELIN-1048: Pandas support for python interpreter ### What is this PR for? Display Pandas DataFrame using Zeppelin's Table Display system. ### What type of PR is it? Feature ### Todos * [x] fix NPE in logs on empty paragraph execution * [x] matplotlib: refactor `zeppelin_show(plt)` -> `z.show(plt)` * [x] pandas: support `z.show(df)` * [x] update docs ### What is the Jira issue? [ZEPPELIN-1048](https://issues.apache.org/jira/browse/ZEPPELIN-1048) ### How should this be tested? "Zeppelin Tutorial: Python - matplotlib basic" should work, and ```python import pandas as pd rates = pd.read_csv("bank.csv", sep=";") z.show(rates) ``` ### Screenshots (if appropriate) ![screen shot 2016-06-23 at 10 29 00](https://cloud.githubusercontent.com/assets/5582506/16289133/85f0ddbc-392d-11e6-86a3-28d10e73f68d.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Alexander Bezzubov <bzz@apache.org> Closes #1067 from bzz/python/pandas-support and squashes the following commits: 3b1ad36 [Alexander Bezzubov] Python: update docs to reffer new API ee6668b [Alexander Bezzubov] Python: update docs, add Pandas integration 71be418 [Alexander Bezzubov] Python: limit 1000 for table display system on DataFrame 52e787d [Alexander Bezzubov] Python: pandas DataFrame using Table display system bc91b86 [Alexander Bezzubov] Python: skip interpreting empty paragraphs a7248cd [Alexander Bezzubov] Python: draft of pandas support 15646a1 [Alexander Bezzubov] Python: refactoring to z.show()
2016-06-23 01:25:49 +00:00
## Pandas integration
[Zeppelin Display System]({{BASE_PATH}}/displaysystem/basicdisplaysystem.html#table) provides simple API to visualize data in Pandas DataFrames, same as in Matplotlib.
Example:
```python
import pandas as pd
rates = pd.read_csv("bank.csv", sep=";")
z.show(rates)
```
## Technical description
[ZEPPELIN-502] Python interpreter group ### What is this PR for? Adding a python 2 &3 interpreter. It's a basic implementation (no py4j for example), with a java ProcessBuilder object used to instantiate a python REPL. The interpreter doesn't bring it own python binary but uses the python specified by python.path configutation. Thus, you can still use your specific installed python modules (scikit-learn, matplotlib...) and the interpreter is able to work with python 2 & 3 without change. I had a python helper function (zeppelin_show() ) to easily display matplotlib graph as SVG. ### What type of PR is it? [Feature] ### Todos * [x] - Code review * [x] - Improve bootstrap.py : choose available helper functions and their names * [x] - Unit / IT tests ? * [x] documentation updates needed, that AhyoungRyu pointed out * [X] LICENSE needs to be updated to include all non-apache licensed dependencies (i.e AFAIK Py4j is BSD ) in bin-license * [x] double-check that code formatting conforms project style guide * [x] the branch need to be rebased on latest master. ### What is the Jira issue? [ZEPPELIN-502](https://issues.apache.org/jira/browse/ZEPPELIN-502?jql=project%20%3D%20ZEPPELIN%20AND%20text%20~%20%22python%22) ### How should this be tested? 1. In interpreter screen, in Python section, specify in python.path the python binary you want to use 2. In a paragraph, you can use the interpreter with **_%python_**. Calling help() will describe you the interpreter functionnalities. 3. Install py4j (pip install py4j) if you want to use input form ### Screenshots ![image](https://cloud.githubusercontent.com/assets/12515751/14936724/5108fb60-0ef4-11e6-93ea-232a037f7957.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943716/98a75c4a-0fe0-11e6-9d4b-e10c39d53a15.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14936715/0eec90de-0ef4-11e6-811b-7ebe46f0d279.png) ![image](https://cloud.githubusercontent.com/assets/12515751/14943722/b89b7824-0fe0-11e6-9c73-c12f7372d487.png) ### Questions: * Does the licenses files need update? Yes, only bin-license (py4j) * Is there breaking changes for older versions? No * Does this needs documentation? Yes Author: Hervé RIVIERE <hriviere@users.noreply.github.com> Closes #869 from hriviere/PR_interpreter_python and squashes the following commits: 80b6e75 [Hervé RIVIERE] [ZEPPELIN-502] move BSD py4j license to zeppelin-distribution/src/bin_license/license a4b82a5 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 3252353 [Hervé RIVIERE] [ZEPPELIN-502] Formatting code to respect project convention 54ec4f1 [Hervé RIVIERE] [ZEPPELIN-502]Improving doc following @AhyoungRyu review 6a831bc [Hervé RIVIERE] [ZEPPELIN-502] Add BSD py4j license 11e1b9c [Hervé RIVIERE] [ZEPPELIN-502] minor changes in python.md e5d0bdb [Hervé RIVIERE] [ZEPPELIN-502] change PYTHON_PATH to ZEPPELIN_PYTHON c62ac98 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md 5008125 [Hervé RIVIERE] [ZEPPELIN-502] Improve python.md with features not yet supported and technical description 7d533e1 [Hervé RIVIERE] [ZEPPELIN-502] Add tests and reformating code to help tests writing fecaf25 [Hervé RIVIERE] [ZEPPELIN-502] Rename python.path to python and default from /usr/bin/python to python 02d1320 [Hervé RIVIERE] [ZEPPELIN-502] Input form, change from simple input form to native (pyspark syntax) 60d2956 [Hervé RIVIERE] [ZEPPELIN-502] Indent as pep8 convention 9bdb192 [Hervé RIVIERE] [ZEPPELIN-502] Add python.md to _navigation.html 7142aa5 [Hervé RIVIERE] [ZEPPELIN-502] Catch exception in logger.error 1a86ad7 [Hervé RIVIERE] [ZEPPELIN-502] Python interpreter group
2016-05-30 20:07:26 +00:00
[ZEPPELIN-1018] Apply auto "Table of Contents" generator to Zeppelin docs website ### What is this PR for? I added auto TOC(Table of Contents) generator for Zeppelin documentation website. TOC can help people looking through whole contents at a glance and finding what they want quickly. I just added `<div id="toc"></div>` to the each documentation header. [`toc`](https://github.com/apache/zeppelin/compare/master...AhyoungRyu:ZEPPELIN-1018?expand=1#diff-85af09fb498a5667ea455391533f945dR3) recognize `<h2>` & `<h3>` as a title in the docs and it automatically generate TOC. So I set a rule for this work. (I'll write this rule on `docs/CONTRIBUTING.md` or [docs/howtocontributewebsite](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/howtocontributewebsite.html)). ``` # Level-1 Heading <- Use only for the main title of the page ## Level-2 Heading <- Start with this one ### Level-3 heading <- Only use this one for child of Level-2 toc only recognize Level-2 & Level-3 ``` Please see the below attached screenshot image. ### What type of PR is it? Improvement & Documentation ### Todos * [x] - Add TOC generator * [x] - Apply TOC(`<div id="toc"></div>`) to every documentation and reorganize each headers(apply the above rule) * [x] - Fix some broken code block in several docs * [x] - Apply TOC to `r.md` (Currently R docs has some duplicated info since [this one](https://github.com/apache/zeppelin/commit/d5e87fb8ba98f08db5b0a4995104ce19f182c678) and [this one](https://github.com/apache/zeppelin/commit/7d6cc7e99154e2d337c11fdf8be1a874ed3e9ada) ) * [x] - Apply TOC to `install.md` after #1010 merged * [x] - Apply TOC to `interpreterinstallation.md` after #1042 merged ### What is the Jira issue? [ZEPPELIN-1018](https://issues.apache.org/jira/browse/ZEPPELIN-1018) ### How should this be tested? 1. Apply this patch and build `docs/` with [this guide](https://github.com/apache/zeppelin/tree/master/docs#build-documentation) 2. Visit some docs page. Then you can see TOC in the header of page. ### Screenshots (if appropriate) - Automatically generated TOC in Spark interpreter docs page <img width="831" alt="screen shot 2016-06-16 at 9 37 18 pm" src="https://cloud.githubusercontent.com/assets/10060731/16140902/945b9c7a-340a-11e6-91f3-b6174738bed0.png"> ### Questions: * Does the licenses files need update? No. Actually I used [jekyll-table-of-contents#copyright](https://github.com/ghiculescu/jekyll-table-of-contents#copyright). But I don't need to add a license for this :) * Is there breaking changes for older versions? No * Does this needs documentation? Maybe Author: AhyoungRyu <fbdkdud93@hanmail.net> Closes #1031 from AhyoungRyu/ZEPPELIN-1018 and squashes the following commits: e66397b [AhyoungRyu] Apply TOC to interpreterinstallation.md 009579b [AhyoungRyu] Add more info to 'What is the next?' in install.md 04cf501 [AhyoungRyu] Revert 'where to start' section b7cbe5f [AhyoungRyu] Fix typo cf0911c [AhyoungRyu] Rename license file 388f35a [AhyoungRyu] Add jekyll-table-of-contents license info 6394c70 [AhyoungRyu] Fix image path in python.md d00e4b1 [AhyoungRyu] Move interpreter/screenshot/ -> asset/../img/docs-img/ 3ffb383 [AhyoungRyu] Remove duplicated info in r.md & apply toc a03ca99 [AhyoungRyu] Exclude toc.js from pom.xml 3fae7df [AhyoungRyu] Apply auto generated toc to install.md d114a9d [AhyoungRyu] Address @felixcheung feedback 6a788fe [AhyoungRyu] Resize TOC tab indent 6760c00 [AhyoungRyu] Apply auto TOC to all of docs under docs/storage/ fbde57f [AhyoungRyu] Apply auto TOC to all of docs under docs/quickstart/ db76eb6 [AhyoungRyu] Apply auto TOC to all of docs under docs/install/ f35db47 [AhyoungRyu] Apply auto TOC to all of docs under docs/displaysystem/ b05365f [AhyoungRyu] Apply auto TOC to all of docs under docs/rest-api/ 163691c [AhyoungRyu] Apply auto TOC to all of docs under docs/manual/ bef398e [AhyoungRyu] Apply auto TOC to all of docs under docs/development/ 9c5f76b [AhyoungRyu] Apply auto TOC to all of docs under docs/interpreter/ 587d4ba [AhyoungRyu] Apply auto TOC to all of docs under docs/security/ 1f10b97 [AhyoungRyu] Change toc configuration 78dca9e [AhyoungRyu] Add toc.js for auto generating TOC
2016-06-25 19:44:53 +00:00
For in-depth technical details on current implementation plese reffer [python/README.md](https://github.com/apache/zeppelin/blob/master/python/README.md).