Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Python: move technical details from doc to README.md

Browse source
This commit is contained in:
Alexander Bezzubov 2016-06-16 11:51:28 +09:00
parent 0f74e5dfd7
commit 744f7d23d6
2 changed files with 46 additions and 26 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

30
docs/interpreter/python.md
View file

@ -68,8 +68,8 @@ print("".join(z.checkbox("f3", [("o1","1"), ("o2","2")],["1"])))
* Code-completion is currently not implemented.
## Matplotlib integration
The python interpreter can display matplotlib graph with the function **_zeppelin_show()_**
You need to already have matplotlib module installed and a running XServer to use this functionality !
The python interpreter can display matplotlib graph with the function `zeppelin_show()`.
You need to have matplotlib module installed and a XServer running to use this functionality !
```python
%python
@ -90,28 +90,6 @@ zeppelin_show(plt,height='150px')
[![pythonmatplotlib](../interpreter/screenshots/pythonMatplotlib.png)](/docs/interpreter/screenshots/pythonMatplotlib.png)
## Technical description - Interpreter architecture
## Technical description
### Dev prerequisites
* Python 2 and 3 installed with py4j (0.9.2) and matplotlib (1.31 or later) installed on each
* Tests only checks the interpreter logic and starts any Python process ! Python process is mocked with a class that simply output it input.
* Make sure the code wrote in bootstrap.py and bootstrap_input.py is Python2 and 3 compliant.
* Use PEP8 convention for python code.
### Technical overview
* When interpreter is starting it launches a python process inside a Java ProcessBuilder. Python is started with -i (interactive mode) and -u (unbuffered stdin, stdout and stderr) options. Thus the interpreter has a "sleeping" python process.
* Interpreter sends command to python with a Java `outputStreamWiter` and read from an `InputStreamReader`. To know when stop reading stdout, interpreter sends `print "*!?flush reader!?*"`after each command and reads stdout until he receives back the `*!?flush reader!?*`.
* When interpreter is starting, it sends some Python code (bootstrap.py and bootstrap_input.py) to initialize default behavior and functions (`help(), z.input()...`). bootstrap_input.py is sent only if py4j library is detected inside Python process.
* [Py4J](https://www.py4j.org/) python and java libraries is used to load Input zeppelin Java class into the python process (make java code with python code !). Therefore the interpreter can directly create Zeppelin input form inside the Python process (and eventually with some python variable already defined). JVM opens a random open port to be accessible from python process.
* JavaBuilder can't send SIGINT signal to interrupt paragraph execution. Therefore interpreter directly send a `kill SIGINT PID` to python process to interrupt execution. Python process catch SIGINT signal with some code defined in bootstrap.py
* Matplotlib display feature is made with SVG export (in string) and then displays it with html code.
For in-depth technical details on current implementation plese reffer [python/README.md](https://github.com/apache/zeppelin/blob/master/python/README.md)

42
python/README.md Normal file
View file

@ -0,0 +1,42 @@
# Overview
Python interpreter for Apache Zeppelin
# Architecture
Current interpreter implementation spawns new system python process through `ProcessBuilder` and re-directs it's stdin\strout to Zeppelin
# Details
- **Py4j support**
[Py4j](https://www.py4j.org/) enables Python programs to dynamically access Java objects in a JVM.
It is required in order to use Zeppelin [dynamic forms](http://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/manual/dynamicform.html) feature.
- bootstrap process
Interpreter environment is setup with thex [bootstrap.py](https://github.com/apache/zeppelin/blob/master/python/src/main/resources/bootstrap.py)
It defines `help()` and `z` convenience functions
### Dev prerequisites
* Python 2 or 3 installed with py4j (0.9.2) and matplotlib (1.31 or later) installed on each
* Tests only checks the interpreter logic and starts any Python process! Python process is mocked with a class that simply output it input.
* Code wrote in `bootstrap.py` and `bootstrap_input.py` should always be Python 2 and 3 compliant.
* Use PEP8 convention for python code.
### Technical overview
* When interpreter is starting it launches a python process inside a Java ProcessBuilder. Python is started with -i (interactive mode) and -u (unbuffered stdin, stdout and stderr) options. Thus the interpreter has a "sleeping" python process.
* Interpreter sends command to python with a Java `outputStreamWiter` and read from an `InputStreamReader`. To know when stop reading stdout, interpreter sends `print "*!?flush reader!?*"`after each command and reads stdout until he receives back the `*!?flush reader!?*`.
* When interpreter is starting, it sends some Python code (bootstrap.py and bootstrap_input.py) to initialize default behavior and functions (`help(), z.input()...`). bootstrap_input.py is sent only if py4j library is detected inside Python process.
* [Py4J](https://www.py4j.org/) python and java libraries is used to load Input zeppelin Java class into the python process (make java code with python code !). Therefore the interpreter can directly create Zeppelin input form inside the Python process (and eventually with some python variable already defined). JVM opens a random open port to be accessible from python process.
* JavaBuilder can't send SIGINT signal to interrupt paragraph execution. Therefore interpreter directly send a `kill SIGINT PID` to python process to interrupt execution. Python process catch SIGINT signal with some code defined in bootstrap.py
* Matplotlib display feature is made with SVG export (in string) and then displays it with html code.