[ZEPPELIN-502] Improve python.md with features not yet supported and technical description

This commit is contained in:
Hervé RIVIERE 2016-05-05 21:48:02 +02:00
parent 7d533e128a
commit 50081253e5

View file

@ -54,6 +54,13 @@ print (z.select("f1",[("o1","1"),("o2","2")],"2"))
print("".join(z.checkbox("f3", [("o1","1"), ("o2","2")],["1"])))
```
## Zeppelin features not fully supported by the Python Interpreter
* Interrupt a paragraph execution (`cancel()` method) is currently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter.
* Progression bar in webUI (`getProgress()` method) is currently not implemented.
* Code-completion is currently not implemented.
* Zeppelin Dynamic Form can only be used if py4j Python library is installed in your system. If not, you can install it with `pip install py4j`.
## Matplotlib integration
The python interpreter can display matplotlib graph with the function **_zeppelin_show()_**
You need to already have matplotlib module installed to use this functionality !
@ -80,4 +87,31 @@ Matplotlib needs a X server to plot graph.
If you don't have any XServer and you don't want to install one (typically, a server environment) , you can use [Xvfb](http://www.x.org/archive/X11R7.6/doc/man/man1/Xvfb.1.xhtml) to simulate it.
Althought Xvfb doesn't display anything, you will have any issue to use the python interpreter, matplotlib and Xvfb. The interpreter plot the graph on X and then extracts svg path (in strings) to send it to the zeppelin webapp.
Althought Xvfb doesn't display anything, you will have any issue to use the python interpreter, matplotlib and Xvfb. The interpreter plot the graph on X and then extracts svg path (in strings) to send it to the zeppelin webapp.
## Technical description - Interpreter architecture
### Dev prerequisites
* Python 2 and 3 installed with py4j (0.9.2) and matplotlib (1.31 or later) installed on each
* Tests only checks the interpreter logic and starts any Python process ! Python process is mocked with a class that simply output it input.
* Make sure the code wrote in bootstrap.py and bootstrap_input.py is Python2 and 3 compliant.
* Use PEP8 convention for python code.
### Technical overview
* When interpreter is starting it launches a python process inside a Java ProcessBuilder. Python is started with -i (interactive mode) and -u (unbuffered stdin, stdout and stderr) options. Thus the interpreter has a "sleeping" python process.
* Interpreter sends command to python with a Java `outputStreamWiter` and read from an `InputStreamReader`. To know when stop reading stdout, interpreter sends `print "*!?flush reader!?*"`after each command and reads stdout until he receives back the `*!?flush reader!?*`.
* When interpreter is starting, it sends some Python code (bootstrap.py and bootstrap_input.py) to initialize default behavior and functions (`help(), z.input()...`). bootstrap_input.py is sent only if py4j library is detected inside Python process.
* [Py4J](https://www.py4j.org/) python and java libraries is used to load Input zeppelin Java class into the python process (make java code with python code !). Therefore the interpreter can directly create Zeppelin input form inside the Python process (and eventually with some python variable already defined). JVM opens a random open port to be accessible from python process.
* JavaBuilder can't send SIGINT signal to interrupt paragraph execution. Therefore interpreter directly send a `kill SIGINT PID` to python process to interrupt execution. Python process catch SIGINT signal with some code defined in bootstrap.py
* Matplotlib display feature is made with SVG export (in string) and then displays it with html code.