Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
zeppelin/beam
History
Jeff Zhang 4a3057fa87 ZEPPELIN-2982. Copy interpreter-setting.json to interpreter dir 
### What is this PR for?
2 main changes.
* Introduce `interpreter-parent` module, so that all the interpreter can reuse the plugin defined in `interpreter-parent`
* Add new plugin for `copying interpreter-setting.json` to interpreter dir

### What type of PR is it?
[ Improvement | Refactoring]

### Todos
* [ ] - Task

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-2982

### How should this be tested?
* First time? Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed behavior
* Outline any manual steps to test the PR here.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jeff Zhang <zjffdu@apache.org>

Closes #2613 from zjffdu/ZEPPELIN-2982 and squashes the following commits:

ccccd32 [Jeff Zhang] update doc
8f643a7 [Jeff Zhang] ZEPPELIN-2982. Copy interpreter-setting.json to interpreter dir
2017-10-13 13:51:22 +08:00
..
src/main [ZEPPELIN-2403] interpreter property widgets 2017-07-06 15:54:55 +09:00
pom.xml ZEPPELIN-2982. Copy interpreter-setting.json to interpreter dir 2017-10-13 13:51:22 +08:00
README.md [ZEPPELIN-2865] upgrade Beam interpreter to latest version 2017-08-20 20:16:32 +09:00

README.md

Overview

Beam interpreter for Apache Zeppelin

Architecture

Current interpreter implementation supports the static repl. It compiles the code in memory, execute it and redirect the output to zeppelin.

Building the Beam Interpreter

You have to first build the Beam interpreter by enable the beam profile as follows:

mvn clean package -Pbeam -DskipTests -Pscala-2.10

Notice

  • Flink runner comes with binary compiled for scala 2.10. So, currently we support only Scala 2.10

Technical overview

  • Upon starting an interpreter, an instance of JavaCompiler is created.

  • When the user runs commands with beam, the JavaParser go through the code to get a class that contains the main method.

  • Then it replaces the class name with random class name to avoid overriding while compilation. it creates new out & err stream to get the data in new stream instead of the console, to redirect output to zeppelin.

  • If there is any error during compilation, it can catch and redirect to zeppelin.