zeppelin/docs/development/writingzeppelinapplication.md

188 lines
5.6 KiB
Markdown
Raw Normal View History

[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
---
layout: page
[ZEPPELIN-1219] Add searching feature to Zeppelin docs site ### What is this PR for? As more and more document pages are added, it's really hard to find specific pages. So I added searching feature to Zeppelin documentation site([jekyll](https://jekyllrb.com/) based site) using [lunr.js](http://lunrjs.com/). - **How does it work?** I created [`search_data.json`](https://github.com/AhyoungRyu/zeppelin/blob/6e02423f541cc406e4e41031629609a276a9f481/docs/search_data.json) which is used for docs info template. `lunr.js` combines all of the text from all of the docs in `docs/` into `_site/search_data.json`. It looks like below. ![screen shot 2016-08-03 at 4 49 59 am](https://cloud.githubusercontent.com/assets/10060731/17342828/f2908be8-5935-11e6-8eee-b189677c0531.png) All the info are comes from [Jekyll YAML front matter](https://jekyllrb.com/docs/frontmatter/) variables. (i.e. title, group, description.. that's why I rewrote all docs' title and description.) [search.js](https://github.com/AhyoungRyu/zeppelin/blob/6e02423f541cc406e4e41031629609a276a9f481/docs/assets/themes/zeppelin/js/search.js) will do this job using this data! ### What type of PR is it? Improvement & Feature ### Todos * [x] - Keep consistency for all docs pages' `Title` * [x] - Add some overview sentences to all docs pages' `Description` section (this will be used as the result preview) * [x] - Add apache license header to all docs page (some pages are missing the license header currently) * [x] - Add LICENSE for `lunr.min.js` ### What is the Jira issue? [ZEPPELIN-1219](https://issues.apache.org/jira/browse/ZEPPELIN-1219) ### How should this be tested? 1. Apply this patch and build `ZEPPELIN_HOME/docs` dir -> please see [docs/README.md#build-documentation](https://github.com/apache/zeppelin/tree/master/docs#build-documentation) 2. Click `search` icon in navbar and go to `search.html` page 3. Type anything you want to search in the search bar (i.e. type `python`, `spark`, `dynamic` ... ) ### Screenshots (if appropriate) ![screen shot 2016-08-03 at 4 42 28 pm](https://cloud.githubusercontent.com/assets/10060731/17357851/d092e2ca-5999-11e6-9917-a3d4113e6e43.png) ![search](https://cloud.githubusercontent.com/assets/10060731/17357828/b2486cd6-5999-11e6-873b-121fac033b03.gif) ### Questions: * Does the licenses files need update? Yes, for `lunr.min.js` * Is there breaking changes for older versions? no * Does this needs documentation? no Author: AhyoungRyu <fbdkdud93@hanmail.net> Closes #1266 from AhyoungRyu/ZEPPELIN-1219 and squashes the following commits: 7ec8854 [AhyoungRyu] Modify 'no result' sentence 91b71a7 [AhyoungRyu] Remove Apache license header since JSON doesn't allow comment 34afd5d [AhyoungRyu] Add Apache license header to search_data.json 6784282 [AhyoungRyu] Minor search page UI update 0389d28 [AhyoungRyu] Make index.md not to be searched 9f1ba42 [AhyoungRyu] Disable enterkey press & change icon bd4956a [AhyoungRyu] Add docs.js & search.js to exclude list in pom.xml 624b051 [AhyoungRyu] Add Apache license header to search.js 1381152 [AhyoungRyu] Fix search result skipping issue 6e775f5 [AhyoungRyu] Make pleasecontribute.md not to be searched ee11136 [AhyoungRyu] Fix some typos fa01299 [AhyoungRyu] Refine 'description' in some docs as @bzz suggested da0cff9 [AhyoungRyu] Exclude lunr.min.js 36ba7f1 [AhyoungRyu] Add lunr.min.js license info f6a05a6 [AhyoungRyu] Apply css style for the search results 68eb997 [AhyoungRyu] Attach 'Apache Zeppelin ZEPPELIN_VERSION Documentation: ' to title d908c37 [AhyoungRyu] Add searching page a951fa6 [AhyoungRyu] Add search icon to navbar 0688a79 [AhyoungRyu] Keep consistency all docs' front matter for the right search result 040f532 [AhyoungRyu] Add template for storing docs info based on jekyll front matter 0705bd6 [AhyoungRyu] Add js files: lunr.min.js & search.js
2016-08-06 05:50:25 +00:00
title: "Writing a new Application(Experimental)"
description: "Apache Zeppelin Application is a package that runs on Interpreter process and displays it's output inside of the notebook. Make your own Application in Apache Zeppelin is quite easy."
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
group: development
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
{% include JB/setup %}
# Writing a new Application (Experimental)
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
<div id="toc"></div>
## What is Apache Zeppelin Application
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
Apache Zeppelin Application is a package that runs on Interpreter process and displays it's output inside of the notebook. While application runs on Interpreter process, it's able to access resources provided by Interpreter through ResourcePool. Output is always rendered by AngularDisplaySystem. Therefore application provides all the possiblities of making interactive graphical application that uses data and processing power of any Interpreter.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
## Make your own Application
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
Writing Application means extending `org.apache.zeppelin.helium.Application`. You can use your favorite IDE and language while Java class files are packaged into jar. `Application` class looks like
```java
/**
* Constructor. Invoked when application is loaded
*/
public Application(ApplicationContext context);
/**
* Invoked when there're (possible) updates in required resource set.
* i.e. invoked after application load and after paragraph finishes.
*/
public abstract void run(ResourceSet args);
/**
* Invoked before application unload.
* Application is automatically unloaded with paragraph/notebook removal
*/
public abstract void unload();
```
You can check example applications under [./zeppelin-examples](https://github.com/apache/incubator-zeppelin/tree/master/zeppelin-examples) directory.
## Development mode
In the development mode, you can run your Application in your IDE as a normal java application and see the result inside of Zeppelin notebook.
`org.apache.zeppelin.interpreter.dev.ZeppelinApplicationDevServer` can run Zeppelin Application in development mode.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
```java
// entry point for development mode
public static void main(String[] args) throws Exception {
// add resources for development mode
LocalResourcePool pool = new LocalResourcePool("dev");
pool.put("date", new Date());
// run application in devlopment mode with given resource
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
// in this case, Clock.class.getName() will be the application class name
ZeppelinApplicationDevServer devServer = new ZeppelinApplicationDevServer(
Clock.class.getName(), pool.getAll());
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
// start development mode
devServer.start();
devServer.join();
}
```
In the Zeppelin notebook, run `%dev run` will connect to application running in development mode.
## Package file
Package file is a json file that provides information about the application.
Json file contains the following information
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
```
{
name : "[organization].[name]",
description : "Description",
artifact : "groupId:artifactId:version",
className : "your.package.name.YourApplicationClass",
resources : [
["resource.name", ":resource.class.name"],
["alternative.resource.name", ":alternative.class.name"]
],
icon : "<i class="icon"></i>"
}
```
#### name
Name is a string in `[group].[name]` format.
`[group]` and `[name]` allow only `[A-Za-z0-9_]`.
Group is normally the name of an organization who creates this application.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
#### description
A short description about the application
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
#### artifact
Location of the jar artifact.
`"groupId:artifactId:version"` will load artifact from maven repository.
If jar exists in the local filesystem, absolute/relative can be used.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
e.g.
When artifact exists in Maven repository
```
artifact: "org.apache.zeppelin:zeppelin-examples:0.6.0"
```
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
When artifact exists in the local filesystem
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
```
artifact: "zeppelin-example/target/zeppelin-example-0.6.0.jar"
```
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
#### className
Entry point. Class that extends `org.apache.zeppelin.helium.Application`
#### resources
Two dimensional array that defines required resources by name or by className. Helium Application launcher will compare resources in the ResourcePool with the information in this field and suggest application only when all required resources are available in the ResourcePool.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
Resouce name is a string which will be compared with the name of objects in the ResourcePool. className is a string with ":" prepended, which will be compared with className of the objects in the ResourcePool.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
Application may require two or more resources. Required resources can be listed inside of the json array. For example, if the application requires object "name1", "name2" and "className1" type of object to run, resources field can be
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
```
resources: [
[ "name1", "name2", ":className1", ...]
]
```
If Application can handle alternative combination of required resources, alternative set can be listed as below.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
```
resources: [
[ "name", ":className"],
[ "altName", ":altClassName1"],
...
]
```
Easier way to understand this scheme is
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
```
resources: [
[ 'resource' AND 'resource' AND ... ] OR
[ 'resource' AND 'resource' AND ... ] OR
...
]
```
#### icon
Icon to be used on the application button. String in this field will be rendered as a HTML tag.
[ZEPPELIN-732] Helium Application ### What is this PR for? This PR implements pluggable Application Framework. Here's summary of the change. **Applicaiton** Base class of user application. User application should extends `org.apache.zeppelin.helium.Application`. It looks like ```java public abstract class Application { /** * When application loaded * params context Application context */ public Application(ApplicationContext context) { this.context = context; } /** * This method can be invoked multiple times before unload(), * Invoked just after application loaded or when paragraph output is updated * param args Required resources from paragraph */ public abstract void run(ResourceSet args) throws ApplicationException, IOException; /** * this method is invoked just before application is removed */ public abstract void unload() throws ApplicationException; } ``` **HeliumPackage** `org.apache.zeppelin.helium.HeliumPackage` keeps information of application package. Package informations includes ``` { type : "APPLICATION", // type of this package name : "zeppelin.app", // application name. [organizationName].[appName] format description : "App description", // description artifact : "groupId:artifactId:version", // artifact name for maven repository className : "org.apache.zeppelin.helium.MyApp", // application class that extends org.apache.zeppelin.helium.Application resources : [[]], // required resource icon : '<i class="fa fa-clock-o"></i>' // icon } ``` `resources` field defines what kind of data this application requires, from ResourcePool. "resourceName" and ":className" works in the array. inner array combines each resourceName, :className with 'AND' outer array combines inner array with 'OR'. For example, ``` resources : [ [ "name1", ":java.util.Date"], [ "name1", "name2"] ] ``` Then Zeppelin searches ResourcePool, first for resource name "name1" and resource type "java.util.Date" from resourcePool then resource name "name1" and "name2". Once resources are found, they'll be passed to `Application.run()` method **Package Registry** `org.apache.zeppelin.helium.HeliumRegistry` provides list of available HeliumPackages. Currently `HeliumLocalRegistry` is implemented and it provides list by reading HeliumPackage json files under ./helium directory in the file system. Later there will be some class like `HeliumOnlineRegistry' which reads available package from a community managed online central registry. **Development mode** `org.apache.zeppelin.interpreter.dev.*` package is provided for development support. Developer can run their app inside of their IDE, that connects to running Zeppelin instance and display output. **Application Suggestion** Once paragraph becomes FINISHED status after run, Zeppelin searches resources from ResourcesPool that belongs to current paragraph. And compare all HeliumPackage if these resources satisfies their requirements ('resources' field). If there're available app, Helium button will be displayed. ![image](https://cloud.githubusercontent.com/assets/1540981/14518452/d8f85d5a-020f-11e6-8313-00b37eb4f077.png) User can see and select available application for this paragraph. ![image](https://cloud.githubusercontent.com/assets/1540981/14518465/ee607fce-020f-11e6-8ad4-fe98723c7c94.png) **Application selection** After application is loaded, application icon will be displayed. So user can select and switch between apps. ![image](https://cloud.githubusercontent.com/assets/1540981/14518547/6f3fd766-0210-11e6-9a08-0dc2ba6d15e3.png) if built-in visualization is available, application icon will be displayed next to built-in visualizations ![image](https://cloud.githubusercontent.com/assets/1540981/14518500/27505746-0210-11e6-9738-fb514376b906.png) **Experimental** While this is new feature and API and specification may change, i'd suggest mark this feature as a experimental for a while. ### What type of PR is it? Feature ### Todos * [x] - Helium Application framework * [x] - App launcher * [x] - App display and selectors * [x] - Package registry * [x] - Development mode * [x] - Make CI green * [x] - Improve comment in source codes * [x] - Documentation * [x] - Examples * [x] - Mark experimental * [ ] - Review ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-732 ### How should this be tested? Will be updated with examples ### Screenshots (if appropriate) ![helium](https://cloud.githubusercontent.com/assets/1540981/14518685/49c2a788-0211-11e6-81c8-6cae48bf8a48.gif) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? yes Author: Lee moon soo <moon@apache.org> Closes #836 from Leemoonsoo/ZEPPELIN-732-up and squashes the following commits: 3577777 [Lee moon soo] Address Ahyoung's comment 0665380 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 83b529c [Lee moon soo] Update unittest e1b0e79 [Lee moon soo] Update unittest 6785441 [Lee moon soo] interpreter order changed 3249ddb [Lee moon soo] Remove incubating 4030fff [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 4e98cfe [Lee moon soo] Update DependencyResolver c714a19 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up d558591 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 0467077 [Lee moon soo] update test d43ab57 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f999f71 [Lee moon soo] Update getParagraphXpath eaa68c6 [Lee moon soo] print processes 5b5f6c8 [Lee moon soo] trigger ci 1bdea70 [Lee moon soo] Print error message in the test 562f083 [Lee moon soo] CLose interpreter after test 385dab3 [Lee moon soo] Fix xpath in the SparkParagraphIt test ac963ff [Lee moon soo] Remove unnecessary stacktrace and provide more meaningful message 644add4 [Lee moon soo] Takecare csv/tsv download button position after merge 6a5a19e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 5bef173 [Lee moon soo] Update NoteTest e54d633 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 88dbc71 [Lee moon soo] change style 6f1343f [Lee moon soo] add license header 4571781 [Lee moon soo] double quote the keys in json 710487b [Lee moon soo] fix syntax error 6c5019e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 3619afd [Lee moon soo] Exclude test data file from rat check b6811a6 [Lee moon soo] set SPARK_HOME to find bin/interpreter.sh 9110897 [Lee moon soo] arrange buttons 14efb6c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 76de11c [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up bff2217 [Lee moon soo] Fix SparkParagraphIT e60cd8e [Lee moon soo] Add docs 7f174e3 [Lee moon soo] Add more comments 0671dfd [Lee moon soo] Fix compile error bfcdf34 [Lee moon soo] Fix style 1dc4409 [Lee moon soo] Remove incubating from example pom.xml e36d2fb [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e0a2046 [Lee moon soo] Add annotation to api 93fa766 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 99dd0b0 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up e89177e [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up f782fb0 [Lee moon soo] clean pom.xml file b974eb1 [Lee moon soo] Add horizontal bar visualization example 5225551 [Lee moon soo] let instead of var b068065 [Lee moon soo] Replace . to _ b6e4141 [Lee moon soo] helper for printing javascript and inject $z 8186daf [Lee moon soo] Update Application Status from RemoteInterpreterProcess event f07ada1 [Lee moon soo] Construct classpath correctly 7451479 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up ab1de03 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 51fc113 [Lee moon soo] Include examples in CI build 83eba98 [Lee moon soo] UnloadApp on interpreter restart 3b891de [Lee moon soo] Provide resource pool for testing 0730ece [Lee moon soo] Add license into package json 620f79f [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up 01a1646 [Lee moon soo] Add example 1f1a3b5 [Lee moon soo] Fix test 864bea1 [Lee moon soo] Merge branch 'master' into ZEPPELIN-732-up c30f53c [Lee moon soo] null check 03be3a1 [Lee moon soo] Pass required resource to run() method fade3c1 [Lee moon soo] Handle output update in angular mode 0e4d81c [Lee moon soo] Remove unnecessary log ec2fdea [Lee moon soo] Match classname correctly b4ff52f [Lee moon soo] Put last value of scala repl into resource pool f2ab95d [Lee moon soo] Prevent unnecessary output update 71f814d [Lee moon soo] Better way to find resource dir for InterpreterOutput watcher 024d7fc [Lee moon soo] Dev mode 98f3872 [Lee moon soo] Managed interpreter process and Running interpreter process b47ca74 [Lee moon soo] Fix tests 5503f9c [Lee moon soo] Improved be3a1fa [Lee moon soo] Add license header 412480a [Lee moon soo] Fix style 16f6887 [Lee moon soo] Angular object update for helium app 6223cd4 [Lee moon soo] App output display bd0f467 [Lee moon soo] Style 94b490d [Lee moon soo] initial rest api impl 134bbe6 [Lee moon soo] Change HeliumRegistry constructor argument type 7aeb64a [Lee moon soo] Unload app on paragraph / note removal as well as interpreter unbind 4eaeea7 [Lee moon soo] sync -> async api 9f5c493 [Lee moon soo] Application output b239f1b [Lee moon soo] Helium application factory b891b98 [Lee moon soo] HeliumRegistry 568ee54 [Lee moon soo] ApplicationLoader 7424af2 [Lee moon soo] Remove resource on note / paragraph removal
2016-07-03 14:58:25 +00:00
e.g.
```
icon: "<i class='fa fa-clock-o'></i>"
```