Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

updated to code to lastest master version

Browse source
This commit is contained in:
maocorte 2015-12-23 16:15:01 +01:00
commit 2895a592b5
122 changed files with 6537 additions and 2913 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

81
CONTRIBUTING.md
View file

@ -7,25 +7,77 @@ Contributing to Zeppelin (Source code, Documents, Image, Website) means you agre
2. If not, create a ticket describing the change you're proposing in the [Jira issue tracker](https://issues.apache.org/jira/browse/ZEPPELIN)
3. Contribute your patch via Pull Request.
## Creating a Pull Request
In order to make the review process easier, please follow this template when making a Pull Request:
## SourceControl Workflow
```
### What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
### What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]
### Todos
* [ ] - Task
### Is there a relevant Jira issue?
### How should this be tested?
Outline the steps to test the PR here.
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?
```
You can also use this small bookmarklet tool to fill your Pull Request fields automatically:
```
javascript:(function() {var e = document.getElementById('pull_request_body');if (e) {e.value += '### What is this PR for?\nA few sentences describing the overall goals of the pull request\'s commits.\n\n### What type of PR is it?\n[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]\n\n### Todos\n* [ ] - Task\n\n### Is there a relevant Jira issue?\n\n### How should this be tested?\nOutline the steps to test the PR here.\n\n### Screenshots (if appropriate)\n\n### Questions:\n* Does the licenses files need update?\n* Is there breaking changes for older versions?\n* Does this needs documentation?';}})();
```
## Source Control Workflow
Zeppelin follows [Fork & Pull] (https://github.com/sevntu-checkstyle/sevntu.checkstyle/wiki/Development-workflow-with-Git:-Fork,-Branching,-Commits,-and-Pull-Request) model.
## The Review Process
When a Pull Request is submitted, it is being merged or rejected by following review process.
* Anybody can be a reviewer and may comment on the change and suggest modifications.
* Reviewer can indicate that a patch looks suitable for merging with a comment such as: "Looks good", "LGTM", "+1".
* At least one indication of suitable for merging (e.g. "LGTM") from committer is required to be merged.
* Pull request is open for 1 or 2 days for potential additional review, unless it's got enough indication of suitable for merging.
* Committer can initiate lazy consensus ("Merge if there is no more discussion") and the code can be merged after certain time (normally 24 hours) when there is no review exists.
* Contributor can ping reviewers (including committer) by commenting 'Ready to review' or suitable indication.
## Becoming a Committer
The PPMC adds new committers from the active contributors, based on their contribution to Zeppelin. The qualifications for new committers include:
1. Sustained contributions: Committers should have a history of constant contributions to Zeppelin.
2. Quality of contributions: Committers more than any other community member should submit simple, well-tested, and well-designed patches.
3. Community involvement: Committers should have a constructive and friendly attitude in all community interactions. They should also be active on the dev, user list and reviewing patches. Also help new contributors and users.
## Setting up
Here are some things you will need to build and test Zeppelin.
Here are some things you will need to build and test Zeppelin.
### Software Configuration Management(SCM)
### Software Configuration Management (SCM)
Zeppelin uses Git for it's SCM system. Hosted by github.com. `https://github.com/apache/incubator-zeppelin` You'll need git client installed in your development machine.
Zeppelin uses Git for its SCM system. Hosted by github.com. `https://github.com/apache/incubator-zeppelin` you'll need git client installed in your development machine.
### Integrated Development Environment(IDE)
### Integrated Development Environment (IDE)
You are free to use whatever IDE you prefer, or your favorite command line editor.
### Project Structure
Zeppelin project is based on Maven. Maven works by convention & defines [directory structure] (https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html) for a project.
The top level pom.xml describes the basic project structure. Currently Zeppelin has the following modules.
The top-level pom.xml describes the basic project structure. Currently Zeppelin has the following modules.
<module>zeppelin-interpreter</module>
<module>zeppelin-zengine</module>
@ -43,13 +95,16 @@ The top level pom.xml describes the basic project structure. Currently Zeppelin
<module>zeppelin-server</module>
<module>zeppelin-distribution</module>
### Web Project Contribution Guidelines
If you plan on making a contribution to Zeppelin's WebApplication,
please check [its own contribution guidelines](https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-web/CONTRIBUTING.md)
### Code convention
We are following Google Code style:
* [Java style](http://google-styleguide.googlecode.com/svn/trunk/javaguide.html)
* [Shell style](https://google-styleguide.googlecode.com/svn/trunk/shell.xml)
Checkstyle report location are in `${submodule}/target/site/checkstyle.html`
Check style report location are in `${submodule}/target/site/checkstyle.html`
Test coverage report location are in `${submodule}/target/site/cobertura/index.html`
#### Build Tools
@ -105,9 +160,9 @@ Each new File should have its own accompanying unit tests. Each new interpreter
Zeppelin has 3 types of tests:
1. Unit Tests: The unit tests run as part of each package's build. E.g SparkInterpeter Module's unit test is SparkInterpreterTest
2. Integration Tests: The intergration tests run after all modules are build. The integration tests launch an instance of Zeppelin server. ZeppelinRestApiTest is an example integration test.
3. GUI integration tests: These tests validate the Zeppelin UI elements. These tests require a running Zepplein server and lauches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example.
1. Unit Tests: The unit tests run as part of each package's build. E.g. SparkInterpeter Module's unit test is SparkInterpreterTest
2. Integration Tests: The integration tests run after all modules are build. The integration tests launch an instance of Zeppelin server. ZeppelinRestApiTest is an example integration test.
3. GUI integration tests: These tests validate the Zeppelin UI elements. These tests require a running Zeppelin server and launches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example.
Currently the GUI integration tests are not run in the Maven and are only run in the CI environment when the pull request is submitted to github. Make sure to watch the [CI results] (https://travis-ci.org/apache/incubator-zeppelin/pull_requests) for your pull request.
@ -116,7 +171,7 @@ Currently the GUI integration tests are not run in the Maven and are only run in
Zeppelin uses Travis for CI. In the project root there is .travis.yml that configures CI and [publishes CI results] (https://travis-ci.org/apache/incubator-zeppelin/builds)
## Run Zepplin server in development mode
## Run Zeppelin server in development mode
```
cd zeppelin-server
@ -136,8 +191,8 @@ Server will be run on http://localhost:8080
Zeppelin manages it's issues in Jira. [https://issues.apache.org/jira/browse/ZEPPELIN](https://issues.apache.org/jira/browse/ZEPPELIN)
## Stay involved
Everyone is welcome to join our mailling list:
Everyone is welcome to join our mailing list:
* [users@zeppelin.incubator.apache.org](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-users/) is for usage questions, help, and announcements [ [subscribe](mailto:users-subscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20subscribe), [unsubscribe](mailto:users-unsubscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20unsubscribe), [archive](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-users/) ]
* [dev@zeppelin.incubator.apache.org](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-users/) is for people who want to contribute code to Zeppelin.[ [subscribe](mailto:dev-subscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20subscribe), [unsubscribe](mailto:dev-unsubscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20unsubscribe), [archive](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-dev/) ]
* [commits@zeppelin.incubator.apache.org](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-commits/) is for commit messages and patches to Zeppelin. [ [subscribe](mailto:commits-subscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20subscribe), [unsubscribe](mailto:commits-unsubscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20unsubscribe), [archive](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-commits/) ]
* [commits@zeppelin.incubator.apache.org](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-commits/) is for commit messages and patches to Zeppelin. [ [subscribe](mailto:commits-subscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20subscribe), [unsubscribe](mailto:commits-unsubscribe@zeppelin.incubator.apache.org?subject=send%20this%20email%20to%20unsubscribe), [archive](http://mail-archives.apache.org/mod_mbox/incubator-zeppelin-commits/) ]

3
README.md
View file

@ -1,9 +1,10 @@
#Zeppelin
**Documentation:** [User Guide](http://zeppelin.incubator.apache.org/docs/index.html)<br/>
**Mailing List:** [User and Dev mailing list](http://zeppelin.incubator.apache.org/community.html)<br/>
**Mailing Lists:** [User and Dev mailing list](http://zeppelin.incubator.apache.org/community.html)<br/>
**Continuous Integration:** [![Build Status](https://secure.travis-ci.org/apache/incubator-zeppelin.png?branch=master)](https://travis-ci.org/apache/incubator-zeppelin) <br/>
**Contributing:** [Contribution Guide](https://github.com/apache/incubator-zeppelin/blob/master/CONTRIBUTING.md)<br/>
**Issue Tracker:** [Jira](https://issues.apache.org/jira/browse/ZEPPELIN)<br/>
**License:** [Apache 2.0](https://github.com/apache/incubator-zeppelin/blob/master/LICENSE)

13
bin/common.sh
View file

@ -70,6 +70,15 @@ function addEachJarInDir(){
fi
}
function addEachJarInDirRecursive(){
if [[ -d "${1}" ]]; then
for jar in $(find -L "${1}" -type f -name '*jar'); do
ZEPPELIN_CLASSPATH="$jar:$ZEPPELIN_CLASSPATH"
done
fi
}
function addJarInDir(){
if [[ -d "${1}" ]]; then
ZEPPELIN_CLASSPATH="${1}/*:${ZEPPELIN_CLASSPATH}"
@ -84,7 +93,7 @@ if [[ -z "${ZEPPELIN_ENCODING}" ]]; then
fi
if [[ -z "$ZEPPELIN_MEM" ]]; then
export ZEPPELIN_MEM="-Xmx1024m -XX:MaxPermSize=512m"
export ZEPPELIN_MEM="-Xms1024m -Xmx1024m -XX:MaxPermSize=512m"
fi
JAVA_OPTS+=" ${ZEPPELIN_JAVA_OPTS} -Dfile.encoding=${ZEPPELIN_ENCODING} ${ZEPPELIN_MEM}"
@ -99,7 +108,7 @@ if [[ -z "${ZEPPELIN_INTP_MEM}" ]]; then
export ZEPPELIN_INTP_MEM="${ZEPPELIN_MEM}"
fi
JAVA_INTP_OPTS+=" ${ZEPPELIN_INTP_JAVA_OPTS} -Dfile.encoding=${ZEPPELIN_ENCODING} ${ZEPPELIN_INTP_MEM}"
JAVA_INTP_OPTS+=" ${ZEPPELIN_INTP_JAVA_OPTS} -Dfile.encoding=${ZEPPELIN_ENCODING}"
export JAVA_INTP_OPTS

7
bin/interpreter.sh
View file

@ -52,6 +52,9 @@ ZEPPELIN_CLASSPATH+=":${ZEPPELIN_CONF_DIR}"
# construct classpath
if [[ -d "${ZEPPELIN_HOME}/zeppelin-interpreter/target/classes" ]]; then
ZEPPELIN_CLASSPATH+=":${ZEPPELIN_HOME}/zeppelin-interpreter/target/classes"
else
ZEPPELIN_INTERPRETER_JAR="$(ls ${ZEPPELIN_HOME}/lib/zeppelin-interpreter*.jar)"
ZEPPELIN_CLASSPATH+=":${ZEPPELIN_INTERPRETER_JAR}"
fi
addJarInDir "${ZEPPELIN_HOME}/zeppelin-interpreter/target/lib"
@ -84,7 +87,7 @@ if [[ "${INTERPRETER_ID}" == "spark" ]]; then
# add Hadoop jars into classpath
if [[ -n "${HADOOP_HOME}" ]]; then
# Apache
addEachJarInDir "${HADOOP_HOME}/share"
addEachJarInDirRecursive "${HADOOP_HOME}/share"
# CDH
addJarInDir "${HADOOP_HOME}"
@ -123,7 +126,7 @@ CLASSPATH+=":${ZEPPELIN_CLASSPATH}"
if [[ -n "${SPARK_SUBMIT}" ]]; then
${SPARK_SUBMIT} --class ${ZEPPELIN_SERVER} --driver-class-path "${ZEPPELIN_CLASSPATH_OVERRIDES}:${CLASSPATH}" --driver-java-options "${JAVA_INTP_OPTS}" ${SPARK_SUBMIT_OPTIONS} ${SPARK_APP_JAR} ${PORT} &
else
${ZEPPELIN_RUNNER} ${JAVA_INTP_OPTS} -cp ${ZEPPELIN_CLASSPATH_OVERRIDES}:${CLASSPATH} ${ZEPPELIN_SERVER} ${PORT} &
${ZEPPELIN_RUNNER} ${JAVA_INTP_OPTS} ${ZEPPELIN_INTP_MEM} -cp ${ZEPPELIN_CLASSPATH_OVERRIDES}:${CLASSPATH} ${ZEPPELIN_SERVER} ${PORT} &
fi
pid=$!

2
cassandra/pom.xml
View file

@ -40,7 +40,7 @@
<lz4.version>1.2.0</lz4.version>
<scala.version>2.11.7</scala.version>
<scala.binary.version>2.11</scala.binary.version>
<commons-lang.version>3.3.2</commons-lang.version>
<commons-lang.version>3.4</commons-lang.version>
<scalate.version>1.7.1</scalate.version>
<!--TEST-->

1
conf/zeppelin-env.sh.template
View file

@ -25,6 +25,7 @@
# export ZEPPELIN_LOG_DIR # Where log files are stored. PWD by default.
# export ZEPPELIN_PID_DIR # The pid files are stored. /tmp by default.
# export ZEPPELIN_WAR_TEMPDIR # The location of jetty temporary directory.
# export ZEPPELIN_NOTEBOOK_DIR # Where notebook saved
# export ZEPPELIN_NOTEBOOK_HOMESCREEN # Id of notebook to be displayed in homescreen. ex) 2A94M5J1Z
# export ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE # hide homescreen notebook from list when this value set to "true". default "false"

16
conf/zeppelin-site.xml.template
View file

@ -37,6 +37,12 @@
<description>Context Path of the Web Application</description>
</property>
<property>
<name>zeppelin.war.tempdir</name>
<value>webapps</value>
<description>Location of jetty temporary directory</description>
</property>
<property>
<name>zeppelin.notebook.dir</name>
<value>notebook</value>
@ -77,6 +83,14 @@
</property>
-->
<!-- For versioning your local norebook storage using Git repository
<property>
<name>zeppelin.notebook.storage</name>
<value>org.apache.zeppelin.notebook.repo.GitNotebookRepo</value>
<description>notebook persistence layer implementation</description>
</property>
-->
<property>
<name>zeppelin.notebook.storage</name>
<value>org.apache.zeppelin.notebook.repo.VFSNotebookRepo</value>
@ -91,7 +105,7 @@
<property>
<name>zeppelin.interpreters</name>
<value>org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.angular.AngularInterpreter,org.apache.zeppelin.shell.ShellInterpreter,org.apache.zeppelin.hive.HiveInterpreter,org.apache.zeppelin.tajo.TajoInterpreter,org.apache.zeppelin.flink.FlinkInterpreter,org.apache.zeppelin.lens.LensInterpreter,org.apache.zeppelin.ignite.IgniteInterpreter,org.apache.zeppelin.ignite.IgniteSqlInterpreter,org.apache.zeppelin.cassandra.CassandraInterpreter,org.apache.zeppelin.geode.GeodeOqlInterpreter,org.apache.zeppelin.postgresql.PostgreSqlInterpreter,org.apache.zeppelin.phoenix.PhoenixInterpreter,org.apache.zeppelin.kylin.KylinInterpreter,org.apache.zeppelin.tachyon.TachyonInterpreter</value>
<value>org.apache.zeppelin.spark.SparkInterpreter,org.apache.zeppelin.spark.PySparkInterpreter,org.apache.zeppelin.spark.SparkSqlInterpreter,org.apache.zeppelin.spark.DepInterpreter,org.apache.zeppelin.markdown.Markdown,org.apache.zeppelin.angular.AngularInterpreter,org.apache.zeppelin.shell.ShellInterpreter,org.apache.zeppelin.hive.HiveInterpreter,org.apache.zeppelin.tajo.TajoInterpreter,org.apache.zeppelin.flink.FlinkInterpreter,org.apache.zeppelin.lens.LensInterpreter,org.apache.zeppelin.ignite.IgniteInterpreter,org.apache.zeppelin.ignite.IgniteSqlInterpreter,org.apache.zeppelin.cassandra.CassandraInterpreter,org.apache.zeppelin.geode.GeodeOqlInterpreter,org.apache.zeppelin.postgresql.PostgreSqlInterpreter,org.apache.zeppelin.phoenix.PhoenixInterpreter,org.apache.zeppelin.kylin.KylinInterpreter,org.apache.zeppelin.elasticsearch.ElasticsearchInterpreter,org.apache.zeppelin.tachyon.TachyonInterpreter</value>
<description>Comma separated interpreter configurations. First interpreter become a default</description>
</property>

104
dev/test_zeppelin_pr.py Executable file
View file

@ -0,0 +1,104 @@
#!/usr/bin/python
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# This utility creates a local branch from specified pullrequest, to help the test and review
# You'll need to run this utility from master branch with command
#
# dev/test_zeppelin_pr.py [#PR]
#
# then pr[#PR] branch will be created.
#
import sys, os, subprocess
import json, urllib
if len(sys.argv) == 1:
print "usage) " + sys.argv[0] + " [#PR]"
print " eg) " + sys.argv[0] + " 122"
sys.exit(1)
pr=sys.argv[1]
githubApi="https://api.github.com/repos/apache/incubator-zeppelin"
prInfo = json.load(urllib.urlopen(githubApi + "/pulls/" + pr))
if "message" in prInfo and prInfo["message"] == "Not Found":
sys.stderr.write("PullRequest #" + pr + " not found\n")
sys.exit(1)
prUser=prInfo['user']['login']
prRepoUrl=prInfo['head']['repo']['clone_url']
prBranch=prInfo['head']['label'].replace(":", "/")
# create local branch
exitCode = os.system("git checkout -b pr" + pr)
if exitCode != 0:
sys.exit(1)
# add remote repository and fetch
exitCode = os.system("git remote remove " + prUser)
exitCode = os.system("git remote add " + prUser + " " + prRepoUrl)
if exitCode != 0:
sys.stderr.write("Can not add remote repository.\n")
sys.exit(1)
exitCode = os.system("git fetch " + prUser)
if exitCode != 0:
sys.stderr.write("Can't fetch remote repository.\n")
sys.exit(1)
currentBranch = subprocess.check_output(["git rev-parse --abbrev-ref HEAD"], shell=True).rstrip()
print "Merge branch " + prBranch + " into " + currentBranch
rev = subprocess.check_output(["git rev-parse " + prBranch], shell=True).rstrip()
prAuthor = subprocess.check_output(["git --no-pager show -s --format='%an <%ae>' " + rev], shell=True).rstrip()
prAuthorDate = subprocess.check_output(["git --no-pager show -s --format='%ad' " + rev], shell=True).rstrip()
prTitle = prInfo['title']
prBody = prInfo['body']
commitList = subprocess.check_output(["git log --pretty=format:'%h' " + currentBranch + ".." + prBranch], shell=True).rstrip()
authorList = []
for commitHash in commitList.split("\n"):
a = subprocess.check_output(["git show -s --pretty=format:'%an <%ae>' "+commitHash], shell=True).rstrip()
if a not in authorList:
authorList.append(a)
commitMsg = prTitle + "\n"
if prBody :
commitMsg += prBody + "\n\n"
for author in authorList:
commitMsg += "Author: " + author+"\n"
commitMsg += "\n"
commitMsg += "Closes #" + pr + " from " + prBranch + " and squashes the following commits:\n\n"
commitMsg += subprocess.check_output(["git log --pretty=format:'%h [%an] %s' " + currentBranch + ".." + prBranch], shell=True).rstrip()
exitCode = os.system("git merge --no-commit --squash " + prBranch)
if exitCode != 0:
sys.stderr.write("Can not merge\n")
sys.exit(1)
exitCode = os.system('git commit -a --author "' + prAuthor + '" --date "' + prAuthorDate + '" -m"' + commitMsg.encode('utf-8') + '"')
if exitCode != 0:
sys.stderr.write("Commit failed\n")
sys.exit(1)
os.system("git remote remove " + prUser)
print "Branch " + prBranch + " is merged into " + currentBranch

2
docs/_config.yml
View file

@ -5,6 +5,8 @@ permalink: /:categories/:year/:month/:day/:title
exclude: [".rvmrc", ".rbenv-version", "README.md", "Rakefile", "changelog.md", "vendor", "node_modules", "scss"]
pygments: true
markdown: redcarpet
redcarpet:
extensions: ["tables"]
encoding: utf-8
# Themes are encouraged to use these universal variables

7
docs/_includes/themes/zeppelin/_navigation.html
View file

@ -24,6 +24,7 @@
<!-- li><span><b>Install</b><span></li -->
<li><a href="{{BASE_PATH}}/install/install.html">Install</a></li>
<li><a href="{{BASE_PATH}}/install/yarn_install.html">YARN Install</a></li>
<li><a href="{{BASE_PATH}}/install/virtual_machine.html">Virtual Machine Install</a></li>
<li role="separator" class="divider"></li>
<!-- li><span><b>Tutorial</b><span></li -->
<li><a href="{{BASE_PATH}}/tutorial/tutorial.html">Tutorial</a></li>
@ -36,9 +37,10 @@
<a href="#" data-toggle="dropdown" class="dropdown-toggle">Interpreter <b class="caret"></b></a>
<ul class="dropdown-menu">
<li><a href="{{BASE_PATH}}/interpreter/cassandra.html">Cassandra</a></li>
<li><a href="{{BASE_PATH}}/interpreter/elasticsearch.html">Elasticsearch</a></li>
<li><a href="{{BASE_PATH}}/interpreter/flink.html">Flink</a></li>
<li><a href="{{BASE_PATH}}/interpreter/geode.html">Geode</a></li>
<li><a href="{{BASE_PATH}}/pleasecontribute.html">Hive</a></li>
<li><a href="{{BASE_PATH}}/interpreter/hive.html">Hive</a></li>
<li><a href="{{BASE_PATH}}/interpreter/ignite.html">Ignite</a></li>
<li><a href="{{BASE_PATH}}/interpreter/lens.html">Lens</a></li>
<li><a href="{{BASE_PATH}}/interpreter/markdown.html">Markdown</a></li>
@ -64,7 +66,8 @@
<li><a href="{{BASE_PATH}}/manual/notebookashomepage.html">Notebook as Homepage</a></li>
<li role="separator" class="divider"></li>
<!-- li><span><b>Notebook Storage</b><span></li -->
<li><a href="{{BASE_PATH}}/storage/storage.html">S3 Storage</a></li>
<li><a href="{{BASE_PATH}}/storage/storage.html#Git">Git Storage</a></li>
<li><a href="{{BASE_PATH}}/storage/storage.html#S3">S3 Storage</a></li>
<li role="separator" class="divider"></li>
<!-- li><span><b>REST API</b><span></li -->
<li><a href="{{BASE_PATH}}/rest-api/rest-interpreter.html">Interpreter API</a></li>

1
docs/_includes/themes/zeppelin/default.html
View file

@ -29,6 +29,7 @@
<!-- Js -->
<script src="https://code.jquery.com/jquery-1.10.2.min.js"></script>
<script src="{{ ASSET_PATH }}/bootstrap/js/bootstrap.min.js"></script>
<script src="{{ ASSET_PATH }}/js/docs.js"></script>
<!-- atom & rss feed -->
<link href="{{ BASE_PATH }}{{ site.JB.atom_path }}" type="application/atom+xml" rel="alternate" title="Sitewide ATOM Feed">

18
docs/assets/themes/zeppelin/css/style.css
View file

@ -15,6 +15,10 @@ body {
padding-bottom: 10px;
}
.navbar-brand img {
margin: 0;
}
.navbar {
background: #3071a9;
border-bottom: 0px;
@ -209,13 +213,8 @@ body {
/* Table for property */
.table-configuration {
<<<<<<< HEAD
width : 100%;
border : 1px solid gray;
=======
width: 800px;
border: 1px solid gray;
>>>>>>> PR_TOOL_MERGE_PR_483
}
.table-configuration tr td {
border: 1px solid gray;
@ -238,7 +237,6 @@ body {
}
/* Custom container */
<<<<<<< HEAD
/* <a> */
.container a {
color: #4183C4; }
@ -359,8 +357,6 @@ a.anchor {
margin: 12px 0;
}
=======
>>>>>>> PR_TOOL_MERGE_PR_483
.container-narrow {
margin: 0 auto;
}
@ -518,5 +514,9 @@ and (max-width: 1024px) {
}
#menu .navbar-brand {
margin-right: 50px;
margin-right: 50px;
}
.row img {
max-width: 100%;
}

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-config.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 67 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-count-with-query.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 54 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-count.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 58 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-get.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 74 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-query-string.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 113 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-search-json-query-table.png Executable file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 56 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-search-pie.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 47 KiB

BIN
docs/assets/themes/zeppelin/img/docs-img/elasticsearch-search-table.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 257 KiB

121
docs/assets/themes/zeppelin/js/docs.js Normal file
View file

@ -0,0 +1,121 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/* Note: This file is originally from the Apache Spark project. */
/* Custom JavaScript code in the MarkDown docs */
// Enable language-specific code tabs
function codeTabs() {
var counter = 0;
var langImages = {
"scala": "img/scala-sm.png",
"python": "img/python-sm.png",
"java": "img/java-sm.png"
};
$("div.codetabs").each(function() {
$(this).addClass("tab-content");
// Insert the tab bar
var tabBar = $('<ul class="nav nav-tabs" data-tabs="tabs"></ul>');
$(this).before(tabBar);
// Add each code sample to the tab bar:
var codeSamples = $(this).children("div");
codeSamples.each(function() {
$(this).addClass("tab-pane");
var lang = $(this).data("lang");
var image = $(this).data("image");
var notabs = $(this).data("notabs");
var capitalizedLang = lang.substr(0, 1).toUpperCase() + lang.substr(1);
var id = "tab_" + lang + "_" + counter;
$(this).attr("id", id);
if (image != null && langImages[lang]) {
var buttonLabel = "<img src='" +langImages[lang] + "' alt='" + capitalizedLang + "' />";
} else if (notabs == null) {
var buttonLabel = "<b>" + capitalizedLang + "</b>";
} else {
var buttonLabel = ""
}
tabBar.append(
'<li><a class="tab_' + lang + '" href="#' + id + '">' + buttonLabel + '</a></li>'
);
});
codeSamples.first().addClass("active");
tabBar.children("li").first().addClass("active");
counter++;
});
$("ul.nav-tabs a").click(function (e) {
// Toggling a tab should switch all tabs corresponding to the same language
// while retaining the scroll position
e.preventDefault();
var scrollOffset = $(this).offset().top - $(document).scrollTop();
$("." + $(this).attr('class')).tab('show');
$(document).scrollTop($(this).offset().top - scrollOffset);
});
}
function makeCollapsable(elt, accordionClass, accordionBodyId, title) {
$(elt).addClass("accordion-inner");
$(elt).wrap('<div class="accordion ' + accordionClass + '"></div>')
$(elt).wrap('<div class="accordion-group"></div>')
$(elt).wrap('<div id="' + accordionBodyId + '" class="accordion-body collapse"></div>')
$(elt).parent().before(
'<div class="accordion-heading">' +
'<a class="accordion-toggle" data-toggle="collapse" href="#' + accordionBodyId + '">' +
title +
'</a>' +
'</div>'
);
}
// Enable "view solution" sections (for exercises)
function viewSolution() {
var counter = 0
$("div.solution").each(function() {
var id = "solution_" + counter
makeCollapsable(this, "", id,
'<i class="icon-ok-sign" style="text-decoration: none; color: #0088cc">' +
'</i>' + "View Solution");
counter++;
});
}
// A script to fix internal hash links because we have an overlapping top bar.
// Based on https://github.com/twitter/bootstrap/issues/193#issuecomment-2281510
function maybeScrollToHash() {
console.log("HERE");
if (window.location.hash && $(window.location.hash).length) {
console.log("HERE2", $(window.location.hash), $(window.location.hash).offset().top);
var newTop = $(window.location.hash).offset().top - 57;
$(window).scrollTop(newTop);
}
}
$(function() {
codeTabs();
viewSolution();
$(window).bind('hashchange', function() {
maybeScrollToHash();
});
// Scroll now too in case we had opened the page on a hash, but wait a bit because some browsers
// will try to do *their* initial scroll after running the onReady handler.
$(window).load(function() { setTimeout(function() { maybeScrollToHash(); }, 25); });
});

3
docs/docs.md
View file

@ -36,7 +36,7 @@ limitations under the License.
* [cassandra](./interpreter/cassandra.html)
* [flink](./interpreter/flink.html)
* [geode](./interpreter/geode.html)
* [hive](./pleasecontribute.html)
* [hive](./interpreter/hive.html)
* [ignite](./interpreter/ignite.html)
* [lens](./interpreter/lens.html)
* [md](./interpreter/markdown.html)
@ -44,6 +44,7 @@ limitations under the License.
* [sh](./pleasecontribute.html)
* [spark](./interpreter/spark.html)
* [tajo](./pleasecontribute.html)
* [elasticsearch](./interpreter/elasticsearch.html)
### Storage
* [S3 Storage](./storage/storage.html)

8
docs/install/install.md
View file

@ -145,7 +145,13 @@ Configuration can be done by both environment variable(conf/zeppelin-env.sh) and
<td>zeppelin.notebook.homescreen.hide</td>
<td>false</td>
<td>hide homescreen notebook from list when this value set to "true"</td>
</tr>
</tr>
<tr>
<td>ZEPPELIN_WAR_TEMPDIR</td>
<td>zeppelin.war.tempdir</td>
<td>webapps</td>
<td>The location of jetty temporary directory.</td>
</tr>
<tr>
<td>ZEPPELIN_NOTEBOOK_DIR</td>
<td>zeppelin.notebook.dir</td>

236
docs/interpreter/elasticsearch.md Normal file
View file

@ -0,0 +1,236 @@
---
layout: page
title: "Elasticsearch Interpreter"
description: ""
group: manual
---
{% include JB/setup %}
## Elasticsearch Interpreter for Apache Zeppelin
### 1. Configuration
<br/>
<table class="table-configuration">
<tr>
<th>Property</th>
<th>Default</th>
<th>Description</th>
</tr>
<tr>
<td>elasticsearch.cluster.name</td>
<td>elasticsearch</td>
<td>Cluster name</td>
</tr>
<tr>
<td>elasticsearch.host</td>
<td>localhost</td>
<td>Host of a node in the cluster</td>
</tr>
<tr>
<td>elasticsearch.port</td>
<td>9300</td>
<td>Connection port <b>(important: this is not the HTTP port, but the transport port)</b></td>
</tr>
<tr>
<td>elasticsearch.result.size</td>
<td>10</td>
<td>The size of the result set of a search query</td>
</tr>
</table>
<center>
![Interpreter configuration](../assets/themes/zeppelin/img/docs-img/elasticsearch-config.png)
</center>
> Note #1: you can add more properties to configure the Elasticsearch client.
> Note #2: if you use Shield, you can add a property named `shield.user` with a value containing the name and the password (format: `username:password`). For more details about Shield configuration, consult the [Shield reference guide](https://www.elastic.co/guide/en/shield/current/_using_elasticsearch_java_clients_with_shield.html). Do not forget, to copy the shield client jar in the interpreter directory (`ZEPPELIN_HOME/interpreters/elasticsearch`).
<hr/>
### 2. Enabling the Elasticsearch Interpreter
In a notebook, to enable the **Elasticsearch** interpreter, click the **Gear** icon and select **Elasticsearch**.
<hr/>
### 3. Using the Elasticsearch Interpreter
In a paragraph, use `%elasticsearch` to select the Elasticsearch interpreter and then input all commands. To get the list of available commands, use `help`.
```bash
| %elasticsearch
| help
Elasticsearch interpreter:
General format: <command> /<indices>/<types>/<id> <option> <JSON>
- indices: list of indices separated by commas (depends on the command)
- types: list of document types separated by commas (depends on the command)
Commands:
- search /indices/types <query>
. indices and types can be omitted (at least, you have to provide '/')
. a query is either a JSON-formatted query, nor a lucene query
- size <value>
. defines the size of the result set (default value is in the config)
. if used, this command must be declared before a search command
- count /indices/types <query>
. same comments as for the search
- get /index/type/id
- delete /index/type/id
- index /ndex/type/id <json-formatted document>
. the id can be omitted, elasticsearch will generate one
```
> Tip: use (CTRL + .) for completion
#### get
With the `get` command, you can find a document by id. The result is a JSON document.
```bash
| %elasticsearch
| get /index/type/id
```
Example:
![Elasticsearch - Get](../assets/themes/zeppelin/img/docs-img/elasticsearch-get.png)
#### search
With the `search` command, you can send a search query to Elasticsearch. There are two formats of query:
* You can provide a JSON-formatted query, that is exactly what you provide when you use the REST API of Elasticsearch.
* See [Elasticsearch search API reference document](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html) for more details about the content of the search queries.
* You can also provide the content of a `query_string`
* This is a shortcut to a query like that: `{ "query": { "query_string": { "query": "__HERE YOUR QUERY__", "analyze_wildcard": true } } }`
* See [Elasticsearch query string syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax) for more details about the content of such a query.
```bash
| %elasticsearch
| search /index1,index2,.../type1,type2,... <JSON document containing the query or query_string elements>
```
If you want to modify the size of the result set, you can add a line that is setting the size, before your search command.
```bash
| %elasticsearch
| size 50
| search /index1,index2,.../type1,type2,... <JSON document containing the query or query_string elements>
```
Examples:
* With a JSON query:
```bash
| %elasticsearch
| search / { "query": { "match_all": {} } }
|
| %elasticsearch
| search /logs { "query": { "query_string": { "query": "request.method:GET AND status:200" } } }
```
* With query_string elements:
```bash
| %elasticsearch
| search /logs request.method:GET AND status:200
|
| %elasticsearch
| search /logs (404 AND (POST OR DELETE))
```
> **Important**: a document in Elasticsearch is a JSON document, so it is hierarchical, not flat as a row in a SQL table.
For the Elastic interpreter, the result of a search query is flattened.
Suppose we have a JSON document:
```
{
"date": "2015-12-08T21:03:13.588Z",
"request": {
"method": "GET",
"url": "/zeppelin/4cd001cd-c517-4fa9-b8e5-a06b8f4056c4",
"headers": [ "Accept: *.*", "Host: apache.org"]
},
"status": "403"
}
```
The data will be flattened like this:
date | request.headers[0] | request.headers[1] | request.method | request.url | status
-----|--------------------|--------------------|----------------|-------------|-------
2015-12-08T21:03:13.588Z | Accept: \*.\* | Host: apache.org | GET | /zeppelin/4cd001cd-c517-4fa9-b8e5-a06b8f4056c4 | 403
Examples:
* With a table containing the results:
![Elasticsearch - Search - table](../assets/themes/zeppelin/img/docs-img/elasticsearch-search-table.png)
* You can also use a predefined diagram:
![Elasticsearch - Search - diagram](../assets/themes/zeppelin/img/docs-img/elasticsearch-search-pie.png)
* With a JSON query:
![Elasticsearch - Search with query](../assets/themes/zeppelin/img/docs-img/elasticsearch-search-json-query-table.png)
* With a query string:
![Elasticsearch - Search with query string](../assets/themes/zeppelin/img/docs-img/elasticsearch-query-string.png)
#### count
With the `count` command, you can count documents available in some indices and types. You can also provide a query.
```bash
| %elasticsearch
| count /index1,index2,.../type1,type2,... <JSON document containing the query OR a query string>
```
Examples:
* Without query:
![Elasticsearch - Count](../assets/themes/zeppelin/img/docs-img/elasticsearch-count.png)
* With a query:
![Elasticsearch - Count with query](../assets/themes/zeppelin/img/docs-img/elasticsearch-count-with-query.png)
#### index
With the `index` command, you can insert/update a document in Elasticsearch.
```bash
| %elasticsearch
| index /index/type/id <JSON document>
|
| %elasticsearch
| index /index/type <JSON document>
```
#### delete
With the `delete` command, you can delete a document.
```bash
| %elasticsearch
| delete /index/type/id
```
#### Apply Zeppelin Dynamic Forms
You can leverage [Zeppelin Dynamic Form]({{BASE_PATH}}/manual/dynamicform.html) inside your queries. You can use both the `text input` and `select form` parameterization features
```bash
| %elasticsearch
| size ${limit=10}
| search /index/type { "query": { "match_all": {} } }
```

105
docs/interpreter/hive.md Normal file
View file

@ -0,0 +1,105 @@
---
layout: page
title: "Hive Interpreter"
description: ""
group: manual
---
{% include JB/setup %}
## Hive Interpreter for Apache Zeppelin
### Configuration
<br/>
<table class="table-configuration">
<tr>
<th>Property</th>
<th>Default</th>
<th>Description</th>
</tr>
<tr>
<td>default.driver</td>
<td>org.apache.hive.jdbc.HiveDriver</td>
<td>Class path of JDBC driver</td>
</tr>
<tr>
<td>default.url</td>
<td>jdbc:hive2://localhost:10000</td>
<td>Url for connection</td>
</tr>
<tr>
<td>default.user</td>
<td></td>
<td><b>(Optional)</b>Username of the connection</td>
</tr>
<tr>
<td>default.password</td>
<td></td>
<td><b>(Optional)</b>Password of the connection</td>
</tr>
<tr>
<td>default.xxx</td>
<td></td>
<td><b>(Optional)</b>Other properties used by the driver</td>
</tr>
<tr>
<td>${prefix}.driver</td>
<td></td>
<td>Driver class path of `%hive(${prefix})`</td>
</tr>
<tr>
<td>${prefix}.url</td>
<td></td>
<td>Url of `%hive(${prefix})`</td>
</tr>
<tr>
<td>${prefix}.user</td>
<td></td>
<td><b>(Optional)</b>Username of the connection of `%hive(${prefix})`</td>
</tr>
<tr>
<td>${prefix}.password</td>
<td></td>
<td><b>(Optional)</b>Password of the connection of `%hive(${prefix})`</td>
</tr>
<tr>
<td>${prefix}.xxx</td>
<td></td>
<td><b>(Optional)</b>Other properties used by the driver of `%hive(${prefix})`</td>
</tr>
</table>
This interpreter provides multiple configuration with ${prefix}. User can set a multiple connection properties by this prefix. It can be used like `%hive(${prefix})`.
### How to use
Basically, you can use
```sql
%hive
select * from my_table;
```
or
```sql
%hive(etl)
-- 'etl' is a ${prefix}
select * from my_table;
```
You can also run multiple queries up to 10 by default. Changing these settings is not implemented yet.
#### Apply Zeppelin Dynamic Forms
You can leverage [Zeppelin Dynamic Form]({{BASE_PATH}}/manual/dynamicform.html) inside your queries. You can use both the `text input` and `select form` parameterization features
```sql
%hive
SELECT ${group_by}, count(*) as count
FROM retail_demo.order_lineitems_pxf
GROUP BY ${group_by=product_id,product_id|product_name|customer_id|store_id}
ORDER BY count ${order=DESC,DESC|ASC}
LIMIT ${limit=10};
```

64
docs/manual/dynamicform.md
View file

@ -19,18 +19,15 @@ limitations under the License.
-->
{% include JB/setup %}
## Dynamic Form
Zeppelin dynamically creates input forms. Depending on language backend, there're two different ways to create dynamic form.
Custom language backend can select which type of form creation it wants to use.
<br />
### Using form Templates
This mode creates form using simple template language. It's simple and easy to use. For example Markdown, Shell, SparkSql language backend uses it.
<br />
#### Text input form
To create text input form, use _${formName}_ templates.
@ -45,7 +42,6 @@ Also you can provide default value, using _${formName=defaultValue}_.
<img src="/assets/themes/zeppelin/img/screenshots/form_input_default.png" />
<br />
#### Select form
To create select form, use _${formName=defaultValue,option1|option2...}_
@ -58,54 +54,59 @@ Also you can separate option's display name and value, using _${formName=default
<img src="/assets/themes/zeppelin/img/screenshots/form_select_displayname.png" />
<br />
### Creates Programmatically
Some language backend uses programmatic way to create form. For example [ZeppelinContext](../interpreter/spark.html#zeppelincontext) provides form creation API
Here're some examples.
Text input form
####Text input form
<div class="codetabs">
<div data-lang="scala" markdown="1">
You can do this in Scala
```scala
{% highlight scala %}
%spark
println("Hello "+z.input("name"))
```
{% endhighlight %}
Or Python
</div>
<div data-lang="python" markdown="1">
```python
{% highlight python %}
%pyspark
print("Hello "+z.input("name"))
```
{% endhighlight %}
</div>
</div>
<img src="/assets/themes/zeppelin/img/screenshots/form_input_prog.png" />
Text input form with default value
####Text input form with default value
<div class="codetabs">
<div data-lang="scala" markdown="1">
Scala
```scala
{% highlight scala %}
%spark
println("Hello "+z.input("name", "sun"))
```
println("Hello "+z.input("name", "sun"))
{% endhighlight %}
Python
</div>
<div data-lang="python" markdown="1">
```python
{% highlight python %}
%pyspark
print("Hello "+z.input("name", "sun"))
```
{% endhighlight %}
</div>
</div>
<img src="/assets/themes/zeppelin/img/screenshots/form_input_default_prog.png" />
Select form
####Select form
<div class="codetabs">
<div data-lang="scala" markdown="1">
Scala
```scala
{% highlight scala %}
%spark
println("Hello "+z.select("day", Seq(("1","mon"),
("2","tue"),
@ -114,11 +115,12 @@ println("Hello "+z.select("day", Seq(("1","mon"),
("5","fri"),
("6","sat"),
("7","sun"))))
```
{% endhighlight %}
Python
</div>
<div data-lang="python" markdown="1">
```python
{% highlight python %}
%pyspark
print("Hello "+z.select("day", [("1","mon"),
("2","tue"),
@ -127,6 +129,8 @@ print("Hello "+z.select("day", [("1","mon"),
("5","fri"),
("6","sat"),
("7","sun")]))
```
{% endhighlight %}
</div>
</div>
<img src="/assets/themes/zeppelin/img/screenshots/form_select_prog.png" />

34
docs/rest-api/rest-interpreter.md
View file

@ -361,3 +361,37 @@ limitations under the License.
</td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Restart an interpreter</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```PUT``` method restart the given interpreter id.</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/interpreter/setting/restart/[interpreter ID]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response
</td>
<td>
<pre>{"status":"OK"}</pre>
</td>
</tr>
</table>

443
docs/rest-api/rest-notebook.md
View file

@ -20,12 +20,12 @@ limitations under the License.
{% include JB/setup %}
## Zeppelin REST API
Zeppelin provides several REST API's for interaction and remote activation of zeppelin functionality.
Zeppelin provides several REST APIs for interaction and remote activation of zeppelin functionality.
All REST API are available starting with the following endpoint ```http://[zeppelin-server]:[zeppelin-port]/api```
All REST APIs are available starting with the following endpoint ```http://[zeppelin-server]:[zeppelin-port]/api```
Note that zeppein REST API receive or return JSON objects, it it recommended you install some JSON view such as
[JSONView](https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc)
Note that zeppelin REST APIs receive or return JSON objects, it is recommended for you to install some JSON viewer
such as [JSONView](https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc)
If you work with zeppelin and find a need for an additional REST API please [file an issue or send us mail](../../community.html)
@ -33,7 +33,7 @@ limitations under the License.
<br />
### Notebook REST API list
Notebooks REST API supports the following operations: List, Create, Delete & Clone as detailed in the following table
Notebooks REST API supports the following operations: List, Create, Get, Delete, Clone, Run as detailed in the following table
<table class="table-configuration">
<col width="200">
@ -43,7 +43,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method list the available notebooks on your server.
<td>This ```GET``` method lists the available notebooks on your server.
Notebook JSON contains the ```name``` and ```id``` of all notebooks.
</td>
</tr>
@ -75,8 +75,8 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method create a new notebook using the given name or default name if none given.
The body field of the returned JSON contain the new notebook id.
<td>This ```POST``` method creates a new notebook using the given name or default name if none given.
The body field of the returned JSON contains the new notebook id.
</td>
</tr>
<tr>
@ -92,8 +92,26 @@ limitations under the License.
<td> 500 </td>
</tr>
<tr>
<td> sample JSON input </td>
<td><pre>{"name": "name of new notebook"}</pre></td>
<td> sample JSON input (without paragraphs) </td>
<td><pre>{ "name": "name of new notebook" }</pre></td>
</tr>
<tr>
<td> sample JSON input (with initial paragraphs) </td>
<td><pre>
{
"name": "name of new notebook",
"paragraphs": [
{
"title": "paragraph title1",
"text": "paragraph text1"
},
{
"title": "paragraph title2",
"text": "paragraph text2"
}
]
}
</pre></td>
</tr>
<tr>
<td> sample JSON response </td>
@ -101,6 +119,108 @@ limitations under the License.
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Get notebook</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method retrieves an existing notebook's information using the given id.
The body field of the returned JSON contain information about paragraphs in the notebook.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>
{
"status": "OK",
"message": "",
"body": {
"paragraphs": [
{
"text": "%sql \nselect age, count(1) value\nfrom bank \nwhere age < 30 \ngroup by age \norder by age",
"config": {
"colWidth": 4,
"graph": {
"mode": "multiBarChart",
"height": 300,
"optionOpen": false,
"keys": [
{
"name": "age",
"index": 0,
"aggr": "sum"
}
],
"values": [
{
"name": "value",
"index": 1,
"aggr": "sum"
}
],
"groups": [],
"scatter": {
"xAxis": {
"name": "age",
"index": 0,
"aggr": "sum"
},
"yAxis": {
"name": "value",
"index": 1,
"aggr": "sum"
}
}
}
},
"settings": {
"params": {},
"forms": {}
},
"jobName": "paragraph_1423500782552_-1439281894",
"id": "20150210-015302_1492795503",
"result": {
"code": "SUCCESS",
"type": "TABLE",
"msg": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t20\n24\t24\n25\t44\n26\t77\n27\t94\n28\t103\n29\t97\n"
},
"dateCreated": "Feb 10, 2015 1:53:02 AM",
"dateStarted": "Jul 3, 2015 1:43:17 PM",
"dateFinished": "Jul 3, 2015 1:43:23 PM",
"status": "FINISHED",
"progressUpdateIntervalMs": 500
}
],
"name": "Zeppelin Tutorial",
"id": "2A94M5J1Z",
"angularObjects": {},
"config": {
"looknfeel": "default"
},
"info": {}
}
}
</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
@ -111,7 +231,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method delete a notebook by the given notebook id.
<td>This ```DELETE``` method deletes a notebook by the given notebook id.
</td>
</tr>
<tr>
@ -142,9 +262,9 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method clone a notebook by the given id and create a new notebook using the given name
<td>This ```POST``` method clones a notebook by the given id and create a new notebook using the given name
or default name if none given.
The body field of the returned JSON contain the new notebook id.
The body field of the returned JSON contains the new notebook id.
</td>
</tr>
<tr>
@ -169,3 +289,300 @@ limitations under the License.
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Run notebook job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method runs all paragraph in the given notebook id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK"}</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Stop notebook job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method stops all paragraph in the given notebook id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK"}</pre></td>
</tr>
</table>
<br/>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Get notebook job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method gets all paragraph status by the given notebook id.
The body field of the returned JSON contains of the array that compose of the paragraph id, paragraph status, paragraph finish date, paragraph started date.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK","body":[{"id":"20151121-212654_766735423","status":"FINISHED","finished":"Tue Nov 24 14:21:40 KST 2015","started":"Tue Nov 24 14:21:39 KST 2015"},{"id":"20151121-212657_730976687","status":"FINISHED","finished":"Tue Nov 24 14:21:40 KST 2015","started":"Tue Nov 24 14:21:40 KST 2015"}]}</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Run paragraph job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method runs the paragraph by given notebook and paragraph id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON input (optional, only needed when if you want to update dynamic form's value) </td>
<td><pre>
{
"name": "name of new notebook",
"params": {
"formLabel1": "value1",
"formLabel2": "value2"
}
}
</pre></td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK"}</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Stop paragraph job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method stops the paragraph by given notebook and paragraph id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK"}</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Add cron job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method adds cron job by the given notebook id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON input </td>
<td><pre>{"cron": "cron expression of notebook"}</pre></td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK"}</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Remove cron job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method removes cron job by the given notebook id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK"}</pre></td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Get cron job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method gets cron job expression of given notebook id.
The body field of the returned JSON contains the cron expression.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[notebookId]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response </td>
<td><pre>{"status":"OK","body":"* * * * * ?"}</pre></td>
</tr>
</table>
<table class="table-configuration">
<col width="200">
<tr>
<th>Full-text search through the paragraphs in all notebooks</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>```GET``` request will return list of matching paragraphs
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/search?q=[query]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td>Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td>Sample JSON response </td>
<td><pre>{"status":"OK", body: [{"id":"<noteId>/paragraph/<paragraphId>", "name":"Notebook Name", "snippet":"", "text":""}]}</pre></td>
</tr>
</table>

24
docs/storage/storage.md
View file

@ -19,10 +19,30 @@ limitations under the License.
-->
### Notebook Storage
In Zeppelin there are two option for storage Notebook, by default the notebook is storage in the notebook folder in your local File System and the second option is S3.
Zeppelin has a pluggable notebook storage mechanism controlled by `zeppelin.notebook.storage` configuration option with multiple implementations.
There are few Notebook storages avaialble for a use out of the box:
- (default) all notes are saved in the notebook folder in your local File System - `VFSNotebookRepo`
- there is also an option to version it using local Git repository - `GitNotebookRepo`
- another option is Amazon S3 service - `S3NotebookRepo`
Multiple storages can be used at the same time by providing a comma-separated list of the calss-names in the confiruration.
By default, only first two of them will be automatically kept in sync by Zeppelin.
</br>
#### Notebook Storage in S3
#### Notebook Storage in local Git repository <a name="Git"></a>
To enable versioning for all your local notebooks though a standard Git repository - uncomment the next property in `zeppelin-site.xml` in order to use GitNotebookRepo class:
```
<property>
<name>zeppelin.notebook.storage</name>
<value>org.apache.zeppelin.notebook.repo.GitNotebookRepo</value>
<description>notebook persistence layer implementation</description>
</property>
```
</br>
#### Notebook Storage in S3 <a name="S3"></a>
For notebook storage in S3 you need the AWS credentials, for this there are three options, the enviroment variable ```AWS_ACCESS_KEY_ID``` and ```AWS_ACCESS_SECRET_KEY```, credentials file in the folder .aws in you home and IAM role for your instance. For complete the need steps is necessary:

147
elasticsearch/pom.xml Normal file
View file

@ -0,0 +1,147 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>zeppelin</artifactId>
<groupId>org.apache.zeppelin</groupId>
<version>0.6.0-incubating-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>
<groupId>org.apache.zeppelin</groupId>
<artifactId>zeppelin-elasticsearch</artifactId>
<packaging>jar</packaging>
<version>0.6.0-incubating-SNAPSHOT</version>
<name>Zeppelin: Elasticsearch interpreter</name>
<url>http://www.apache.org</url>
<properties>
<elasticsearch.version>2.1.0</elasticsearch.version>
<guava.version>18.0</guava.version>
<json-flattener.version>0.1.1</json-flattener.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.zeppelin</groupId>
<artifactId>zeppelin-interpreter</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
</dependency>
<dependency>
<groupId>com.github.wnameless</groupId>
<artifactId>json-flattener</artifactId>
<version>${json-flattener.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.7</version>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.3.1</version>
<executions>
<execution>
<id>enforce</id>
<phase>none</phase>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/../../interpreter/elasticsearch</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
<includeScope>runtime</includeScope>
</configuration>
</execution>
<execution>
<id>copy-artifact</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/../../interpreter/elasticsearch</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
<includeScope>runtime</includeScope>
<artifactItems>
<artifactItem>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<version>${project.version}</version>
<type>${project.packaging}</type>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

465
elasticsearch/src/main/java/org/apache/zeppelin/elasticsearch/ElasticsearchInterpreter.java Normal file
View file

@ -0,0 +1,465 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.elasticsearch;
import java.io.IOException;
import java.net.InetAddress;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import java.util.TreeSet;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.interpreter.InterpreterContext;
import org.apache.zeppelin.interpreter.InterpreterPropertyBuilder;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchAction;
import org.elasticsearch.action.search.SearchRequestBuilder;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.github.wnameless.json.flattener.JsonFlattener;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonParseException;
/**
* Elasticsearch Interpreter for Zeppelin.
*/
public class ElasticsearchInterpreter extends Interpreter {
private static Logger logger = LoggerFactory.getLogger(ElasticsearchInterpreter.class);
private static final String HELP = "Elasticsearch interpreter:\n"
+ "General format: <command> /<indices>/<types>/<id> <option> <JSON>\n"
+ " - indices: list of indices separated by commas (depends on the command)\n"
+ " - types: list of document types separated by commas (depends on the command)\n"
+ "Commands:\n"
+ " - search /indices/types <query>\n"
+ " . indices and types can be omitted (at least, you have to provide '/')\n"
+ " . a query is either a JSON-formatted query, nor a lucene query\n"
+ " - size <value>\n"
+ " . defines the size of the result set (default value is in the config)\n"
+ " . if used, this command must be declared before a search command\n"
+ " - count /indices/types <query>\n"
+ " . same comments as for the search\n"
+ " - get /index/type/id\n"
+ " - delete /index/type/id\n"
+ " - index /ndex/type/id <json-formatted document>\n"
+ " . the id can be omitted, elasticsearch will generate one";
private static final List<String> COMMANDS = Arrays.asList(
"count", "delete", "get", "help", "index", "search");
public static final String ELASTICSEARCH_HOST = "elasticsearch.host";
public static final String ELASTICSEARCH_PORT = "elasticsearch.port";
public static final String ELASTICSEARCH_CLUSTER_NAME = "elasticsearch.cluster.name";
public static final String ELASTICSEARCH_RESULT_SIZE = "elasticsearch.result.size";
static {
Interpreter.register(
"elasticsearch",
"elasticsearch",
ElasticsearchInterpreter.class.getName(),
new InterpreterPropertyBuilder()
.add(ELASTICSEARCH_HOST, "localhost", "The host for Elasticsearch")
.add(ELASTICSEARCH_PORT, "9300", "The port for Elasticsearch")
.add(ELASTICSEARCH_CLUSTER_NAME, "elasticsearch", "The cluster name for Elasticsearch")
.add(ELASTICSEARCH_RESULT_SIZE, "10", "The size of the result set of a search query")
.build());
}
private final Gson gson = new GsonBuilder().setPrettyPrinting().create();
private Client client;
private String host = "localhost";
private int port = 9300;
private String clusterName = "elasticsearch";
private int resultSize = 10;
public ElasticsearchInterpreter(Properties property) {
super(property);
this.host = getProperty(ELASTICSEARCH_HOST);
this.port = Integer.parseInt(getProperty(ELASTICSEARCH_PORT));
this.clusterName = getProperty(ELASTICSEARCH_CLUSTER_NAME);
this.resultSize = Integer.parseInt(getProperty(ELASTICSEARCH_RESULT_SIZE));
}
@Override
public void open() {
try {
logger.info("prop={}", getProperty());
final Settings settings = Settings.settingsBuilder()
.put("cluster.name", clusterName)
.put(getProperty())
.build();
client = TransportClient.builder().settings(settings).build()
.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(host), port));
}
catch (IOException e) {
logger.error("Open connection with Elasticsearch", e);
}
}
@Override
public void close() {
if (client != null) {
client.close();
}
}
@Override
public InterpreterResult interpret(String cmd, InterpreterContext interpreterContext) {
logger.info("Run Elasticsearch command '" + cmd + "'");
int currentResultSize = resultSize;
if (client == null) {
return new InterpreterResult(InterpreterResult.Code.ERROR,
"Problem with the Elasticsearch client, please check your configuration (host, port,...)");
}
String[] items = StringUtils.split(cmd.trim(), " ", 3);
// Process some specific commands (help, size, ...)
if ("help".equalsIgnoreCase(items[0])) {
return processHelp(InterpreterResult.Code.SUCCESS, null);
}
if ("size".equalsIgnoreCase(items[0])) {
// In this case, the line with size must be followed by a search,
// so we will continue with the next lines
final String[] lines = StringUtils.split(cmd.trim(), "\n", 2);
if (lines.length < 2) {
return processHelp(InterpreterResult.Code.ERROR,
"Size cmd must be followed by a search");
}
final String[] sizeLine = StringUtils.split(lines[0], " ", 2);
if (sizeLine.length != 2) {
return processHelp(InterpreterResult.Code.ERROR, "Right format is : size <value>");
}
currentResultSize = Integer.parseInt(sizeLine[1]);
items = StringUtils.split(lines[1].trim(), " ", 3);
}
if (items.length < 2) {
return processHelp(InterpreterResult.Code.ERROR, "Arguments missing");
}
final String method = items[0];
final String url = items[1];
final String data = items.length > 2 ? items[2].trim() : null;
final String[] urlItems = StringUtils.split(url.trim(), "/");
try {
if ("get".equalsIgnoreCase(method)) {
return processGet(urlItems);
}
else if ("count".equalsIgnoreCase(method)) {
return processCount(urlItems, data);
}
else if ("search".equalsIgnoreCase(method)) {
return processSearch(urlItems, data, currentResultSize);
}
else if ("index".equalsIgnoreCase(method)) {
return processIndex(urlItems, data);
}
else if ("delete".equalsIgnoreCase(method)) {
return processDelete(urlItems);
}
return processHelp(InterpreterResult.Code.ERROR, "Unknown command");
}
catch (Exception e) {
return new InterpreterResult(InterpreterResult.Code.ERROR, "Error : " + e.getMessage());
}
}
@Override
public void cancel(InterpreterContext interpreterContext) {
// Nothing to do
}
@Override
public FormType getFormType() {
return FormType.SIMPLE;
}
@Override
public int getProgress(InterpreterContext interpreterContext) {
return 0;
}
@Override
public List<String> completion(String s, int i) {
final List<String> suggestions = new ArrayList<>();
if (StringUtils.isEmpty(s)) {
suggestions.addAll(COMMANDS);
}
else {
for (String cmd : COMMANDS) {
if (cmd.toLowerCase().contains(s)) {
suggestions.add(cmd);
}
}
}
return suggestions;
}
private InterpreterResult processHelp(InterpreterResult.Code code, String additionalMessage) {
final StringBuffer buffer = new StringBuffer();
if (additionalMessage != null) {
buffer.append(additionalMessage).append("\n");
}
buffer.append(HELP).append("\n");
return new InterpreterResult(code, InterpreterResult.Type.TEXT, buffer.toString());
}
/**
* Processes a "get" request.
*
* @param urlItems Items of the URL
* @return Result of the get request, it contains a JSON-formatted string
*/
private InterpreterResult processGet(String[] urlItems) {
if (urlItems.length != 3
|| StringUtils.isEmpty(urlItems[0])
|| StringUtils.isEmpty(urlItems[1])
|| StringUtils.isEmpty(urlItems[2])) {
return new InterpreterResult(InterpreterResult.Code.ERROR,
"Bad URL (it should be /index/type/id)");
}
final GetResponse response = client
.prepareGet(urlItems[0], urlItems[1], urlItems[2])
.get();
if (response.isExists()) {
final String json = gson.toJson(response.getSource());
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TEXT,
json);
}
return new InterpreterResult(InterpreterResult.Code.ERROR, "Document not found");
}
/**
* Processes a "count" request.
*
* @param urlItems Items of the URL
* @param data May contains the JSON of the request
* @return Result of the count request, it contains the total hits
*/
private InterpreterResult processCount(String[] urlItems, String data) {
if (urlItems.length > 2) {
return new InterpreterResult(InterpreterResult.Code.ERROR,
"Bad URL (it should be /index1,index2,.../type1,type2,...)");
}
final SearchResponse response = searchData(urlItems, data, 0);
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TEXT,
"" + response.getHits().getTotalHits());
}
/**
* Processes a "search" request.
*
* @param urlItems Items of the URL
* @param data May contains the limit and the JSON of the request
* @return Result of the search request, it contains a tab-formatted string of the matching hits
*/
private InterpreterResult processSearch(String[] urlItems, String data, int size) {
if (urlItems.length > 2) {
return new InterpreterResult(InterpreterResult.Code.ERROR,
"Bad URL (it should be /index1,index2,.../type1,type2,...)");
}
final SearchResponse response = searchData(urlItems, data, size);
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TABLE,
buildResponseMessage(response.getHits().getHits()));
}
/**
* Processes a "index" request.
*
* @param urlItems Items of the URL
* @param data JSON to be indexed
* @return Result of the index request, it contains the id of the document
*/
private InterpreterResult processIndex(String[] urlItems, String data) {
if (urlItems.length < 2 || urlItems.length > 3) {
return new InterpreterResult(InterpreterResult.Code.ERROR,
"Bad URL (it should be /index/type or /index/type/id)");
}
final IndexResponse response = client
.prepareIndex(urlItems[0], urlItems[1], urlItems.length == 2 ? null : urlItems[2])
.setSource(data)
.get();
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TEXT,
response.getId());
}
/**
* Processes a "delete" request.
*
* @param urlItems Items of the URL
* @return Result of the delete request, it contains the id of the deleted document
*/
private InterpreterResult processDelete(String[] urlItems) {
if (urlItems.length != 3
|| StringUtils.isEmpty(urlItems[0])
|| StringUtils.isEmpty(urlItems[1])
|| StringUtils.isEmpty(urlItems[2])) {
return new InterpreterResult(InterpreterResult.Code.ERROR,
"Bad URL (it should be /index/type/id)");
}
final DeleteResponse response = client
.prepareDelete(urlItems[0], urlItems[1], urlItems[2])
.get();
if (response.isFound()) {
return new InterpreterResult(
InterpreterResult.Code.SUCCESS,
InterpreterResult.Type.TEXT,
response.getId());
}
return new InterpreterResult(InterpreterResult.Code.ERROR, "Document not found");
}
private SearchResponse searchData(String[] urlItems, String query, int size) {
final SearchRequestBuilder reqBuilder = new SearchRequestBuilder(
client, SearchAction.INSTANCE);
reqBuilder.setIndices();
if (urlItems.length >= 1) {
reqBuilder.setIndices(StringUtils.split(urlItems[0], ","));
}
if (urlItems.length > 1) {
reqBuilder.setTypes(StringUtils.split(urlItems[1], ","));
}
if (!StringUtils.isEmpty(query)) {
// The query can be either JSON-formatted, nor a Lucene query
// So, try to parse as a JSON => if there is an error, consider the query a Lucene one
try {
final Map source = gson.fromJson(query, Map.class);
reqBuilder.setExtraSource(source);
}
catch (JsonParseException e) {
// This is not a JSON (or maybe not well formatted...)
reqBuilder.setQuery(QueryBuilders.queryStringQuery(query).analyzeWildcard(true));
}
}
reqBuilder.setSize(size);
final SearchResponse response = reqBuilder.get();
return response;
}
private String buildResponseMessage(SearchHit[] hits) {
if (hits == null || hits.length == 0) {
return "";
}
//First : get all the keys in order to build an ordered list of the values for each hit
//
final List<Map<String, Object>> flattenHits = new LinkedList<>();
final Set<String> keys = new TreeSet<>();
for (SearchHit hit : hits) {
final String json = hit.getSourceAsString();
final Map<String, Object> flattenMap = JsonFlattener.flattenAsMap(json);
flattenHits.add(flattenMap);
for (String key : flattenMap.keySet()) {
keys.add(key);
}
}
// Next : build the header of the table
//
final StringBuffer buffer = new StringBuffer();
for (String key : keys) {
buffer.append(key).append('\t');
}
buffer.replace(buffer.lastIndexOf("\t"), buffer.lastIndexOf("\t") + 1, "\n");
// Finally : build the result by using the key set
//
for (Map<String, Object> hit : flattenHits) {
for (String key : keys) {
final Object val = hit.get(key);
if (val != null) {
buffer.append(val);
}
buffer.append('\t');
}
buffer.replace(buffer.lastIndexOf("\t"), buffer.lastIndexOf("\t") + 1, "\n");
}
return buffer.toString();
}
}

171
elasticsearch/src/test/java/org/apache/zeppelin/elasticsearch/ElasticsearchInterpreterTest.java Normal file
View file

@ -0,0 +1,171 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.elasticsearch;
import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.junit.Assert.assertEquals;
import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.util.Arrays;
import java.util.Date;
import java.util.Properties;
import java.util.UUID;
import org.apache.commons.lang.math.RandomUtils;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.apache.zeppelin.interpreter.InterpreterResult.Code;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.client.Client;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.node.Node;
import org.elasticsearch.node.NodeBuilder;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
public class ElasticsearchInterpreterTest {
private static Client elsClient;
private static Node elsNode;
private static ElasticsearchInterpreter interpreter;
private static final String[] METHODS = { "GET", "PUT", "DELETE", "POST" };
private static final String[] STATUS = { "200", "404", "500", "403" };
private static final String ELS_CLUSTER_NAME = "zeppelin-elasticsearch-interpreter-test";
private static final String ELS_HOST = "localhost";
private static final String ELS_TRANSPORT_PORT = "10300";
private static final String ELS_HTTP_PORT = "10200";
private static final String ELS_PATH = "/tmp/els";
@BeforeClass
public static void populate() throws IOException {
final Settings settings = Settings.settingsBuilder()
.put("cluster.name", ELS_CLUSTER_NAME)
.put("network.host", ELS_HOST)
.put("http.port", ELS_HTTP_PORT)
.put("transport.tcp.port", ELS_TRANSPORT_PORT)
.put("path.home", ELS_PATH)
.build();
elsNode = NodeBuilder.nodeBuilder().settings(settings).node();
elsClient = elsNode.client();
for (int i = 0; i < 50; i++) {
elsClient.prepareIndex("logs", "http", "" + i)
.setRefresh(true)
.setSource(jsonBuilder()
.startObject()
.field("date", new Date())
.startObject("request")
.field("method", METHODS[RandomUtils.nextInt(METHODS.length)])
.field("url", "/zeppelin/" + UUID.randomUUID().toString())
.field("headers", Arrays.asList("Accept: *.*", "Host: apache.org"))
.endObject()
.field("status", STATUS[RandomUtils.nextInt(STATUS.length)])
)
.get();
}
final Properties props = new Properties();
props.put(ElasticsearchInterpreter.ELASTICSEARCH_HOST, ELS_HOST);
props.put(ElasticsearchInterpreter.ELASTICSEARCH_PORT, ELS_TRANSPORT_PORT);
props.put(ElasticsearchInterpreter.ELASTICSEARCH_CLUSTER_NAME, ELS_CLUSTER_NAME);
interpreter = new ElasticsearchInterpreter(props);
interpreter.open();
}
@AfterClass
public static void clean() {
if (interpreter != null) {
interpreter.close();
}
if (elsClient != null) {
elsClient.admin().indices().delete(new DeleteIndexRequest("logs")).actionGet();
elsClient.close();
}
if (elsNode != null) {
elsNode.close();
}
}
@Test
public void testCount() {
InterpreterResult res = interpreter.interpret("count /unknown", null);
assertEquals(Code.ERROR, res.code());
res = interpreter.interpret("count /logs", null);
assertEquals("50", res.message());
}
@Test
public void testGet() {
InterpreterResult res = interpreter.interpret("get /logs/http/unknown", null);
assertEquals(Code.ERROR, res.code());
res = interpreter.interpret("get /logs/http/10", null);
assertEquals(Code.SUCCESS, res.code());
}
@Test
public void testSearch() {
InterpreterResult res = interpreter.interpret("size 10\nsearch /logs *", null);
assertEquals(Code.SUCCESS, res.code());
res = interpreter.interpret("search /logs {{{hello}}}", null);
assertEquals(Code.ERROR, res.code());
res = interpreter.interpret("search /logs { \"query\": { \"match\": { \"status\": 500 } } }", null);
assertEquals(Code.SUCCESS, res.code());
res = interpreter.interpret("search /logs status:404", null);
assertEquals(Code.SUCCESS, res.code());
}
@Test
public void testIndex() {
InterpreterResult res = interpreter.interpret("index /logs { \"date\": \"" + new Date() + "\", \"method\": \"PUT\", \"status\": \"500\" }", null);
assertEquals(Code.ERROR, res.code());
res = interpreter.interpret("index /logs/http { \"date\": \"2015-12-06T14:54:23.368Z\", \"method\": \"PUT\", \"status\": \"500\" }", null);
assertEquals(Code.SUCCESS, res.code());
}
@Test
public void testDelete() {
InterpreterResult res = interpreter.interpret("delete /logs/http/unknown", null);
assertEquals(Code.ERROR, res.code());
res = interpreter.interpret("delete /logs/http/11", null);
assertEquals("11", res.message());
}
}

2
geode/pom.xml
View file

@ -54,7 +54,7 @@
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.1</version>
<version>1.3</version>
</dependency>
<dependency>

8
hive/pom.xml
View file

@ -59,7 +59,7 @@
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.1</version>
<version>1.3</version>
</dependency>
<dependency>
@ -87,6 +87,12 @@
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.4.190</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>

384
hive/src/main/java/org/apache/zeppelin/hive/HiveInterpreter.java
View file

@ -1,13 +1,12 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@ -15,6 +14,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.hive;
import java.sql.Connection;
@ -23,10 +23,14 @@ import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.interpreter.InterpreterContext;
import org.apache.zeppelin.interpreter.InterpreterPropertyBuilder;
@ -37,152 +41,319 @@ import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import static org.apache.commons.lang.StringUtils.containsIgnoreCase;
/**
* Hive interpreter for Zeppelin.
*/
public class HiveInterpreter extends Interpreter {
Logger logger = LoggerFactory.getLogger(HiveInterpreter.class);
int commandTimeOut = 600000;
static final String HIVESERVER_URL = "hive.hiveserver2.url";
static final String HIVESERVER_USER = "hive.hiveserver2.user";
static final String HIVESERVER_PASSWORD = "hive.hiveserver2.password";
static final String COMMON_KEY = "common";
static final String MAX_LINE_KEY = "max_count";
static final String MAX_LINE_DEFAULT = "1000";
static final String DEFAULT_KEY = "default";
static final String DRIVER_KEY = "driver";
static final String URL_KEY = "url";
static final String USER_KEY = "user";
static final String PASSWORD_KEY = "password";
static final String DOT = ".";
static final char TAB = '\t';
static final char NEWLINE = '\n';
static final String EXPLAIN_PREDICATE = "EXPLAIN ";
static final String TABLE_MAGIC_TAG = "%table ";
static final String UPDATE_COUNT_HEADER = "Update Count";
static final String COMMON_MAX_LINE = COMMON_KEY + DOT + MAX_LINE_KEY;
static final String DEFAULT_DRIVER = DEFAULT_KEY + DOT + DRIVER_KEY;
static final String DEFAULT_URL = DEFAULT_KEY + DOT + URL_KEY;
static final String DEFAULT_USER = DEFAULT_KEY + DOT + USER_KEY;
static final String DEFAULT_PASSWORD = DEFAULT_KEY + DOT + PASSWORD_KEY;
private final HashMap<String, Properties> propertiesMap;
private final Map<String, Statement> paragraphIdStatementMap;
private final Map<String, ArrayList<Connection>> propertyKeyUnusedConnectionListMap;
private final Map<String, Connection> paragraphIdConnectionMap;
static {
Interpreter.register(
"hql",
"hive",
HiveInterpreter.class.getName(),
new InterpreterPropertyBuilder()
.add(HIVESERVER_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.")
.add(HIVESERVER_USER, "hive", "The hive user")
.add(HIVESERVER_PASSWORD, "", "The password for the hive user").build());
"hql",
"hive",
HiveInterpreter.class.getName(),
new InterpreterPropertyBuilder()
.add(COMMON_MAX_LINE, MAX_LINE_DEFAULT, "Maximum line of results")
.add(DEFAULT_DRIVER, "org.apache.hive.jdbc.HiveDriver", "Hive JDBC driver")
.add(DEFAULT_URL, "jdbc:hive2://localhost:10000", "The URL for HiveServer2.")
.add(DEFAULT_USER, "hive", "The hive user")
.add(DEFAULT_PASSWORD, "", "The password for the hive user").build());
}
public HiveInterpreter(Properties property) {
super(property);
propertiesMap = new HashMap<>();
propertyKeyUnusedConnectionListMap = new HashMap<>();
paragraphIdStatementMap = new HashMap<>();
paragraphIdConnectionMap = new HashMap<>();
}
Connection jdbcConnection;
Exception exceptionOnConnect;
//Test only method
public Connection getJdbcConnection()
throws SQLException {
String url = getProperty(HIVESERVER_URL);
String user = getProperty(HIVESERVER_USER);
String password = getProperty(HIVESERVER_PASSWORD);
return DriverManager.getConnection(url, user, password);
public HashMap<String, Properties> getPropertiesMap() {
return propertiesMap;
}
@Override
public void open() {
logger.info("Jdbc open connection called!");
try {
String driverName = "org.apache.hive.jdbc.HiveDriver";
Class.forName(driverName);
} catch (ClassNotFoundException e) {
logger.error("Can not open connection", e);
exceptionOnConnect = e;
return;
logger.debug("property: {}", property);
for (String propertyKey : property.stringPropertyNames()) {
logger.debug("propertyKey: {}", propertyKey);
String[] keyValue = propertyKey.split("\\.", 2);
if (2 == keyValue.length) {
logger.debug("key: {}, value: {}", keyValue[0], keyValue[1]);
Properties prefixProperties;
if (propertiesMap.containsKey(keyValue[0])) {
prefixProperties = propertiesMap.get(keyValue[0]);
} else {
prefixProperties = new Properties();
propertiesMap.put(keyValue[0], prefixProperties);
}
prefixProperties.put(keyValue[1], property.getProperty(propertyKey));
}
}
try {
jdbcConnection = getJdbcConnection();
exceptionOnConnect = null;
logger.info("Successfully created Jdbc connection");
Set<String> removeKeySet = new HashSet<>();
for (String key : propertiesMap.keySet()) {
if (!COMMON_KEY.equals(key)) {
Properties properties = propertiesMap.get(key);
if (!properties.containsKey(DRIVER_KEY) || !properties.containsKey(URL_KEY)) {
logger.error("{} will be ignored. {}.{} and {}.{} is mandatory.",
key, DRIVER_KEY, key, key, URL_KEY);
removeKeySet.add(key);
}
}
}
catch (SQLException e) {
logger.error("Cannot open connection", e);
exceptionOnConnect = e;
for (String key : removeKeySet) {
propertiesMap.remove(key);
}
logger.debug("propertiesMap: {}", propertiesMap);
}
@Override
public void close() {
try {
if (jdbcConnection != null) {
jdbcConnection.close();
for (List<Connection> connectionList : propertyKeyUnusedConnectionListMap.values()) {
for (Connection c : connectionList) {
c.close();
}
}
}
catch (SQLException e) {
logger.error("Cannot close connection", e);
}
finally {
jdbcConnection = null;
exceptionOnConnect = null;
for (Statement statement : paragraphIdStatementMap.values()) {
statement.close();
}
paragraphIdStatementMap.clear();
for (Connection connection : paragraphIdConnectionMap.values()) {
connection.close();
}
paragraphIdConnectionMap.clear();
} catch (SQLException e) {
logger.error("Error while closing...", e);
}
}
Statement currentStatement;
private InterpreterResult executeSql(String sql) {
public Connection getConnection(String propertyKey) throws ClassNotFoundException, SQLException {
Connection connection = null;
if (propertyKeyUnusedConnectionListMap.containsKey(propertyKey)) {
ArrayList<Connection> connectionList = propertyKeyUnusedConnectionListMap.get(propertyKey);
if (0 != connectionList.size()) {
connection = propertyKeyUnusedConnectionListMap.get(propertyKey).remove(0);
if (null != connection && connection.isClosed()) {
connection.close();
connection = null;
}
}
}
if (null == connection) {
Properties properties = propertiesMap.get(propertyKey);
Class.forName(properties.getProperty(DRIVER_KEY));
String url = properties.getProperty(URL_KEY);
String user = properties.getProperty(USER_KEY);
String password = properties.getProperty(PASSWORD_KEY);
if (null != user && null != password) {
connection = DriverManager.getConnection(url, user, password);
} else {
connection = DriverManager.getConnection(url, properties);
}
}
return connection;
}
public Statement getStatement(String propertyKey, String paragraphId)
throws SQLException, ClassNotFoundException {
Connection connection;
if (paragraphIdConnectionMap.containsKey(paragraphId)) {
// Never enter for now.
connection = paragraphIdConnectionMap.get(paragraphId);
} else {
connection = getConnection(propertyKey);
}
Statement statement = connection.createStatement();
if (isStatementClosed(statement)) {
connection = getConnection(propertyKey);
statement = connection.createStatement();
}
paragraphIdConnectionMap.put(paragraphId, connection);
paragraphIdStatementMap.put(paragraphId, statement);
return statement;
}
private boolean isStatementClosed(Statement statement) {
try {
if (exceptionOnConnect != null) {
return new InterpreterResult(Code.ERROR, exceptionOnConnect.getMessage());
}
currentStatement = jdbcConnection.createStatement();
StringBuilder msg = null;
if (StringUtils.containsIgnoreCase(sql, "EXPLAIN ")) {
//return the explain as text, make this visual explain later
return statement.isClosed();
} catch (Throwable t) {
logger.debug("{} doesn't support isClosed method", statement);
return false;
}
}
public InterpreterResult executeSql(String propertyKey, String sql,
InterpreterContext interpreterContext) {
String paragraphId = interpreterContext.getParagraphId();
try {
Statement statement = getStatement(propertyKey, paragraphId);
statement.setMaxRows(getMaxResult());
StringBuilder msg;
if (containsIgnoreCase(sql, EXPLAIN_PREDICATE)) {
msg = new StringBuilder();
} else {
msg = new StringBuilder(TABLE_MAGIC_TAG);
}
else {
msg = new StringBuilder("%table ");
}
ResultSet res = currentStatement.executeQuery(sql);
ResultSet resultSet = null;
try {
ResultSetMetaData md = res.getMetaData();
for (int i = 1; i < md.getColumnCount() + 1; i++) {
if (i == 1) {
msg.append(md.getColumnName(i));
} else {
msg.append("\t" + md.getColumnName(i));
}
}
msg.append("\n");
while (res.next()) {
boolean isResultSetAvailable = statement.execute(sql);
if (isResultSetAvailable) {
resultSet = statement.getResultSet();
ResultSetMetaData md = resultSet.getMetaData();
for (int i = 1; i < md.getColumnCount() + 1; i++) {
msg.append(res.getString(i) + "\t");
if (i > 1) {
msg.append(TAB);
}
msg.append(md.getColumnName(i));
}
msg.append("\n");
msg.append(NEWLINE);
int displayRowCount = 0;
while (resultSet.next() && displayRowCount < getMaxResult()) {
for (int i = 1; i < md.getColumnCount() + 1; i++) {
msg.append(resultSet.getString(i));
if (i != md.getColumnCount()) {
msg.append(TAB);
}
}
msg.append(NEWLINE);
displayRowCount++;
}
} else {
// Response contains either an update count or there are no results.
int updateCount = statement.getUpdateCount();
msg.append(UPDATE_COUNT_HEADER).append(NEWLINE);
msg.append(updateCount).append(NEWLINE);
}
}
finally {
} finally {
try {
res.close();
currentStatement.close();
}
finally {
currentStatement = null;
if (resultSet != null) {
resultSet.close();
}
statement.close();
} finally {
moveConnectionToUnused(propertyKey, paragraphId);
}
}
InterpreterResult rett = new InterpreterResult(Code.SUCCESS, msg.toString());
return rett;
}
catch (SQLException ex) {
logger.error("Can not run " + sql, ex);
return new InterpreterResult(Code.SUCCESS, msg.toString());
} catch (SQLException | ClassNotFoundException ex) {
logger.error("Cannot run " + sql, ex);
return new InterpreterResult(Code.ERROR, ex.getMessage());
}
}
private void moveConnectionToUnused(String propertyKey, String paragraphId) {
if (paragraphIdConnectionMap.containsKey(paragraphId)) {
Connection connection = paragraphIdConnectionMap.remove(paragraphId);
if (null != connection) {
if (propertyKeyUnusedConnectionListMap.containsKey(propertyKey)) {
propertyKeyUnusedConnectionListMap.get(propertyKey).add(connection);
} else {
ArrayList<Connection> connectionList = new ArrayList<>();
connectionList.add(connection);
propertyKeyUnusedConnectionListMap.put(propertyKey, connectionList);
}
}
}
}
@Override
public InterpreterResult interpret(String cmd, InterpreterContext contextInterpreter) {
logger.info("Run SQL command '" + cmd + "'");
return executeSql(cmd);
String propertyKey = getPropertyKey(cmd);
if (null != propertyKey) {
cmd = cmd.substring(propertyKey.length() + 2);
} else {
propertyKey = DEFAULT_KEY;
}
cmd = cmd.trim();
logger.info("PropertyKey: {}, SQL command: '{}'", propertyKey, cmd);
return executeSql(propertyKey, cmd, contextInterpreter);
}
private int getMaxResult() {
return Integer.valueOf(
propertiesMap.get(COMMON_KEY).getProperty(MAX_LINE_KEY, MAX_LINE_DEFAULT));
}
public String getPropertyKey(String cmd) {
int firstLineIndex = cmd.indexOf("\n");
if (-1 == firstLineIndex) {
firstLineIndex = cmd.length();
}
int configStartIndex = cmd.indexOf("(");
int configLastIndex = cmd.indexOf(")");
if (configStartIndex != -1 && configLastIndex != -1
&& configLastIndex < firstLineIndex && configLastIndex < firstLineIndex) {
return cmd.substring(configStartIndex + 1, configLastIndex);
}
return null;
}
@Override
public void cancel(InterpreterContext context) {
if (currentStatement != null) {
try {
currentStatement.cancel();
}
catch (SQLException ex) {
}
finally {
currentStatement = null;
}
String paragraphId = context.getParagraphId();
try {
paragraphIdStatementMap.get(paragraphId).cancel();
} catch (SQLException e) {
logger.error("Error while cancelling...", e);
}
}
@ -198,13 +369,12 @@ public class HiveInterpreter extends Interpreter {
@Override
public Scheduler getScheduler() {
return SchedulerFactory.singleton().createOrGetFIFOScheduler(
HiveInterpreter.class.getName() + this.hashCode());
return SchedulerFactory.singleton().createOrGetParallelScheduler(
HiveInterpreter.class.getName() + this.hashCode(), 10);
}
@Override
public List<String> completion(String buf, int cursor) {
return null;
}
}

1748
hive/src/test/java/org/apache/zeppelin/hive/HiveInterpreterTest.java
View file

File diff suppressed because it is too large Load diff

12
pom.xml
View file

@ -101,6 +101,7 @@
<module>kylin</module>
<module>lens</module>
<module>cassandra</module>
<module>elasticsearch</module>
<module>zeppelin-web</module>
<module>zeppelin-server</module>
<module>zeppelin-distribution</module>
@ -702,4 +703,15 @@
</build>
</profile>
</profiles>
<distributionManagement>
<site>
<id>Website</id>
<url>${site_url}</url>
</site>
<repository>
<id>${repoid}</id>
<name>${reponame}</name>
<url>${repourl}</url>
</repository>
</distributionManagement>
</project>

2
shell/pom.xml
View file

@ -44,7 +44,7 @@
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.1</version>
<version>1.3</version>
</dependency>
<dependency>

2
spark/pom.xml
View file

@ -190,7 +190,7 @@
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.1</version>
<version>1.3</version>
</dependency>
<!-- to compile and test code.

93
spark/src/main/java/org/apache/zeppelin/spark/PySparkInterpreter.java
View file

@ -58,6 +58,11 @@ import org.apache.zeppelin.spark.dep.DependencyContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.Gson;
import com.google.gson.JsonElement;
import com.google.gson.JsonParseException;
import com.google.gson.JsonParser;
import py4j.GatewayServer;
/**
@ -368,12 +373,96 @@ public class PySparkInterpreter extends Interpreter implements ExecuteResultHand
return sparkInterpreter.getProgress(context);
}
@Override
public List<String> completion(String buf, int cursor) {
// not supported
return new LinkedList<String>();
if (buf.length() < cursor) {
cursor = buf.length();
}
String completionString = getCompletionTargetString(buf, cursor);
String completionCommand = "completion.getCompletion('" + completionString + "')";
//start code for completion
SparkInterpreter sparkInterpreter = getSparkInterpreter();
if (sparkInterpreter.getSparkVersion().isUnsupportedVersion() == false
&& pythonscriptRunning == false) {
return new LinkedList<String>();
}
outputStream.reset();
pythonInterpretRequest = new PythonInterpretRequest(completionCommand, "");
statementOutput = null;
synchronized (statementSetNotifier) {
statementSetNotifier.notify();
}
synchronized (statementFinishedNotifier) {
while (statementOutput == null) {
try {
statementFinishedNotifier.wait(1000);
} catch (InterruptedException e) {
// not working
logger.info("wait drop");
return new LinkedList<String>();
}
}
}
if (statementError) {
return new LinkedList<String>();
}
InterpreterResult completionResult = new InterpreterResult(Code.SUCCESS, statementOutput);
//end code for completion
Gson gson = new Gson();
return gson.fromJson(completionResult.message(), LinkedList.class);
}
private String getCompletionTargetString(String text, int cursor) {
String[] completionSeqCharaters = {" ", "\n", "\t"};
int completionEndPosition = cursor;
int completionStartPosition = cursor;
int indexOfReverseSeqPostion = cursor;
String resultCompletionText = "";
String completionScriptText = "";
try {
completionScriptText = text.substring(0, cursor);
}
catch (Exception e) {
logger.error(e.toString());
return null;
}
completionEndPosition = completionScriptText.length();
String tempReverseCompletionText = new StringBuilder(completionScriptText).reverse().toString();
for (String seqCharacter : completionSeqCharaters) {
indexOfReverseSeqPostion = tempReverseCompletionText.indexOf(seqCharacter);
if (indexOfReverseSeqPostion < completionStartPosition && indexOfReverseSeqPostion > 0) {
completionStartPosition = indexOfReverseSeqPostion;
}
}
if (completionStartPosition == completionEndPosition) {
completionStartPosition = 0;
}
else
{
completionStartPosition = completionEndPosition - completionStartPosition;
}
resultCompletionText = completionScriptText.substring(
completionStartPosition , completionEndPosition);
return resultCompletionText;
}
private SparkInterpreter getSparkInterpreter() {
InterpreterGroup intpGroup = getInterpreterGroup();
LazyOpenInterpreter lazy = null;

51
spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
View file

@ -580,11 +580,60 @@ public class SparkInterpreter extends Interpreter {
@Override
public List<String> completion(String buf, int cursor) {
if (buf.length() < cursor) {
cursor = buf.length();
}
String completionText = getCompletionTargetString(buf, cursor);
if (completionText == null) {
completionText = "";
cursor = completionText.length();
}
ScalaCompleter c = completor.completer();
Candidates ret = c.complete(buf, cursor);
Candidates ret = c.complete(completionText, cursor);
return scala.collection.JavaConversions.asJavaList(ret.candidates());
}
private String getCompletionTargetString(String text, int cursor) {
String[] completionSeqCharaters = {" ", "\n", "\t"};
int completionEndPosition = cursor;
int completionStartPosition = cursor;
int indexOfReverseSeqPostion = cursor;
String resultCompletionText = "";
String completionScriptText = "";
try {
completionScriptText = text.substring(0, cursor);
}
catch (Exception e) {
logger.error(e.toString());
return null;
}
completionEndPosition = completionScriptText.length();
String tempReverseCompletionText = new StringBuilder(completionScriptText).reverse().toString();
for (String seqCharacter : completionSeqCharaters) {
indexOfReverseSeqPostion = tempReverseCompletionText.indexOf(seqCharacter);
if (indexOfReverseSeqPostion < completionStartPosition && indexOfReverseSeqPostion > 0) {
completionStartPosition = indexOfReverseSeqPostion;
}
}
if (completionStartPosition == completionEndPosition) {
completionStartPosition = 0;
}
else
{
completionStartPosition = completionEndPosition - completionStartPosition;
}
resultCompletionText = completionScriptText.substring(
completionStartPosition , completionEndPosition);
return resultCompletionText;
}
public Object getValue(String name) {
Object ret = intp.valueOfTerm(name);
if (ret instanceof None) {

51
spark/src/main/resources/python/zeppelin_pyspark.py
View file

@ -15,7 +15,7 @@
# limitations under the License.
#
import sys, getopt, traceback
import sys, getopt, traceback, json, re
from py4j.java_gateway import java_import, JavaGateway, GatewayClient
from py4j.protocol import Py4JJavaError
@ -52,9 +52,9 @@ class PyZeppelinContext(dict):
def show(self, obj):
from pyspark.sql import DataFrame
if isinstance(obj, DataFrame):
print gateway.jvm.org.apache.zeppelin.spark.ZeppelinContext.showDF(self.z, obj._jdf)
print(gateway.jvm.org.apache.zeppelin.spark.ZeppelinContext.showDF(self.z, obj._jdf))
else:
print str(obj)
print(str(obj))
# By implementing special methods it makes operating on it more Pythonic
def __setitem__(self, key, item):
@ -107,6 +107,50 @@ class SparkVersion(object):
def isImportAllPackageUnderSparkSql(self):
return self.version >= self.SPARK_1_3_0
class PySparkCompletion:
def getGlobalCompletion(self):
objectDefList = []
try:
for completionItem in list(globals().iterkeys()):
objectDefList.append(completionItem)
except:
return None
else:
return objectDefList
def getMethodCompletion(self, text_value):
objectDefList = []
completion_target = text_value
try:
if len(completion_target) <= 0:
return None
if text_value[-1] == ".":
completion_target = text_value[:-1]
exec("%s = %s(%s)" % ("objectDefList", "dir", completion_target))
except:
return None
else:
return objectDefList
def getCompletion(self, text_value):
completionList = set()
globalCompletionList = self.getGlobalCompletion()
if globalCompletionList != None:
for completionItem in list(globalCompletionList):
completionList.add(completionItem)
if text_value != None:
objectCompletionList = self.getMethodCompletion(text_value)
if objectCompletionList != None:
for completionItem in list(objectCompletionList):
completionList.add(completionItem)
if len(completionList) <= 0:
print ""
else:
print json.dumps(filter(lambda x : not re.match("^__.*", x), list(completionList)))
output = Logger()
sys.stdout = output
@ -149,6 +193,7 @@ sc = SparkContext(jsc=jsc, gateway=gateway, conf=conf)
sqlc = SQLContext(sc, intp.getSQLContext())
sqlContext = sqlc
completion = PySparkCompletion()
z = PyZeppelinContext(intp.getZeppelinContext())
while True :

52
zeppelin-distribution/src/bin_license/LICENSE
View file

@ -1,7 +1,7 @@
(Apache 2.0) nvd3.js v1.1.15-beta (http://nvd3.org/) - https://github.com/novus/nvd3/blob/v1.1.15-beta/LICENSE.md
(Apache 2.0) gson v2.2 (com.google.code.gson:gson:jar:2.2 - https://github.com/google/gson) - https://github.com/google/gson/blob/gson-2.2/LICENSE
(Apache 2.0) Amazon Web Services SDK for Java v1.10.1 (https://aws.amazon.com/sdk-for-java/) - https://raw.githubusercontent.com/aws/aws-sdk-java/1.10.1/LICENSE.txt
(Apache 2.0) JavaEWAH v0.7.9 (https://github.com/lemire/javaewah) - https://github.com/lemire/javaewah/blob/master/LICENSE-2.0.txt
The following components are provided under Apache License.
@ -13,11 +13,11 @@ The following components are provided under Apache License.
(Apache 2.0) Apache Commons Compress (org.apache.commons:commons-compress:1.9 - http://commons.apache.org/proper/commons-compress/)
(Apache 2.0) Apache Commons Configuration (commons-configuration:commons-configuration:1.9 - http://commons.apache.org/configuration/)
(Apache 2.0) Apache Commons CLI (commons-cli:commons-cli:1.2 - http://commons.apache.org/cli/)
(Apache 2.0) Apache Commons Exec (commons-exec:commons-exec:1.1 - http://commons.apache.org/exec/)
(Apache 2.0) Apache Commons Exec (commons-exec:commons-exec:1.3 - http://commons.apache.org/exec/)
(Apache 2.0) Http Components (org.apache.httpcomponents:httpcore:4.3.3 - https://github.com/apache/httpclient)
(Apache 2.0) Http Components (org.apache.httpcomponents:httpclient:4.3.6 - https://github.com/apache/httpclient)
(Apache 2.0) Apache Commons Lang (org.apache.commons:commons-lang:2.5 - http://commons.apache.org/proper/commons-lang/)
(Apache 2.0) Apache Commons Lang 3 (org.apache.commons:commons-lang3:3.3.2 - http://commons.apache.org/proper/commons-lang/)
(Apache 2.0) Apache Commons Lang 3 (org.apache.commons:commons-lang3:3.4 - http://commons.apache.org/proper/commons-lang/)
(Apache 2.0) Apache Commons Math 3 (org.apache.commons:commons-math3:3.4.1 - http://commons.apache.org/proper/commons-math/)
(Apache 2.0) Apache Commons Net (commons-net:commons-net:2.2 - http://commons.apache.org/proper/commons-net/)
(Apache 2.0) Apache log4j (log4j:log4j:1.2.17 - http://logging.apache.org/log4j/1.2/)
@ -44,6 +44,7 @@ The following components are provided under Apache License.
(Apache 2.0) Apache Tajo (http://tajo.apache.org/)
(Apache 2.0) Apache Flink (http://flink.apache.org/)
(Apache 2.0) Apache Thrift (http://thrift.apache.org/)
(Apache 2.0) Apache Lucene (https://lucene.apache.org/)
(Apache 2.0) Apache Zookeeper (org.apache.zookeeper:zookeeper:jar:3.4.5 - http://zookeeper.apache.org/)
(Apache 2.0) Chill (com.twitter:chill-java:jar:0.5.0 - https://github.com/twitter/chill/)
(Apache 2.0) Codehaus Plexus (org.codehaus.plexus:plexus:jar:1.5.6 - https://codehaus-plexus.github.io/)
@ -66,6 +67,31 @@ The following components are provided under Apache License.
(Apache 2.0) lz4-java (net.jpountz.lz4:lz4:jar:1.3.0 - https://github.com/jpountz/lz4-java)
(Apache 2.0) RoaringBitmap (org.roaringbitmap:RoaringBitmap:jar:0.4.5 - https://github.com/lemire/RoaringBitmap)
(Apache 2.0) json4s (org.json4s:json4s-ast_2.10:jar:3.2.10 - https://github.com/json4s/json4s)
(Apache 2.0) HPPC Collections (com.carrotsearch:hppc:0.7.1 - http://labs.carrotsearch.com/hppc.html/hppc)
(Apache 2.0) Jackson-dataformat-CBOR (com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.6.2 - http://wiki.fasterxml.com/JacksonForCbor)
(Apache 2.0) Jackson-dataformat-Smile (com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.6.2 - http://wiki.fasterxml.com/JacksonForSmile)
(Apache 2.0) Jackson-dataformat-YAML (com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.6.2 - https://github.com/FasterXML/jackson)
(Apache 2.0) json-flattener (com.github.wnameless:json-flattener:0.1.1 - https://github.com/wnameless/json-flattener)
(Apache 2.0) Spatial4J (com.spatial4j:spatial4j:0.4.1 - https://github.com/spatial4j/spatial4j)
(Apache 2.0) T-Digest (com.tdunning:t-digest:3.0 - https://github.com/tdunning/t-digest)
(Apache 2.0) Netty (io.netty:netty:3.10.5.Final - http://netty.io/)
(Apache 2.0) Lucene Common Analyzers (org.apache.lucene:lucene-analyzers-common:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-analyzers-common)
(Apache 2.0) Lucene Memory (org.apache.lucene:lucene-backward-codecs:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-backward-codecs)
(Apache 2.0) Lucene Core (org.apache.lucene:lucene-core:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-core)
(Apache 2.0) Lucene Grouping (org.apache.lucene:lucene-grouping:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-grouping)
(Apache 2.0) Lucene Highlighter (org.apache.lucene:lucene-highlighter:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-highlighter)
(Apache 2.0) Lucene Join (org.apache.lucene:lucene-join:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-join)
(Apache 2.0) Lucene Memory (org.apache.lucene:lucene-memory:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-memory)
(Apache 2.0) Lucene Miscellaneous (org.apache.lucene:lucene-misc:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-misc)
(Apache 2.0) Lucene Queries (org.apache.lucene:lucene-queries:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-queries)
(Apache 2.0) Lucene QueryParsers (org.apache.lucene:lucene-queryparser:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-queryparser)
(Apache 2.0) Lucene Sandbox (org.apache.lucene:lucene-sandbox:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-sandbox)
(Apache 2.0) Lucene Spatial (org.apache.lucene:lucene-spatial:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-spatial)
(Apache 2.0) Lucene Spatial 3D (org.apache.lucene:lucene-spatial3d:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-spatial3d)
(Apache 2.0) Lucene Suggest (org.apache.lucene:lucene-suggest:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-suggest)
(Apache 2.0) Elasticsearch: Core (org.elasticsearch:elasticsearch:2.1.0 - http://nexus.sonatype.org/oss-repository-hosting.html/parent/elasticsearch)
(Apache 2.0) Joda convert (org.joda:joda-convert:1.2 - http://joda-convert.sourceforge.net)
(Apache 2.0) SnakeYAML (org.yaml:snakeyaml:1.15 - http://www.snakeyaml.org)
@ -81,6 +107,7 @@ The text of each license is also included at licenses/LICENSE-[project]-[version
(The MIT License) es5-shim v3.1.0 (https://github.com/es-shims/es5-shim) - https://github.com/es-shims/es5-shim/blob/v3.1.0/LICENSE
(The MIT License) bootstrap v3.2.0 (http://getbootstrap.com/) - https://github.com/twbs/bootstrap/blob/v3.2.0/LICENSE
(The MIT License) UI Bootstrap v0.13.0 (http://angular-ui.github.io/bootstrap/) - https://github.com/angular-ui/bootstrap/blob/0.13.0/LICENSE
(The MIT License) bootstrap3-dialog v1.34.7 (https://github.com/nakupanda/bootstrap3-dialog/tree/v1.34.7) - https://github.com/nakupanda/bootstrap3-dialog/tree/v1.34.7
(The MIT License) Angular Websocket v1.0.13 (http://angularclass.github.io/angular-websocket/) - https://github.com/AngularClass/angular-websocket/blob/v1.0.13/LICENSE
(The MIT License) UI.Ace v0.1.1 (http://angularclass.github.io/angular-websocket/) - https://github.com/angular-ui/ui-ace/blob/master/LICENSE
(The MIT License) jquery.scrollTo v1.4.13 (https://github.com/flesler/jquery.scrollTo) - https://github.com/flesler/jquery.scrollTo/blob/1.4.13/LICENSE
@ -88,7 +115,8 @@ The text of each license is also included at licenses/LICENSE-[project]-[version
(The MIT License) perfect-scrollbar v0.5.4 (http://noraesae.github.io/perfect-scrollbar/) - https://github.com/noraesae/perfect-scrollbar/tree/0.5.4
(The MIT License) ng-sortable v1.1.9 (https://github.com/a5hik/ng-sortable) - https://github.com/a5hik/ng-sortable/blob/1.1.9/LICENSE
(The MIT License) angular-elastic v2.4.2 (https://github.com/monospaced/angular-elastic) - https://github.com/monospaced/angular-elastic/blob/v2.4.2/LICENCE.txt
(The MIT License) angular-elastic-input v2.0.1 (https://github.com/jacek-pulit/angular-elastic-input) - https://github.com/jacek-pulit/angular-elastic-input/blob/v2.0.1/LICENSE
(The MIT License) angular-elastic-input v2.2.0 (https://github.com/jacek-pulit/angular-elastic-input) - https://github.com/jacek-pulit/angular-elastic-input/blob/v2.2.0/LICENSE
(The MIT License) ng-focus-if v1.0.2 (https://github.com/hiebj/ng-focus-if) - https://github.com/hiebj/ng-focus-if/blob/v1.0.2/LICENSE
(The MIT License) angular-xeditable v0.1.8 (http://vitalets.github.io/angular-xeditable/) - https://github.com/vitalets/angular-xeditable/tree/0.1.8
(The MIT License) lodash v3.9.3 (https://lodash.com/) - https://github.com/lodash/lodash/blob/3.9.3/LICENSE.txt
(The MIT License) angular-filter v0.5.4 (https://github.com/a8m/angular-filter) - https://github.com/a8m/angular-filter/blob/v0.5.4/license.md
@ -102,7 +130,8 @@ The following components are provided under the MIT License.
(The MIT License) Objenesis (org.objenesis:objenesis:2.1 - https://github.com/easymock/objenesis) - Copyright (c) 2006-2015 the original author and authors
(The MIT License) JCL 1.1.1 implemented over SLF4J (org.slf4j:jcl-over-slf4j:1.7.5 - http://www.slf4j.org)
(The MIT License) JUL to SLF4J bridge (org.slf4j:jul-to-slf4j:1.7.5 - http://www.slf4j.org)
(The MIT License) angular-resource (angular-resource - https://github.com/angular/angular.js/tree/master/src/ngResource)
(The MIT License) minimal-json (com.eclipsesource.minimal-json:minimal-json:0.9.4 - https://github.com/ralfstx/minimal-json)
========================================================================
@ -115,13 +144,15 @@ The text of each license is also included at licenses/LICENSE-[project]-[version
(BSD 3 Clause) d3 v2.10.2 (https://d3js.org/) - https://github.com/mbostock/d3/blob/v2.10.2/LICENSE
(BSD 3 Clause) ace-builds v1.1.9 (https://github.com/ajaxorg/ace-builds) - https://github.com/ajaxorg/ace-builds/blob/v1.1.9/LICENSE
(BSD 3 Clause) Ace v1.1.9 (http://ace.c9.io/) - https://github.com/ajaxorg/ace/blob/v1.1.9/LICENSE
(BSD Style) dom4j v1.6.1 (http://www.dom4j.org) - https://github.com/dom4j/dom4j/blob/dom4j_1_6_1/LICENSE.txt
(BSD Style) dom4j v1.6.1 (http://www.dom4j.org) - https://github.com/dom4j/dom4j/blob/dom4j_1_6_1/LICENSE.txt
(BSD Style) JSch v0.1.53 (http://www.jcraft.com) - http://www.jcraft.com/jsch/LICENSE.txt
(BSD 3 Clause) highlightjs v8.4.0 (https://highlightjs.org/) - https://github.com/isagalaev/highlight.js/blob/8.4/LICENSE
The following components are provided under the BSD-style License.
(New BSD License) JGit (org.eclipse.jgit:org.eclipse.jgit:jar:4.1.1.201511131810-r - https://eclipse.org/jgit/)
(New BSD License) Kryo (com.esotericsoftware.kryo:kryo:2.21 - http://code.google.com/p/kryo/)
(New BSD License) MinLog (com.esotericsoftware.minlog:minlog:1.2 - http://code.google.com/p/minlog/)
(New BSD License) ReflectASM (com.esotericsoftware.reflectasm:reflectasm:1.07 - http://code.google.com/p/reflectasm/)
@ -155,7 +186,7 @@ EPL license
The following components are provided under the EPL License.
(EPL 1.0) Aether (org.sonatype.aether - http://www.eclipse.org/aether/)
(EPL 1.0) JDT Annotations For Enhanced Null Analysis (org.eclipse.jdt:org.eclipse.jdt.annotation:1.1.0 - https://repo.eclipse.org/content/repositories/eclipse-releases/org/eclipse/jdt/org.eclipse.jdt.annotation)
========================================================================
@ -174,3 +205,10 @@ See licenses/LICENSE-postgresql
========================================================================
Creative Commons CC0 (http://creativecommons.org/publicdomain/zero/1.0/)
========================================================================
(CC0 1.0 Universal) JSR166e (com.twitter:jsr166e:1.1.0 - http://github.com/twitter/jsr166e)
(Public Domain, per Creative Commons CC0) HdrHistogram (org.hdrhistogram:HdrHistogram:2.1.6 - http://hdrhistogram.github.io/HdrHistogram/)

62
zeppelin-interpreter/pom.xml
View file

@ -36,6 +36,10 @@
<description>Zeppelin Interpreter</description>
<url>http://zeppelin.incubator.apache.org</url>
<properties>
<commons-lang.version>3.4</commons-lang.version>
</properties>
<dependencies>
<dependency>
@ -51,7 +55,7 @@
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.1</version>
<version>1.3</version>
</dependency>
<dependency>
@ -87,10 +91,64 @@
<version>1.9.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.3.2</version>
<version>${commons-lang.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.rat</groupId>
<artifactId>apache-rat-plugin</artifactId>
<configuration>
<excludes>
<exclude>**/.idea/</exclude>
<exclude>**/*.iml</exclude>
<exclude>.gitignore</exclude>
<exclude>**/.settings/*</exclude>
<exclude>**/.classpath</exclude>
<exclude>**/.project</exclude>
<exclude>**/target/**</exclude>
<exclude>*.md</exclude>
<exclude>dependency-reduced-pom.xml</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<configuration>
<artifactSet>
<includes>
<include>*:*</include>
</includes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

17
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterContext.java
View file

@ -27,6 +27,21 @@ import org.apache.zeppelin.display.GUI;
* Interpreter context
*/
public class InterpreterContext {
private static final ThreadLocal<InterpreterContext> threadIC =
new ThreadLocal<InterpreterContext>();
public static InterpreterContext get() {
return threadIC.get();
}
public static void set(InterpreterContext ic) {
threadIC.set(ic);
}
public static void remove() {
threadIC.remove();
}
private final String noteId;
private final String paragraphTitle;
private final String paragraphId;
@ -55,7 +70,7 @@ public class InterpreterContext {
this.runners = runners;
}
public String getNoteId() {
return noteId;
}

3
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java
View file

@ -193,6 +193,7 @@ public class RemoteInterpreter extends Interpreter {
@Override
public InterpreterResult interpret(String st, InterpreterContext context) {
logger.debug("st: {}", st);
FormType form = getFormType();
RemoteInterpreterProcess interpreterProcess = getInterpreterProcess();
Client client = null;
@ -333,7 +334,7 @@ public class RemoteInterpreter extends Interpreter {
return null;
} else {
return SchedulerFactory.singleton().createOrGetRemoteScheduler(
"remoteinterpreter_" + interpreterProcess.hashCode(), getInterpreterProcess(),
"remoteinterpreter_" + interpreterProcess.hashCode(), interpreterProcess,
maxConcurrency);
}
}

10
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterServer.java
View file

@ -203,6 +203,7 @@ public class RemoteInterpreterServer
@Override
public RemoteInterpreterResult interpret(String className, String st,
RemoteInterpreterContext interpreterContext) throws TException {
logger.debug("st: {}", st);
Interpreter intp = getInterpreter(className);
InterpreterContext context = convert(interpreterContext);
@ -294,8 +295,13 @@ public class RemoteInterpreterServer
@Override
protected Object jobRun() throws Throwable {
InterpreterResult result = interpreter.interpret(script, context);
return result;
try {
InterpreterContext.set(context);
InterpreterResult result = interpreter.interpret(script, context);
return result;
} finally {
InterpreterContext.remove();
}
}
@Override

1
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java
View file

@ -37,7 +37,6 @@ import org.slf4j.LoggerFactory;
* and saving/loading jobs from disk.
* Changing/adding/deleting non transitive field name need consideration of that.
*
* @author Leemoonsoo
*/
public abstract class Job {
/**

37
zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/InterpreterContextTest.java Normal file
View file

@ -0,0 +1,37 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.interpreter;
import static org.junit.Assert.*;
import org.junit.Test;
public class InterpreterContextTest {
@Test
public void testThreadLocal() {
assertNull(InterpreterContext.get());
InterpreterContext.set(new InterpreterContext(null, null, null, null, null, null, null, null));
assertNotNull(InterpreterContext.get());
InterpreterContext.remove();
assertNull(InterpreterContext.get());
}
}

79
zeppelin-server/pom.xml
View file

@ -197,42 +197,6 @@
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.44.0</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-android-driver</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
<exclusion>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency> <!-- because there are two of them above -->
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.4.01</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.6</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-servlet</artifactId>
@ -250,6 +214,48 @@
<artifactId>scala-library</artifactId>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.48.2</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-android-driver</artifactId>
</exclusion>
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
<exclusion>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-exec</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency> <!-- because there are two of them above -->
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
<version>1.4.01</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.6</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.10</artifactId>
@ -261,6 +267,7 @@
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.9.0</version>
<scope>test</scope>
</dependency>
</dependencies>

287
zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java
View file

@ -22,19 +22,31 @@ import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import javax.ws.rs.*;
import javax.ws.rs.DELETE;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.Status;
import org.apache.commons.lang3.StringUtils;
import org.apache.zeppelin.interpreter.InterpreterSetting;
import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.Notebook;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.rest.message.CronRequest;
import org.apache.zeppelin.rest.message.InterpreterSettingListForNoteBind;
import org.apache.zeppelin.rest.message.NewInterpreterSettingRequest;
import org.apache.zeppelin.rest.message.NewNotebookRequest;
import org.apache.zeppelin.rest.message.NewParagraphRequest;
import org.apache.zeppelin.rest.message.RunParagraphWithParametersRequest;
import org.apache.zeppelin.search.SearchService;
import org.apache.zeppelin.server.JsonResponse;
import org.apache.zeppelin.server.ZeppelinServer;
import org.apache.zeppelin.socket.NotebookServer;
import org.quartz.CronExpression;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -47,17 +59,18 @@ import com.google.gson.reflect.TypeToken;
@Path("/notebook")
@Produces("application/json")
public class NotebookRestApi {
Logger logger = LoggerFactory.getLogger(NotebookRestApi.class);
private static final Logger LOG = LoggerFactory.getLogger(NotebookRestApi.class);
Gson gson = new Gson();
private Notebook notebook;
private NotebookServer notebookServer;
private SearchService notebookIndex;
public NotebookRestApi() {}
public NotebookRestApi(Notebook notebook, NotebookServer notebookServer) {
public NotebookRestApi(Notebook notebook, NotebookServer notebookServer, SearchService search) {
this.notebook = notebook;
this.notebookServer = notebookServer;
this.notebookIndex = search;
}
/**
@ -69,7 +82,7 @@ public class NotebookRestApi {
public Response bind(@PathParam("noteId") String noteId, String req) throws IOException {
List<String> settingIdList = gson.fromJson(req, new TypeToken<List<String>>(){}.getType());
notebook.bindInterpretersToNote(noteId, settingIdList);
return new JsonResponse(Status.OK).build();
return new JsonResponse<>(Status.OK).build();
}
/**
@ -112,14 +125,25 @@ public class NotebookRestApi {
);
}
}
return new JsonResponse(Status.OK, "", settingList).build();
return new JsonResponse<>(Status.OK, "", settingList).build();
}
@GET
@Path("/")
public Response getNotebookList() throws IOException {
List<Map<String, String>> notesInfo = notebookServer.generateNotebooksInfo();
return new JsonResponse(Status.OK, "", notesInfo ).build();
return new JsonResponse<>(Status.OK, "", notesInfo ).build();
}
@GET
@Path("{notebookId}")
public Response getNotebook(@PathParam("notebookId") String notebookId) throws IOException {
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
return new JsonResponse<>(Status.OK, "", note).build();
}
/**
@ -131,11 +155,19 @@ public class NotebookRestApi {
@POST
@Path("/")
public Response createNote(String message) throws IOException {
logger.info("Create new notebook by JSON {}" , message);
LOG.info("Create new notebook by JSON {}" , message);
NewNotebookRequest request = gson.fromJson(message,
NewNotebookRequest.class);
Note note = notebook.createNote();
note.addParagraph(); // it's an empty note. so add one paragraph
List<NewParagraphRequest> initialParagraphs = request.getParagraphs();
if (initialParagraphs != null) {
for (NewParagraphRequest paragraphRequest : initialParagraphs) {
Paragraph p = note.addParagraph();
p.setTitle(paragraphRequest.getTitle());
p.setText(paragraphRequest.getText());
}
}
note.addParagraph(); // add one paragraph to the last
String noteName = request.getName();
if (noteName.isEmpty()) {
noteName = "Note " + note.getId();
@ -144,7 +176,7 @@ public class NotebookRestApi {
note.persist();
notebookServer.broadcastNote(note);
notebookServer.broadcastNoteList();
return new JsonResponse(Status.CREATED, "", note.getId() ).build();
return new JsonResponse<>(Status.CREATED, "", note.getId() ).build();
}
/**
@ -156,7 +188,7 @@ public class NotebookRestApi {
@DELETE
@Path("{notebookId}")
public Response deleteNote(@PathParam("notebookId") String notebookId) throws IOException {
logger.info("Delete notebook {} ", notebookId);
LOG.info("Delete notebook {} ", notebookId);
if (!(notebookId.isEmpty())) {
Note note = notebook.getNote(notebookId);
if (note != null) {
@ -164,25 +196,246 @@ public class NotebookRestApi {
}
}
notebookServer.broadcastNoteList();
return new JsonResponse(Status.OK, "").build();
return new JsonResponse<>(Status.OK, "").build();
}
/**
* Clone note REST API
* @param
* @return JSON with status.CREATED
* @throws IOException
* @throws IOException, CloneNotSupportedException, IllegalArgumentException
*/
@POST
@Path("{notebookId}")
public Response cloneNote(@PathParam("notebookId") String notebookId, String message) throws
IOException, CloneNotSupportedException, IllegalArgumentException {
logger.info("clone notebook by JSON {}" , message);
LOG.info("clone notebook by JSON {}" , message);
NewNotebookRequest request = gson.fromJson(message,
NewNotebookRequest.class);
String newNoteName = request.getName();
Note newNote = notebook.cloneNote(notebookId, newNoteName);
notebookServer.broadcastNote(newNote);
notebookServer.broadcastNoteList();
return new JsonResponse(Status.CREATED, "", newNote.getId()).build();
return new JsonResponse<>(Status.CREATED, "", newNote.getId()).build();
}
/**
* Run notebook jobs REST API
* @param
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("job/{notebookId}")
public Response runNoteJobs(@PathParam("notebookId") String notebookId) throws
IOException, IllegalArgumentException {
LOG.info("run notebook jobs {} ", notebookId);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
note.runAll();
return new JsonResponse<>(Status.OK).build();
}
/**
* Stop(delete) notebook jobs REST API
* @param
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@DELETE
@Path("job/{notebookId}")
public Response stopNoteJobs(@PathParam("notebookId") String notebookId) throws
IOException, IllegalArgumentException {
LOG.info("stop notebook jobs {} ", notebookId);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
for (Paragraph p : note.getParagraphs()) {
if (!p.isTerminated()) {
p.abort();
}
}
return new JsonResponse<>(Status.OK).build();
}
/**
* Get notebook job status REST API
* @param
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@GET
@Path("job/{notebookId}")
public Response getNoteJobStatus(@PathParam("notebookId") String notebookId) throws
IOException, IllegalArgumentException {
LOG.info("get notebook job status.");
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
return new JsonResponse<>(Status.OK, null, note.generateParagraphsInfo()).build();
}
/**
* Run paragraph job REST API
*
* @param message - JSON with params if user wants to update dynamic form's value
* null, empty string, empty json if user doesn't want to update
*
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("job/{notebookId}/{paragraphId}")
public Response runParagraph(@PathParam("notebookId") String notebookId,
@PathParam("paragraphId") String paragraphId,
String message) throws
IOException, IllegalArgumentException {
LOG.info("run paragraph job {} {} {}", notebookId, paragraphId, message);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
Paragraph paragraph = note.getParagraph(paragraphId);
if (paragraph == null) {
return new JsonResponse<>(Status.NOT_FOUND, "paragraph not found.").build();
}
// handle params if presented
if (!StringUtils.isEmpty(message)) {
RunParagraphWithParametersRequest request = gson.fromJson(message,
RunParagraphWithParametersRequest.class);
Map<String, Object> paramsForUpdating = request.getParams();
if (paramsForUpdating != null) {
paragraph.settings.getParams().putAll(paramsForUpdating);
note.persist();
}
}
note.run(paragraph.getId());
return new JsonResponse<>(Status.OK).build();
}
/**
* Stop(delete) paragraph job REST API
* @param
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@DELETE
@Path("job/{notebookId}/{paragraphId}")
public Response stopParagraph(@PathParam("notebookId") String notebookId,
@PathParam("paragraphId") String paragraphId) throws
IOException, IllegalArgumentException {
LOG.info("stop paragraph job {} ", notebookId);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
Paragraph p = note.getParagraph(paragraphId);
if (p == null) {
return new JsonResponse<>(Status.NOT_FOUND, "paragraph not found.").build();
}
p.abort();
return new JsonResponse<>(Status.OK).build();
}
/**
* Register cron job REST API
* @param message - JSON with cron expressions.
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("cron/{notebookId}")
public Response registerCronJob(@PathParam("notebookId") String notebookId, String message) throws
IOException, IllegalArgumentException {
LOG.info("Register cron job note={} request cron msg={}", notebookId, message);
CronRequest request = gson.fromJson(message,
CronRequest.class);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
if (!CronExpression.isValidExpression(request.getCronString())) {
return new JsonResponse<>(Status.BAD_REQUEST, "wrong cron expressions.").build();
}
Map<String, Object> config = note.getConfig();
config.put("cron", request.getCronString());
note.setConfig(config);
notebook.refreshCron(note.id());
return new JsonResponse<>(Status.OK).build();
}
/**
* Remove cron job REST API
* @param
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@DELETE
@Path("cron/{notebookId}")
public Response removeCronJob(@PathParam("notebookId") String notebookId) throws
IOException, IllegalArgumentException {
LOG.info("Remove cron job note {}", notebookId);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
Map<String, Object> config = note.getConfig();
config.put("cron", null);
note.setConfig(config);
notebook.refreshCron(note.id());
return new JsonResponse<>(Status.OK).build();
}
/**
* Get cron job REST API
* @param
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@GET
@Path("cron/{notebookId}")
public Response getCronJob(@PathParam("notebookId") String notebookId) throws
IOException, IllegalArgumentException {
LOG.info("Get cron job note {}", notebookId);
Note note = notebook.getNote(notebookId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
return new JsonResponse<>(Status.OK, note.getConfig().get("cron")).build();
}
/**
* Search for a Notes
*/
@GET
@Path("search")
public Response search(@QueryParam("q") String queryTerm) {
LOG.info("Searching notebooks for: {}", queryTerm);
List<Map<String, String>> notebooksFound = notebookIndex.query(queryTerm);
LOG.info("{} notbooks found", notebooksFound.size());
return new JsonResponse<>(Status.OK, notebooksFound).build();
}
}

1
zeppelin-server/src/main/java/org/apache/zeppelin/rest/ZeppelinRestApi.java
View file

@ -24,7 +24,6 @@ import javax.ws.rs.core.Response;
/**
* Zeppelin root rest api endpoint.
*
* @author anthonycorbacho
* @since 0.3.4
*/
@Path("/")

38
zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/CronRequest.java Normal file
View file

@ -0,0 +1,38 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest.message;
import java.util.Map;
import org.apache.zeppelin.interpreter.InterpreterOption;
/**
* CronRequest rest api request message
*
*/
public class CronRequest {
String cron;
public CronRequest (){
}
public String getCronString() {
return cron;
}
}

6
zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/NewNotebookRequest.java
View file

@ -17,6 +17,7 @@
package org.apache.zeppelin.rest.message;
import java.util.List;
import java.util.Map;
import org.apache.zeppelin.interpreter.InterpreterOption;
@ -27,6 +28,7 @@ import org.apache.zeppelin.interpreter.InterpreterOption;
*/
public class NewNotebookRequest {
String name;
List<NewParagraphRequest> paragraphs;
public NewNotebookRequest (){
@ -35,4 +37,8 @@ public class NewNotebookRequest {
public String getName() {
return name;
}
public List<NewParagraphRequest> getParagraphs() {
return paragraphs;
}
}

41
zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/NewParagraphRequest.java Normal file
View file

@ -0,0 +1,41 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest.message;
/**
* NewParagraphRequest rest api request message
*
* It is used for NewNotebookRequest with initial paragraphs
*
*/
public class NewParagraphRequest {
String title;
String text;
public NewParagraphRequest() {
}
public String getTitle() {
return title;
}
public String getText() {
return text;
}
}

35
zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/RunParagraphWithParametersRequest.java Normal file
View file

@ -0,0 +1,35 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest.message;
import java.util.Map;
/**
* RunParagraphWithParametersRequest rest api request message
*/
public class RunParagraphWithParametersRequest {
Map<String, Object> params;
public RunParagraphWithParametersRequest() {
}
public Map<String, Object> getParams() {
return params;
}
}

102
zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
View file

@ -19,14 +19,12 @@ package org.apache.zeppelin.server;
import java.io.File;
import java.io.IOException;
import java.lang.reflect.Constructor;
import java.util.EnumSet;
import java.util.HashSet;
import java.util.Set;
import javax.net.ssl.SSLContext;
import javax.servlet.DispatcherType;
import javax.servlet.Servlet;
import javax.ws.rs.core.Application;
import org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet;
@ -40,6 +38,8 @@ import org.apache.zeppelin.rest.InterpreterRestApi;
import org.apache.zeppelin.rest.NotebookRestApi;
import org.apache.zeppelin.rest.ZeppelinRestApi;
import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.apache.zeppelin.search.SearchService;
import org.apache.zeppelin.search.LuceneSearch;
import org.apache.zeppelin.socket.NotebookServer;
import org.eclipse.jetty.server.AbstractConnector;
import org.eclipse.jetty.server.Handler;
@ -60,59 +60,67 @@ import org.slf4j.LoggerFactory;
/**
* Main class of Zeppelin.
*
* @author Leemoonsoo
*
*/
public class ZeppelinServer extends Application {
private static final Logger LOG = LoggerFactory.getLogger(ZeppelinServer.class);
private SchedulerFactory schedulerFactory;
public static Notebook notebook;
public static Server jettyWebServer;
public static NotebookServer notebookWsServer;
public static NotebookServer notebookServer;
public static Server jettyServer;
private SchedulerFactory schedulerFactory;
private InterpreterFactory replFactory;
private NotebookRepo notebookRepo;
private SearchService notebookIndex;
public static void main(String[] args) throws Exception {
public ZeppelinServer() throws Exception {
ZeppelinConfiguration conf = ZeppelinConfiguration.create();
this.schedulerFactory = new SchedulerFactory();
this.replFactory = new InterpreterFactory(conf, notebookWsServer);
this.notebookRepo = new NotebookRepoSync(conf);
this.notebookIndex = new LuceneSearch();
notebook = new Notebook(conf,
notebookRepo, schedulerFactory, replFactory, notebookWsServer, notebookIndex);
}
public static void main(String[] args) throws InterruptedException {
ZeppelinConfiguration conf = ZeppelinConfiguration.create();
conf.setProperty("args", args);
jettyServer = setupJettyServer(conf);
// REST api
final ServletContextHandler restApi = setupRestApiContextHandler(conf);
final ServletContextHandler restApiContext = setupRestApiContextHandler(conf);
// Notebook server
final ServletContextHandler notebook = setupNotebookServer(conf);
final ServletContextHandler notebookContext = setupNotebookServer(conf);
// Web UI
final WebAppContext webApp = setupWebAppContext(conf);
// add all handlers
ContextHandlerCollection contexts = new ContextHandlerCollection();
contexts.setHandlers(new Handler[]{restApi, notebook, webApp});
jettyServer.setHandler(contexts);
contexts.setHandlers(new Handler[]{restApiContext, notebookContext, webApp});
LOG.info("Start zeppelin server");
jettyWebServer = setupJettyServer(conf);
jettyWebServer.setHandler(contexts);
LOG.info("Starting zeppelin server");
try {
jettyServer.start();
jettyWebServer.start(); //Instantiates ZeppelinServer
} catch (Exception e) {
LOG.error("Error while running jettyServer", e);
System.exit(-1);
}
LOG.info("Started zeppelin server");
LOG.info("Done, zeppelin server started");
Runtime.getRuntime().addShutdownHook(new Thread(){
@Override public void run() {
LOG.info("Shutting down Zeppelin Server ... ");
try {
jettyServer.stop();
ZeppelinServer.notebook.getInterpreterFactory().close();
jettyWebServer.stop();
notebook.getInterpreterFactory().close();
notebook.close();
} catch (Exception e) {
LOG.error("Error while stopping servlet container", e);
}
@ -131,18 +139,15 @@ public class ZeppelinServer extends Application {
System.exit(0);
}
jettyServer.join();
jettyWebServer.join();
ZeppelinServer.notebook.getInterpreterFactory().close();
}
private static Server setupJettyServer(ZeppelinConfiguration conf)
throws Exception {
private static Server setupJettyServer(ZeppelinConfiguration conf) {
AbstractConnector connector;
if (conf.useSsl()) {
connector = new SslSelectChannelConnector(getSslContextFactory(conf));
}
else {
} else {
connector = new SelectChannelConnector();
}
@ -159,11 +164,9 @@ public class ZeppelinServer extends Application {
return server;
}
private static ServletContextHandler setupNotebookServer(ZeppelinConfiguration conf)
throws Exception {
notebookServer = new NotebookServer();
final ServletHolder servletHolder = new ServletHolder(notebookServer);
private static ServletContextHandler setupNotebookServer(ZeppelinConfiguration conf) {
notebookWsServer = new NotebookServer();
final ServletHolder servletHolder = new ServletHolder(notebookWsServer);
servletHolder.setInitParameter("maxTextMessageSize", "1024000");
final ServletContextHandler cxfContext = new ServletContextHandler(
@ -177,9 +180,8 @@ public class ZeppelinServer extends Application {
return cxfContext;
}
private static SslContextFactory getSslContextFactory(ZeppelinConfiguration conf)
throws Exception {
@SuppressWarnings("deprecation")
private static SslContextFactory getSslContextFactory(ZeppelinConfiguration conf) {
// Note that the API for the SslContextFactory is different for
// Jetty version 9
SslContextFactory sslContextFactory = new SslContextFactory();
@ -200,6 +202,7 @@ public class ZeppelinServer extends Application {
return sslContextFactory;
}
@SuppressWarnings("unused") //TODO(bzz) why unused?
private static SSLContext getSslContext(ZeppelinConfiguration conf)
throws Exception {
@ -240,25 +243,16 @@ public class ZeppelinServer extends Application {
} else {
// use packaged WAR
webApp.setWar(warPath.getAbsolutePath());
File warTempDirectory = new File(conf.getRelativeDir(ConfVars.ZEPPELIN_WAR_TEMPDIR));
warTempDirectory.mkdir();
LOG.info("ZeppelinServer Webapp path: {}", warTempDirectory.getPath());
webApp.setTempDirectory(warTempDirectory);
}
// Explicit bind to root
webApp.addServlet(
new ServletHolder(new DefaultServlet()),
"/*"
);
webApp.addServlet(new ServletHolder(new DefaultServlet()), "/*");
return webApp;
}
public ZeppelinServer() throws Exception {
ZeppelinConfiguration conf = ZeppelinConfiguration.create();
this.schedulerFactory = new SchedulerFactory();
this.replFactory = new InterpreterFactory(conf, notebookServer);
this.notebookRepo = new NotebookRepoSync(conf);
notebook = new Notebook(conf, notebookRepo, schedulerFactory, replFactory, notebookServer);
}
@Override
public Set<Class<?>> getClasses() {
Set<Class<?>> classes = new HashSet<Class<?>>();
@ -266,14 +260,14 @@ public class ZeppelinServer extends Application {
}
@Override
public java.util.Set<java.lang.Object> getSingletons() {
Set<Object> singletons = new HashSet<Object>();
public Set<Object> getSingletons() {
Set<Object> singletons = new HashSet<>();
/** Rest-api root endpoint */
ZeppelinRestApi root = new ZeppelinRestApi();
singletons.add(root);
NotebookRestApi notebookApi = new NotebookRestApi(notebook, notebookServer);
NotebookRestApi notebookApi = new NotebookRestApi(notebook, notebookWsServer, notebookIndex);
singletons.add(notebookApi);
InterpreterRestApi interpreterApi = new InterpreterRestApi(replFactory);

40
zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
View file

@ -15,10 +15,13 @@
* limitations under the License.
*/
package org.apache.zeppelin.socket;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.UnknownHostException;
import java.util.*;
import java.util.concurrent.ConcurrentLinkedQueue;
import javax.servlet.http.HttpServletRequest;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
@ -44,27 +47,27 @@ import org.eclipse.jetty.websocket.WebSocketServlet;
import org.quartz.SchedulerException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.base.Strings;
import com.google.gson.Gson;
/**
* Zeppelin websocket service.
*
* @author anthonycorbacho
*/
public class NotebookServer extends WebSocketServlet implements
NotebookSocketListener, JobListenerFactory, AngularObjectRegistryListener {
private static final Logger LOG = LoggerFactory
.getLogger(NotebookServer.class);
private static final Logger LOG = LoggerFactory.getLogger(NotebookServer.class);
Gson gson = new Gson();
final Map<String, List<NotebookSocket>> noteSocketMap = new HashMap<>();
final List<NotebookSocket> connectedSockets = new LinkedList<>();
final Queue<NotebookSocket> connectedSockets = new ConcurrentLinkedQueue<>();
private Notebook notebook() {
return ZeppelinServer.notebook;
}
@Override
public boolean checkOrigin(HttpServletRequest request, String origin) {
try {
return SecurityUtils.isValidOrigin(origin, ZeppelinConfiguration.create());
} catch (UnknownHostException e) {
@ -72,7 +75,6 @@ public class NotebookServer extends WebSocketServlet implements
} catch (URISyntaxException e) {
e.printStackTrace();
}
return false;
}
@ -85,9 +87,7 @@ public class NotebookServer extends WebSocketServlet implements
public void onOpen(NotebookSocket conn) {
LOG.info("New connection from {} : {}", conn.getRequest().getRemoteAddr(),
conn.getRequest().getRemotePort());
synchronized (connectedSockets) {
connectedSockets.add(conn);
}
connectedSockets.add(conn);
}
@Override
@ -147,8 +147,7 @@ public class NotebookServer extends WebSocketServlet implements
completion(conn, notebook, messagereceived);
break;
case PING:
pong();
break;
break; //do nothing
case ANGULAR_OBJECT_UPDATED:
angularObjectUpdated(conn, notebook, messagereceived);
break;
@ -166,9 +165,7 @@ public class NotebookServer extends WebSocketServlet implements
LOG.info("Closed connection to {} : {}. ({}) {}", conn.getRequest()
.getRemoteAddr(), conn.getRequest().getRemotePort(), code, reason);
removeConnectionFromAllNote(conn);
synchronized (connectedSockets) {
connectedSockets.remove(conn);
}
connectedSockets.remove(conn);
}
protected Message deserializeMessage(String msg) {
@ -285,13 +282,11 @@ public class NotebookServer extends WebSocketServlet implements
}
private void broadcastAll(Message m) {
synchronized (connectedSockets) {
for (NotebookSocket conn : connectedSockets) {
try {
conn.send(serializeMessage(m));
} catch (IOException e) {
LOG.error("socket error", e);
}
for (NotebookSocket conn : connectedSockets) {
try {
conn.send(serializeMessage(m));
} catch (IOException e) {
LOG.error("socket error", e);
}
}
}
@ -730,6 +725,7 @@ public class NotebookServer extends WebSocketServlet implements
public static class ParagraphJobListener implements JobListener {
private NotebookServer notebookServer;
private Note note;
public ParagraphJobListener(NotebookServer notebookServer, Note note) {
this.notebookServer = notebookServer;
this.note = note;
@ -771,8 +767,6 @@ public class NotebookServer extends WebSocketServlet implements
public JobListener getParagraphJobListener(Note note) {
return new ParagraphJobListener(this, note);
}
private void pong() {
}
private void sendAllAngularObjects(Note note, NotebookSocket conn) throws IOException {
List<InterpreterSetting> settings = note.getNoteReplLoader()

116
zeppelin-server/src/test/java/org/apache/zeppelin/ZeppelinIT.java
View file

@ -17,17 +17,23 @@
package org.apache.zeppelin;
import static org.junit.Assert.*;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;
import java.util.Iterator;
import java.io.File;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.TimeUnit;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.ElementNotVisibleException;
import org.openqa.selenium.Keys;
import org.openqa.selenium.NoSuchElementException;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.TimeoutException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
@ -36,22 +42,33 @@ import org.openqa.selenium.firefox.FirefoxBinary;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.safari.SafariDriver;
import org.openqa.selenium.support.ui.ExpectedCondition;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.FluentWait;
import org.openqa.selenium.support.ui.Wait;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.base.Function;
/**
* Test Zeppelin with web brower.
*
* Test Zeppelin with web browser.
*
* To test, ZeppelinServer should be running on port 8080
* On OSX, you'll need firefox 31.0 installed.
* On OSX, you'll need firefox 42.0 installed, then you can run with
*
* PATH=~/Applications/Firefox.app/Contents/MacOS/:$PATH CI="" \
* mvn -Dtest=org.apache.zeppelin.ZeppelinIT -Denforcer.skip=true \
* test -pl zeppelin-server
*
*/
public class ZeppelinIT {
private static final Logger LOG = LoggerFactory.getLogger(ZeppelinIT.class);
private static final long MAX_BROWSER_TIMEOUT_SEC = 30;
private static final long MAX_PARAGRAPH_TIMEOUT_SEC = 60;
private WebDriver driver;
private WebDriver getWebDriver() {
WebDriver driver = null;
private void setWebDriver() {
if (driver == null) {
try {
@ -63,6 +80,7 @@ public class ZeppelinIT {
FirefoxProfile profile = new FirefoxProfile();
driver = new FirefoxDriver(ffox, profile);
} catch (Exception e) {
LOG.error("Starting Firefox failed",e);
}
}
@ -70,6 +88,7 @@ public class ZeppelinIT {
try {
driver = new ChromeDriver();
} catch (Exception e) {
LOG.error("Starting Chrome failed",e);
}
}
@ -77,6 +96,7 @@ public class ZeppelinIT {
try {
driver = new SafariDriver();
} catch (Exception e) {
LOG.error("Starting Safari failed",e);
}
}
@ -92,16 +112,9 @@ public class ZeppelinIT {
driver.get(url);
while (System.currentTimeMillis() - start < 60 * 1000) {
// wait for page load
try {
(new WebDriverWait(driver, 5)).until(new ExpectedCondition<Boolean>() {
@Override
public Boolean apply(WebDriver d) {
return d.findElement(By.partialLinkText("Create new note"))
.isDisplayed();
}
});
loaded = true;
try { // wait for page load
WebElement element = pollingWait(By.partialLinkText("Create new note"), MAX_BROWSER_TIMEOUT_SEC);
loaded = element.isDisplayed();
break;
} catch (TimeoutException e) {
driver.navigate().to(url);
@ -111,8 +124,6 @@ public class ZeppelinIT {
if (loaded == false) {
fail();
}
return driver;
}
@Before
@ -120,8 +131,7 @@ public class ZeppelinIT {
if (!endToEndTestEnabled()) {
return;
}
driver = getWebDriver();
setWebDriver();
}
@After
@ -137,39 +147,45 @@ public class ZeppelinIT {
return "//div[@ng-controller=\"ParagraphCtrl\"][" + paragraphNo +"]";
}
void waitForParagraph(final int paragraphNo, final String state) {
(new WebDriverWait(driver, 60)).until(new ExpectedCondition<Boolean>() {
public Boolean apply(WebDriver d) {
return driver.findElement(By.xpath(getParagraphXPath(paragraphNo)
+ "//div[@class=\"control\"]//span[1][text()=\" " + state + " \"]"))
.isDisplayed();
};
});
boolean waitForParagraph(final int paragraphNo, final String state) {
By locator = By.xpath(getParagraphXPath(paragraphNo)
+ "//div[contains(@class, 'control')]//span[1][contains(.,'" + state + "')]");
WebElement element = pollingWait(locator, MAX_PARAGRAPH_TIMEOUT_SEC);
return element.isDisplayed();
}
boolean endToEndTestEnabled() {
return null != System.getenv("CI");
}
boolean waitForText(final String txt, final By by) {
boolean waitForText(final String txt, final By locator) {
try {
new WebDriverWait(driver, 5).until(new ExpectedCondition<Boolean>() {
@Override
public Boolean apply(WebDriver d) {
return txt.equals(driver.findElement(by).getText());
}
});
return true;
WebElement element = pollingWait(locator, MAX_BROWSER_TIMEOUT_SEC);
return txt.equals(element.getText());
} catch (TimeoutException e) {
return false;
}
}
public WebElement pollingWait(final By locator, final long timeWait) {
Wait<WebDriver> wait = new FluentWait<WebDriver>(driver)
.withTimeout(timeWait, TimeUnit.SECONDS)
.pollingEvery(1, TimeUnit.SECONDS)
.ignoring(NoSuchElementException.class);
return wait.until(new Function<WebDriver, WebElement>() {
public WebElement apply(WebDriver driver) {
return driver.findElement(locator);
}
});
};
boolean endToEndTestEnabled() {
return null != System.getenv("CI");
}
@Test
public void testAngularDisplay() throws InterruptedException{
if (!endToEndTestEnabled()) {
return;
}
try {
createNewNote();
// wait for first paragraph's " READY " status text
@ -213,7 +229,7 @@ public class ZeppelinIT {
*/
WebElement paragraph3Editor = driver.findElement(By.xpath(getParagraphXPath(3) + "//textarea"));
paragraph3Editor.sendKeys(
"print" + Keys.chord(Keys.SHIFT, "9") + "\"myVar=\"" + Keys.chord(Keys.ADD)
"print" + Keys.chord(Keys.SHIFT, "9") + "\"myVar=\"" + Keys.chord(Keys.ADD)
+ "z.angular" + Keys.chord(Keys.SHIFT, "9") + "\"myVar\"))");
paragraph3Editor.sendKeys(Keys.chord(Keys.SHIFT, Keys.ENTER));
waitForParagraph(3, "FINISHED");
@ -241,7 +257,7 @@ public class ZeppelinIT {
WebElement paragraph4Editor = driver.findElement(By.xpath(getParagraphXPath(4) + "//textarea"));
paragraph4Editor.sendKeys(
"z.angularWatch" + Keys.chord(Keys.SHIFT, "9") + "\"myVar\", "
+ Keys.chord(Keys.SHIFT, "9")
+ Keys.chord(Keys.SHIFT, "9")
+ "before:Object, after:Object, context:org.apache.zeppelin.interpreter.InterpreterContext)"
+ Keys.EQUALS + ">{ z.run" +Keys.chord(Keys.SHIFT, "9") + "2, context)}");
paragraph4Editor.sendKeys(Keys.chord(Keys.SHIFT, Keys.ENTER));
@ -289,20 +305,24 @@ public class ZeppelinIT {
By.xpath(getParagraphXPath(1) + "//div[@id=\"angularTestButton\"]"));
System.out.println("testCreateNotebook Test executed");
} catch (ElementNotVisibleException e) {
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
}
}
private void createNewNote() {
List<WebElement> notebookLinks = driver.findElements(By
.xpath("//div[contains(@class, \"col-md-4\")]/div/ul/li"));
.xpath("//div[contains(@class, \"col-md-4\")]/div/ul/li"));
List<String> notebookTitles = new LinkedList<String>();
for (WebElement el : notebookLinks) {
notebookTitles.add(el.getText());
}
WebElement createNoteLink = driver.findElement(By.xpath("//div[contains(@class, \"col-md-4\")]/div/h5/a[contains(.,'Create new note')]"));
createNoteLink.click();
WebDriverWait block = new WebDriverWait(driver, 10);
WebDriverWait block = new WebDriverWait(driver, MAX_BROWSER_TIMEOUT_SEC);
WebElement modal = block.until(ExpectedConditions.visibilityOfElementLocated(By.id("noteNameModal")));
WebElement createNoteButton = modal.findElement(By.id("createNoteButton"));
createNoteButton.click();
@ -310,6 +330,6 @@ public class ZeppelinIT {
try {
Thread.sleep(500); // wait for notebook list updated
} catch (InterruptedException e) {
}
}
}
}

14
zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java
View file

@ -29,8 +29,12 @@ import java.util.concurrent.Executors;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethodBase;
import org.apache.commons.httpclient.methods.*;
import org.apache.zeppelin.interpreter.Interpreter.RegisteredInterpreter;
import org.apache.commons.httpclient.methods.ByteArrayRequestEntity;
import org.apache.commons.httpclient.methods.DeleteMethod;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.PutMethod;
import org.apache.commons.httpclient.methods.RequestEntity;
import org.apache.zeppelin.interpreter.InterpreterGroup;
import org.apache.zeppelin.interpreter.InterpreterOption;
import org.apache.zeppelin.interpreter.InterpreterSetting;
@ -207,7 +211,7 @@ public abstract class AbstractTestRestApi {
}
LOG.info("Terminating test Zeppelin...");
ZeppelinServer.jettyServer.stop();
ZeppelinServer.jettyWebServer.stop();
executor.shutdown();
long s = System.currentTimeMillis();
@ -393,6 +397,10 @@ public abstract class AbstractTestRestApi {
protected Matcher<? super HttpMethodBase> isCreated() { return responsesWith(201); }
protected Matcher<? super HttpMethodBase> isBadRequest() { return responsesWith(400); }
protected Matcher<? super HttpMethodBase> isNotFound() { return responsesWith(404); }
protected Matcher<? super HttpMethodBase> isNotAllowed() {
return responsesWith(405);
}

237
zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinRestApiTest.java
View file

@ -17,8 +17,13 @@
package org.apache.zeppelin.rest;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@ -26,7 +31,7 @@ import org.apache.commons.httpclient.methods.DeleteMethod;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.PutMethod;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.commons.lang3.StringUtils;
import org.apache.zeppelin.interpreter.InterpreterSetting;
import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.Paragraph;
@ -41,13 +46,9 @@ import org.junit.runners.MethodSorters;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import static org.junit.Assert.*;
/**
* BASIC Zeppelin rest api tests
*
* @author anthonycorbacho
*
*/
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class ZeppelinRestApiTest extends AbstractTestRestApi {
@ -190,6 +191,39 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
ZeppelinServer.notebook.removeNote(note.getId());
}
@Test
public void testGetNotebookInfo() throws IOException {
LOG.info("testGetNotebookInfo");
// Create note to get info
Note note = ZeppelinServer.notebook.createNote();
assertNotNull("can't create new note", note);
note.setName("note");
Paragraph paragraph = note.addParagraph();
Map config = paragraph.getConfig();
config.put("enabled", true);
paragraph.setConfig(config);
String paragraphText = "%md This is my new paragraph in my new note";
paragraph.setText(paragraphText);
note.persist();
String sourceNoteID = note.getId();
GetMethod get = httpGet("/notebook/" + sourceNoteID);
LOG.info("testGetNotebookInfo \n" + get.getResponseBodyAsString());
assertThat("test notebook get method:", get, isAllowed());
Map<String, Object> resp = gson.fromJson(get.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
assertNotNull(resp);
assertEquals("OK", resp.get("status"));
Map<String, Object> body = (Map<String, Object>) resp.get("body");
List<Map<String, Object>> paragraphs = (List<Map<String, Object>>) body.get("paragraphs");
assertTrue(paragraphs.size() > 0);
assertEquals(paragraphText, paragraphs.get(0).get("text"));
}
@Test
public void testNotebookCreateWithName() throws IOException {
String noteName = "Test note name";
@ -201,6 +235,46 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
testNotebookCreate("");
}
@Test
public void testNotebookCreateWithParagraphs() throws IOException {
// Call Create Notebook REST API
String noteName = "test";
String jsonRequest = "{\"name\":\"" + noteName + "\", \"paragraphs\": [" +
"{\"title\": \"title1\", \"text\": \"text1\"}," +
"{\"title\": \"title2\", \"text\": \"text2\"}" +
"]}";
PostMethod post = httpPost("/notebook/", jsonRequest);
LOG.info("testNotebookCreate \n" + post.getResponseBodyAsString());
assertThat("test notebook create method:", post, isCreated());
Map<String, Object> resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
String newNotebookId = (String) resp.get("body");
LOG.info("newNotebookId:=" + newNotebookId);
Note newNote = ZeppelinServer.notebook.getNote(newNotebookId);
assertNotNull("Can not find new note by id", newNote);
// This is partial test as newNote is in memory but is not persistent
String newNoteName = newNote.getName();
LOG.info("new note name is: " + newNoteName);
String expectedNoteName = noteName;
if (noteName.isEmpty()) {
expectedNoteName = "Note " + newNotebookId;
}
assertEquals("compare note name", expectedNoteName, newNoteName);
assertEquals("initial paragraph check failed", 3, newNote.getParagraphs().size());
for (Paragraph p : newNote.getParagraphs()) {
if (StringUtils.isEmpty(p.getText())) {
continue;
}
assertTrue("paragraph title check failed", p.getTitle().startsWith("title"));
assertTrue("paragraph text check failed", p.getText().startsWith("text"));
}
// cleanup
ZeppelinServer.notebook.removeNote(newNotebookId);
post.releaseConnection();
}
private void testNotebookCreate(String noteName) throws IOException {
// Call Create Notebook REST API
String jsonRequest = "{\"name\":\"" + noteName + "\"}";
@ -264,7 +338,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
LOG.info("testCloneNotebook");
// Create note to clone
Note note = ZeppelinServer.notebook.createNote();
assertNotNull("cant create new note", note);
assertNotNull("can't create new note", note);
note.setName("source note for clone");
Paragraph paragraph = note.addParagraph();
Map config = paragraph.getConfig();
@ -308,5 +382,154 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
get.releaseConnection();
}
@Test
public void testNoteJobs() throws IOException, InterruptedException {
LOG.info("testNoteJobs");
// Create note to run test.
Note note = ZeppelinServer.notebook.createNote();
assertNotNull("can't create new note", note);
note.setName("note for run test");
Paragraph paragraph = note.addParagraph();
Map config = paragraph.getConfig();
config.put("enabled", true);
paragraph.setConfig(config);
paragraph.setText("%md This is test paragraph.");
note.persist();
String noteID = note.getId();
note.runAll();
// wait until job is finished or timeout.
int timeout = 1;
while (!paragraph.isTerminated()) {
Thread.sleep(1000);
if (timeout++ > 10) {
LOG.info("testNoteJobs timeout job.");
break;
}
}
// Call Run Notebook Jobs REST API
PostMethod postNoteJobs = httpPost("/notebook/job/" + noteID, "");
assertThat("test notebook jobs run:", postNoteJobs, isAllowed());
postNoteJobs.releaseConnection();
// Call Stop Notebook Jobs REST API
DeleteMethod deleteNoteJobs = httpDelete("/notebook/job/" + noteID);
assertThat("test notebook stop:", deleteNoteJobs, isAllowed());
deleteNoteJobs.releaseConnection();
Thread.sleep(1000);
// Call Run paragraph REST API
PostMethod postParagraph = httpPost("/notebook/job/" + noteID + "/" + paragraph.getId(), "");
assertThat("test paragraph run:", postParagraph, isAllowed());
postParagraph.releaseConnection();
Thread.sleep(1000);
// Call Stop paragraph REST API
DeleteMethod deleteParagraph = httpDelete("/notebook/job/" + noteID + "/" + paragraph.getId());
assertThat("test paragraph stop:", deleteParagraph, isAllowed());
deleteParagraph.releaseConnection();
Thread.sleep(1000);
//cleanup
ZeppelinServer.notebook.removeNote(note.getId());
}
@Test
public void testRunParagraphWithParams() throws IOException, InterruptedException {
LOG.info("testRunParagraphWithParams");
// Create note to run test.
Note note = ZeppelinServer.notebook.createNote();
assertNotNull("can't create new note", note);
note.setName("note for run test");
Paragraph paragraph = note.addParagraph();
Map config = paragraph.getConfig();
config.put("enabled", true);
paragraph.setConfig(config);
paragraph.setText("%spark\nval param = z.input(\"param\").toString\nprintln(param)");
note.persist();
String noteID = note.getId();
note.runAll();
// wait until job is finished or timeout.
int timeout = 1;
while (!paragraph.isTerminated()) {
Thread.sleep(1000);
if (timeout++ > 120) {
LOG.info("testRunParagraphWithParams timeout job.");
break;
}
}
// Call Run paragraph REST API
PostMethod postParagraph = httpPost("/notebook/job/" + noteID + "/" + paragraph.getId(),
"{\"params\": {\"param\": \"hello\", \"param2\": \"world\"}}");
assertThat("test paragraph run:", postParagraph, isAllowed());
postParagraph.releaseConnection();
Thread.sleep(1000);
Note retrNote = ZeppelinServer.notebook.getNote(noteID);
Paragraph retrParagraph = retrNote.getParagraph(paragraph.getId());
Map<String, Object> params = retrParagraph.settings.getParams();
assertEquals("hello", params.get("param"));
assertEquals("world", params.get("param2"));
//cleanup
ZeppelinServer.notebook.removeNote(note.getId());
}
@Test
public void testCronJobs() throws InterruptedException, IOException{
// create a note and a paragraph
Note note = ZeppelinServer.notebook.createNote();
note.setName("note for run test");
Paragraph paragraph = note.addParagraph();
paragraph.setText("%md This is test paragraph.");
Map config = paragraph.getConfig();
config.put("enabled", true);
paragraph.setConfig(config);
note.runAll();
// wait until job is finished or timeout.
int timeout = 1;
while (!paragraph.isTerminated()) {
Thread.sleep(1000);
if (timeout++ > 10) {
LOG.info("testNoteJobs timeout job.");
break;
}
}
String jsonRequest = "{\"cron\":\"* * * * * ?\" }";
// right cron expression but not exist note.
PostMethod postCron = httpPost("/notebook/cron/notexistnote", jsonRequest);
assertThat("", postCron, isNotFound());
postCron.releaseConnection();
// right cron expression.
postCron = httpPost("/notebook/cron/" + note.getId(), jsonRequest);
assertThat("", postCron, isAllowed());
postCron.releaseConnection();
Thread.sleep(1000);
// wrong cron expression.
jsonRequest = "{\"cron\":\"a * * * * ?\" }";
postCron = httpPost("/notebook/cron/" + note.getId(), jsonRequest);
assertThat("", postCron, isBadRequest());
postCron.releaseConnection();
Thread.sleep(1000);
// remove cron job.
DeleteMethod deleteCron = httpDelete("/notebook/cron/" + note.getId());
assertThat("", deleteCron, isAllowed());
deleteCron.releaseConnection();
ZeppelinServer.notebook.removeNote(note.getId());
}
}

2
zeppelin-server/src/test/java/org/apache/zeppelin/socket/NotebookServerTest.java
View file

@ -60,7 +60,7 @@ public class NotebookServerTest extends AbstractTestRestApi {
AbstractTestRestApi.startUp();
gson = new Gson();
notebook = ZeppelinServer.notebook;
notebookServer = ZeppelinServer.notebookServer;
notebookServer = ZeppelinServer.notebookWsServer;
}
@AfterClass

3
zeppelin-web/.jshintrc
View file

@ -30,6 +30,7 @@
"alert": false,
"nv": false,
"ace": false,
"d3": false
"d3": false,
"BootstrapDialog": false
}
}

15
zeppelin-web/CONTRIBUTING.md
View file

@ -1,4 +1,11 @@
# Contributing to Zeppelin-Web
## Dev Mode
When working on Zeppelin's WebApplication, it is recommended to run in dev mode.
For that, start Zeppelin server normally, then use ``./grunt serve`` in _zeppelin-web_ directory.
This will launch a Zeppelin WebApplication on port **9000** that will update on code changes.
## Technologies
@ -16,8 +23,10 @@ So you might want to get familiar with it.
But don't worry, JSHint will make you remember it for the most part.
There is also a rule of **No JQuery except in directives**, If you want to include a library,
please search for its **angularJS** directive first and if it doesn't exist, make one :)
We try not to have **JQuery except in directives**, If you want to include a library,
please search for its **angularJS** directive first.
If you still need to use it, then please use ``angular.element()`` instead of ``$()``
## Folder Structure & Code Organization
@ -94,7 +103,7 @@ The `components` folder is here to contains any reusable component (used more th
Fonts files and their css are mixed together in the `fonts` folder
## Compiling and using dev mode
## New files includes
As we do not use yeoman to generate controllers or other type of files with this new structure,
we need to do some includes manually in `index.html` in order to use dev mode and compile correctly.

2
zeppelin-web/README.md
View file

@ -24,7 +24,7 @@ Here are the basic commands to compile the WebApplication with a configured envi
``./grunt serve``
This will launch a Zeppelin WebApplication on port **9000** and update on code changes.
This will launch a Zeppelin WebApplication on port **9000** that will update on code changes.
(You will need to have Zeppelin running on the side)

7
zeppelin-web/bower.json
View file

@ -11,6 +11,7 @@
"angular-animate": "1.3.8",
"angular-touch": "1.3.8",
"angular-route": "1.3.8",
"angular-resource": "1.3.8",
"angular-bootstrap": "~0.13.0",
"angular-websocket": "~1.0.13",
"ace-builds": "1.1.9",
@ -21,12 +22,14 @@
"perfect-scrollbar": "~0.5.4",
"ng-sortable": "~1.1.9",
"angular-elastic": "~2.4.2",
"angular-elastic-input": "~2.0.1",
"angular-elastic-input": "~2.2.0",
"angular-xeditable": "0.1.8",
"highlightjs": "~8.4.0",
"lodash": "~3.9.3",
"angular-filter": "~0.5.4",
"ngtoast": "~1.5.5"
"ngtoast": "~1.5.5",
"ng-focus-if": "~1.0.2",
"bootstrap3-dialog": "bootstrap-dialog#~1.34.7"
},
"devDependencies": {
"angular-mocks": "1.3.8"

7
zeppelin-web/src/app/app.controller.js
View file

@ -20,7 +20,7 @@ angular.module('zeppelinWebApp').controller('MainCtrl', function($scope, $rootSc
var init = function() {
$scope.asIframe = (($window.location.href.indexOf('asIframe') > -1) ? true : false);
};
init();
$rootScope.$on('setIframe', function(event, data) {
@ -36,10 +36,13 @@ angular.module('zeppelinWebApp').controller('MainCtrl', function($scope, $rootSc
event.preventDefault();
}
});
// Set The lookAndFeel to default on every page
$rootScope.$on('$routeChangeStart', function(event, next, current) {
$rootScope.$broadcast('setLookAndFeel', 'default');
});
BootstrapDialog.defaultOptions.onshown = function() {
angular.element('#' + this.id).find('.btn:last').focus();
};
});

14
zeppelin-web/src/app/app.js
View file

@ -31,7 +31,9 @@ angular.module('zeppelinWebApp', [
'monospaced.elastic',
'puElasticInput',
'xeditable',
'ngToast'
'ngToast',
'focus-if',
'ngResource'
])
.filter('breakFilter', function() {
return function (text) {
@ -49,6 +51,10 @@ angular.module('zeppelinWebApp', [
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/notebook/:noteId/paragraph?=:paragraphId', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/notebook/:noteId/paragraph/:paragraphId?', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
@ -57,10 +63,14 @@ angular.module('zeppelinWebApp', [
templateUrl: 'app/interpreter/interpreter.html',
controller: 'InterpreterCtrl'
})
.when('/search/:searchTerm', {
templateUrl: 'app/search/result-list.html',
controller: 'SearchResultCtrl'
})
.otherwise({
redirectTo: '/'
});
ngToastProvider.configure({
dismissButton: true,
dismissOnClick: false,

16
zeppelin-web/src/app/home/home.css
View file

@ -275,10 +275,6 @@ kbd {
border-radius: 2px;
}
.home {
min-height: 400px;
}
/*
ngToast Style
*/
@ -318,6 +314,14 @@ This part should be removed when new version of bootstrap handles this issue.
float: left;
}
.modal-backdrop {
z-index: 10002 !important;
}
.modal-dialog, .modal {
z-index: 10003 !important;
}
#noteImportModal .modal-body {
min-height: 420px;
overflow: hidden;
@ -683,3 +687,7 @@ This part should be removed when new version of bootstrap handles this issue.
-webkit-transform: translateY(0);
opacity: 1;
}
.bootstrap-dialog.type-primary .modal-header {
background: #3071a9;
}

2
zeppelin-web/src/app/home/home.html
View file

@ -44,7 +44,7 @@ limitations under the License.
<div class="col-md-6">
<h4>Help</h4>
Get started with <a style="text-decoration: none;" target="_blank"
href="http://zeppelin.incubator.apache.org/docs/index.html">Zeppelin documentation</a><br>
href="http://zeppelin.incubator.apache.org/docs/latest/index.html">Zeppelin documentation</a><br>
<h4>Community</h4>
Please feel free to help us to improve Zeppelin, <br>

110
zeppelin-web/src/app/interpreter/interpreter.controller.js
View file

@ -20,6 +20,7 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
$scope.interpreterSettings = [];
$scope.availableInterpreters = {};
$scope.showAddNewSetting = false;
$scope._ = _;
var getInterpreterSettings = function() {
$http.get(baseUrlSrv.getRestApiBase()+'/interpreter/setting').
@ -55,25 +56,27 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
};
$scope.updateInterpreterSetting = function(settingId) {
var result = confirm('Do you want to update this interpreter and restart with new settings?');
if (!result) {
return;
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to update this interpreter and restart with new settings?',
callback: function(result) {
if (result) {
var index = _.findIndex($scope.interpreterSettings, {'id': settingId});
var request = {
properties: angular.copy($scope.interpreterSettings[index].properties),
};
var index = _.findIndex($scope.interpreterSettings, { 'id': settingId });
var request = {
properties : angular.copy($scope.interpreterSettings[index].properties),
};
$http.put(baseUrlSrv.getRestApiBase() + '/interpreter/setting/' + settingId, request).
success(function(data, status, headers, config) {
$scope.interpreterSettings[index] = data.body;
removeTMPSettings(index);
}).
error(function(data, status, headers, config) {
console.log('Error %o %o', status, data.message);
$http.put(baseUrlSrv.getRestApiBase() + '/interpreter/setting/' + settingId, request).
success(function (data, status, headers, config) {
$scope.interpreterSettings[index] = data.body;
removeTMPSettings(index);
}).
error(function (data, status, headers, config) {
console.log('Error %o %o', status, data.message);
});
}
}
});
};
@ -86,19 +89,23 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
};
$scope.removeInterpreterSetting = function(settingId) {
var result = confirm('Do you want to delete this interpreter setting?');
if (!result) {
return;
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this interpreter setting?',
callback: function(result) {
if (result) {
$http.delete(baseUrlSrv.getRestApiBase() + '/interpreter/setting/' + settingId).
success(function(data, status, headers, config) {
$http.delete(baseUrlSrv.getRestApiBase() + '/interpreter/setting/' + settingId).
success(function(data, status, headers, config) {
var index = _.findIndex($scope.interpreterSettings, { 'id': settingId });
$scope.interpreterSettings.splice(index, 1);
}).
error(function(data, status, headers, config) {
console.log('Error %o %o', status, data.message);
var index = _.findIndex($scope.interpreterSettings, { 'id': settingId });
$scope.interpreterSettings.splice(index, 1);
}).
error(function(data, status, headers, config) {
console.log('Error %o %o', status, data.message);
});
}
}
});
};
@ -120,29 +127,41 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
};
$scope.restartInterpreterSetting = function(settingId) {
var result = confirm('Do you want to restart this interpreter?');
if (!result) {
return;
}
$http.put(baseUrlSrv.getRestApiBase() + '/interpreter/setting/restart/' + settingId).
success(function(data, status, headers, config) {
var index = _.findIndex($scope.interpreterSettings, { 'id': settingId });
$scope.interpreterSettings[index] = data.body;
}).
error(function(data, status, headers, config) {
console.log('Error %o %o', status, data.message);
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to restart this interpreter?',
callback: function(result) {
if (result) {
$http.put(baseUrlSrv.getRestApiBase() + '/interpreter/setting/restart/' + settingId).
success(function(data, status, headers, config) {
var index = _.findIndex($scope.interpreterSettings, { 'id': settingId });
$scope.interpreterSettings[index] = data.body;
}).
error(function(data, status, headers, config) {
console.log('Error %o %o', status, data.message);
});
}
}
});
};
$scope.addNewInterpreterSetting = function() {
if (!$scope.newInterpreterSetting.name || !$scope.newInterpreterSetting.group) {
alert('Please determine name and interpreter');
BootstrapDialog.alert({
closable: true,
title: 'Add interpreter',
message: 'Please determine name and interpreter'
});
return;
}
if (_.findIndex($scope.interpreterSettings, { 'name': $scope.newInterpreterSetting.name }) >= 0) {
alert('Name ' + $scope.newInterpreterSetting.name + ' already exists');
BootstrapDialog.alert({
closable: true,
title: 'Add interpreter',
message: 'Name ' + $scope.newInterpreterSetting.name + ' already exists'
});
return;
}
@ -203,6 +222,9 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
var index = _.findIndex($scope.interpreterSettings, { 'id': settingId });
var setting = $scope.interpreterSettings[index];
if (!setting.propertyKey || setting.propertyKey === '') {
return;
}
setting.properties[setting.propertyKey] = setting.propertyValue;
emptyNewProperty(setting);
}

18
zeppelin-web/src/app/interpreter/interpreter.css
View file

@ -62,10 +62,6 @@
overflow-y: auto;
}
.interpreter table tr {
height : 45px;
}
.interpreterSettingAdd {
margin : 5px 5px 5px 5px;
padding : 10px 10px 10px 10px;
@ -74,3 +70,17 @@
.editable-wrap {
width : 100%;
}
.interpreter h5 {
font-weight: bold;
}
.new_h3 {
margin-top: 1px;
padding-top: 7px;
float: left;
}
.empty-properties-message {
color: #666;
}

24
zeppelin-web/src/app/interpreter/interpreter.html
View file

@ -15,7 +15,7 @@ limitations under the License.
<div class="header">
<div class="row">
<div class="col-md-12">
<h3 class="new_h3" style="float:left">
<h3 class="new_h3">
Interpreters
</h3>
<span class="btn btn-default fa fa-plus"
@ -65,17 +65,21 @@ limitations under the License.
</span>
</div>
</div>
<br />
<div class="row interpreter">
<div class="col-md-12">
<b>Properties</b>
<div ng-show="_.isEmpty(setting.properties) || valueform.$hidden" class="col-md-12 empty-properties-message">
<em>Currently there are no properties set for this interpreter</em>
</div>
<div class="col-md-12" ng-show="!_.isEmpty(setting.properties) || valueform.$visible">
<h5>Properties</h5>
<table class="table table-striped">
<tr>
<th style="width:30%">name</th>
<th>value</th>
<th ng-if="valueform.$visible">action</th>
</tr>
<tr ng-repeat="(key, value) in setting.properties">
<thead>
<tr>
<th style="width:30%">name</th>
<th>value</th>
<th ng-if="valueform.$visible">action</th>
</tr>
</thead>
<tr ng-repeat="(key, value) in setting.properties" >
<td>{{key}}</td>
<td>
<span editable-textarea="setting.properties[key]" e-form="valueform" e-msd-elastic>

135
zeppelin-web/src/app/notebook/notebook-actionBar.html Normal file
View file

@ -0,0 +1,135 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div class="noteAction" ng-show="note.id && !paragraphUrl">
<h3>
<input type="text" pu-elastic-input class="form-control2" placeholder="{{note.name || 'Note ' + note.id}}" style="min-width: 200px; max-width: 600px;"
ng-show="showEditor" ng-model="note.name" ng-blur="sendNewName();showEditor = false;" ng-enter="sendNewName();showEditor = false;" ng-escape="note.name = oldName; showEditor = false" focus-if="showEditor" />
<p class="form-control-static2" ng-click="showEditor = true; oldName = note.name" ng-show="!showEditor">{{note.name || 'Note ' + note.id}}</p>
<span class="labelBtn btn-group">
<button type="button"
class="btn btn-default btn-xs"
ng-click="runNote()"
ng-class="{'disabled':isNoteRunning()}"
tooltip-placement="bottom" tooltip="Run all paragraphs">
<i class="icon-control-play"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="toggleAllEditor()"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Show/hide the code">
<i ng-class="editorToggled ? 'fa icon-size-fullscreen' :'fa icon-size-actual'"></i></button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="toggleAllTable()"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Show/hide the output">
<i ng-class="tableToggled ? 'fa icon-notebook' : 'fa icon-book-open'"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="clearAllParagraphOutput()"
ng-hide="viewOnly"
ng-class="{'disabled':isNoteRunning()}"
tooltip-placement="bottom" tooltip="Clear output">
<i class="fa fa-eraser"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="removeNote(note.id)"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Remove the notebook">
<i class="icon-trash"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Clone the notebook"
data-toggle="modal" data-target="#noteNameModal" data-clone="true"
>
<i class="fa fa-copy"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
ng-click="exportNotebook()"
tooltip-placement="bottom" tooltip="Export the notebook">
<i class="fa fa-download"></i>
</button>
</span>
<span ng-hide="viewOnly">
<div class="labelBtn btn-group">
<div class="btn btn-default btn-xs dropdown-toggle"
type="button"
data-toggle="dropdown"
ng-class="{ 'btn-info' : note.config.cron, 'btn-danger' : note.info.cron, 'btn-default' : !note.config.cron}"
tooltip-placement="bottom" tooltip="Run scheduler">
<span class="fa fa-clock-o"></span> {{getCronOptionNameFromValue(note.config.cron)}}
</div>
<ul class="dropdown-menu" role="menu" style="width:300px">
<li>
<div class="cron-preset-container">
Run note with cron scheduler.
Either choose from<br/>preset or write your own <a href="http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger" target=_blank>cron expression</a>.
<div>
<span>- Preset</span>
<a class="cron-preset" ng-repeat="cr in cronOption"
type="button"
ng-click="setCronScheduler(cr.value)"
dropdown-input ng-class="{ 'selected' : cr.value == note.config.cron}">{{cr.name}}</a>
</div>
<div>
<span>- Cron expression</span>
<input type="text"
ng-model="note.config.cron"
ng-change="setCronScheduler(note.config.cron)"
dropdown-input ng-model-options="{ debounce: 1000 }" />
<p ng-show="note.info.cron" class="text-danger cron-info">
{{note.info.cron}}
</p>
</div>
</div>
</li>
</ul>
</div>
</span>
<div class="pull-right" style="margin-top:15px; margin-right:15px; font-size:15px;">
<span style="position:relative; top:3px; margin-right:4px; cursor:pointer"
data-toggle="modal"
data-target="#shortcutModal"
tooltip-placement="bottom" tooltip="List of shortcut">
<i class="icon-question"></i>
</span>
<span style="position:relative; top:2px; margin-right:4px; cursor:pointer;"
ng-click="toggleSetting()"
tooltip-placement="bottom" tooltip="Interpreter binding">
<i class="fa fa-cog" ng-style="{color: showSetting ? '#3071A9' : 'black' }"></i>
</span>
<span class="btn-group">
<button type="button" class="btn btn-default btn-xs dropdown-toggle"
data-toggle="dropdown">
{{note.config.looknfeel}} <span class="caret"></span>
</button>
<ul class="dropdown-menu pull-right" role="menu">
<li ng-repeat="looknfeel in looknfeelOption">
<a style="cursor:pointer" ng-click="setLookAndFeel(looknfeel)">{{looknfeel}}</a>
</li>
</ul>
</span>
</div>
</h3>
</div>

127
zeppelin-web/src/app/notebook/notebook.controller.js
View file

@ -1,4 +1,5 @@
/* jshint loopfunc: true */
/* global $: false */
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
@ -14,9 +15,9 @@
*/
'use strict';
angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $route, $routeParams, $location,
$rootScope, $http, websocketMsgSrv, baseUrlSrv,
$timeout, SaveAsService) {
angular.module('zeppelinWebApp').controller('NotebookCtrl',
function($scope, $route, $routeParams, $location, $rootScope, $http,
websocketMsgSrv, baseUrlSrv, $timeout, SaveAsService) {
$scope.note = null;
$scope.showEditor = false;
$scope.editorToggled = false;
@ -66,6 +67,26 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
/** Init the new controller */
var initNotebook = function() {
websocketMsgSrv.getNotebook($routeParams.noteId);
var currentRoute = $route.current;
if (currentRoute) {
setTimeout(
function() {
var routeParams = currentRoute.params;
var $id = $('#' + routeParams.paragraph + '_container');
if ($id.length > 0) {
// adjust for navbar
var top = $id.offset().top - 103;
$('html, body').scrollTo({top: top, left: 0});
}
},
1000
);
}
};
initNotebook();
@ -73,11 +94,17 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
/** Remove the note and go back tot he main page */
/** TODO(anthony): In the nearly future, go back to the main page and telle to the dude that the note have been remove */
$scope.removeNote = function(noteId) {
var result = confirm('Do you want to delete this notebook?');
if (result) {
websocketMsgSrv.deleteNotebook(noteId);
$location.path('/#');
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this notebook?',
callback: function(result) {
if (result) {
websocketMsgSrv.deleteNotebook(noteId);
$location.path('/#');
}
}
});
};
//Export notebook
@ -88,20 +115,32 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
//Clone note
$scope.cloneNote = function(noteId) {
var result = confirm('Do you want to clone this notebook?');
if (result) {
websocketMsgSrv.cloneNotebook(noteId);
$location.path('/#');
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to clone this notebook?',
callback: function(result) {
if (result) {
websocketMsgSrv.cloneNotebook(noteId);
$location.path('/#');
}
}
});
};
$scope.runNote = function() {
var result = confirm('Run all paragraphs?');
if (result) {
_.forEach($scope.note.paragraphs, function(n, key) {
angular.element('#' + n.id + '_paragraphColumn_main').scope().runParagraph(n.text);
});
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Run all paragraphs?',
callback: function(result) {
if (result) {
_.forEach($scope.note.paragraphs, function (n, key) {
angular.element('#' + n.id + '_paragraphColumn_main').scope().runParagraph(n.text);
});
}
}
});
};
$scope.saveNote = function() {
@ -114,12 +153,18 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
};
$scope.clearAllParagraphOutput = function() {
var result = confirm('Do you want to clear all output?');
if (result) {
_.forEach($scope.note.paragraphs, function(n, key) {
angular.element('#' + n.id + '_paragraphColumn_main').scope().clearParagraphOutput();
});
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to clear all output?',
callback: function(result) {
if (result) {
_.forEach($scope.note.paragraphs, function(n, key) {
angular.element('#' + n.id + '_paragraphColumn_main').scope().clearParagraphOutput();
});
}
}
});
};
$scope.toggleAllEditor = function() {
@ -216,7 +261,6 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
/** Update the note name */
$scope.sendNewName = function() {
$scope.showEditor = false;
if ($scope.note.name) {
websocketMsgSrv.updateNotebook($scope.note.id, $scope.note.name, $scope.note.config);
}
@ -291,11 +335,16 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
});
// create new paragraph on current position
$scope.$on('insertParagraph', function(event, paragraphId) {
$scope.$on('insertParagraph', function(event, paragraphId, position) {
var newIndex = -1;
for (var i=0; i<$scope.note.paragraphs.length; i++) {
if ($scope.note.paragraphs[i].id === paragraphId) {
newIndex = i+1;
if ( $scope.note.paragraphs[i].id === paragraphId ) {
//determine position of where to add new paragraph; default is below
if ( position === 'above' ) {
newIndex = i;
} else {
newIndex = i+1;
}
break;
}
}
@ -384,6 +433,7 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
$scope.note.paragraphs.splice(index, 0, note.paragraphs[index]);
break;
}
$scope.$broadcast('updateParagraph', {paragraph: note.paragraphs[index]});
}
}
@ -471,12 +521,21 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
$scope.closeSetting = function() {
if (isSettingDirty()) {
var result = confirm('Changes will be discarded');
if (!result) {
return;
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Changes will be discarded.',
callback: function(result) {
if (result) {
$scope.$apply(function() {
$scope.showSetting = false;
});
}
}
});
} else {
$scope.showSetting = false;
}
$scope.showSetting = false;
};
$scope.saveSetting = function() {

77
zeppelin-web/src/app/notebook/notebook.css
View file

@ -37,6 +37,34 @@
margin-top: 20px;
}
/*
Add New Paragraph
*/
.new-paragraph{
text-align: center;
height: 7px;
margin: -7px 2px 0 2px;
cursor: pointer;
}
.new-paragraph:hover{
background: #d3e5ea;
}
.new-paragraph:hover .plus-sign{
display: block;
}
.plus-sign{
display: none;
height: 7px;
margin: 0;
color: #3071a9;
line-height: 5px;
font-size: 11.5px;
}
.noteBtnfa {
margin-left: 3px;
}
@ -46,6 +74,11 @@
pointer-events: none;
}
.navbar-fixed-top,
.navbar-fixed-top .dropdown-menu {
z-index: 10002;
}
.noteAction {
margin-left: -10px;
margin-right: -10px;
@ -55,7 +88,7 @@
top: 50px;
width: 100%;
height: 54px;
z-index: 3;
z-index: 10001;
}
.noteAction h3 {
@ -71,25 +104,19 @@
color: #333333;
}
.new_h3 {
margin-top: 1px;
padding-top: 7px;
}
.form-control2 {
width: 100%;
margin-left: 15px;
height: 40px;
font-size: 29px;
line-height: 1.42857143;
line-height: 1.2;
color: #555;
background: #fff;
border: 1px solid #ccc;
border-radius: 4px;
border-radius: 0;
-webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, .075);
box-shadow: inset 0 1px 1px rgba(0, 0, 0, .075);
-webkit-transition: border-color ease-in-out .15s, -webkit-box-shadow ease-in-out .15s;
-o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
padding: 7px 0;
margin: 2px 0 0 14px;
}
.form-control-static2 {
@ -146,3 +173,29 @@
.setting .modal-footer {
border: 0;
}
.cron-preset-container {
padding: 10px 20px 0 20px;
font-weight: normal;
word-wrap: break-word;
}
.cron-preset-container > div {
margin: 10px 0;
}
.cron-preset {
margin: 0px 2px;
cursor: pointer;
}
.cron-info {
margin: 5px;
font-size: 11px;
}
.cron-preset.selected {
color: #000;
text-decoration: none;
cursor: default;
}

130
zeppelin-web/src/app/notebook/notebook.html
View file

@ -12,129 +12,7 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Here the controller <NotebookCtrl> is not needed because explicitly set in the app.js (route) -->
<div class="noteAction" ng-show="note.id && !paragraphUrl">
<h3>
<input type="text" class="form-control2" placeholder="{{note.name || 'Note ' + note.id}}" style="width:200px;"
ng-show="showEditor" ng-model="note.name" ng-enter="sendNewName()" ng-escape="showEditor = false" autofocus/>
<p class="form-control-static2" ng-click="showEditor = true" ng-show="!showEditor">{{note.name || 'Note ' + note.id}}</p>
<span class="labelBtn btn-group">
<button type="button"
class="btn btn-default btn-xs"
ng-click="runNote()"
ng-class="{'disabled':isNoteRunning()}"
tooltip-placement="bottom" tooltip="Run all the notes">
<i class="icon-control-play"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="toggleAllEditor()"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Show/hide the code">
<i ng-class="editorToggled ? 'fa icon-size-fullscreen' :'fa icon-size-actual'"></i></button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="toggleAllTable()"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Show/hide the output">
<i ng-class="tableToggled ? 'fa icon-notebook' : 'fa icon-book-open'"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="clearAllParagraphOutput()"
ng-hide="viewOnly"
ng-class="{'disabled':isNoteRunning()}"
tooltip-placement="bottom" tooltip="Clear output">
<i class="fa fa-eraser"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-click="removeNote(note.id)"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Remove the notebook">
<i class="icon-trash"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Clone the notebook"
data-toggle="modal" data-target="#noteNameModal" data-clone="true"
>
<i class="fa fa-copy"></i>
</button>
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
ng-click="exportNotebook()"
tooltip-placement="bottom" tooltip="Export the notebook">
<i class="fa fa-download"></i>
</button>
</span>
<span ng-hide="viewOnly">
<div class="labelBtn btn-group">
<div class="btn btn-default btn-xs dropdown-toggle"
type="button"
data-toggle="dropdown"
ng-class="{ 'btn-info' : note.config.cron, 'btn-danger' : note.info.cron, 'btn-default' : !note.config.cron}"
tooltip-placement="bottom" tooltip="Run scheduler">
<span class="fa fa-clock-o"></span> {{getCronOptionNameFromValue(note.config.cron)}}
</div>
<ul class="dropdown-menu" role="menu" style="width:300px">
<li>
<div style="padding:10px 20px 0 20px;font-weight:normal;word-wrap:break-word">
Run note with cron scheduler.
Either choose from<br/>preset or write your own <a href="http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger" target=_blank>cron expression</a>.
<br/><br/>
<span>- Preset</span>
<a ng-repeat="cr in cronOption"
type="button"
ng-click="setCronScheduler(cr.value)"
style="cursor:pointer"
dropdown-input>{{cr.name}}</a>
<br/><br/>
<span>- Cron expression</span>
<input type="text"
ng-model="note.config.cron"
ng-change="setCronScheduler(note.config.cron)"
dropdown-input />
<p ng-show="note.info.cron"
style="color:red">
{{note.info.cron}}
</p>
</div>
</li>
</ul>
</div>
</span>
<div class="pull-right" style="margin-top:15px; margin-right:15px; font-size:15px;">
<span style="position:relative; top:3px; margin-right:4px; cursor:pointer"
data-toggle="modal"
data-target="#shortcutModal"
tooltip-placement="bottom" tooltip="List of shortcut">
<i class="icon-question"></i>
</span>
<span style="position:relative; top:2px; margin-right:4px; cursor:pointer;"
ng-click="toggleSetting()"
tooltip-placement="bottom" tooltip="Interpreter binding">
<i class="fa fa-cog" ng-style="{color: showSetting ? '#3071A9' : 'black' }"></i>
</span>
<span class="btn-group">
<button type="button" class="btn btn-default btn-xs dropdown-toggle"
data-toggle="dropdown">
{{note.config.looknfeel}} <span class="caret"></span>
</button>
<ul class="dropdown-menu pull-right" role="menu">
<li ng-repeat="looknfeel in looknfeelOption">
<a style="cursor:pointer" ng-click="setLookAndFeel(looknfeel)">{{looknfeel}}</a>
</li>
</ul>
</span>
</div>
</h3>
</div>
<div ng-include src="'app/notebook/notebook-actionBar.html'"></div>
<div style="padding-top: 36px;">
<!-- settings -->
<div ng-if="showSetting" class="setting">
@ -188,12 +66,18 @@ limitations under the License.
ng-Init="init(currentParagraph)"
ng-class="columnWidthClass(currentParagraph.config.colWidth)"
class="paragraph-col">
<div class="new-paragraph" ng-click="insertNew('above')" ng-hide="viewOnly">
<h4 class="plus-sign">&#43;</h4>
</div>
<div id="{{currentParagraph.id}}_paragraphColumn"
ng-include src="'app/notebook/paragraph/paragraph.html'"
ng-class="{'paragraph-space box paragraph-margin': !asIframe, 'focused': paragraphFocused,
'lastEmptyParagraph': !paragraph.text && !paragraph.result}"
ng-hide="currentParagraph.config.tableHide && viewOnly">
</div>
<div class="new-paragraph" ng-click="insertNew('below');" ng-hide="!$last || viewOnly">
<h4 class="plus-sign">&#43;</h4>
</div>
</div>
<div style="clear:both;height:10px"></div>
</div>

49
zeppelin-web/src/app/notebook/paragraph/paragraph-chart-selector.html Normal file
View file

@ -0,0 +1,49 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="{{paragraph.id}}_switch"
ng-if="paragraph.result.type == 'TABLE' && !asIframe && !viewOnly"
class="btn-group"
style='margin-bottom: 10px;'>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('table')}"
ng-click="setGraphMode('table', true)" ><i class="fa fa-table"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('multiBarChart')}"
ng-click="setGraphMode('multiBarChart', true)"><i class="fa fa-bar-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('pieChart')}"
ng-click="setGraphMode('pieChart', true)"><i class="fa fa-pie-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('stackedAreaChart')}"
ng-click="setGraphMode('stackedAreaChart', true)"><i class="fa fa-area-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('lineChart') || isGraphMode('lineWithFocusChart')}"
ng-click="paragraph.config.graph.lineWithFocus ? setGraphMode('lineWithFocusChart', true) : setGraphMode('lineChart', true)"><i class="fa fa-line-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('scatterChart')}"
ng-click="setGraphMode('scatterChart', true)"><i class="cf cf-scatter-chart"></i>
</button>
</div>
<span ng-if="getResultType()=='TABLE' && getGraphMode()!='table' && !asIframe && !viewOnly"
style="margin-left:10px; cursor:pointer; display: inline-block; vertical-align:top; position: relative; line-height:30px;">
<a class="btnText" ng-click="toggleGraphOption()">
settings <span ng-class="paragraph.config.graph.optionOpen ? 'fa fa-caret-up' : 'fa fa-caret-down'"></span>
</a>
</span>

90
zeppelin-web/src/app/notebook/paragraph/paragraph-control.html Normal file
View file

@ -0,0 +1,90 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="{{paragraph.id}}_control" class="control" ng-show="!asIframe">
<span>
{{paragraph.status}}
</span>
<span ng-if="paragraph.status=='RUNNING'">
{{getProgress()}}%
</span>
<!-- Run / Cancel button -->
<span class="icon-control-play" style="cursor:pointer;color:#3071A9" tooltip-placement="top" tooltip="Run this paragraph (Shift+Enter)"
ng-click="runParagraph(getEditorValue())"
ng-show="paragraph.status!='RUNNING' && paragraph.status!='PENDING' && paragraph.config.enabled"></span>
<span class="icon-control-pause" style="cursor:pointer;color:#CD5C5C" tooltip-placement="top" tooltip="Cancel"
ng-click="cancelParagraph()"
ng-show="paragraph.status=='RUNNING' || paragraph.status=='PENDING'"></span>
<span class="{{paragraph.config.editorHide ? 'icon-size-fullscreen' : 'icon-size-actual'}}" style="cursor:pointer;" tooltip-placement="top" tooltip="{{(paragraph.config.editorHide ? 'Show' : 'Hide') + ' editor'}}"
ng-click="toggleEditor()"></span>
<span class="{{paragraph.config.tableHide ? 'icon-notebook' : 'icon-book-open'}}" style="cursor:pointer;" tooltip-placement="top" tooltip="{{(paragraph.config.tableHide ? 'Show' : 'Hide') + ' output'}}"
ng-click="toggleOutput()"></span>
<span class="dropdown navbar-right">
<span class="icon-settings" style="cursor:pointer"
data-toggle="dropdown"
type="button">
</span>
<ul class="dropdown-menu" role="menu" style="width:200px;">
<li>
<a ng-click="$event.stopPropagation()" class="dropdown"><span class="fa fa-arrows-h"></span> Width
<form style="display:inline; margin-left:5px;">
<select ng-model="paragraph.config.colWidth"
class="selectpicker"
ng-change="changeColWidth()"
ng-options="col for col in colWidthOption"></select>
</form>
</a>
</li>
<li>
<a ng-click="moveUp()"><span class="icon-arrow-up"></span> Move Up</a>
</li>
<li>
<a ng-click="moveDown()"><span class="icon-arrow-down"></span> Move Down</a>
</li>
<li>
<a ng-click="insertNew()"><span class="icon-plus"></span> Insert New</a>
</li>
<li>
<!-- paragraph handler -->
<a ng-click="hideTitle()"
ng-show="paragraph.config.title"><span class="fa fa-font"></span> Hide title</a>
<a ng-click="showTitle()"
ng-show="!paragraph.config.title"><span class="fa fa-font"></span> Show title</a>
</li>
<li>
<a ng-click="hideLineNumbers()"
ng-show="paragraph.config.lineNumbers"><span class="fa fa-list-ol"></span> Hide line numbers</a>
<a ng-click="showLineNumbers()"
ng-show="!paragraph.config.lineNumbers"><span class="fa fa-list-ol"></span> Show line numbers</a>
</li>
<li>
<a ng-click="toggleEnableDisable()"><span class="icon-control-play"></span>
{{paragraph.config.enabled ? "Disable" : "Enable"}} run</a>
</li>
<li>
<a ng-click="goToSingleParagraph()"><span class="icon-share-alt"></span> Link this paragraph</a>
</li>
<li>
<a ng-click="clearParagraphOutput()"><span class="fa fa-eraser"></span> Clear output</a>
</li>
<li>
<!-- remove paragraph -->
<a ng-click="removeParagraph()"><span class="fa fa-times"></span> Remove</a>
</li>
</ul>
</span>
</div>

55
zeppelin-web/src/app/notebook/paragraph/paragraph-graph.html Normal file
View file

@ -0,0 +1,55 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="p{{paragraph.id}}_graph"
class="graphContainer"
ng-class="{'noOverflow': getGraphMode()=='table'}"
ng-if="getResultType()=='TABLE'"
allowresize="{{!asIframe && !viewOnly}}"
resizable on-resize="setGraphHeight();">
<div ng-if="getGraphMode()=='table'"
id="p{{paragraph.id}}_table"
class="table">
</div>
<div ng-if="getGraphMode()=='multiBarChart'"
id="p{{paragraph.id}}_multiBarChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='pieChart'"
id="p{{paragraph.id}}_pieChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='stackedAreaChart'"
id="p{{paragraph.id}}_stackedAreaChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='lineChart'"
id="p{{paragraph.id}}_lineChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='lineWithFocusChart'"
id="p{{paragraph.id}}_lineWithFocusChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='scatterChart'"
id="p{{paragraph.id}}_scatterChart">
<svg></svg>
</div>
</div>

31
zeppelin-web/src/app/notebook/paragraph/paragraph-graphOptions.html Normal file
View file

@ -0,0 +1,31 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div>
<div ng-if="isGraphMode('lineChart') || isGraphMode('lineWithFocusChart')">
<label>
<input type="checkbox"
ng-model="paragraph.config.graph.forceY"
ng-click="onGraphOptionChange()">
force Y to 0
</label>
<br/>
<label>
<input type="checkbox"
ng-model="paragraph.config.graph.lineWithFocus"
ng-click="toggleLineWithFocus()">
show line chart with focus
</label>
</div>
</div>

40
zeppelin-web/src/app/notebook/paragraph/paragraph-parameterizedQueryForm.html Normal file
View file

@ -0,0 +1,40 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<form id="{{paragraph.id}}_form" role="form"
ng-show="!paragraph.config.tableHide"
class=" paragraphForm form-horizontal row">
<div class="form-group col-sm-6 col-md-6 col-lg-4"
ng-repeat="formulaire in paragraph.settings.forms"
ng-Init="loadForm(formulaire, paragraph.settings.params)">
<label class="control-label input-sm" ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING' }">{{formulaire.name}}</label>
<div>
<input class="form-control input-sm"
ng-if="!paragraph.settings.forms[formulaire.name].options"
ng-enter="runParagraph(getEditorValue())"
ng-model="paragraph.settings.params[formulaire.name]"
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING' }"
name="{{formulaire.name}}" />
<select class="form-control input-sm"
ng-if="paragraph.settings.forms[formulaire.name].options"
ng-change="runParagraph(getEditorValue())"
ng-model="paragraph.settings.params[formulaire.name]"
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING' }"
name="{{formulaire.name}}"
ng-options="option.value as (option.displayName||option.value) for option in paragraph.settings.forms[formulaire.name].options">
</select>
</div>
</div>
</form>

168
zeppelin-web/src/app/notebook/paragraph/paragraph-pivot.html Normal file
View file

@ -0,0 +1,168 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div class="option lightBold" style="overflow: visible;"
ng-if="getResultType()=='TABLE' && getGraphMode()!='table'
&& paragraph.config.graph.optionOpen && !asIframe && !viewOnly">
<div ng-include src="'app/notebook/paragraph/paragraph-graphOptions.html'"></div>
All fields:
<div class="allFields row">
<ul class="noDot">
<li class="liVertical" ng-repeat="col in paragraph.result.columnNames">
<div class="btn btn-default btn-xs"
data-drag="true"
data-jqyoui-options="{revert: 'invalid', helper: 'clone'}"
ng-model="paragraph.result.columnNames"
jqyoui-draggable="{index: {{$index}}, placeholder: 'keep'}">
{{col.name | limitTo: 30}}{{col.name.length > 30 ? '...' : ''}}
</div>
</li>
</ul>
</div>
<div class="row" ng-if="getGraphMode()!='scatterChart'">
<div class="col-md-4">
<span class="columns lightBold">
Keys
<ul data-drop="true"
ng-model="paragraph.config.graph.keys"
jqyoui-droppable="{multiple:true, onDrop:'onGraphOptionChange()'}"
class="list-unstyled">
<li ng-repeat="item in paragraph.config.graph.keys">
<div class="btn btn-primary btn-xs">
{{item.name}} <span class="fa fa-close" ng-click="removeGraphOptionKeys($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-4">
<span class="columns lightBold">
Groups
<ul data-drop="true"
ng-model="paragraph.config.graph.groups"
jqyoui-droppable="{multiple:true, onDrop:'onGraphOptionChange()'}"
class="list-unstyled">
<li ng-repeat="item in paragraph.config.graph.groups">
<div class="btn btn-success btn-xs">
{{item.name}} <span class="fa fa-close" ng-click="removeGraphOptionGroups($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-4">
<span class="columns lightBold">
Values
<ul data-drop="true"
ng-model="paragraph.config.graph.values"
jqyoui-droppable="{multiple:true, onDrop:'onGraphOptionChange()'}"
class="list-unstyled">
<li ng-repeat="item in paragraph.config.graph.values">
<div class="btn-group">
<div class="btn btn-info btn-xs dropdown-toggle"
type="button"
data-toggle="dropdown">
{{item.name | limitTo: 30}}{{item.name.length > 30 ? '...' : ''}}
<font style="color:#EEEEEE;"><span class="lightBold" style="text-transform: uppercase;">{{item.aggr}}</span></font>
<span class="fa fa-close" ng-click="removeGraphOptionValues($index)"></span>
</div>
<ul class="dropdown-menu" role="menu">
<li ng-click="setGraphOptionValueAggr($index, 'sum')"><a>sum</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'count')"><a>count</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'avg')"><a>avg</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'min')"><a>min</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'max')"><a>max</a></li>
</ul>
</div>
</li>
</ul>
</span>
</div>
</div>
<div class="row" ng-if="getGraphMode()=='scatterChart'">
<div class="col-md-3">
<span class="columns lightBold">
xAxis
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.xAxis"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.xAxis">
<div class="btn btn-primary btn-xs">
{{paragraph.config.graph.scatter.xAxis.name}} <span class="fa fa-close" ng-click="removeScatterOptionXaxis($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-3">
<span class="columns lightBold">
yAxis
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.yAxis"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.yAxis">
<div class="btn btn-success btn-xs">
{{paragraph.config.graph.scatter.yAxis.name}} <span class="fa fa-close" ng-click="removeScatterOptionYaxis($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-3">
<span class="columns lightBold">
group
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.group"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.group">
<div class="btn btn-info btn-xs">
{{paragraph.config.graph.scatter.group.name}} <span class="fa fa-close" ng-click="removeScatterOptionGroup($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-3">
<span class="columns lightBold">
size
<a tabindex="0" class="fa fa-info-circle" role="button" popover-placement="top"
popover-trigger="focus"
popover-html-unsafe="<li>Size option is valid only when you drop numeric field here.</li>
<li>When data in each axis are discrete, 'number of values in corresponding coordinate' will be used as size.</li>
<li>Zeppelin consider values as discrete when the values contain string value or the number of distinct values are bigger than 5% of total number of values.</li>
<li>Size field button turns to grey when the option you chose is not valid.</li>"></a>
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.size"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.size">
<div class="btn btn-xs" style="color:white" ng-class="{'btn-warning': isValidSizeOption(paragraph.config.graph.scatter, paragraph.result.rows)}">
{{paragraph.config.graph.scatter.size.name}} <span class="fa fa-close" ng-click="removeScatterOptionSize($index)"></span>
</div>
</li>
</ul>
</span>
</div>
</div>
</div>

22
zeppelin-web/src/app/notebook/paragraph/paragraph-progressBar.html Normal file
View file

@ -0,0 +1,22 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="{{paragraph.id}}_runControl" class="runControl" ng-show="paragraph.status=='RUNNING'">
<div id="{{paragraph.id}}_progress" class="progress">
<div ng-if="getProgress()>0 && getProgress()<100 && paragraph.status=='RUNNING'"
class="progress-bar" role="progressbar" style="width:{{getProgress()}}%;"></div>
<div ng-if="(getProgress()<=0 || getProgress()>=100) && (paragraph.status=='RUNNING' )"
class="progress-bar progress-bar-striped active" role="progressbar" style="width:100%;"></div>
</div>
</div>

47
zeppelin-web/src/app/notebook/paragraph/paragraph-results.html Normal file
View file

@ -0,0 +1,47 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div ng-include src="'app/notebook/paragraph/paragraph-graph.html'"></div>
<div id="{{paragraph.id}}_comment"
class="text"
ng-if="getResultType()=='TABLE' && paragraph.result.comment"
ng-bind-html="paragraph.result.comment">
</div>
<div id="{{paragraph.id}}_text"
class="text"
ng-if="paragraph.result.type == 'TEXT'"
ng-bind="paragraph.result.msg">
</div>
<div id="p{{paragraph.id}}_html"
class="resultContained"
ng-if="paragraph.result.type == 'HTML'">
</div>
<div id="p{{paragraph.id}}_angular"
class="resultContained"
ng-if="paragraph.result.type == 'ANGULAR'">
</div>
<img id="{{paragraph.id}}_img"
ng-if="paragraph.result.type == 'IMG'"
ng-src="{{getBase64ImageSrc(paragraph.result.msg)}}">
</img>
<div id="{{paragraph.id}}_error"
class="error"
ng-if="paragraph.status == 'ERROR'"
ng-bind="paragraph.errorMessage">
</div>

102
zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
View file

@ -19,6 +19,7 @@ angular.module('zeppelinWebApp')
$timeout, $compile, websocketMsgSrv) {
$scope.paragraph = null;
$scope.originalText = '';
$scope.editor = null;
var editorModes = {
@ -31,6 +32,7 @@ angular.module('zeppelinWebApp')
// Controller init
$scope.init = function(newParagraph) {
$scope.paragraph = newParagraph;
$scope.originalText = angular.copy(newParagraph.text);
$scope.chart = {};
$scope.colWidthOption = [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 ];
$scope.showTitleEditor = false;
@ -65,10 +67,10 @@ angular.module('zeppelinWebApp')
console.log('HTML rendering error %o', err);
}
} else {
$timeout(retryRenderer, 10);
}
};
$timeout(retryRenderer);
};
$scope.renderAngular = function() {
@ -82,11 +84,10 @@ angular.module('zeppelinWebApp')
console.log('ANGULAR rendering error %o', err);
}
} else {
$timeout(retryRenderer,10);
$timeout(retryRenderer, 10);
}
};
$timeout(retryRenderer);
};
@ -181,11 +182,13 @@ angular.module('zeppelinWebApp')
if ($scope.dirtyText === data.paragraph.text ) { // when local update is the same from remote, clear local update
$scope.paragraph.text = data.paragraph.text;
$scope.dirtyText = undefined;
$scope.originalText = angular.copy(data.paragraph.text);
} else { // if there're local update, keep it.
$scope.paragraph.text = $scope.dirtyText;
}
} else {
$scope.paragraph.text = data.paragraph.text;
$scope.originalText = angular.copy(data.paragraph.text);
}
}
@ -261,14 +264,16 @@ angular.module('zeppelinWebApp')
$scope.runParagraph = function(data) {
websocketMsgSrv.runParagraph($scope.paragraph.id, $scope.paragraph.title,
data, $scope.paragraph.config, $scope.paragraph.settings.params);
$scope.originalText = angular.copy(data);
$scope.dirtyText = undefined;
};
$scope.saveParagraph = function(){
if($scope.dirtyText === undefined){
if($scope.dirtyText === undefined || $scope.dirtyText === $scope.originalText){
return;
}
commitParagraph($scope.paragraph.title, $scope.dirtyText, $scope.paragraph.config, $scope.paragraph.settings.params);
$scope.originalText = angular.copy($scope.dirtyText);
$scope.dirtyText = undefined;
};
@ -287,16 +292,22 @@ angular.module('zeppelinWebApp')
$scope.$emit('moveParagraphDown', $scope.paragraph.id);
};
$scope.insertNew = function() {
$scope.$emit('insertParagraph', $scope.paragraph.id);
$scope.insertNew = function(position) {
$scope.$emit('insertParagraph', $scope.paragraph.id, position || 'below');
};
$scope.removeParagraph = function() {
var result = confirm('Do you want to delete this paragraph?');
if (result) {
console.log('Remove paragraph');
websocketMsgSrv.removeParagraph($scope.paragraph.id);
}
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this paragraph?',
callback: function(result) {
if (result) {
console.log('Remove paragraph');
websocketMsgSrv.removeParagraph($scope.paragraph.id);
}
}
});
};
$scope.clearParagraphOutput = function() {
@ -400,7 +411,7 @@ angular.module('zeppelinWebApp')
};
$scope.changeColWidth = function() {
angular.element('.navbar-right.open').removeClass('open');
var newParams = angular.copy($scope.paragraph.settings.params);
var newConfig = angular.copy($scope.paragraph.config);
@ -459,7 +470,6 @@ angular.module('zeppelinWebApp')
};
$scope.aceChanged = function() {
$scope.dirtyText = $scope.editor.getSession().getValue();
$scope.startSaveTimer();
@ -587,7 +597,9 @@ angular.module('zeppelinWebApp')
exec: function(editor) {
var editorValue = editor.getValue();
if (editorValue) {
$scope.runParagraph(editorValue);
if (!($scope.paragraph.status === 'RUNNING' || $scope.paragraph.status === 'PENDING')) {
$scope.runParagraph(editorValue);
}
}
},
readOnly: false
@ -930,14 +942,18 @@ angular.module('zeppelinWebApp')
var renderTable = function() {
var html = '';
html += '<table class="table table-hover table-condensed">';
html += '<table class="table table-hover table-condensed" style="top: 0; position: absolute;">';
html += ' <thead>';
html += ' <tr style="background-color: #F6F6F6; font-weight: bold;">';
for (var c in $scope.paragraph.result.columnNames) {
html += '<th>'+$scope.paragraph.result.columnNames[c].name+'</th>';
for (var titleIndex in $scope.paragraph.result.columnNames) {
html += '<th>'+$scope.paragraph.result.columnNames[titleIndex].name+'</th>';
}
html += ' </tr>';
html += ' </thead>';
html += '</table>';
html += '<table class="table table-hover table-condensed" style="margin-top: 31px;">';
for (var r in $scope.paragraph.result.msgTable) {
var row = $scope.paragraph.result.msgTable[r];
@ -983,6 +999,31 @@ angular.module('zeppelinWebApp')
};
var integerFormatter = d3.format(',.1d');
var customAbbrevFormatter = function(x) {
var s = d3.format('.3s')(x);
switch (s[s.length - 1]) {
case 'G': return s.slice(0, -1) + 'B';
}
return s;
};
var xAxisTickFormat = function(d, xLabels) {
if (xLabels[d] && (isNaN(parseFloat(xLabels[d])) || !isFinite(xLabels[d]))) { // to handle string type xlabel
return xLabels[d];
} else {
return d;
}
};
var yAxisTickFormat = function(d) {
if(d >= Math.pow(10,6)){
return customAbbrevFormatter(d);
}
return integerFormatter(d);
};
var setD3Chart = function(type, data, refresh) {
if (!$scope.chart[type]) {
var chart = nv.models[type]();
@ -1000,21 +1041,8 @@ angular.module('zeppelinWebApp')
yLabels = scatterData.yLabels;
d3g = scatterData.d3g;
$scope.chart[type].xAxis.tickFormat(function(d) {
if (xLabels[d] && (isNaN(parseFloat(xLabels[d])) || !isFinite(xLabels[d]))) {
return xLabels[d];
} else {
return d;
}
});
$scope.chart[type].yAxis.tickFormat(function(d) {
if (yLabels[d] && (isNaN(parseFloat(yLabels[d])) || !isFinite(yLabels[d]))) {
return yLabels[d];
} else {
return d;
}
});
$scope.chart[type].xAxis.tickFormat(function(d) {return xAxisTickFormat(d, xLabels);});
$scope.chart[type].yAxis.tickFormat(function(d) {return xAxisTickFormat(d, yLabels);});
// configure how the tooltip looks.
$scope.chart[type].tooltipContent(function(key, x, y, data) {
@ -1051,17 +1079,13 @@ angular.module('zeppelinWebApp')
} else if (type === 'multiBarChart') {
d3g = pivotDataToD3ChartFormat(p, true, false, type).d3g;
$scope.chart[type].yAxis.axisLabelDistance(50);
$scope.chart[type].yAxis.tickFormat(function(d) {return yAxisTickFormat(d);});
} else if (type === 'lineChart' || type === 'stackedAreaChart' || type === 'lineWithFocusChart') {
var pivotdata = pivotDataToD3ChartFormat(p, false, true);
xLabels = pivotdata.xLabels;
d3g = pivotdata.d3g;
$scope.chart[type].xAxis.tickFormat(function(d) {
if (xLabels[d] && (isNaN(parseFloat(xLabels[d])) || !isFinite(xLabels[d]))) { // to handle string type xlabel
return xLabels[d];
} else {
return d;
}
});
$scope.chart[type].xAxis.tickFormat(function(d) {return xAxisTickFormat(d, xLabels);});
$scope.chart[type].yAxis.tickFormat(function(d) {return yAxisTickFormat(d);});
$scope.chart[type].yAxis.axisLabelDistance(50);
if ($scope.chart[type].useInteractiveGuideline) { // lineWithFocusChart hasn't got useInteractiveGuideline
$scope.chart[type].useInteractiveGuideline(true); // for better UX and performance issue. (https://github.com/novus/nvd3/issues/691)

38
zeppelin-web/src/app/notebook/paragraph/paragraph.css
View file

@ -32,18 +32,16 @@
*/
.paragraph .text {
white-space: pre;
display: block;
unicode-bidi: embed;
display: block !important;
margin: 0 0 0 !important;
font-family: "Monaco","Menlo","Ubuntu Mono","Consolas","source-code-pro",monospace;
font-size: 12px !important;
line-height: 1.42857143 !important;
margin: 0 0 5px !important;
padding-top: 2px;
unicode-bidi: embed;
white-space: pre-wrap;
word-break: break-all !important;
word-wrap: break-word !important;
font-family: 'Monaco', 'Menlo', 'Ubuntu Mono', 'Consolas', 'source-code-pro', monospace;
font-size: 12px !important;
margin-bottom: 5px !important;
padding-top: 2px;
}
.paragraph table {
@ -142,7 +140,7 @@
font-size: 1px;
color: #AAAAAA;
height:4px;
margin: 1px 0 0 0;
margin: 0px 0px 3px 0px;
}
.paragraph .runControl .progress {
@ -153,10 +151,6 @@
border-radius: 0;
}
.paragraph .runControl .progress .progress-bar {
z-index: 100;
}
.paragraph .control span {
margin-left: 4px;
}
@ -209,6 +203,7 @@
.paragraph .title input {
width: 80%;
height: 24px;
line-height: 1.42857143;
color: #555;
background: #fff none;
@ -216,12 +211,12 @@
border-radius: 0;
-webkit-box-shadow: inset 0 1px 1px rgba(0, 0, 0, .075);
box-shadow: inset 0 1px 1px rgba(0, 0, 0, .075);
-webkit-transition: border-color ease-in-out .15s, -webkit-box-shadow ease-in-out .15s;
-o-transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
transition: border-color ease-in-out .15s, box-shadow ease-in-out .15s;
text-transform: capitalize;
font-family: 'Roboto', sans-serif;
font-size: 14px!important;
font-size: 17px !important;
font-weight: bold;
padding: 0 2px;
margin-left: -3px;
}
/*
@ -404,3 +399,12 @@
.lightBold {
font-weight: 500;
}
.dropdown-menu > li:first-child > a:hover {
background-color: transparent;
}
table.table-striped {
border-top: 1px solid #ddd;
margin-top: 20px;
}

450
zeppelin-web/src/app/notebook/paragraph/paragraph.html
View file

@ -19,19 +19,23 @@ limitations under the License.
id="{{paragraph.id}}_title"
class="title">
<input type="text"
placeholder="Edit title"
pu-elastic-input
style="min-width: 400px; max-width: 80%;"
placeholder="Untitled"
ng-model="paragraph.title"
ng-show="showTitleEditor"
ng-escape="showTitleEditor = false"
ng-enter="setTitle(); showTitleEditor = false"/>
<div ng-click="showTitleEditor = !asIframe && !viewOnly"
ng-escape="showTitleEditor = false; paragraph.title = oldTitle;"
ng-blur="setTitle(); showTitleEditor = false"
ng-enter="setTitle(); showTitleEditor = false"
focus-if="showTitleEditor" />
<div ng-click="showTitleEditor = !asIframe && !viewOnly; oldTitle = paragraph.title;"
ng-show="!showTitleEditor"
ng-bind-html="paragraph.title || 'Untitled'">
</div>
</div>
<div>
<div ng-show="!paragraph.config.editorHide && !viewOnly">
<div ng-show="!paragraph.config.editorHide && !viewOnly" style="margin-bottom:3px;">
<div id="{{paragraph.id}}_editor"
style="opacity: 1;"
class="editor"
@ -41,442 +45,28 @@ limitations under the License.
require : ['ace/ext/language_tools']
}"
ng-model="paragraph.text"
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING', 'paragraph-text--dirty' : dirtyText !== undefined}">
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING',
'paragraph-text--dirty' : dirtyText !== originalText && dirtyText !== undefined}">
</div>
</div>
<div id="{{paragraph.id}}_runControl" class="runControl">
<div ng-if="(getProgress()<=0 || getProgress()>=100) && (paragraph.status=='RUNNING' )">
<div id="{{paragraph.id}}_progress"
class="progress">
<div class="progress-bar progress-bar-striped active" role="progressbar" style="width:100%;"></div>
<span class="sr-only"></span>
</div>
</div>
<div ng-if="getProgress()>0 && getProgress()<100 && paragraph.status=='RUNNING'">
<div id="{{paragraph.id}}_progress"
class="progress">
<div class="progress-bar" role="progressbar" style="width:{{getProgress()}}%;"></div>
<span class="sr-only">{{getProgress()}}%</span>
</div>
</div>
</div>
<form id="{{paragraph.id}}_form" role="form"
ng-show="!paragraph.config.tableHide"
class=" paragraphForm form-horizontal row">
<div class="form-group col-sm-6 col-md-6 col-lg-4"
ng-repeat="formulaire in paragraph.settings.forms"
ng-Init="loadForm(formulaire, paragraph.settings.params)">
<label class="control-label input-sm" ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING' }">{{formulaire.name}}</label>
<div>
<input class="form-control input-sm"
ng-if="!paragraph.settings.forms[formulaire.name].options"
ng-enter="runParagraph(getEditorValue())"
ng-model="paragraph.settings.params[formulaire.name]"
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING' }"
name="{{formulaire.name}}">
</input>
<select class="form-control input-sm"
ng-if="paragraph.settings.forms[formulaire.name].options"
ng-change="runParagraph(getEditorValue())"
ng-model="paragraph.settings.params[formulaire.name]"
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING' }"
name="{{formulaire.name}}"
ng-options="option.value as (option.displayName||option.value) for option in paragraph.settings.forms[formulaire.name].options"
>
<!--
<option
ng-repeat="option in paragraph.settings.forms[formulaire.name].options"
value="{{option.value}}"
>{{option.displayName || option.value}}
</option>
-->
</select>
</div>
</div>
</form>
<div ng-include src="'app/notebook/paragraph/paragraph-progressBar.html'"></div>
<div ng-include src="'app/notebook/paragraph/paragraph-parameterizedQueryForm.html'"></div>
<!-- Rendering -->
<div class='tableDisplay' ng-show="!paragraph.config.tableHide">
<div id="{{paragraph.id}}_switch"
ng-if="paragraph.result.type == 'TABLE' && !asIframe && !viewOnly"
class="btn-group"
style='margin-bottom: 10px;'>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('table')}"
ng-click="setGraphMode('table', true)" ><i class="fa fa-table"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('multiBarChart')}"
ng-click="setGraphMode('multiBarChart', true)"><i class="fa fa-bar-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('pieChart')}"
ng-click="setGraphMode('pieChart', true)"><i class="fa fa-pie-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('stackedAreaChart')}"
ng-click="setGraphMode('stackedAreaChart', true)"><i class="fa fa-area-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('lineChart') || isGraphMode('lineWithFocusChart')}"
ng-click="paragraph.config.graph.lineWithFocus ? setGraphMode('lineWithFocusChart', true) : setGraphMode('lineChart', true)"><i class="fa fa-line-chart"></i>
</button>
<button type="button" class="btn btn-default btn-sm"
ng-class="{'active': isGraphMode('scatterChart')}"
ng-click="setGraphMode('scatterChart', true)"><i class="cf cf-scatter-chart"></i>
</button>
</div>
<span ng-if="getResultType()=='TABLE' && getGraphMode()!='table' && !asIframe && !viewOnly"
style="margin-left:10px; cursor:pointer; display: inline-block; vertical-align:top; position: relative; line-height:30px;">
<a ng-if="paragraph.config.graph.optionOpen"
ng-click="toggleGraphOption()">
settings <span class="fa fa-caret-up"></span>
</a>
<a ng-if="!paragraph.config.graph.optionOpen"
ng-click="toggleGraphOption()" >
settings <span class="fa fa-caret-down"></span>
</a>
</span>
<div class="option lightBold" style="overflow: visible;"
ng-if="getResultType()=='TABLE' && getGraphMode()!='table'
&& paragraph.config.graph.optionOpen && !asIframe && !viewOnly">
<div ng-if="isGraphMode('lineChart') || isGraphMode('lineWithFocusChart')">
<label>
<input type="checkbox"
ng-model="paragraph.config.graph.forceY"
ng-click="onGraphOptionChange()">
force Y to 0
</label>
<br/>
<label>
<input type="checkbox"
ng-model="paragraph.config.graph.lineWithFocus"
ng-click="toggleLineWithFocus()">
show line chart with focus
</label>
</div>
All fields:
<div class="allFields row">
<ul class="noDot">
<li class="liVertical" ng-repeat="col in paragraph.result.columnNames">
<div class="btn btn-default btn-xs"
data-drag="true"
data-jqyoui-options="{revert: 'invalid', helper: 'clone'}"
ng-model="paragraph.result.columnNames"
jqyoui-draggable="{index: {{$index}}, placeholder: 'keep'}">
{{col.name | limitTo: 30}}{{col.name.length > 30 ? '...' : ''}}
</div>
</li>
</ul>
</div>
<div class="row" ng-if="getGraphMode()!='scatterChart'">
<div class="col-md-4">
<span class="columns lightBold">
Keys
<ul data-drop="true"
ng-model="paragraph.config.graph.keys"
jqyoui-droppable="{multiple:true, onDrop:'onGraphOptionChange()'}"
class="list-unstyled">
<li ng-repeat="item in paragraph.config.graph.keys">
<div class="btn btn-primary btn-xs">
{{item.name}} <span class="fa fa-close" ng-click="removeGraphOptionKeys($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-4">
<span class="columns lightBold">
Groups
<ul data-drop="true"
ng-model="paragraph.config.graph.groups"
jqyoui-droppable="{multiple:true, onDrop:'onGraphOptionChange()'}"
class="list-unstyled">
<li ng-repeat="item in paragraph.config.graph.groups">
<div class="btn btn-success btn-xs">
{{item.name}} <span class="fa fa-close" ng-click="removeGraphOptionGroups($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-4">
<span class="columns lightBold">
Values
<ul data-drop="true"
ng-model="paragraph.config.graph.values"
jqyoui-droppable="{multiple:true, onDrop:'onGraphOptionChange()'}"
class="list-unstyled">
<li ng-repeat="item in paragraph.config.graph.values">
<div class="btn-group">
<div class="btn btn-info btn-xs dropdown-toggle"
type="button"
data-toggle="dropdown">
{{item.name | limitTo: 30}}{{item.name.length > 30 ? '...' : ''}}
<font style="color:#EEEEEE;"><span class="lightBold" style="text-transform: uppercase;">{{item.aggr}}</span></font>
<span class="fa fa-close" ng-click="removeGraphOptionValues($index)"></span>
</div>
<ul class="dropdown-menu" role="menu">
<li ng-click="setGraphOptionValueAggr($index, 'sum')"><a>sum</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'count')"><a>count</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'avg')"><a>avg</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'min')"><a>min</a></li>
<li ng-click="setGraphOptionValueAggr($index, 'max')"><a>max</a></li>
</ul>
</div>
</li>
</ul>
</span>
</div>
</div>
<div class="row" ng-if="getGraphMode()=='scatterChart'">
<div class="col-md-3">
<span class="columns lightBold">
xAxis
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.xAxis"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.xAxis">
<div class="btn btn-primary btn-xs">
{{paragraph.config.graph.scatter.xAxis.name}} <span class="fa fa-close" ng-click="removeScatterOptionXaxis($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-3">
<span class="columns lightBold">
yAxis
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.yAxis"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.yAxis">
<div class="btn btn-success btn-xs">
{{paragraph.config.graph.scatter.yAxis.name}} <span class="fa fa-close" ng-click="removeScatterOptionYaxis($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-3">
<span class="columns lightBold">
group
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.group"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.group">
<div class="btn btn-info btn-xs">
{{paragraph.config.graph.scatter.group.name}} <span class="fa fa-close" ng-click="removeScatterOptionGroup($index)"></span>
</div>
</li>
</ul>
</span>
</div>
<div class="col-md-3">
<span class="columns lightBold">
size
<a tabindex="0" class="fa fa-info-circle" role="button" popover-placement="top"
popover-trigger="focus"
popover-html-unsafe="<li>Size option is valid only when you drop numeric field here.</li>
<li>When data in each axis are discrete, 'number of values in corresponding coordinate' will be used as size.</li>
<li>Zeppelin consider values as discrete when the values contain string value or the number of distinct values are bigger than 5% of total number of values.</li>
<li>Size field button turns to grey when the option you chose is not valid.</li>"></a>
<ul data-drop="true"
ng-model="paragraph.config.graph.scatter.size"
jqyoui-droppable="{onDrop:'onGraphOptionChange()'}"
class="list-unstyled"
style="height:36px">
<li ng-if="paragraph.config.graph.scatter.size">
<div class="btn btn-xs" style="color:white" ng-class="{'btn-warning': isValidSizeOption(paragraph.config.graph.scatter, paragraph.result.rows)}">
{{paragraph.config.graph.scatter.size.name}} <span class="fa fa-close" ng-click="removeScatterOptionSize($index)"></span>
</div>
</li>
</ul>
</span>
</div>
</div>
</div>
<div id="p{{paragraph.id}}_graph"
class="graphContainer"
ng-class="{'noOverflow': getGraphMode()=='table'}"
ng-if="getResultType()=='TABLE'"
allowresize="{{!asIframe && !viewOnly}}"
resizable on-resize="setGraphHeight();">
<div ng-if="getGraphMode()=='table'"
id="p{{paragraph.id}}_table"
class="table">
</div>
<div ng-if="getGraphMode()=='multiBarChart'"
id="p{{paragraph.id}}_multiBarChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='pieChart'"
id="p{{paragraph.id}}_pieChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='stackedAreaChart'"
id="p{{paragraph.id}}_stackedAreaChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='lineChart'"
id="p{{paragraph.id}}_lineChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='lineWithFocusChart'"
id="p{{paragraph.id}}_lineWithFocusChart">
<svg></svg>
</div>
<div ng-if="getGraphMode()=='scatterChart'"
id="p{{paragraph.id}}_scatterChart">
<svg></svg>
</div>
</div>
<div id="{{paragraph.id}}_comment"
class="text"
ng-if="getResultType()=='TABLE' && paragraph.result.comment"
ng-Init="loadResultType(paragraph.result)"
ng-bind-html="paragraph.result.comment">
</div>
<div id="{{paragraph.id}}_text"
class="text"
ng-if="paragraph.result.type == 'TEXT'"
ng-Init="loadResultType(paragraph.result)"
ng-bind="paragraph.result.msg">
</div>
<div id="p{{paragraph.id}}_html"
class="resultContained"
ng-if="paragraph.result.type == 'HTML'"
ng-Init="loadResultType(paragraph.result)">
</div>
<div id="p{{paragraph.id}}_angular"
class="resultContained"
ng-if="paragraph.result.type == 'ANGULAR'"
ng-Init="loadResultType(paragraph.result)">
</div>
<img id="{{paragraph.id}}_img"
ng-if="paragraph.result.type == 'IMG'"
ng-Init="loadResultType(paragraph.result)"
ng-src="{{getBase64ImageSrc(paragraph.result.msg)}}">
</img>
<div id="{{paragraph.id}}_error"
class="error"
ng-if="paragraph.status == 'ERROR'"
ng-bind="paragraph.errorMessage">
</div>
<div ng-include src="'app/notebook/paragraph/paragraph-chart-selector.html'"></div>
<div ng-include src="'app/notebook/paragraph/paragraph-pivot.html'"></div>
<div ng-include src="'app/notebook/paragraph/paragraph-results.html'"></div>
</div>
</div>
<div id="{{paragraph.id}}_control" class="control" ng-show="!asIframe">
<span>
{{paragraph.status}}
</span>
<span ng-if="paragraph.status=='RUNNING'">
{{getProgress()}}%
</span>
<!-- Run / Cancel button -->
<span class="icon-control-play" style="cursor:pointer;color:#3071A9" tooltip-placement="top" tooltip="Run this paragraph (Shift+Enter)"
ng-click="runParagraph(getEditorValue())"
ng-show="paragraph.status!='RUNNING' && paragraph.status!='PENDING' && paragraph.config.enabled"></span>
<span class="icon-control-pause" style="cursor:pointer;color:#CD5C5C" tooltip-placement="top" tooltip="Cancel"
ng-click="cancelParagraph()"
ng-show="paragraph.status=='RUNNING' || paragraph.status=='PENDING'"></span>
<span class="{{paragraph.config.editorHide ? 'icon-size-fullscreen' : 'icon-size-actual'}}" style="cursor:pointer;" tooltip-placement="top" tooltip="{{(paragraph.config.editorHide ? 'Show' : 'Hide') + ' editor'}}"
ng-click="toggleEditor()"></span>
<span class="{{paragraph.config.tableHide ? 'icon-notebook' : 'icon-book-open'}}" style="cursor:pointer;" tooltip-placement="top" tooltip="{{(paragraph.config.tableHide ? 'Show' : 'Hide') + ' output'}}"
ng-click="toggleOutput()"></span>
<span class="dropdown navbar-right">
<span class="icon-settings" style="cursor:pointer"
data-toggle="dropdown"
type="button">
</span>
<ul class="dropdown-menu" role="menu" style="width:200px;">
<li>
<a class="dropdown"><span class="fa fa-arrows-h"></span> Width
<form style="display:inline; margin-left:5px;">
<select ng-model="paragraph.config.colWidth"
class="selectpicker"
ng-change="changeColWidth()"
ng-options="col for col in colWidthOption"></select>
</form>
</a>
</li>
<li>
<a ng-click="moveUp()"><span class="icon-arrow-up"></span> Move Up</a>
</li>
<li>
<a ng-click="moveDown()"><span class="icon-arrow-down"></span> Move Down</a>
</li>
<li>
<a ng-click="insertNew()"><span class="icon-plus"></span> Insert New</a>
</li>
<li>
<!-- paragraph handler -->
<a ng-click="hideTitle()"
ng-show="paragraph.config.title"><span class="fa fa-font"></span> Hide title</a>
<a ng-click="showTitle()"
ng-show="!paragraph.config.title"><span class="fa fa-font"></span> Show title</a>
</li>
<li>
<a ng-click="hideLineNumbers()"
ng-show="paragraph.config.lineNumbers"><span class="fa fa-list-ol"></span> Hide line numbers</a>
<a ng-click="showLineNumbers()"
ng-show="!paragraph.config.lineNumbers"><span class="fa fa-list-ol"></span> Show line numbers</a>
</li>
<li>
<a ng-click="toggleEnableDisable()"><span class="icon-control-play"></span>
{{paragraph.config.enabled ? "Disable" : "Enable"}} run</a>
</li>
<li>
<a ng-click="goToSingleParagraph()"><span class="icon-share-alt"></span> Link this paragraph</a>
</li>
<li>
<a ng-click="clearParagraphOutput()"><span class="fa fa-eraser"></span> Clear output</a>
</li>
<li>
<!-- remove paragraph -->
<a ng-click="removeParagraph()"><span class="fa fa-times"></span> Remove</a>
</li>
</ul>
</span>
</div>
<div ng-include src="'app/notebook/paragraph/paragraph-control.html'"></div>
<div ng-if="!asIframe" class="paragraphFooter">
<div ng-show="!paragraph.config.tableHide && !viewOnly" id="{{paragraph.id}}_executionTime" class="executionTime" ng-bind-html="getExecutionTime()">
<div ng-show="!paragraph.config.tableHide && !viewOnly"
id="{{paragraph.id}}_executionTime"
class="executionTime" ng-bind-html="getExecutionTime()">
</div>
</div>
</div>

119
zeppelin-web/src/app/search/result-list.controller.js Normal file
View file

@ -0,0 +1,119 @@
/* jshint loopfunc: true */
/*
* Licensed under the Apache License, Version 2.0 (the 'License');
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an 'AS IS' BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
'use strict';
angular
.module('zeppelinWebApp')
.controller('SearchResultCtrl', function($scope, $routeParams, searchService) {
var results = searchService.search({'q': $routeParams.searchTerm}).query();
results.$promise.then(function(result) {
$scope.notes = result.body.map(function(note) {
// redirect to notebook when search result is a notebook itself,
// not a paragraph
if (!/\/paragraph\//.test(note.id)) {
return note;
}
note.id = note.id.replace('paragraph/', '?paragraph=') +
'&term=' +
$routeParams.searchTerm;
return note;
});
});
$scope.page = 0;
$scope.allResults = false;
$scope.highlightSearchResults = function(note) {
return function(_editor) {
function getEditorMode(text) {
var editorModes = {
'ace/mode/scala': /^%spark/,
'ace/mode/sql': /^%(\w*\.)?\wql/,
'ace/mode/markdown': /^%md/,
'ace/mode/sh': /^%sh/
};
return Object.keys(editorModes).reduce(function(res, mode) {
return editorModes[mode].test(text)? mode : res;
}, 'ace/mode/scala');
}
var Range = ace.require('ace/range').Range;
_editor.setOption('highlightActiveLine', false);
_editor.$blockScrolling = Infinity;
_editor.setReadOnly(true);
_editor.renderer.setShowGutter(false);
_editor.setTheme('ace/theme/chrome');
_editor.getSession().setMode(getEditorMode(note.text));
function getIndeces(term) {
return function(str) {
var indeces = [];
var i = -1;
while((i = str.indexOf(term, i + 1)) >= 0) {
indeces.push(i);
}
return indeces;
};
}
var lines = note.snippet
.split('\n')
.map(function(line, row) {
var match = line.match(/<B>(.+?)<\/B>/);
// return early if nothing to highlight
if (!match) {
return line;
}
var term = match[1];
var __line = line
.replace(/<B>/g, '')
.replace(/<\/B>/g, '');
var indeces = getIndeces(term)(__line);
indeces.forEach(function(start) {
var end = start + term.length;
_editor
.getSession()
.addMarker(
new Range(row, start, row, end),
'search-results-highlight',
'line'
);
});
return __line;
});
// resize editor based on content length
_editor.setOption(
'maxLines',
lines.reduce(function(len, line) {return len + line.length;}, 0)
);
_editor.getSession().setValue(lines.join('\n'));
};
};
});

42
zeppelin-web/src/app/search/result-list.html Normal file
View file

@ -0,0 +1,42 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div ng-controller="SearchResultCtrl" class="searchResults">
<div class="row">
<div class="col-sm-8" style="margin: 0 auto; float: none">
<ul class="search-results">
<li class="panel panel-default" ng-repeat="note in notes">
<div class="panel-heading">
<h4>
<i style="font-size: 10px;" class="icon-doc"></i>
<a class="search-results-header"
href="#/notebook/{{note.id}}">
{{note.name || 'Note ' + note.id}}
</a>
</h4>
</div>
<div class="panel-body">
<div
class="search-result"
ui-ace="{
onLoad: highlightSearchResults(note),
require: ['ace/ext/language_tools']
}"
ng-model="_"
>
</div>
</div>
</li>
</div>
</div>
</div>

37
zeppelin-web/src/app/search/search.css Normal file
View file

@ -0,0 +1,37 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
.search-results {
list-style-type: none;
margin: 10% auto 0;
padding: 0;
}
.search-result {
height: 200px;
}
.search-results-header {
text-decoration: none;
}
.search-results-highlight {
background-color: yellow;
position: absolute;
}
/* remove error highlighting */
.search-results .ace_invalid {
background: none !important;
}

1
zeppelin-web/src/assets/styles/looknfeel/default.css
View file

@ -20,7 +20,6 @@ body {
box-shadow: 0 2px 4px rgba(0, 0, 0, 0.15);
color: #2c3e50;
border-bottom: 1px solid #E5E5E5;
z-index: 300;
}
.editor,

2
zeppelin-web/src/assets/styles/looknfeel/simple.css
View file

@ -45,7 +45,7 @@ body {
.paragraph .control {
visibility: hidden;
top: 7px !important;
top: 5px !important;
right: 10px !important;
}

16
zeppelin-web/src/components/navbar/navbar.controller.js
View file

@ -15,8 +15,7 @@
'use strict';
angular.module('zeppelinWebApp').controller('NavCtrl', function($scope, $rootScope, $routeParams,
notebookListDataFactory, websocketMsgSrv,
arrayOrderingSrv) {
$location, notebookListDataFactory, websocketMsgSrv, arrayOrderingSrv) {
/** Current list of notes (ids) */
var vm = this;
@ -35,6 +34,19 @@ angular.module('zeppelinWebApp').controller('NavCtrl', function($scope, $rootSco
vm.connected = param;
});
$rootScope.$on('$locationChangeSuccess', function () {
var path = $location.path();
// hacky solution to clear search bar
// TODO(felizbear): figure out how to make ng-click work in navbar
if (path === '/') {
$scope.searchTerm = '';
}
});
$scope.search = function() {
$location.url(/search/ + $scope.searchTerm);
};
function loadNotes() {
websocketMsgSrv.getNotebookList();
}

27
zeppelin-web/src/components/navbar/navbar.html
View file

@ -43,7 +43,34 @@ limitations under the License.
<a href="#/interpreter">Interpreter</a>
</li>
</ul>
<ul class="nav navbar-nav navbar-right" style="margin-top:10px; margin-right:5px;">
<li>
<!--TODO(bzz): move to Typeahead https://angular-ui.github.io/bootstrap -->
<form role="search"
style="width: 300px; display: inline-block; margin: 0 10px"
ng-submit="search()">
<div class="input-group">
<input
type="text"
ng-model="searchTerm"
ng-disabled="!navbar.connected"
class="form-control"
placeholder="Search in your notebooks"
/>
<span class="input-group-btn">
<button
type="submit"
class="btn btn-default"
ng-disabled="!navbar.connected"
>
<i class="glyphicon glyphicon-search"></i>
</button>
</span>
</div>
</form>
</li>
<li class="server-status">
<i class="fa fa-circle" ng-class="{'server-connected':navbar.connected, 'server-disconnected':!navbar.connected}"></i>
<span ng-show="navbar.connected">Connected</span>

2
zeppelin-web/src/components/ngescape/ngescape.directive.js
View file

@ -18,7 +18,7 @@ angular.module('zeppelinWebApp').directive('ngEscape', function() {
element.bind('keydown keyup', function(event) {
if (event.which === 27) {
scope.$apply(function() {
scope.$eval(attrs.ngEnter);
scope.$eval(attrs.ngEscape);
});
event.preventDefault();
}

4
zeppelin-web/src/components/noteName-create/note-name-dialog.html
View file

@ -11,7 +11,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="noteNameModal" class="modal fade" role="dialog" modalvisible previsiblecallback="notenamectrl.preVisible"
<div id="noteNameModal" class="modal fade" role="dialog" modalvisible previsiblecallback="notenamectrl.preVisible"
targetinput="noteName" tabindex='-1'>
<div class="modal-dialog">
@ -26,7 +26,7 @@ limitations under the License.
<div class="form-group">
<label for="noteName">Note Name</label> <input
placeholder="Note name" type="text" class="form-control"
id="noteName" ng-model="note.notename">
id="noteName" ng-model="note.notename" ng-enter="notenamectrl.handleNameEnter()">
</div>
</div>
<div class="modal-footer">

8
zeppelin-web/src/components/noteName-create/notename.controller.js
View file

@ -14,7 +14,8 @@
'use strict';
angular.module('zeppelinWebApp').controller('NotenameCtrl', function($scope, $rootScope, $routeParams, websocketMsgSrv, $location) {
angular.module('zeppelinWebApp').controller('NotenameCtrl', function($scope, $rootScope, $routeParams, websocketMsgSrv,
$location) {
var vm = this;
vm.websocketMsgSrv = websocketMsgSrv;
$scope.note = {};
@ -28,6 +29,11 @@ angular.module('zeppelinWebApp').controller('NotenameCtrl', function($scope, $ro
}
};
vm.handleNameEnter = function(){
angular.element('#noteNameModal').modal('toggle');
vm.createNote();
};
$scope.$on('setNoteContent', function(event, note) {
//a hack, to make it run only after notebook creation
//it should not run i.e in case of linking to the paragraph

51
zeppelin-web/src/components/resizable/resizable.directive.js
View file

@ -13,29 +13,34 @@
*/
'use strict';
angular.module('zeppelinWebApp').directive('resizable', function () {
var resizableConfig = {
autoHide: true,
handles: 'se',
helper: 'resizable-helper',
minHeight:100,
grid: [10000, 10] // allow only vertical
};
angular.module('zeppelinWebApp').directive('resizable', function() {
var resizableConfig = {
autoHide: true,
handles: 'se',
helper: 'resizable-helper',
minHeight: 100,
grid: [10000, 10], // allow only vertical
stop: function() {
angular.element(this).css({'width': '100%', 'height': '100%'});
}
};
return {
restrict: 'A',
scope: {
callback: '&onResize'
},
link: function postLink(scope, elem, attrs) {
attrs.$observe('allowresize', function(isAllowed) {
if (isAllowed === 'true') {
elem.resizable(resizableConfig);
elem.on('resizestop', function () {
if (scope.callback) { scope.callback(); }
});
}
});
return {
restrict: 'A',
scope: {
callback: '&onResize'
},
link: function postLink(scope, elem, attrs) {
attrs.$observe('allowresize', function(isAllowed) {
if (isAllowed === 'true') {
elem.resizable(resizableConfig);
elem.on('resizestop', function() {
if (scope.callback) {
scope.callback();
}
});
}
};
});
}
};
});

29
zeppelin-web/src/components/searchService/search.service.js Normal file
View file

@ -0,0 +1,29 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
'use strict';
angular.module('zeppelinWebApp').service('searchService', function($resource, baseUrlSrv) {
this.search = function(term) {
console.log('Searching for: %o', term.q);
if (!term.q) { //TODO(bzz): empty string check
return;
}
var encQuery = window.encodeURIComponent(term.q);
return $resource(baseUrlSrv.getRestApiBase()+'/notebook/search?q='+encQuery, {}, {
query: {method:'GET'}
});
};
});

Some files were not shown because too many files have changed in this diff Show more