mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
Merge branch 'master' into ZEPPELIN-960
This commit is contained in:
CloverHearts
commit
33b0732b2c
20 changed files with 416 additions and 160 deletions
|
|
@ -1,4 +1,4 @@
|
|||
#Zeppelin
|
||||
# Apache Zeppelin
|
||||
|
||||
**Documentation:** [User Guide](http://zeppelin.apache.org/docs/latest/index.html)<br/>
|
||||
**Mailing Lists:** [User and Dev mailing list](http://zeppelin.apache.org/community.html)<br/>
|
||||
|
|
|
|||
94
docs/CONTRIBUTING.md
Normal file
94
docs/CONTRIBUTING.md
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
# Contributing to Apache Zeppelin Documentation
|
||||
|
||||
## Folder Structure
|
||||
`docs/` folder is organized as below:
|
||||
|
||||
```
|
||||
docs/
|
||||
├── _includes/themes/zeppelin
|
||||
│ ├── _navigation.html
|
||||
│ └── default.html
|
||||
├── _layouts
|
||||
├── _plugins
|
||||
├── assets/themes/zeppelin -> {ASSET_PATH}
|
||||
│ ├── bootstrap
|
||||
│ ├── css
|
||||
│ ├── img
|
||||
│ └── js
|
||||
├── development/ *.md
|
||||
├── displaysystem/ *.md
|
||||
├── install/ *.md
|
||||
├── interpreter/ *.md
|
||||
├── manual/ *.md
|
||||
├── quickstart/ *.md
|
||||
├── rest-api/ *.md
|
||||
├── security/ *.md
|
||||
├── storage/ *.md
|
||||
├── Gemfile
|
||||
├── Gemfile.lock
|
||||
├── _config.yml
|
||||
├── index.md
|
||||
└── ...
|
||||
```
|
||||
|
||||
- `_navigation.html`: the dropdown menu in navbar
|
||||
- `default.html` & `_layouts/`: define default HTML layout
|
||||
- `_plugins/`: custom plugin `*.rb` files can be placed in this folder. See [jekyll/plugins](https://jekyllrb.com/docs/plugins/) for the further information.
|
||||
- `{ASSET_PATH}/css/style.css`: extra css components can be defined
|
||||
- `{ASSET_PATH}/img/docs-img/`: image files used for document pages can be placed in this folder
|
||||
- `{ASSET_PATH}/js/`: extra `.js` files can be placed
|
||||
- `Gemfile`: defines bundle dependencies. They will be installed by `bundle install`.
|
||||
- `Gemfile.lock`: when you run `bundle install`, bundler will persist all gems name and their version to this file. For the more details, see [Bundle "The Gemfile Lock"](http://bundler.io/v1.10/man/bundle-install.1.html#THE-GEMFILE-LOCK)
|
||||
- `documentation_group`: `development/`, `displaysystem/`, `install/`, `interpreter/`...
|
||||
- `_config.yml`: defines configuration options for docs website. See [jekyll/configuration](https://jekyllrb.com/docs/configuration/) for the other available config variables.
|
||||
- `index.md`: the main page of `http://zeppelin.apache.org/docs/<ZEPPELIN_VERSION>/`
|
||||
|
||||
|
||||
## Markdown
|
||||
Zeppelin documentation pages are written with [Markdown](http://daringfireball.net/projects/markdown/). It is possible to use [GitHub flavored syntax](https://help.github.com/categories/writing-on-github/) and intermix plain HTML.
|
||||
|
||||
## Front matter
|
||||
Every page contains [YAML front matter](https://jekyllrb.com/docs/frontmatter/) block in their header. Don't forget to wrap the front matter list with triple-dashed lines(`---`) like below.
|
||||
The document page should start this triple-dashed lines. Or you will face 404 error, since Jekyll can't find the page.
|
||||
|
||||
```
|
||||
---
|
||||
layout: page
|
||||
title: "Apache Zeppelin Tutorial"
|
||||
description: "This tutorial page contains a short walk-through tutorial that uses Apache Spark backend. Please note that this tutorial is valid for Spark 1.3 and higher."
|
||||
group: quickstart
|
||||
---
|
||||
```
|
||||
|
||||
- `layout`: the default layout is `page` which is defined in `_layout/page.html`.
|
||||
- `title`: the title for the document. Please note that if it needs to include `Zeppelin`, it should be `Apache Zeppelin`, not `Zeppelin`.
|
||||
- `description`: a short description for the document. One or two sentences would be enough. This description also will be shown as an extract sentence when people search pages.
|
||||
- `group`: a category of the document page
|
||||
|
||||
## Headings
|
||||
All documents are structured with headings. From these headings, you can automatically generate a **Table of Contents**. There is a simple rule for Zeppelin docs headings.
|
||||
|
||||
```
|
||||
# Level-1 heading <- used only for the main title
|
||||
## Level-2 heading <- start with this
|
||||
### Level-3 heading
|
||||
#### Level-4 heading <- won't be converted in TOC from this level
|
||||
```
|
||||
|
||||
## Table of contents(TOC)
|
||||
|
||||
```
|
||||
<div id="toc"></div>
|
||||
```
|
||||
|
||||
Add this line below `# main title` in order to generate a **Table of Contents**. Headings until `### (Level-3 heading)` are included to TOC.
|
||||
|
||||
|
||||
Default setting options for TOC are definded in [here](https://github.com/apache/zeppelin/blob/master/docs/assets/themes/zeppelin/js/toc.js#L4).
|
||||
|
||||
|
||||
## Adding new pages
|
||||
If you're going to create new pages, there are some spots you need to add the location of the page.
|
||||
|
||||
- **Dropdown menu in navbar**: add your docs location to [_navigation.html](https://github.com/apache/zeppelin/blob/master/docs/_includes/themes/zeppelin/_navigation.html)
|
||||
- **Main index**: add your docs below [What is the next?](http://zeppelin.apache.org/docs/latest/#what-is-the-next) section in [index.md](https://github.com/apache/zeppelin/blob/master/docs/index.md) with a short description. No need to do this if the page is for **Interpreters**.
|
||||
|
|
@ -1,41 +1,55 @@
|
|||
## Apache Zeppelin documentation
|
||||
|
||||
This readme will walk you through building the Zeppelin documentation, which is included here with the Zeppelin source code.
|
||||
# Apache Zeppelin documentation
|
||||
|
||||
This README will walk you through building the documentation of Apache Zeppelin. The documentation is included here with Apache Zeppelin source code. The online documentation at [https://zeppelin.apache.org/docs/<ZEPPELIN_VERSION>](https://zeppelin.apache.org/docs/latest) is also generated from the files found in here.
|
||||
|
||||
## Build documentation
|
||||
See https://help.github.com/articles/using-jekyll-with-pages#installing-jekyll
|
||||
Zeppelin is using [Jekyll](https://jekyllrb.com/) which is a static site generator and [Github Pages](https://pages.github.com/) as a site publisher. For the more details, see [help.github.com/articles/about-github-pages-and-jekyll/](https://help.github.com/articles/about-github-pages-and-jekyll/).
|
||||
|
||||
**Requirements**
|
||||
|
||||
```
|
||||
ruby --version >= 2.0.0
|
||||
gem install bundler
|
||||
# go to /docs under your Zeppelin source
|
||||
bundle install
|
||||
# ruby --version >= 2.0.0
|
||||
# Install Bundler using gem
|
||||
gem install bundler
|
||||
|
||||
cd $ZEPPELIN_HOME/docs
|
||||
# Install all dependencies declared in the Gemfile
|
||||
bundle install
|
||||
```
|
||||
|
||||
For the further information about requirements, please see [here](https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/#requirements).
|
||||
For the further information about requirements, please see [here](https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/#requirements).
|
||||
|
||||
*On OS X 10.9 you may need to do "xcode-select --install"*
|
||||
On OS X 10.9, you may need to do
|
||||
|
||||
```
|
||||
xcode-select --install
|
||||
```
|
||||
|
||||
## Run website locally
|
||||
If you don't want to encounter uglily rendered pages, run the documentation site in your local first.
|
||||
|
||||
In `$ZEPPELIN_HOME/docs`,
|
||||
|
||||
```
|
||||
bundle exec jekyll serve --watch
|
||||
```
|
||||
|
||||
Using the above command, Jekyll will start a web server at `http://localhost:4000` and watch the `/docs` directory to update.
|
||||
|
||||
|
||||
## Run website
|
||||
|
||||
bundle exec jekyll serve --watch
|
||||
## Contribute to Zeppelin documentation
|
||||
If you wish to help us and contribute to Zeppelin Documentation, please look at [Zeppelin Documentation's contribution guideline](https://github.com/apache/zeppelin/blob/master/docs/CONTRIBUTING.md).
|
||||
|
||||
|
||||
## Adding a new page
|
||||
|
||||
rake page name="new-page.md"
|
||||
|
||||
|
||||
## Bumping up version in a new release
|
||||
## For committers only
|
||||
### Bumping up version in a new release
|
||||
|
||||
* `ZEPPELIN_VERSION` and `BASE_PATH` property in _config.yml
|
||||
|
||||
## Deploy to ASF svnpubsub infra (for committers only)
|
||||
### Deploy to ASF svnpubsub infra
|
||||
1. generate static website in `./_site`
|
||||
|
||||
```
|
||||
# go to /docs under Zeppelin source
|
||||
bundle exec jekyll build --safe
|
||||
|
|
|
|||
|
|
@ -412,6 +412,43 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
<br/>
|
||||
### Get the status of a single paragraph
|
||||
<table class="table-configuration">
|
||||
<col width="200">
|
||||
<tr>
|
||||
<td>Description</td>
|
||||
<td>This ```GET``` method gets the status of a single paragraph by the given notebook and paragraph id.
|
||||
The body field of the returned JSON contains of the array that compose of the paragraph id, paragraph status, paragraph finish date, paragraph started date.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>URL</td>
|
||||
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Success code</td>
|
||||
<td>200</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td> Fail code</td>
|
||||
<td> 500 </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td> sample JSON response </td>
|
||||
<td><pre>
|
||||
{
|
||||
"status": "OK",
|
||||
"body": {
|
||||
"id":"20151121-212654\_766735423",
|
||||
"status":"FINISHED",
|
||||
"finished":"Tue Nov 24 14:21:40 KST 2015",
|
||||
"started":"Tue Nov 24 14:21:39 KST 2015"
|
||||
}
|
||||
}</pre></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br/>
|
||||
### Run a paragraph
|
||||
<table class="table-configuration">
|
||||
|
|
|
|||
|
|
@ -54,23 +54,15 @@ import java.util.Properties;
|
|||
* zeppelin.hbase.test.mode: (Testing only) Disable checks for unit and manual tests. Default: false
|
||||
*/
|
||||
public class HbaseInterpreter extends Interpreter {
|
||||
public static final String HBASE_HOME = "hbase.home";
|
||||
public static final String HBASE_RUBY_SRC = "hbase.ruby.sources";
|
||||
public static final String HBASE_TEST_MODE = "zeppelin.hbase.test.mode";
|
||||
|
||||
private Logger logger = LoggerFactory.getLogger(HbaseInterpreter.class);
|
||||
private ScriptingContainer scriptingContainer;
|
||||
|
||||
private StringWriter writer;
|
||||
|
||||
static {
|
||||
Interpreter.register("hbase", "hbase", HbaseInterpreter.class.getName(),
|
||||
new InterpreterPropertyBuilder()
|
||||
.add("hbase.home",
|
||||
getSystemDefault("HBASE_HOME", "hbase.home", "/usr/lib/hbase/"),
|
||||
"Installation directory of HBase")
|
||||
.add("hbase.ruby.sources", "lib/ruby",
|
||||
"Path to Ruby scripts relative to 'hbase.home'")
|
||||
.add("zeppelin.hbase.test.mode", "false", "Disable checks for unit and manual tests")
|
||||
.build());
|
||||
}
|
||||
|
||||
public HbaseInterpreter(Properties property) {
|
||||
super(property);
|
||||
}
|
||||
|
|
@ -81,9 +73,9 @@ public class HbaseInterpreter extends Interpreter {
|
|||
this.writer = new StringWriter();
|
||||
scriptingContainer.setOutput(this.writer);
|
||||
|
||||
if (!Boolean.parseBoolean(getProperty("zeppelin.hbase.test.mode"))) {
|
||||
String hbase_home = getProperty("hbase.home");
|
||||
String ruby_src = getProperty("hbase.ruby.sources");
|
||||
if (!Boolean.parseBoolean(getProperty(HBASE_TEST_MODE))) {
|
||||
String hbase_home = getProperty(HBASE_HOME);
|
||||
String ruby_src = getProperty(HBASE_RUBY_SRC);
|
||||
Path abs_ruby_src = Paths.get(hbase_home, ruby_src).toAbsolutePath();
|
||||
|
||||
logger.info("Home:" + hbase_home);
|
||||
|
|
@ -98,7 +90,7 @@ public class HbaseInterpreter extends Interpreter {
|
|||
logger.info("Absolute Ruby Source:" + abs_ruby_src.toString());
|
||||
// hirb.rb:41 requires the following system property to be set.
|
||||
Properties sysProps = System.getProperties();
|
||||
sysProps.setProperty("hbase.ruby.sources", abs_ruby_src.toString());
|
||||
sysProps.setProperty(HBASE_RUBY_SRC, abs_ruby_src.toString());
|
||||
|
||||
Path abs_hirb_path = Paths.get(hbase_home, "bin/hirb.rb");
|
||||
try {
|
||||
|
|
|
|||
25
hbase/src/main/resources/interpreter-setting.json
Normal file
25
hbase/src/main/resources/interpreter-setting.json
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
[
|
||||
{
|
||||
"group": "hbase",
|
||||
"name": "hbase",
|
||||
"className": "org.apache.zeppelin.hbase.HbaseInterpreter",
|
||||
"properties": {
|
||||
"hbase.home": {
|
||||
"envName": "HBASE_HOME",
|
||||
"propertyName": "hbase.home",
|
||||
"defaultValue": "/usr/lib/hbase/",
|
||||
"description": "Installation directory of HBase"
|
||||
},
|
||||
"hbase.ruby.sources": {
|
||||
"propertyName": "hbase.ruby.sources",
|
||||
"defaultValue": "lib/ruby",
|
||||
"description": "Path to Ruby scripts relative to 'hbase.home'"
|
||||
},
|
||||
"zeppelin.hbase.test.mode": {
|
||||
"propertyName": "zeppelin.hbase.test.mode",
|
||||
"defaultValue": "false",
|
||||
"description": "Disable checks for unit and manual tests"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
1
pom.xml
1
pom.xml
|
|
@ -486,6 +486,7 @@
|
|||
<exclude>docs/sitemap.txt</exclude>
|
||||
<exclude>docs/search_data.json</exclude>
|
||||
<exclude>**/dependency-reduced-pom.xml</exclude>
|
||||
<exclude>docs/CONTRIBUTING.md</exclude>
|
||||
|
||||
<!-- bundled from anchor -->
|
||||
<exclude>docs/assets/themes/zeppelin/js/anchor.min.js</exclude>
|
||||
|
|
|
|||
|
|
@ -20,11 +20,11 @@
|
|||
import sys
|
||||
import signal
|
||||
import base64
|
||||
|
||||
from io import BytesIO
|
||||
try:
|
||||
import StringIO as io
|
||||
from StringIO import StringIO
|
||||
except ImportError:
|
||||
import io as io
|
||||
from io import StringIO
|
||||
|
||||
def intHandler(signum, frame): # Set the signal handler
|
||||
print ("Paragraph interrupted")
|
||||
|
|
@ -117,6 +117,7 @@ class PyZeppelinContext(object):
|
|||
|
||||
def __init__(self):
|
||||
self.max_result = 1000
|
||||
self.py3 = bool(sys.version_info >= (3,))
|
||||
|
||||
def input(self, name, defaultValue=""):
|
||||
print(self.errorMsg)
|
||||
|
|
@ -141,14 +142,14 @@ class PyZeppelinContext(object):
|
|||
"""Pretty prints DF using Table Display System
|
||||
"""
|
||||
limit = len(df) > self.max_result
|
||||
header_buf = io.StringIO("")
|
||||
header_buf = StringIO("")
|
||||
header_buf.write(str(df.columns[0]))
|
||||
for col in df.columns[1:]:
|
||||
header_buf.write("\t")
|
||||
header_buf.write(str(col))
|
||||
header_buf.write("\n")
|
||||
|
||||
body_buf = io.StringIO("")
|
||||
body_buf = StringIO("")
|
||||
rows = df.head(self.max_result).values if limit else df.values
|
||||
for row in rows:
|
||||
body_buf.write(str(row[0]))
|
||||
|
|
@ -168,13 +169,18 @@ class PyZeppelinContext(object):
|
|||
fmt='png', **kwargs):
|
||||
"""Matplotlib show function
|
||||
"""
|
||||
img = io.StringIO()
|
||||
if fmt == 'png':
|
||||
img = BytesIO()
|
||||
p.savefig(img, format=fmt)
|
||||
html = "%html <img src={img} width={width}, height={height}>"
|
||||
img_str = "data:image/png;base64,"
|
||||
img_str = b"data:image/png;base64,"
|
||||
img_str += base64.b64encode(img.getvalue().strip())
|
||||
# Need to do this for python3 compatibility
|
||||
if self.py3:
|
||||
img_str = img_str.decode('ascii')
|
||||
|
||||
elif fmt == 'svg':
|
||||
img = StringIO()
|
||||
p.savefig(img, format=fmt)
|
||||
html = "%html <div style='width:{width};height:{height}'>{img}<div>"
|
||||
img_str = img.getvalue()
|
||||
|
|
|
|||
|
|
@ -118,8 +118,8 @@ public class NotebookRestApi {
|
|||
* TODO(jl): Fixed the type of HashSet
|
||||
* https://issues.apache.org/jira/browse/ZEPPELIN-1162
|
||||
*/
|
||||
HashMap<String, HashSet> permMap =
|
||||
gson.fromJson(req, new TypeToken<HashMap<String, HashSet>>() {
|
||||
HashMap<String, HashSet<String>> permMap =
|
||||
gson.fromJson(req, new TypeToken<HashMap<String, HashSet<String>>>() {
|
||||
}.getType());
|
||||
Note note = notebook.getNote(noteId);
|
||||
String principal = SecurityUtils.getPrincipal();
|
||||
|
|
@ -135,9 +135,9 @@ public class NotebookRestApi {
|
|||
ownerPermissionError(userAndRoles, notebookAuthorization.getOwners(noteId))).build();
|
||||
}
|
||||
|
||||
HashSet readers = permMap.get("readers");
|
||||
HashSet owners = permMap.get("owners");
|
||||
HashSet writers = permMap.get("writers");
|
||||
HashSet<String> readers = permMap.get("readers");
|
||||
HashSet<String> owners = permMap.get("owners");
|
||||
HashSet<String> writers = permMap.get("writers");
|
||||
// Set readers, if writers and owners is empty -> set to user requesting the change
|
||||
if (readers != null && !readers.isEmpty()) {
|
||||
if (writers.isEmpty()) {
|
||||
|
|
@ -537,6 +537,35 @@ public class NotebookRestApi {
|
|||
return new JsonResponse<>(Status.OK, null, note.generateParagraphsInfo()).build();
|
||||
}
|
||||
|
||||
/**
|
||||
* Get notebook paragraph job status REST API
|
||||
*
|
||||
* @param notebookId ID of Notebook
|
||||
* @param paragraphId ID of Paragraph
|
||||
* @return JSON with status.OK
|
||||
* @throws IOException, IllegalArgumentException
|
||||
*/
|
||||
@GET
|
||||
@Path("job/{notebookId}/{paragraphId}")
|
||||
@ZeppelinApi
|
||||
public Response getNoteParagraphJobStatus(@PathParam("notebookId") String notebookId,
|
||||
@PathParam("paragraphId") String paragraphId)
|
||||
throws IOException, IllegalArgumentException {
|
||||
LOG.info("get notebook paragraph job status.");
|
||||
Note note = notebook.getNote(notebookId);
|
||||
if (note == null) {
|
||||
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
|
||||
}
|
||||
|
||||
Paragraph paragraph = note.getParagraph(paragraphId);
|
||||
if (paragraph == null) {
|
||||
return new JsonResponse<>(Status.NOT_FOUND, "paragraph not found.").build();
|
||||
}
|
||||
|
||||
return new JsonResponse<>(Status.OK, null, note.generateSingleParagraphInfo(paragraphId)).
|
||||
build();
|
||||
}
|
||||
|
||||
/**
|
||||
* Run paragraph job REST API
|
||||
*
|
||||
|
|
|
|||
|
|
@ -122,6 +122,28 @@ public class NotebookRestApiTest extends AbstractTestRestApi {
|
|||
ZeppelinServer.notebook.removeNote(note2.getId(), null);
|
||||
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testGetNoteParagraphJobStatus() throws IOException {
|
||||
Note note1 = ZeppelinServer.notebook.createNote(null);
|
||||
note1.addParagraph();
|
||||
|
||||
String paragraphId = note1.getLastParagraph().getId();
|
||||
|
||||
GetMethod get = httpGet("/notebook/job/" + note1.getId() + "/" + paragraphId);
|
||||
assertThat(get, isAllowed());
|
||||
Map<String, Object> resp = gson.fromJson(get.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
|
||||
}.getType());
|
||||
Map<String, Set<String>> paragraphStatus = (Map<String, Set<String>>) resp.get("body");
|
||||
|
||||
// Check id and status have proper value
|
||||
assertEquals(paragraphStatus.get("id"), paragraphId);
|
||||
assertEquals(paragraphStatus.get("status"), "READY");
|
||||
|
||||
//cleanup
|
||||
ZeppelinServer.notebook.removeNote(note1.getId(), null);
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -68,7 +68,6 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
|
|||
var initNotebook = function() {
|
||||
websocketMsgSrv.getNotebook($routeParams.noteId);
|
||||
websocketMsgSrv.listRevisionHistory($routeParams.noteId);
|
||||
|
||||
var currentRoute = $route.current;
|
||||
if (currentRoute) {
|
||||
setTimeout(
|
||||
|
|
@ -333,6 +332,7 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl', function($scope, $ro
|
|||
} else {
|
||||
$scope.viewOnly = $scope.note.config.looknfeel === 'report' ? true : false;
|
||||
}
|
||||
$scope.note.paragraphs[0].focus = true;
|
||||
$rootScope.$broadcast('setLookAndFeel', $scope.note.config.looknfeel);
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -99,7 +99,6 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
|
|||
if (newParagraph.focus) {
|
||||
$scope.paragraphFocused = true;
|
||||
}
|
||||
|
||||
if (!$scope.paragraph.config) {
|
||||
$scope.paragraph.config = {};
|
||||
}
|
||||
|
|
@ -556,9 +555,10 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
|
|||
$scope.editor.setHighlightGutterLine(false);
|
||||
$scope.editor.getSession().setUseWrapMode(true);
|
||||
$scope.editor.setTheme('ace/theme/chrome');
|
||||
$scope.editor.setReadOnly($scope.isRunning());
|
||||
if ($scope.paragraphFocused) {
|
||||
$scope.editor.focus();
|
||||
$scope.goToLineEnd();
|
||||
$scope.goToEnd();
|
||||
}
|
||||
|
||||
autoAdjustEditorHeight(_editor.container.id);
|
||||
|
|
@ -839,8 +839,8 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
|
|||
return false;
|
||||
};
|
||||
|
||||
$scope.goToLineEnd = function() {
|
||||
$scope.editor.navigateLineEnd();
|
||||
$scope.goToEnd = function() {
|
||||
$scope.editor.navigateFileEnd();
|
||||
};
|
||||
|
||||
$scope.getResultType = function(paragraph) {
|
||||
|
|
@ -1053,7 +1053,7 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
|
|||
d3g = scatterData.d3g;
|
||||
|
||||
$scope.chart[type].xAxis.tickFormat(function(d) {return xAxisTickFormat(d, xLabels);});
|
||||
$scope.chart[type].yAxis.tickFormat(function(d) {return xAxisTickFormat(d, yLabels);});
|
||||
$scope.chart[type].yAxis.tickFormat(function(d) {return yAxisTickFormat(d, yLabels);});
|
||||
|
||||
// configure how the tooltip looks.
|
||||
$scope.chart[type].tooltipContent(function(key, x, y, graph, data) {
|
||||
|
|
@ -1095,7 +1095,11 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
|
|||
xLabels = pivotdata.xLabels;
|
||||
d3g = pivotdata.d3g;
|
||||
$scope.chart[type].xAxis.tickFormat(function(d) {return xAxisTickFormat(d, xLabels);});
|
||||
$scope.chart[type].yAxis.tickFormat(function(d) {return yAxisTickFormat(d);});
|
||||
if (type === 'stackedAreaChart') {
|
||||
$scope.chart[type].yAxisTickFormat(function(d) {return yAxisTickFormat(d);});
|
||||
} else {
|
||||
$scope.chart[type].yAxis.tickFormat(function(d) {return yAxisTickFormat(d, xLabels);});
|
||||
}
|
||||
$scope.chart[type].yAxis.axisLabelDistance(50);
|
||||
if ($scope.chart[type].useInteractiveGuideline) { // lineWithFocusChart hasn't got useInteractiveGuideline
|
||||
$scope.chart[type].useInteractiveGuideline(true); // for better UX and performance issue. (https://github.com/novus/nvd3/issues/691)
|
||||
|
|
@ -2516,6 +2520,7 @@ angular.module('zeppelinWebApp').controller('ParagraphCtrl', function($scope, $r
|
|||
$scope.paragraph.status = data.paragraph.status;
|
||||
$scope.paragraph.result = data.paragraph.result;
|
||||
$scope.paragraph.settings = data.paragraph.settings;
|
||||
$scope.editor.setReadOnly($scope.isRunning());
|
||||
|
||||
if (!$scope.asIframe) {
|
||||
$scope.paragraph.config = data.paragraph.config;
|
||||
|
|
|
|||
|
|
@ -275,6 +275,10 @@ table.dataTable.table-condensed .sorting_desc:after {
|
|||
background: none !important;
|
||||
}
|
||||
|
||||
.paragraph-disable {
|
||||
opacity : 0.6!important;
|
||||
}
|
||||
|
||||
.ace_marker-layer .ace_selection {
|
||||
z-index: 0 !important;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -38,16 +38,14 @@ limitations under the License.
|
|||
<div>
|
||||
<div ng-show="!paragraph.config.editorHide && !viewOnly" style="margin-bottom:3px;">
|
||||
<div id="{{paragraph.id}}_editor"
|
||||
style="opacity: 1;"
|
||||
class="editor"
|
||||
ui-ace="{
|
||||
onLoad : aceLoaded,
|
||||
require : ['ace/ext/language_tools']
|
||||
}"
|
||||
ng-model="paragraph.text"
|
||||
ng-class="{'disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING',
|
||||
'paragraph-text--dirty' : dirtyText !== originalText && dirtyText !== undefined}">
|
||||
</div>
|
||||
ng-class="{'paragraph-disable': paragraph.status == 'RUNNING' || paragraph.status == 'PENDING',
|
||||
'paragraph-text--dirty' : dirtyText !== originalText && dirtyText !== undefined}"> </div>
|
||||
</div>
|
||||
|
||||
<div ng-include src="'app/notebook/paragraph/paragraph-progressBar.html'"></div>
|
||||
|
|
|
|||
|
|
@ -17,9 +17,11 @@ angular.module('zeppelinWebApp').directive('ngEnter', function() {
|
|||
return function(scope, element, attrs) {
|
||||
element.bind('keydown keypress', function(event) {
|
||||
if (event.which === 13) {
|
||||
scope.$apply(function() {
|
||||
scope.$eval(attrs.ngEnter);
|
||||
});
|
||||
if (!event.shiftKey) {
|
||||
scope.$apply(function() {
|
||||
scope.$eval(attrs.ngEnter);
|
||||
});
|
||||
}
|
||||
event.preventDefault();
|
||||
}
|
||||
});
|
||||
|
|
|
|||
|
|
@ -97,13 +97,13 @@ public class InterpreterFactory implements InterpreterGroupFactory {
|
|||
* This is only references with default settings, name and properties
|
||||
* key: InterpreterSetting.name
|
||||
*/
|
||||
private Map<String, InterpreterSetting> interpreterSettingsRef = new HashMap<>();
|
||||
private final Map<String, InterpreterSetting> interpreterSettingsRef = new HashMap<>();
|
||||
|
||||
/**
|
||||
* This is used by creating and running Interpreters
|
||||
* key: InterpreterSetting.id <- This is becuase backward compatibility
|
||||
*/
|
||||
private Map<String, InterpreterSetting> interpreterSettings = new HashMap<>();
|
||||
private final Map<String, InterpreterSetting> interpreterSettings = new HashMap<>();
|
||||
|
||||
private Map<String, List<String>> interpreterBindings = new HashMap<>();
|
||||
private List<RemoteRepository> interpreterRepositories;
|
||||
|
|
@ -175,7 +175,7 @@ public class InterpreterFactory implements InterpreterGroupFactory {
|
|||
|
||||
registerInterpreterFromResource(cl, interpreterDirString, interpreterJson);
|
||||
|
||||
/**
|
||||
/*
|
||||
* TODO(jongyoul)
|
||||
* - Remove these codes below because of legacy code
|
||||
* - Support ThreadInterpreter
|
||||
|
|
@ -468,8 +468,6 @@ public class InterpreterFactory implements InterpreterGroupFactory {
|
|||
* Return ordered interpreter setting list.
|
||||
* The list does not contain more than one setting from the same interpreter class.
|
||||
* Order by InterpreterClass (order defined by ZEPPELIN_INTERPRETERS), Interpreter setting name
|
||||
*
|
||||
* @return
|
||||
*/
|
||||
public List<String> getDefaultInterpreterSettingList() {
|
||||
// this list will contain default interpreter setting list
|
||||
|
|
@ -529,8 +527,7 @@ public class InterpreterFactory implements InterpreterGroupFactory {
|
|||
}
|
||||
|
||||
/**
|
||||
* @param group InterpreterSetting reference name
|
||||
* @param properties
|
||||
* @param group InterpreterSetting reference name
|
||||
* @return
|
||||
*/
|
||||
public InterpreterSetting add(String group, ArrayList<InterpreterInfo> interpreterInfos,
|
||||
|
|
@ -579,8 +576,8 @@ public class InterpreterFactory implements InterpreterGroupFactory {
|
|||
|
||||
} else {
|
||||
interpreterSetting =
|
||||
new InterpreterSetting(group, null, interpreterInfos, properties, dependencies,
|
||||
option, path);
|
||||
new InterpreterSetting(group, null, interpreterInfos, properties, dependencies, option,
|
||||
path);
|
||||
interpreterSettingsRef.put(group, interpreterSetting);
|
||||
}
|
||||
}
|
||||
|
|
@ -1176,6 +1173,16 @@ public class InterpreterFactory implements InterpreterGroupFactory {
|
|||
return interpreters.get(0);
|
||||
}
|
||||
}
|
||||
|
||||
// Support the legacy way to use it
|
||||
for (InterpreterSetting s : settings) {
|
||||
if (s.getGroup().equals(replName)) {
|
||||
List<Interpreter> interpreters = createOrGetInterpreterList(noteId, s);
|
||||
if (null != interpreters) {
|
||||
return interpreters.get(0);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// dev interpreter
|
||||
|
|
|
|||
|
|
@ -411,24 +411,40 @@ public class Note implements Serializable, ParagraphJobListener {
|
|||
List<Map<String, String>> paragraphsInfo = new LinkedList<>();
|
||||
synchronized (paragraphs) {
|
||||
for (Paragraph p : paragraphs) {
|
||||
Map<String, String> info = new HashMap<>();
|
||||
info.put("id", p.getId());
|
||||
info.put("status", p.getStatus().toString());
|
||||
if (p.getDateStarted() != null) {
|
||||
info.put("started", p.getDateStarted().toString());
|
||||
}
|
||||
if (p.getDateFinished() != null) {
|
||||
info.put("finished", p.getDateFinished().toString());
|
||||
}
|
||||
if (p.getStatus().isRunning()) {
|
||||
info.put("progress", String.valueOf(p.progress()));
|
||||
}
|
||||
Map<String, String> info = populatePragraphInfo(p);
|
||||
paragraphsInfo.add(info);
|
||||
}
|
||||
}
|
||||
return paragraphsInfo;
|
||||
}
|
||||
|
||||
public Map<String, String> generateSingleParagraphInfo(String paragraphId) {
|
||||
synchronized (paragraphs) {
|
||||
for (Paragraph p : paragraphs) {
|
||||
if (p.getId().equals(paragraphId)) {
|
||||
return populatePragraphInfo(p);
|
||||
}
|
||||
}
|
||||
return new HashMap<>();
|
||||
}
|
||||
}
|
||||
|
||||
private Map<String, String> populatePragraphInfo(Paragraph p) {
|
||||
Map<String, String> info = new HashMap<>();
|
||||
info.put("id", p.getId());
|
||||
info.put("status", p.getStatus().toString());
|
||||
if (p.getDateStarted() != null) {
|
||||
info.put("started", p.getDateStarted().toString());
|
||||
}
|
||||
if (p.getDateFinished() != null) {
|
||||
info.put("finished", p.getDateFinished().toString());
|
||||
}
|
||||
if (p.getStatus().isRunning()) {
|
||||
info.put("progress", String.valueOf(p.progress()));
|
||||
}
|
||||
return info;
|
||||
}
|
||||
|
||||
/**
|
||||
* Run all paragraphs sequentially.
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -33,6 +33,8 @@ import org.apache.zeppelin.user.AuthenticationInfo;
|
|||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import com.google.common.base.Joiner;
|
||||
import com.google.common.collect.ImmutableMap;
|
||||
import com.google.gson.Gson;
|
||||
import com.google.gson.reflect.TypeToken;
|
||||
|
||||
|
|
@ -191,20 +193,45 @@ public class ZeppelinHubRepo implements NotebookRepo {
|
|||
@Override
|
||||
public Revision checkpoint(String noteId, String checkpointMsg, AuthenticationInfo subject)
|
||||
throws IOException {
|
||||
// Auto-generated method stub
|
||||
return null;
|
||||
if (StringUtils.isBlank(noteId)) {
|
||||
return null;
|
||||
}
|
||||
String endpoint = Joiner.on("/").join(noteId, "checkpoint");
|
||||
String content = GSON.toJson(ImmutableMap.of("message", checkpointMsg));
|
||||
String response = restApiClient.asyncPutWithResponseBody(endpoint, content);
|
||||
|
||||
return GSON.fromJson(response, Revision.class);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Note get(String noteId, String revId, AuthenticationInfo subject) throws IOException {
|
||||
// Auto-generated method stub
|
||||
return null;
|
||||
if (StringUtils.isBlank(noteId) || StringUtils.isBlank(revId)) {
|
||||
return EMPTY_NOTE;
|
||||
}
|
||||
String endpoint = Joiner.on("/").join(noteId, "checkpoint", revId);
|
||||
String response = restApiClient.asyncGet(endpoint);
|
||||
Note note = GSON.fromJson(response, Note.class);
|
||||
if (note == null) {
|
||||
return EMPTY_NOTE;
|
||||
}
|
||||
LOG.info("ZeppelinHub REST API get note {} revision {}", noteId, revId);
|
||||
return note;
|
||||
}
|
||||
|
||||
@Override
|
||||
public List<Revision> revisionHistory(String noteId, AuthenticationInfo subject) {
|
||||
// Auto-generated method stub
|
||||
return null;
|
||||
if (StringUtils.isBlank(noteId)) {
|
||||
return Collections.emptyList();
|
||||
}
|
||||
String endpoint = Joiner.on("/").join(noteId, "checkpoint");
|
||||
List<Revision> history = Collections.emptyList();
|
||||
try {
|
||||
String response = restApiClient.asyncGet(endpoint);
|
||||
history = GSON.fromJson(response, new TypeToken<List<Revision>>(){}.getType());
|
||||
} catch (IOException e) {
|
||||
LOG.error("Cannot get note history", e);
|
||||
}
|
||||
return history;
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -25,9 +25,8 @@ import java.util.concurrent.TimeoutException;
|
|||
import org.apache.commons.io.IOUtils;
|
||||
import org.apache.commons.lang.StringUtils;
|
||||
import org.eclipse.jetty.client.HttpClient;
|
||||
import org.eclipse.jetty.client.api.Request;
|
||||
import org.eclipse.jetty.client.api.Response;
|
||||
import org.eclipse.jetty.client.api.Result;
|
||||
import org.eclipse.jetty.client.util.BufferingResponseListener;
|
||||
import org.eclipse.jetty.client.util.InputStreamResponseListener;
|
||||
import org.eclipse.jetty.client.util.StringContentProvider;
|
||||
import org.eclipse.jetty.http.HttpMethod;
|
||||
|
|
@ -115,89 +114,66 @@ public class ZeppelinhubRestApiHandler {
|
|||
}
|
||||
|
||||
public String asyncGet(String argument) throws IOException {
|
||||
String note = StringUtils.EMPTY;
|
||||
|
||||
InputStreamResponseListener listener = new InputStreamResponseListener();
|
||||
client.newRequest(zepelinhubUrl + argument)
|
||||
.header(ZEPPELIN_TOKEN_HEADER, token)
|
||||
.send(listener);
|
||||
|
||||
// Wait for the response headers to arrive
|
||||
Response response;
|
||||
try {
|
||||
response = listener.get(30, TimeUnit.SECONDS);
|
||||
} catch (InterruptedException | TimeoutException | ExecutionException e) {
|
||||
LOG.error("Cannot perform Get request to ZeppelinHub", e);
|
||||
throw new IOException("Cannot load note from ZeppelinHub", e);
|
||||
}
|
||||
|
||||
int code = response.getStatus();
|
||||
if (code == 200) {
|
||||
try (InputStream responseContent = listener.getInputStream()) {
|
||||
note = IOUtils.toString(responseContent, "UTF-8");
|
||||
}
|
||||
} else {
|
||||
LOG.error("ZeppelinHub Get {} returned with status {} ", zepelinhubUrl + argument, code);
|
||||
throw new IOException("Cannot load note from ZeppelinHub");
|
||||
}
|
||||
return note;
|
||||
return sendToZeppelinHub(HttpMethod.GET, zepelinhubUrl + argument);
|
||||
}
|
||||
|
||||
|
||||
public String asyncPutWithResponseBody(String url, String json) throws IOException {
|
||||
if (StringUtils.isBlank(url) || StringUtils.isBlank(json)) {
|
||||
LOG.error("Empty note, cannot send it to zeppelinHub");
|
||||
throw new IOException("Cannot send emtpy note to zeppelinHub");
|
||||
}
|
||||
return sendToZeppelinHub(HttpMethod.PUT, zepelinhubUrl + url, json);
|
||||
}
|
||||
|
||||
public void asyncPut(String jsonNote) throws IOException {
|
||||
if (StringUtils.isBlank(jsonNote)) {
|
||||
LOG.error("Cannot save empty note/string to ZeppelinHub");
|
||||
return;
|
||||
}
|
||||
|
||||
client.newRequest(zepelinhubUrl).method(HttpMethod.PUT)
|
||||
.header(ZEPPELIN_TOKEN_HEADER, token)
|
||||
.content(new StringContentProvider(jsonNote, "UTF-8"), "application/json;charset=UTF-8")
|
||||
.send(new BufferingResponseListener() {
|
||||
|
||||
@Override
|
||||
public void onComplete(Result res) {
|
||||
if (!res.isFailed() && res.getResponse().getStatus() == 200) {
|
||||
LOG.info("Successfully saved note to ZeppelinHub with {}",
|
||||
res.getResponse().getStatus());
|
||||
} else {
|
||||
LOG.warn("Failed to save note to ZeppelinHub with HttpStatus {}",
|
||||
res.getResponse().getStatus());
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onFailure(Response response, Throwable failure) {
|
||||
LOG.error("Failed to save note to ZeppelinHub: {}", response.getReason(), failure);
|
||||
}
|
||||
});
|
||||
sendToZeppelinHub(HttpMethod.PUT, zepelinhubUrl, jsonNote);
|
||||
}
|
||||
|
||||
public void asyncDel(String argument) {
|
||||
public void asyncDel(String argument) throws IOException {
|
||||
if (StringUtils.isBlank(argument)) {
|
||||
LOG.error("Cannot delete empty note from ZeppelinHub");
|
||||
return;
|
||||
}
|
||||
client.newRequest(zepelinhubUrl + argument)
|
||||
.method(HttpMethod.DELETE)
|
||||
.header(ZEPPELIN_TOKEN_HEADER, token)
|
||||
.send(new BufferingResponseListener() {
|
||||
sendToZeppelinHub(HttpMethod.DELETE, zepelinhubUrl + argument);
|
||||
}
|
||||
|
||||
private String sendToZeppelinHub(HttpMethod method, String url) throws IOException {
|
||||
return sendToZeppelinHub(method, url, StringUtils.EMPTY);
|
||||
}
|
||||
|
||||
private String sendToZeppelinHub(HttpMethod method, String url, String json) throws IOException {
|
||||
InputStreamResponseListener listener = new InputStreamResponseListener();
|
||||
Response response;
|
||||
String data;
|
||||
|
||||
@Override
|
||||
public void onComplete(Result res) {
|
||||
if (!res.isFailed() && res.getResponse().getStatus() == 200) {
|
||||
LOG.info("Successfully removed note from ZeppelinHub with {}",
|
||||
res.getResponse().getStatus());
|
||||
} else {
|
||||
LOG.warn("Failed to remove note from ZeppelinHub with HttpStatus {}",
|
||||
res.getResponse().getStatus());
|
||||
}
|
||||
}
|
||||
Request request = client.newRequest(url).method(method).header(ZEPPELIN_TOKEN_HEADER, token);
|
||||
if ((method.equals(HttpMethod.PUT) || method.equals(HttpMethod.POST)) &&
|
||||
!StringUtils.isBlank(json)) {
|
||||
request.content(new StringContentProvider(json, "UTF-8"), "application/json;charset=UTF-8");
|
||||
}
|
||||
request.send(listener);
|
||||
|
||||
@Override
|
||||
public void onFailure(Response response, Throwable failure) {
|
||||
LOG.error("Failed to remove note from ZeppelinHub: {}", response.getReason(), failure);
|
||||
}
|
||||
});
|
||||
try {
|
||||
response = listener.get(30, TimeUnit.SECONDS);
|
||||
} catch (InterruptedException | TimeoutException | ExecutionException e) {
|
||||
LOG.error("Cannot perform {} request to ZeppelinHub", method, e);
|
||||
throw new IOException("Cannot perform " + method + " request to ZeppelinHub", e);
|
||||
}
|
||||
|
||||
int code = response.getStatus();
|
||||
if (code == 200) {
|
||||
try (InputStream responseContent = listener.getInputStream()) {
|
||||
data = IOUtils.toString(responseContent, "UTF-8");
|
||||
}
|
||||
} else {
|
||||
LOG.error("ZeppelinHub {} {} returned with status {} ", method, url, code);
|
||||
throw new IOException("Cannot perform " + method + " request to ZeppelinHub");
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
public void close() {
|
||||
|
|
|
|||
|
|
@ -155,5 +155,6 @@ public class InterpreterFactoryTest {
|
|||
}});
|
||||
|
||||
assertEquals("className1", factory.getInterpreter("note", "test-group1").getClassName());
|
||||
assertEquals("className1", factory.getInterpreter("note", "group1").getClassName());
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue