Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge remote-tracking branch 'origin/master' into ZEPPELIN-1320-2

Browse source
This commit is contained in:
Prabhjyot Singh 2016-10-28 11:18:38 +05:30
commit 0ff80ec987
77 changed files with 2011 additions and 715 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
docs/assets/themes/zeppelin/img/ui-img/configuration_menu.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 320 KiB

After

Width:  |  Height:  |  Size: 364 KiB

BIN
docs/assets/themes/zeppelin/img/ui-img/interpreter_menu.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 199 KiB

After

Width:  |  Height:  |  Size: 280 KiB

4
docs/displaysystem/basicdisplaysystem.md
View file

@ -25,7 +25,7 @@ limitations under the License.
## Text
By default, Apache Zeppelin prints interpreter responce as a plain text using `text` display system.
By default, Apache Zeppelin prints interpreter response as a plain text using `text` display system.
<img src="/assets/themes/zeppelin/img/screenshots/display_text.png" />
@ -41,7 +41,7 @@ With `%html` directive, Zeppelin treats your output as HTML
## Table
If you have data that row seprated by '\n' (newline) and column separated by '\t' (tab) with first row as header row, for example
If you have data that row separated by '\n' (newline) and column separated by '\t' (tab) with first row as header row, for example
<img src="/assets/themes/zeppelin/img/screenshots/display_table.png" />

4
docs/install/install.md
View file

@ -311,13 +311,13 @@ You can configure Apache Zeppelin with either **environment variables** in `conf
<td>ZEPPELIN_NOTEBOOK_HOMESCREEN</td>
<td>zeppelin.notebook.homescreen</td>
<td></td>
<td>Display notebook IDs on the Apache Zeppelin homescreen <br />i.e. 2A94M5J1Z</td>
<td>Display note IDs on the Apache Zeppelin homescreen <br />i.e. 2A94M5J1Z</td>
</tr>
<tr>
<td>ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE</td>
<td>zeppelin.notebook.homescreen.hide</td>
<td>false</td>
<td>Hide the notebook ID set by <code>ZEPPELIN_NOTEBOOK_HOMESCREEN</code> on the Apache Zeppelin homescreen. <br />For the further information, please read <a href="../manual/notebookashomepage.html">Customize your Zeppelin homepage</a>.</td>
<td>Hide the note ID set by <code>ZEPPELIN_NOTEBOOK_HOMESCREEN</code> on the Apache Zeppelin homescreen. <br />For the further information, please read <a href="../manual/notebookashomepage.html">Customize your Zeppelin homepage</a>.</td>
</tr>
<tr>
<td>ZEPPELIN_WAR_TEMPDIR</td>

2
docs/install/virtual_machine.md
View file

@ -163,7 +163,7 @@ import matplotlib.pyplot as plt
import numpy as np
import StringIO
# clear out any previous plots on this notebook
# clear out any previous plots on this note
plt.clf()
def show(p):

28
docs/manual/notebookashomepage.md
View file

@ -1,7 +1,7 @@
---
layout: page
title: "Customize Apache Zeppelin homepage"
description: "Apache Zeppelin allows you to use one of the notebooks you create as your Zeppelin Homepage. With that you can brand your Zeppelin installation, adjust the instruction to your users needs and even translate to other languages."
description: "Apache Zeppelin allows you to use one of the notes you create as your Zeppelin Homepage. With that you can brand your Zeppelin installation, adjust the instruction to your users needs and even translate to other languages."
group: manual
---
<!--
@ -23,37 +23,37 @@ limitations under the License.
<div id="toc"></div>
Apache Zeppelin allows you to use one of the notebooks you create as your Zeppelin Homepage.
Apache Zeppelin allows you to use one of the notes you create as your Zeppelin Homepage.
With that you can brand your Zeppelin installation, adjust the instruction to your users needs and even translate to other languages.
## How to set a notebook as your Zeppelin homepage
## How to set a note as your Zeppelin homepage
The process for creating your homepage is very simple as shown below:
1. Create a notebook using Zeppelin
2. Set the notebook id in the config file
1. Create a note using Zeppelin
2. Set the note id in the config file
3. Restart Zeppelin
### Create a notebook using Zeppelin
Create a new notebook using Zeppelin,
### Create a note using Zeppelin
Create a new note using Zeppelin,
you can use ```%md``` interpreter for markdown content or any other interpreter you like.
You can also use the display system to generate [text](../displaysystem/basicdisplaysystem.html#text), [html](../displaysystem/basicdisplaysystem.html#html), [table](../displaysystem/basicdisplaysystem.html#table) or
Angular ([backend API](../displaysystem/back-end-angular.html), [frontend API](../displaysystem/front-end-angular.html)).
Run (shift+Enter) the notebook and see the output. Optionally, change the notebook view to report to hide
Run (shift+Enter) the note and see the output. Optionally, change the note view to report to hide
the code sections.
### Set the notebook id in the config file
To set the notebook id in the config file, you should copy it from the last word in the notebook url.
### Set the note id in the config file
To set the note id in the config file, you should copy it from the last word in the note url.
For example,
<img src="/assets/themes/zeppelin/img/screenshots/homepage_notebook_id.png" width="400px" />
Set the notebook id to the ```ZEPPELIN_NOTEBOOK_HOMESCREEN``` environment variable
Set the note id to the ```ZEPPELIN_NOTEBOOK_HOMESCREEN``` environment variable
or ```zeppelin.notebook.homescreen``` property.
You can also set the ```ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE``` environment variable
or ```zeppelin.notebook.homescreen.hide``` property to hide the new notebook from the notebook list.
or ```zeppelin.notebook.homescreen.hide``` property to hide the new note from the note list.
### Restart Zeppelin
Restart your Zeppelin server
@ -89,7 +89,7 @@ println(
""")
```
After running the notebook you will see output similar to this one:
After running the note you will see output similar to this one:
<img src="/assets/themes/zeppelin/img/screenshots/homepage_notebook_list.png" />
The main trick here relays in linking the ```<div>``` to the controller:
@ -99,4 +99,4 @@ The main trick here relays in linking the ```<div>``` to the controller:
```
Once we have ```home``` as our controller variable in our ```<div></div>```
we can use ```home.notes.list``` to get access to the notebook list.
we can use ```home.notes.list``` to get access to the note list.

114
docs/rest-api/rest-notebook.md
View file

@ -35,13 +35,13 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
Notebooks REST API supports the following operations: List, Create, Get, Delete, Clone, Run, Export, Import as detailed in the following tables.
### List of the notebooks
### List of the notes
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```GET``` method lists the available notebooks on your server.
Notebook JSON contains the ```name``` and ```id``` of all notebooks.
<td>This ```GET``` method lists the available notes on your server.
Notebook JSON contains the ```name``` and ```id``` of all notes.
</td>
</tr>
<tr>
@ -77,13 +77,13 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br/>
### Create a new notebook
### Create a new note
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```POST``` method creates a new notebook using the given name or default name if none given.
The body field of the returned JSON contains the new notebook id.
<td>This ```POST``` method creates a new note using the given name or default name if none given.
The body field of the returned JSON contains the new note id.
</td>
</tr>
<tr>
@ -100,13 +100,13 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</tr>
<tr>
<td> sample JSON input (without paragraphs) </td>
<td><pre>{"name": "name of new notebook"}</pre></td>
<td><pre>{"name": "name of new note"}</pre></td>
</tr>
<tr>
<td> sample JSON input (with initial paragraphs) </td>
<td><pre>
{
"name": "name of new notebook",
"name": "name of new note",
"paragraphs": [
{
"title": "paragraph title1",
@ -131,18 +131,18 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br/>
### Get an existing notebook information
### Get an existing note information
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```GET``` method retrieves an existing notebook's information using the given id.
The body field of the returned JSON contain information about paragraphs in the notebook.
<td>This ```GET``` method retrieves an existing note's information using the given id.
The body field of the returned JSON contain information about paragraphs in the note.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -228,17 +228,17 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br/>
### Delete a notebook
### Delete a note
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```DELETE``` method deletes a notebook by the given notebook id.
<td>This ```DELETE``` method deletes a note by the given note id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -255,19 +255,19 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br/>
### Clone a notebook
### Clone a note
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```POST``` method clones a notebook by the given id and create a new notebook using the given name
<td>This ```POST``` method clones a note by the given id and create a new note using the given name
or default name if none given.
The body field of the returned JSON contains the new notebook id.
The body field of the returned JSON contains the new note id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -279,7 +279,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</tr>
<tr>
<td> sample JSON input </td>
<td><pre>{"name": "name of new notebook"}</pre></td>
<td><pre>{"name": "name of new note"}</pre></td>
</tr>
<tr>
<td> sample JSON response </td>
@ -299,14 +299,14 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<tr>
<td>Description</td>
<td>
This ```POST``` method runs all paragraphs in the given notebook id. <br />
If you can not find Notebook id 404 returns.
This ```POST``` method runs all paragraphs in the given note id. <br />
If you can not find Note id 404 returns.
If there is a problem with the interpreter returns a 412 error.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -345,12 +345,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```DELETE``` method stops all paragraphs in the given notebook id.
<td>This ```DELETE``` method stops all paragraphs in the given note id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -372,13 +372,13 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```GET``` method gets the status of all paragraphs by the given notebook id.
<td>This ```GET``` method gets the status of all paragraphs by the given note id.
The body field of the returned JSON contains of the array that compose of the paragraph id, paragraph status, paragraph finish date, paragraph started date.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -418,13 +418,13 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```GET``` method gets the status of a single paragraph by the given notebook and paragraph id.
<td>This ```GET``` method gets the status of a single paragraph by the given note and paragraph id.
The body field of the returned JSON contains of the array that compose of the paragraph id, paragraph status, paragraph finish date, paragraph started date.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -455,12 +455,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```POST``` method runs the paragraph asynchronously by given notebook and paragraph id. This API always return SUCCESS even if the execution of the paragraph fails later because the API is asynchronous
<td>This ```POST``` method runs the paragraph asynchronously by given note and paragraph id. This API always return SUCCESS even if the execution of the paragraph fails later because the API is asynchronous
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -474,7 +474,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<td> sample JSON input (optional, only needed when if you want to update dynamic form's value) </td>
<td><pre>
{
"name": "name of new notebook",
"name": "name of new note",
"params": {
"formLabel1": "value1",
"formLabel2": "value2"
@ -493,12 +493,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td> This ```POST``` method runs the paragraph synchronously by given notebook and paragraph id. This API can return SUCCESS or ERROR depending on the outcome of the paragraph execution
<td> This ```POST``` method runs the paragraph synchronously by given note and paragraph id. This API can return SUCCESS or ERROR depending on the outcome of the paragraph execution
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/run/[noteId]/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -512,7 +512,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<td> sample JSON input (optional, only needed when if you want to update dynamic form's value) </td>
<td><pre>
{
"name": "name of new notebook",
"name": "name of new note",
"params": {
"formLabel1": "value1",
"formLabel2": "value2"
@ -543,12 +543,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```DELETE``` method stops the paragraph by given notebook and paragraph id.
<td>This ```DELETE``` method stops the paragraph by given note and paragraph id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[notebookId]/[paragraphId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/job/[noteId]/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -570,12 +570,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```POST``` method adds cron job by the given notebook id.
<td>This ```POST``` method adds cron job by the given note id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -587,7 +587,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</tr>
<tr>
<td> sample JSON input </td>
<td><pre>{"cron": "cron expression of notebook"}</pre></td>
<td><pre>{"cron": "cron expression of note"}</pre></td>
</tr>
<tr>
<td> sample JSON response </td>
@ -602,12 +602,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```DELETE``` method removes cron job by the given notebook id.
<td>This ```DELETE``` method removes cron job by the given note id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -630,13 +630,13 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```GET``` method gets cron job expression of given notebook id.
<td>This ```GET``` method gets cron job expression of given note id.
The body field of the returned JSON contains the cron expression.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/cron/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -653,7 +653,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br />
### Full text search through the paragraphs in all notebooks
### Full text search through the paragraphs in all notes
<table class="table-configuration">
<col width="200">
<tr>
@ -681,7 +681,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
"body": [
{
"id": "<noteId>/paragraph/<paragraphId>",
"name":"Notebook Name",
"name":"Note Name",
"snippet":"",
"text":""
}
@ -702,7 +702,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]/paragraph```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]/paragraph```</td>
</tr>
<tr>
<td>Success code</td>
@ -752,7 +752,7 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]/paragraph/[paragraphId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]/paragraph/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -814,12 +814,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```POST``` method moves a paragraph to the specific index (order) from the notebook.
<td>This ```POST``` method moves a paragraph to the specific index (order) from the note.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]/paragraph/[paragraphId]/move/[newIndex]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]/paragraph/[paragraphId]/move/[newIndex]```</td>
</tr>
<tr>
<td>Success code</td>
@ -842,12 +842,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
<col width="200">
<tr>
<td>Description</td>
<td>This ```DELETE``` method deletes a paragraph by the given notebook and paragraph id.
<td>This ```DELETE``` method deletes a paragraph by the given note and paragraph id.
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[notebookId]/paragraph/[paragraphId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/[noteId]/paragraph/[paragraphId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -864,17 +864,17 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br />
### Export a notebook
### Export a note
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```GET``` method exports a notebook by the given id and gernerates a JSON
<td>This ```GET``` method exports a note by the given id and gernerates a JSON
</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/export/[notebookId]```</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/notebook/export/[noteId]```</td>
</tr>
<tr>
<td>Success code</td>
@ -914,12 +914,12 @@ If you work with Apache Zeppelin and find a need for an additional REST API, ple
</table>
<br />
### Import a notebook
### Import a note
<table class="table-configuration">
<col width="200">
<tr>
<td>Description</td>
<td>This ```POST``` method imports a notebook from the notebook JSON input
<td>This ```POST``` method imports a note from the note JSON input
</td>
</tr>
<tr>

27
geode/src/main/java/org/apache/zeppelin/geode/GeodeOqlInterpreter.java
View file

@ -21,7 +21,6 @@ import java.util.Properties;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.interpreter.Interpreter;
import org.apache.zeppelin.interpreter.InterpreterContext;
import org.apache.zeppelin.interpreter.InterpreterPropertyBuilder;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.apache.zeppelin.interpreter.InterpreterResult.Code;
import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion;
@ -38,7 +37,7 @@ import com.gemstone.gemfire.cache.query.Struct;
import com.gemstone.gemfire.pdx.PdxInstance;
/**
* Apache Geode OQL Interpreter (http://geode.incubator.apache.org)
* Apache Geode OQL Interpreter (http://geode.apache.org)
*
* <ul>
* <li>{@code geode.locator.host} - The Geode Locator {@code <HOST>} to connect to.</li>
@ -87,30 +86,12 @@ public class GeodeOqlInterpreter extends Interpreter {
private Logger logger = LoggerFactory.getLogger(GeodeOqlInterpreter.class);
public static final String DEFAULT_PORT = "10334";
public static final String DEFAULT_HOST = "localhost";
public static final String DEFAULT_MAX_RESULT = "1000";
private static final char NEWLINE = '\n';
private static final char TAB = '\t';
private static final char WHITESPACE = ' ';
private static final String TABLE_MAGIC_TAG = "%table ";
public static final String LOCATOR_HOST = "geode.locator.host";
public static final String LOCATOR_PORT = "geode.locator.port";
public static final String MAX_RESULT = "geode.max.result";
static {
Interpreter.register(
"oql",
"geode",
GeodeOqlInterpreter.class.getName(),
new InterpreterPropertyBuilder().add(LOCATOR_HOST, DEFAULT_HOST, "The Geode Locator Host.")
.add(LOCATOR_PORT, DEFAULT_PORT, "The Geode Locator Port")
.add(MAX_RESULT, DEFAULT_MAX_RESULT, "Max number of OQL result to display.").build());
}
private ClientCache clientCache = null;
private QueryService queryService = null;
private Exception exceptionOnConnect;
@ -122,8 +103,8 @@ public class GeodeOqlInterpreter extends Interpreter {
protected ClientCache getClientCache() {
String locatorHost = getProperty(LOCATOR_HOST);
int locatorPort = Integer.valueOf(getProperty(LOCATOR_PORT));
String locatorHost = getProperty("geode.locator.host");
int locatorPort = Integer.valueOf(getProperty("geode.locator.port"));
ClientCache clientCache =
new ClientCacheFactory().addPoolLocator(locatorHost, locatorPort).create();
@ -139,7 +120,7 @@ public class GeodeOqlInterpreter extends Interpreter {
close();
try {
maxResult = Integer.valueOf(getProperty(MAX_RESULT));
maxResult = Integer.valueOf(getProperty("geode.max.result"));
clientCache = getClientCache();
queryService = clientCache.getQueryService();

30
geode/src/main/resources/interpreter-setting.json Normal file
View file

@ -0,0 +1,30 @@
[
{
"group": "geode",
"name": "oql",
"className": "org.apache.zeppelin.geode.GeodeOqlInterpreter",
"properties": {
"geode.locator.host": {
"envName": null,
"propertyName": "geode.locator.host",
"defaultValue": "localhost",
"description": "The Geode Locator Host."
},
"geode.locator.port": {
"envName": null,
"propertyName": "geode.locator.port",
"defaultValue": "10334",
"description": "The Geode Locator Port."
},
"geode.max.result": {
"envName": null,
"propertyName": "geode.max.result",
"defaultValue": "1000",
"description": "Max number of OQL result to display."
}
},
"editor": {
"language": "sql"
}
}
]

6
geode/src/test/java/org/apache/zeppelin/geode/GeodeOqlInterpreterTest.java
View file

@ -58,9 +58,9 @@ public class GeodeOqlInterpreterTest {
public void testOpenCommandIndempotency() {
Properties properties = new Properties();
properties.put(LOCATOR_HOST, DEFAULT_HOST);
properties.put(LOCATOR_PORT, DEFAULT_PORT);
properties.put(MAX_RESULT, DEFAULT_MAX_RESULT);
properties.put("geode.locator.host", "localhost");
properties.put("geode.locator.port", "10334");
properties.put("geode.max.result", "1000");
GeodeOqlInterpreter spyGeodeOqlInterpreter = spy(new GeodeOqlInterpreter(properties));

2
lens/src/main/java/org/apache/zeppelin/lens/LensInterpreter.java
View file

@ -76,7 +76,7 @@ public class LensInterpreter extends Interpreter {
private static Pattern s_queryExecutePattern = Pattern.compile(".*query\\s+execute\\s+(.*)");
private static Map<String, ExecutionDetail> s_paraToQH =
new ConcurrentHashMap<String, ExecutionDetail> (); //tracks paragraphID -> Lens QueryHandle
new ConcurrentHashMap<String, ExecutionDetail> (); //tracks paragraphId -> Lens QueryHandle
private static Map<LensClient, Boolean> s_clientMap =
new ConcurrentHashMap<LensClient, Boolean>();

106
livy/pom.xml
View file

@ -40,6 +40,7 @@
<achilles.version>3.2.4-Zeppelin</achilles.version>
<assertj.version>1.7.0</assertj.version>
<mockito.version>1.9.5</mockito.version>
<livy.version>0.2.0</livy.version>
</properties>
<dependencies>
@ -106,8 +107,58 @@
<version>4.3.0.RELEASE</version>
</dependency>
<dependency>
<groupId>com.cloudera.livy</groupId>
<artifactId>livy-integration-test</artifactId>
<version>${livy.version}</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.cloudera.livy</groupId>
<artifactId>livy-test-lib</artifactId>
<version>${livy.version}</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.cloudera.livy</groupId>
<artifactId>livy-core</artifactId>
<version>${livy.version}</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>org.xerial.snappy</groupId>
<artifactId>snappy-java</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
<repositories>
<repository>
<id>ossrh</id>
<name>ossrh repository</name>
<url>https://oss.sonatype.org/content/repositories/releases/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<build>
<plugins>
<plugin>
@ -165,6 +216,61 @@
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.16</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<systemPropertyVariables>
<java.io.tmpdir>${project.build.directory}/tmp</java.io.tmpdir>
</systemPropertyVariables>
<argLine>-Xmx2048m</argLine>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.8</version>
<executions>
<!-- Cleans up files that tests append to (because we have two test plugins). -->
<execution>
<id>pre-test-clean</id>
<phase>generate-test-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<delete file="${project.build.directory}/unit-tests.log" quiet="true" />
<delete file="${project.build.directory}/jacoco.exec" quiet="true" />
<delete dir="${project.build.directory}/tmp" quiet="true" />
</target>
</configuration>
</execution>
<!-- Create the temp directory to be used by tests. -->
<execution>
<id>create-tmp-dir</id>
<phase>generate-test-resources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<mkdir dir="${project.build.directory}/tmp" />
</target>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

1
livy/src/main/java/org/apache/zeppelin/livy/LivyHelper.java
View file

@ -321,6 +321,7 @@ public class LivyHelper {
+ userSessionMap.get(context.getAuthenticationInfo().getUser())
+ "/statements/" + id,
"GET", null, context.getParagraphId());
LOGGER.debug("statement {} response: {}", id, json);
try {
Map jsonMap = gson.fromJson(json,
new TypeToken<Map>() {

7
livy/src/main/test/org/apache/zeppelin/livy/LivyHelperTest.java → livy/src/test/java/org/apache/zeppelin/livy/LivyHelperTest.java
View file

@ -28,6 +28,7 @@ import org.junit.runner.RunWith;
import org.mockito.Answers;
import org.mockito.Mock;
import org.mockito.runners.MockitoJUnitRunner;
import org.slf4j.LoggerFactory;
import java.util.HashMap;
import java.util.Properties;
@ -63,14 +64,14 @@ public class LivyHelperTest {
livyHelper.property = properties;
livyHelper.paragraphHttpMap = new HashMap<>();
livyHelper.gson = new GsonBuilder().setPrettyPrinting().create();
livyHelper.LOGGER = LoggerFactory.getLogger(LivyHelper.class);
doReturn("{\"id\":1,\"state\":\"idle\",\"kind\":\"spark\",\"proxyUser\":\"null\",\"log\":[]}")
.when(livyHelper)
.executeHTTP(
livyHelper.property.getProperty("zeppelin.livy.url") + "/sessions",
"POST",
"{\"kind\": \"spark\", \"proxyUser\": \"null\"}",
"{\"kind\": \"spark\", \"conf\": {}, \"proxyUser\": null}",
null
);
@ -80,7 +81,7 @@ public class LivyHelperTest {
.executeHTTP(
livyHelper.property.getProperty("zeppelin.livy.url") + "/sessions/1/statements",
"POST",
"{\"code\": \"print(1)\" }",
"{\"code\": \"print(1)\"}",
null
);

242
livy/src/test/java/org/apache/zeppelin/livy/LivyIntegrationTest.java Normal file
View file

@ -0,0 +1,242 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.livy;
import com.cloudera.livy.test.framework.Cluster;
import com.cloudera.livy.test.framework.Cluster$;
import org.apache.zeppelin.interpreter.InterpreterContext;
import org.apache.zeppelin.interpreter.InterpreterOutput;
import org.apache.zeppelin.interpreter.InterpreterOutputListener;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.junit.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Properties;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertTrue;
public class LivyIntegrationTest {
private static Logger LOGGER = LoggerFactory.getLogger(LivyIntegrationTest.class);
private static Cluster cluster;
private static Properties properties;
@BeforeClass
public static void setUp() {
if (!checkPreCondition()) {
return;
}
cluster = Cluster$.MODULE$.get();
LOGGER.info("Starting livy at {}", cluster.livyEndpoint());
properties = new Properties();
properties.setProperty("zeppelin.livy.url", cluster.livyEndpoint());
properties.setProperty("zeppelin.livy.create.session.retries", "120");
properties.setProperty("zeppelin.livy.spark.sql.maxResult", "100");
}
@AfterClass
public static void tearDown() {
if (cluster != null) {
cluster.cleanUp();
}
}
public static boolean checkPreCondition() {
if (System.getenv("LIVY_HOME") == null) {
LOGGER.warn(("livy integration is skipped because LIVY_HOME is not set"));
return false;
}
if (System.getenv("SPARK_HOME") == null) {
LOGGER.warn(("livy integration is skipped because SPARK_HOME is not set"));
return false;
}
return true;
}
@Test
public void testSparkInterpreter() {
if (!checkPreCondition()) {
return;
}
LivySparkInterpreter sparkInterpreter = new LivySparkInterpreter(properties);
AuthenticationInfo authInfo = new AuthenticationInfo("user1");
MyInterpreterOutputListener outputListener = new MyInterpreterOutputListener();
InterpreterOutput output = new InterpreterOutput(outputListener);
InterpreterContext context = new InterpreterContext("noteId", "paragraphId", "title",
"text", authInfo, null, null, null, null, null, output);
sparkInterpreter.open();
InterpreterResult result = sparkInterpreter.interpret("sc.version", context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
// test RDD api
outputListener.reset();
result = sparkInterpreter.interpret("sc.parallelize(1 to 10).sum()", context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertTrue(outputListener.getOutputAppended().contains("Double = 55.0"));
// test DataFrame api
outputListener.reset();
sparkInterpreter.interpret("val sqlContext = new org.apache.spark.sql.SQLContext(sc)\n"
+ "import sqlContext.implicits._", context);
result = sparkInterpreter.interpret("val df=sqlContext.createDataFrame(Seq((\"hello\",20)))\n"
+ "df.collect()" , context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertTrue(outputListener.getOutputAppended()
.contains("Array[org.apache.spark.sql.Row] = Array([hello,20])"));
sparkInterpreter.interpret("df.registerTempTable(\"df\")", context);
// test LivySparkSQLInterpreter which share the same SparkContext with LivySparkInterpreter
outputListener.reset();
LivySparkSQLInterpreter sqlInterpreter = new LivySparkSQLInterpreter(properties);
sqlInterpreter.open();
result = sqlInterpreter.interpret("select * from df", context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TABLE, result.type());
// TODO (zjffdu), \t at the end of each line is not necessary, it is a bug of LivySparkSQLInterpreter
assertEquals("_1\t_2\t\nhello\t20\t\n", result.message());
// single line comment
outputListener.reset();
String singleLineComment = "// my comment";
result = sparkInterpreter.interpret(singleLineComment, context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertNull(result.message());
// multiple line comment
outputListener.reset();
String multipleLineComment = "/* multiple \n" + "line \n" + "comment */";
result = sparkInterpreter.interpret(multipleLineComment, context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertNull(result.message());
// multi-line string
outputListener.reset();
String multiLineString = "val str = \"\"\"multiple\n" +
"line\"\"\"\n" +
"println(str)";
result = sparkInterpreter.interpret(multiLineString, context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertNull(result.message());
assertTrue(outputListener.getOutputAppended().contains("multiple\nline"));
// case class
outputListener.reset();
String caseClassCode = "case class Person(id:Int, \n" +
"name:String)\n" +
"val p=Person(1, \"name_a\")";
result = sparkInterpreter.interpret(caseClassCode, context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertNull(result.message());
assertTrue(outputListener.getOutputAppended().contains("defined class Person"));
// object class
outputListener.reset();
String objectClassCode = "object Person {}";
result = sparkInterpreter.interpret(objectClassCode, context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertNull(result.message());
assertTrue(outputListener.getOutputAppended().contains("defined module Person"));
}
@Test
public void testPySparkInterpreter() {
if (!checkPreCondition()) {
return;
}
LivyPySparkInterpreter pysparkInterpreter = new LivyPySparkInterpreter(properties);
AuthenticationInfo authInfo = new AuthenticationInfo("user1");
MyInterpreterOutputListener outputListener = new MyInterpreterOutputListener();
InterpreterOutput output = new InterpreterOutput(outputListener);
InterpreterContext context = new InterpreterContext("noteId", "paragraphId", "title",
"text", authInfo, null, null, null, null, null, output);
pysparkInterpreter.open();
InterpreterResult result = pysparkInterpreter.interpret("sc.version", context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
// test RDD api
outputListener.reset();
result = pysparkInterpreter.interpret("sc.range(1, 10).sum()", context);
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertEquals("45", result.message());
// test DataFrame api
outputListener.reset();
pysparkInterpreter.interpret("from pyspark.sql import SQLContext\n"
+ "sqlContext = SQLContext(sc)", context);
result = pysparkInterpreter.interpret("df=sqlContext.createDataFrame([(\"hello\",20)])\n"
+ "df.collect()" , context);
assertTrue(result.message().contains("[Row(_1=u'hello', _2=20)]"));
assertEquals(InterpreterResult.Type.TEXT, result.type());
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
}
@Test
public void testSparkRInterpreter() {
if (!checkPreCondition()) {
return;
}
// TODO (zjffdu), Livy's SparkRIntepreter has some issue, do it after livy-0.3 release.
}
public static class MyInterpreterOutputListener implements InterpreterOutputListener {
private StringBuilder outputAppended = new StringBuilder();
private StringBuilder outputUpdated = new StringBuilder();
@Override
public void onAppend(InterpreterOutput out, byte[] line) {
LOGGER.info("onAppend:" + new String(line));
outputAppended.append(new String(line));
}
@Override
public void onUpdate(InterpreterOutput out, byte[] output) {
LOGGER.info("onUpdate:" + new String(output));
outputUpdated.append(new String(output));
}
public String getOutputAppended() {
return outputAppended.toString();
}
public String getOutputUpdated() {
return outputUpdated.toString();
}
public void reset() {
outputAppended = new StringBuilder();
outputUpdated = new StringBuilder();
}
}
}

0
livy/src/main/test/org/apache/zeppelin/livy/LivyInterpreterTest.java → livy/src/test/java/org/apache/zeppelin/livy/LivyInterpreterTest.java
View file

24
livy/src/test/resources/log4j.properties Normal file
View file

@ -0,0 +1,24 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
log4j.rootLogger = INFO, stdout
log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) - %m%n
log4j.logger.org.apache.zeppelin.livy=DEBUG

2
scripts/vagrant/zeppelin-dev/README.md
View file

@ -140,7 +140,7 @@ import matplotlib.pyplot as plt
import numpy as np
import StringIO
# clear out any previous plots on this notebook
# clear out any previous plots on this note
plt.clf()
def show(p):

2
zeppelin-display/src/main/scala/org/apache/zeppelin/display/angular/notebookscope/AngularElem.scala
View file

@ -71,7 +71,7 @@ object AngularElem {
}
/**
* Disassociate (remove) all angular object in this notebook
* Disassociate (remove) all angular object in this note
*/
def disassociate() = {
val ic = InterpreterContext.get

2
zeppelin-display/src/main/scala/org/apache/zeppelin/display/angular/paragraphscope/AngularElem.scala
View file

@ -73,7 +73,7 @@ object AngularElem {
}
/**
* Disassociate (remove) all angular object in this notebook
* Disassociate (remove) all angular object in this note
*/
def disassociate() = {
val ic = InterpreterContext.get

20
zeppelin-interpreter/src/main/java/org/apache/zeppelin/user/AuthenticationInfo.java
View file

@ -18,13 +18,20 @@
package org.apache.zeppelin.user;
import org.apache.commons.lang.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/***
*
*/
public class AuthenticationInfo {
private static final Logger LOG = LoggerFactory.getLogger(AuthenticationInfo.class);
String user;
String ticket;
UserCredentials userCredentials;
public static final AuthenticationInfo ANONYMOUS = new AuthenticationInfo("anonymous",
"anonymous");
public AuthenticationInfo() {}
@ -66,4 +73,17 @@ public class AuthenticationInfo {
this.userCredentials = userCredentials;
}
public static boolean isAnonymous(AuthenticationInfo subject) {
if (subject == null) {
LOG.warn("Subject is null, assuming anonymous. "
+ "Not recommended to use subject as null except in tests");
return true;
}
return subject.isAnonymous();
}
public boolean isAnonymous() {
return ANONYMOUS.equals(this) || "anonymous".equalsIgnoreCase(this.getUser())
|| StringUtils.isEmpty(this.getUser());
}
}

10
zeppelin-server/src/main/java/org/apache/zeppelin/realm/ZeppelinHubRealm.java
View file

@ -20,6 +20,7 @@ import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.HashSet;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.commons.httpclient.HttpClient;
@ -36,6 +37,7 @@ import org.apache.shiro.authc.UsernamePasswordToken;
import org.apache.shiro.authz.AuthorizationInfo;
import org.apache.shiro.realm.AuthorizingRealm;
import org.apache.shiro.subject.PrincipalCollection;
import org.apache.zeppelin.server.ZeppelinServer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@ -135,6 +137,7 @@ public class ZeppelinHubRealm extends AuthorizingRealm {
}
responseBody = put.getResponseBodyAsString();
put.releaseConnection();
} catch (IOException e) {
LOG.error("Cannot login user", e);
throw new AuthenticationException(e.getMessage());
@ -147,6 +150,13 @@ public class ZeppelinHubRealm extends AuthorizingRealm {
LOG.error("Cannot deserialize ZeppelinHub response to User instance", e);
throw new AuthenticationException("Cannot login to ZeppelinHub");
}
/* TODO(khalid): add proper roles and add listener */
HashSet<String> userAndRoles = new HashSet<String>();
userAndRoles.add(account.login);
ZeppelinServer.notebookWsServer.broadcastReloadedNoteList(
new org.apache.zeppelin.user.AuthenticationInfo(account.login), userAndRoles);
return account;
}

115
zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRepoRestApi.java Normal file
View file

@ -0,0 +1,115 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest;
import java.util.Collections;
import java.util.List;
import javax.ws.rs.GET;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Response;
import javax.ws.rs.core.Response.Status;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.annotation.ZeppelinApi;
import org.apache.zeppelin.notebook.repo.NotebookRepoSync;
import org.apache.zeppelin.notebook.repo.NotebookRepoWithSettings;
import org.apache.zeppelin.rest.message.NotebookRepoSettingsRequest;
import org.apache.zeppelin.server.JsonResponse;
import org.apache.zeppelin.socket.NotebookServer;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.apache.zeppelin.utils.SecurityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.collect.ImmutableMap;
import com.google.gson.Gson;
import com.google.gson.JsonSyntaxException;
/**
* NoteRepo rest API endpoint.
*
*/
@Path("/notebook-repositories")
@Produces("application/json")
public class NotebookRepoRestApi {
private static final Logger LOG = LoggerFactory.getLogger(NotebookRepoRestApi.class);
private Gson gson = new Gson();
private NotebookRepoSync noteRepos;
private NotebookServer notebookWsServer;
public NotebookRepoRestApi() {}
public NotebookRepoRestApi(NotebookRepoSync noteRepos, NotebookServer notebookWsServer) {
this.noteRepos = noteRepos;
this.notebookWsServer = notebookWsServer;
}
/**
* List all notebook repository
*/
@GET
@ZeppelinApi
public Response listRepoSettings() {
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
LOG.info("Getting list of NoteRepo with Settings for user {}", subject.getUser());
List<NotebookRepoWithSettings> settings = noteRepos.getNotebookRepos(subject);
return new JsonResponse<>(Status.OK, "", settings).build();
}
/**
* Update a specific note repo.
*
* @param message
* @param settingId
* @return
*/
@PUT
@ZeppelinApi
public Response updateRepoSetting(String payload) {
if (StringUtils.isBlank(payload)) {
return new JsonResponse<>(Status.NOT_FOUND, "", Collections.emptyMap()).build();
}
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
NotebookRepoSettingsRequest newSettings = NotebookRepoSettingsRequest.EMPTY;
try {
newSettings = gson.fromJson(payload, NotebookRepoSettingsRequest.class);
} catch (JsonSyntaxException e) {
LOG.error("Cannot update notebook repo settings", e);
return new JsonResponse<>(Status.NOT_ACCEPTABLE, "",
ImmutableMap.of("error", "Invalid payload structure")).build();
}
if (NotebookRepoSettingsRequest.isEmpty(newSettings)) {
LOG.error("Invalid property");
return new JsonResponse<>(Status.NOT_ACCEPTABLE, "",
ImmutableMap.of("error", "Invalid payload")).build();
}
LOG.info("User {} is going to change repo setting", subject.getUser());
NotebookRepoWithSettings updatedSettings =
noteRepos.updateNotebookRepo(newSettings.name, newSettings.settings, subject);
if (!updatedSettings.isEmpty()) {
LOG.info("Broadcasting note list to user {}", subject.getUser());
notebookWsServer.broadcastReloadedNoteList(subject, null);
}
return new JsonResponse<>(Status.OK, "", updatedSettings).build();
}
}

243
zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java
View file

@ -51,7 +51,7 @@ import org.apache.zeppelin.notebook.NotebookAuthorization;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.rest.message.CronRequest;
import org.apache.zeppelin.types.InterpreterSettingsList;
import org.apache.zeppelin.rest.message.NewNotebookRequest;
import org.apache.zeppelin.rest.message.NewNoteRequest;
import org.apache.zeppelin.rest.message.NewParagraphRequest;
import org.apache.zeppelin.rest.message.RunParagraphWithParametersRequest;
import org.apache.zeppelin.search.SearchService;
@ -61,7 +61,7 @@ import org.apache.zeppelin.user.AuthenticationInfo;
import org.apache.zeppelin.utils.SecurityUtils;
/**
* Rest api endpoint for the noteBook.
* Rest api endpoint for the notebook.
*/
@Path("/notebook")
@Produces("application/json")
@ -70,7 +70,7 @@ public class NotebookRestApi {
Gson gson = new Gson();
private Notebook notebook;
private NotebookServer notebookServer;
private SearchService notebookIndex;
private SearchService noteSearchService;
private NotebookAuthorization notebookAuthorization;
public NotebookRestApi() {
@ -79,7 +79,7 @@ public class NotebookRestApi {
public NotebookRestApi(Notebook notebook, NotebookServer notebookServer, SearchService search) {
this.notebook = notebook;
this.notebookServer = notebookServer;
this.notebookIndex = search;
this.noteSearchService = search;
this.notebookAuthorization = notebook.getNotebookAuthorization();
}
@ -113,10 +113,6 @@ public class NotebookRestApi {
@ZeppelinApi
public Response putNotePermissions(@PathParam("noteId") String noteId, String req)
throws IOException {
/**
* TODO(jl): Fixed the type of HashSet
* https://issues.apache.org/jira/browse/ZEPPELIN-1162
*/
HashMap<String, HashSet<String>> permMap =
gson.fromJson(req, new TypeToken<HashMap<String, HashSet<String>>>() {
}.getType());
@ -181,7 +177,7 @@ public class NotebookRestApi {
}
/**
* list binded setting
* list bound setting
*/
@GET
@Path("interpreter/bind/{noteId}")
@ -196,18 +192,18 @@ public class NotebookRestApi {
@GET
@Path("/")
@ZeppelinApi
public Response getNotebookList() throws IOException {
public Response getNoteList() throws IOException {
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
List<Map<String, String>> notesInfo = notebookServer.generateNotebooksInfo(false, subject,
List<Map<String, String>> notesInfo = notebookServer.generateNotesInfo(false, subject,
SecurityUtils.getRoles());
return new JsonResponse<>(Status.OK, "", notesInfo).build();
}
@GET
@Path("{notebookId}")
@Path("{noteId}")
@ZeppelinApi
public Response getNotebook(@PathParam("notebookId") String notebookId) throws IOException {
Note note = notebook.getNote(notebookId);
public Response getNote(@PathParam("noteId") String noteId) throws IOException {
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -223,9 +219,9 @@ public class NotebookRestApi {
* @throws IOException
*/
@GET
@Path("export/{id}")
@Path("export/{noteId}")
@ZeppelinApi
public Response exportNoteBook(@PathParam("id") String noteId) throws IOException {
public Response exportNote(@PathParam("noteId") String noteId) throws IOException {
String exportJson = notebook.exportNote(noteId);
return new JsonResponse<>(Status.OK, "", exportJson).build();
}
@ -233,14 +229,14 @@ public class NotebookRestApi {
/**
* import new note REST API
*
* @param req - notebook Json
* @param req - note Json
* @return JSON with new note ID
* @throws IOException
*/
@POST
@Path("import")
@ZeppelinApi
public Response importNotebook(String req) throws IOException {
public Response importNote(String req) throws IOException {
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
Note newNote = notebook.importNote(req, null, subject);
return new JsonResponse<>(Status.CREATED, "", newNote.getId()).build();
@ -257,8 +253,8 @@ public class NotebookRestApi {
@Path("/")
@ZeppelinApi
public Response createNote(String message) throws IOException {
LOG.info("Create new notebook by JSON {}", message);
NewNotebookRequest request = gson.fromJson(message, NewNotebookRequest.class);
LOG.info("Create new note by JSON {}", message);
NewNoteRequest request = gson.fromJson(message, NewNoteRequest.class);
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
Note note = notebook.createNote(subject);
List<NewParagraphRequest> initialParagraphs = request.getParagraphs();
@ -285,20 +281,20 @@ public class NotebookRestApi {
/**
* Delete note REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException
*/
@DELETE
@Path("{notebookId}")
@Path("{noteId}")
@ZeppelinApi
public Response deleteNote(@PathParam("notebookId") String notebookId) throws IOException {
LOG.info("Delete notebook {} ", notebookId);
public Response deleteNote(@PathParam("noteId") String noteId) throws IOException {
LOG.info("Delete note {} ", noteId);
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
if (!(notebookId.isEmpty())) {
Note note = notebook.getNote(notebookId);
if (!(noteId.isEmpty())) {
Note note = notebook.getNote(noteId);
if (note != null) {
notebook.removeNote(notebookId, subject);
notebook.removeNote(noteId, subject);
}
}
@ -309,23 +305,23 @@ public class NotebookRestApi {
/**
* Clone note REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.CREATED
* @throws IOException, CloneNotSupportedException, IllegalArgumentException
*/
@POST
@Path("{notebookId}")
@Path("{noteId}")
@ZeppelinApi
public Response cloneNote(@PathParam("notebookId") String notebookId, String message)
public Response cloneNote(@PathParam("noteId") String noteId, String message)
throws IOException, CloneNotSupportedException, IllegalArgumentException {
LOG.info("clone notebook by JSON {}", message);
NewNotebookRequest request = gson.fromJson(message, NewNotebookRequest.class);
LOG.info("clone note by JSON {}", message);
NewNoteRequest request = gson.fromJson(message, NewNoteRequest.class);
String newNoteName = null;
if (request != null) {
newNoteName = request.getName();
}
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
Note newNote = notebook.cloneNote(notebookId, newNoteName, subject);
Note newNote = notebook.cloneNote(noteId, newNoteName, subject);
notebookServer.broadcastNote(newNote);
notebookServer.broadcastNoteList(subject, SecurityUtils.getRoles());
return new JsonResponse<>(Status.CREATED, "", newNote.getId()).build();
@ -339,13 +335,13 @@ public class NotebookRestApi {
* @throws IOException
*/
@POST
@Path("{notebookId}/paragraph")
@Path("{noteId}/paragraph")
@ZeppelinApi
public Response insertParagraph(@PathParam("notebookId") String notebookId, String message)
public Response insertParagraph(@PathParam("noteId") String noteId, String message)
throws IOException {
LOG.info("insert paragraph {} {}", notebookId, message);
LOG.info("insert paragraph {} {}", noteId, message);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -371,18 +367,18 @@ public class NotebookRestApi {
/**
* Get paragraph REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with information of the paragraph
* @throws IOException
*/
@GET
@Path("{notebookId}/paragraph/{paragraphId}")
@Path("{noteId}/paragraph/{paragraphId}")
@ZeppelinApi
public Response getParagraph(@PathParam("notebookId") String notebookId,
public Response getParagraph(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId) throws IOException {
LOG.info("get paragraph {} {}", notebookId, paragraphId);
LOG.info("get paragraph {} {}", noteId, paragraphId);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse(Status.NOT_FOUND, "note not found.").build();
}
@ -403,14 +399,14 @@ public class NotebookRestApi {
* @throws IOException
*/
@POST
@Path("{notebookId}/paragraph/{paragraphId}/move/{newIndex}")
@Path("{noteId}/paragraph/{paragraphId}/move/{newIndex}")
@ZeppelinApi
public Response moveParagraph(@PathParam("notebookId") String notebookId,
public Response moveParagraph(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId, @PathParam("newIndex") String newIndex)
throws IOException {
LOG.info("move paragraph {} {} {}", notebookId, paragraphId, newIndex);
LOG.info("move paragraph {} {} {}", noteId, paragraphId, newIndex);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse(Status.NOT_FOUND, "note not found.").build();
}
@ -436,18 +432,18 @@ public class NotebookRestApi {
/**
* Delete paragraph REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException
*/
@DELETE
@Path("{notebookId}/paragraph/{paragraphId}")
@Path("{noteId}/paragraph/{paragraphId}")
@ZeppelinApi
public Response deleteParagraph(@PathParam("notebookId") String notebookId,
public Response deleteParagraph(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId) throws IOException {
LOG.info("delete paragraph {} {}", notebookId, paragraphId);
LOG.info("delete paragraph {} {}", noteId, paragraphId);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse(Status.NOT_FOUND, "note not found.").build();
}
@ -466,19 +462,19 @@ public class NotebookRestApi {
}
/**
* Run notebook jobs REST API
* Run note jobs REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("job/{notebookId}")
@Path("job/{noteId}")
@ZeppelinApi
public Response runNoteJobs(@PathParam("notebookId") String notebookId)
public Response runNoteJobs(@PathParam("noteId") String noteId)
throws IOException, IllegalArgumentException {
LOG.info("run notebook jobs {} ", notebookId);
Note note = notebook.getNote(notebookId);
LOG.info("run note jobs {} ", noteId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -495,19 +491,19 @@ public class NotebookRestApi {
}
/**
* Stop(delete) notebook jobs REST API
* Stop(delete) note jobs REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@DELETE
@Path("job/{notebookId}")
@Path("job/{noteId}")
@ZeppelinApi
public Response stopNoteJobs(@PathParam("notebookId") String notebookId)
public Response stopNoteJobs(@PathParam("noteId") String noteId)
throws IOException, IllegalArgumentException {
LOG.info("stop notebook jobs {} ", notebookId);
Note note = notebook.getNote(notebookId);
LOG.info("stop note jobs {} ", noteId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -521,19 +517,19 @@ public class NotebookRestApi {
}
/**
* Get notebook job status REST API
* Get note job status REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@GET
@Path("job/{notebookId}")
@Path("job/{noteId}")
@ZeppelinApi
public Response getNoteJobStatus(@PathParam("notebookId") String notebookId)
public Response getNoteJobStatus(@PathParam("noteId") String noteId)
throws IOException, IllegalArgumentException {
LOG.info("get notebook job status.");
Note note = notebook.getNote(notebookId);
LOG.info("get note job status.");
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -542,21 +538,21 @@ public class NotebookRestApi {
}
/**
* Get notebook paragraph job status REST API
* Get note paragraph job status REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @param paragraphId ID of Paragraph
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@GET
@Path("job/{notebookId}/{paragraphId}")
@Path("job/{noteId}/{paragraphId}")
@ZeppelinApi
public Response getNoteParagraphJobStatus(@PathParam("notebookId") String notebookId,
public Response getNoteParagraphJobStatus(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId)
throws IOException, IllegalArgumentException {
LOG.info("get notebook paragraph job status.");
Note note = notebook.getNote(notebookId);
LOG.info("get note paragraph job status.");
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -579,14 +575,14 @@ public class NotebookRestApi {
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("job/{notebookId}/{paragraphId}")
@Path("job/{noteId}/{paragraphId}")
@ZeppelinApi
public Response runParagraph(@PathParam("notebookId") String notebookId,
public Response runParagraph(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId, String message)
throws IOException, IllegalArgumentException {
LOG.info("run paragraph job asynchronously {} {} {}", notebookId, paragraphId, message);
LOG.info("run paragraph job asynchronously {} {} {}", noteId, paragraphId, message);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -620,9 +616,9 @@ public class NotebookRestApi {
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("run/{notebookId}/{paragraphId}")
@Path("run/{noteId}/{paragraphId}")
@ZeppelinApi
public Response runParagraphSynchronously(@PathParam("notebookId") String noteId,
public Response runParagraphSynchronously(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId,
String message) throws
IOException, IllegalArgumentException {
@ -659,22 +655,18 @@ public class NotebookRestApi {
/**
* Stop(delete) paragraph job REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @param paragraphId ID of Paragraph
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@DELETE
@Path("job/{notebookId}/{paragraphId}")
@Path("job/{noteId}/{paragraphId}")
@ZeppelinApi
public Response stopParagraph(@PathParam("notebookId") String notebookId,
public Response stopParagraph(@PathParam("noteId") String noteId,
@PathParam("paragraphId") String paragraphId) throws IOException, IllegalArgumentException {
/**
* TODO(jl): Fixed notebookId to noteId
* https://issues.apache.org/jira/browse/ZEPPELIN-1163
*/
LOG.info("stop paragraph job {} ", notebookId);
Note note = notebook.getNote(notebookId);
LOG.info("stop paragraph job {} ", noteId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -695,16 +687,15 @@ public class NotebookRestApi {
* @throws IOException, IllegalArgumentException
*/
@POST
@Path("cron/{notebookId}")
@Path("cron/{noteId}")
@ZeppelinApi
public Response registerCronJob(@PathParam("notebookId") String notebookId, String message)
public Response registerCronJob(@PathParam("noteId") String noteId, String message)
throws IOException, IllegalArgumentException {
// TODO(jl): Fixed notebookId to noteId
LOG.info("Register cron job note={} request cron msg={}", notebookId, message);
LOG.info("Register cron job note={} request cron msg={}", noteId, message);
CronRequest request = gson.fromJson(message, CronRequest.class);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -724,19 +715,18 @@ public class NotebookRestApi {
/**
* Remove cron job REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@DELETE
@Path("cron/{notebookId}")
@Path("cron/{noteId}")
@ZeppelinApi
public Response removeCronJob(@PathParam("notebookId") String notebookId)
public Response removeCronJob(@PathParam("noteId") String noteId)
throws IOException, IllegalArgumentException {
// TODO(jl): Fixed notebookId to noteId
LOG.info("Remove cron job note {}", notebookId);
LOG.info("Remove cron job note {}", noteId);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -752,19 +742,18 @@ public class NotebookRestApi {
/**
* Get cron job REST API
*
* @param notebookId ID of Notebook
* @param noteId ID of Note
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
*/
@GET
@Path("cron/{notebookId}")
@Path("cron/{noteId}")
@ZeppelinApi
public Response getCronJob(@PathParam("notebookId") String notebookId)
public Response getCronJob(@PathParam("noteId") String noteId)
throws IOException, IllegalArgumentException {
// TODO(jl): Fixed notebookId to noteId
LOG.info("Get cron job note {}", notebookId);
LOG.info("Get cron job note {}", noteId);
Note note = notebook.getNote(notebookId);
Note note = notebook.getNote(noteId);
if (note == null) {
return new JsonResponse<>(Status.NOT_FOUND, "note not found.").build();
}
@ -773,7 +762,7 @@ public class NotebookRestApi {
}
/**
* Get notebook jobs for job manager
* Get note jobs for job manager
*
* @return JSON with status.OK
* @throws IOException, IllegalArgumentException
@ -781,22 +770,22 @@ public class NotebookRestApi {
@GET
@Path("jobmanager/")
@ZeppelinApi
public Response getJobListforNotebook() throws IOException, IllegalArgumentException {
LOG.info("Get notebook jobs for job manager");
public Response getJobListforNote() throws IOException, IllegalArgumentException {
LOG.info("Get note jobs for job manager");
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
List<Map<String, Object>> notebookJobs = notebook
.getJobListByUnixTime(false, 0, subject);
List<Map<String, Object>> noteJobs = notebook
.getJobListByUnixTime(false, 0, subject);
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs);
response.put("jobs", noteJobs);
return new JsonResponse<>(Status.OK, response).build();
}
/**
* Get updated notebook jobs for job manager
* Get updated note jobs for job manager
*
* Return the `Note` change information within the post unix timestamp.
*
@ -806,18 +795,18 @@ public class NotebookRestApi {
@GET
@Path("jobmanager/{lastUpdateUnixtime}/")
@ZeppelinApi
public Response getUpdatedJobListforNotebook(
public Response getUpdatedJobListforNote(
@PathParam("lastUpdateUnixtime") long lastUpdateUnixTime)
throws IOException, IllegalArgumentException {
LOG.info("Get updated notebook jobs lastUpdateTime {}", lastUpdateUnixTime);
LOG.info("Get updated note jobs lastUpdateTime {}", lastUpdateUnixTime);
List<Map<String, Object>> notebookJobs;
List<Map<String, Object>> noteJobs;
AuthenticationInfo subject = new AuthenticationInfo(SecurityUtils.getPrincipal());
notebookJobs = notebook.getJobListByUnixTime(false, lastUpdateUnixTime, subject);
noteJobs = notebook.getJobListByUnixTime(false, lastUpdateUnixTime, subject);
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs);
response.put("jobs", noteJobs);
return new JsonResponse<>(Status.OK, response).build();
}
@ -829,25 +818,25 @@ public class NotebookRestApi {
@Path("search")
@ZeppelinApi
public Response search(@QueryParam("q") String queryTerm) {
LOG.info("Searching notebooks for: {}", queryTerm);
LOG.info("Searching notes for: {}", queryTerm);
String principal = SecurityUtils.getPrincipal();
HashSet<String> roles = SecurityUtils.getRoles();
HashSet<String> userAndRoles = new HashSet<>();
userAndRoles.add(principal);
userAndRoles.addAll(roles);
List<Map<String, String>> notebooksFound = notebookIndex.query(queryTerm);
for (int i = 0; i < notebooksFound.size(); i++) {
String[] Id = notebooksFound.get(i).get("id").split("/", 2);
List<Map<String, String>> notesFound = noteSearchService.query(queryTerm);
for (int i = 0; i < notesFound.size(); i++) {
String[] Id = notesFound.get(i).get("id").split("/", 2);
String noteId = Id[0];
if (!notebookAuthorization.isOwner(noteId, userAndRoles) &&
!notebookAuthorization.isReader(noteId, userAndRoles) &&
!notebookAuthorization.isWriter(noteId, userAndRoles)) {
notebooksFound.remove(i);
notesFound.remove(i);
i--;
}
}
LOG.info("{} notebooks found", notebooksFound.size());
return new JsonResponse<>(Status.OK, notebooksFound).build();
LOG.info("{} notes found", notesFound.size());
return new JsonResponse<>(Status.OK, notesFound).build();
}
@ -856,7 +845,7 @@ public class NotebookRestApi {
// handle params if presented
if (!StringUtils.isEmpty(message)) {
RunParagraphWithParametersRequest request =
gson.fromJson(message, RunParagraphWithParametersRequest.class);
gson.fromJson(message, RunParagraphWithParametersRequest.class);
Map<String, Object> paramsForUpdating = request.getParams();
if (paramsForUpdating != null) {
paragraph.settings.getParams().putAll(paramsForUpdating);

6
zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/NewNotebookRequest.java → zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/NewNoteRequest.java
View file

@ -23,14 +23,14 @@ import java.util.Map;
import org.apache.zeppelin.interpreter.InterpreterOption;
/**
* NewNotebookRequest rest api request message
* NewNoteRequest rest api request message
*
*/
public class NewNotebookRequest {
public class NewNoteRequest {
String name;
List<NewParagraphRequest> paragraphs;
public NewNotebookRequest (){
public NewNoteRequest (){
}

49
zeppelin-server/src/main/java/org/apache/zeppelin/rest/message/NotebookRepoSettingsRequest.java Normal file
View file

@ -0,0 +1,49 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest.message;
import java.util.Collections;
import java.util.Map;
import org.apache.commons.lang.StringUtils;
/**
* Represent payload of a notebook repo settings.
*/
public class NotebookRepoSettingsRequest {
public static final NotebookRepoSettingsRequest EMPTY = new NotebookRepoSettingsRequest();
public String name;
public Map<String, String> settings;
public NotebookRepoSettingsRequest() {
name = StringUtils.EMPTY;
settings = Collections.emptyMap();
}
public boolean isEmpty() {
return this == EMPTY;
}
public static boolean isEmpty(NotebookRepoSettingsRequest repoSetting) {
if (repoSetting == null) {
return true;
}
return repoSetting.isEmpty();
}
}

34
zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
View file

@ -22,9 +22,7 @@ import java.io.IOException;
import java.util.EnumSet;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.ExecutorService;
import javax.net.ssl.SSLContext;
import javax.servlet.DispatcherType;
import javax.ws.rs.core.Application;
@ -37,9 +35,16 @@ import org.apache.zeppelin.helium.HeliumApplicationFactory;
import org.apache.zeppelin.interpreter.InterpreterFactory;
import org.apache.zeppelin.notebook.Notebook;
import org.apache.zeppelin.notebook.NotebookAuthorization;
import org.apache.zeppelin.notebook.repo.NotebookRepo;
import org.apache.zeppelin.notebook.repo.NotebookRepoSync;
import org.apache.zeppelin.rest.*;
import org.apache.zeppelin.rest.ConfigurationsRestApi;
import org.apache.zeppelin.rest.CredentialRestApi;
import org.apache.zeppelin.rest.HeliumRestApi;
import org.apache.zeppelin.rest.InterpreterRestApi;
import org.apache.zeppelin.rest.LoginRestApi;
import org.apache.zeppelin.rest.NotebookRepoRestApi;
import org.apache.zeppelin.rest.NotebookRestApi;
import org.apache.zeppelin.rest.SecurityRestApi;
import org.apache.zeppelin.rest.ZeppelinRestApi;
import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.apache.zeppelin.search.LuceneSearch;
import org.apache.zeppelin.search.SearchService;
@ -47,7 +52,12 @@ import org.apache.zeppelin.socket.NotebookServer;
import org.apache.zeppelin.user.Credentials;
import org.apache.zeppelin.utils.SecurityUtils;
import org.eclipse.jetty.http.HttpVersion;
import org.eclipse.jetty.server.*;
import org.eclipse.jetty.server.HttpConfiguration;
import org.eclipse.jetty.server.HttpConnectionFactory;
import org.eclipse.jetty.server.SecureRequestCustomizer;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
import org.eclipse.jetty.server.SslConnectionFactory;
import org.eclipse.jetty.server.handler.ContextHandlerCollection;
import org.eclipse.jetty.server.session.SessionHandler;
import org.eclipse.jetty.servlet.DefaultServlet;
@ -73,8 +83,8 @@ public class ZeppelinServer extends Application {
private SchedulerFactory schedulerFactory;
private InterpreterFactory replFactory;
private NotebookRepo notebookRepo;
private SearchService notebookIndex;
private SearchService noteSearchService;
private NotebookRepoSync notebookRepo;
private NotebookAuthorization notebookAuthorization;
private Credentials credentials;
private DependencyResolver depResolver;
@ -91,12 +101,12 @@ public class ZeppelinServer extends Application {
this.replFactory = new InterpreterFactory(conf, notebookWsServer,
notebookWsServer, heliumApplicationFactory, depResolver, SecurityUtils.isAuthenticated());
this.notebookRepo = new NotebookRepoSync(conf);
this.notebookIndex = new LuceneSearch();
this.noteSearchService = new LuceneSearch();
this.notebookAuthorization = NotebookAuthorization.init(conf);
this.credentials = new Credentials(conf.credentialsPersist(), conf.getCredentialsPath());
notebook = new Notebook(conf,
notebookRepo, schedulerFactory, replFactory, notebookWsServer,
notebookIndex, notebookAuthorization, credentials);
noteSearchService, notebookAuthorization, credentials);
// to update notebook from application event from remote process.
heliumApplicationFactory.setNotebook(notebook);
@ -304,9 +314,13 @@ public class ZeppelinServer extends Application {
ZeppelinRestApi root = new ZeppelinRestApi();
singletons.add(root);
NotebookRestApi notebookApi = new NotebookRestApi(notebook, notebookWsServer, notebookIndex);
NotebookRestApi notebookApi
= new NotebookRestApi(notebook, notebookWsServer, noteSearchService);
singletons.add(notebookApi);
NotebookRepoRestApi notebookRepoApi = new NotebookRepoRestApi(notebookRepo, notebookWsServer);
singletons.add(notebookRepoApi);
HeliumRestApi heliumApi = new HeliumRestApi(helium, heliumApplicationFactory, notebook);
singletons.add(heliumApi);

118
zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
View file

@ -235,8 +235,8 @@ public class NotebookServer extends WebSocketServlet implements
case LIST_CONFIGURATIONS:
sendAllConfigurations(conn, userAndRoles, notebook);
break;
case CHECKPOINT_NOTEBOOK:
checkpointNotebook(conn, notebook, messagereceived);
case CHECKPOINT_NOTE:
checkpointNote(conn, notebook, messagereceived);
break;
case LIST_REVISION_HISTORY:
listRevisionHistory(conn, notebook, messagereceived);
@ -244,11 +244,11 @@ public class NotebookServer extends WebSocketServlet implements
case NOTE_REVISION:
getNoteByRevision(conn, notebook, messagereceived);
break;
case LIST_NOTEBOOK_JOBS:
unicastNotebookJobInfo(conn, messagereceived);
case LIST_NOTE_JOBS:
unicastNoteJobInfo(conn, messagereceived);
break;
case UNSUBSCRIBE_UPDATE_NOTEBOOK_JOBS:
unsubscribeNotebookJobInfo(conn);
case UNSUBSCRIBE_UPDATE_NOTE_JOBS:
unsubscribeNoteJobInfo(conn);
break;
case GET_INTERPRETER_BINDINGS:
getInterpreterBindings(conn, messagereceived);
@ -430,43 +430,43 @@ public class NotebookServer extends WebSocketServlet implements
}
}
public void unicastNotebookJobInfo(NotebookSocket conn, Message fromMessage) throws IOException {
public void unicastNoteJobInfo(NotebookSocket conn, Message fromMessage) throws IOException {
addConnectionToNote(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(), conn);
AuthenticationInfo subject = new AuthenticationInfo(fromMessage.principal);
List<Map<String, Object>> notebookJobs = notebook()
List<Map<String, Object>> noteJobs = notebook()
.getJobListByUnixTime(false, 0, subject);
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs);
response.put("jobs", noteJobs);
conn.send(serializeMessage(new Message(OP.LIST_NOTEBOOK_JOBS)
.put("notebookJobs", response)));
conn.send(serializeMessage(new Message(OP.LIST_NOTE_JOBS)
.put("noteJobs", response)));
}
public void broadcastUpdateNotebookJobInfo(long lastUpdateUnixTime) throws IOException {
List<Map<String, Object>> notebookJobs = new LinkedList<>();
public void broadcastUpdateNoteJobInfo(long lastUpdateUnixTime) throws IOException {
List<Map<String, Object>> noteJobs = new LinkedList<>();
Notebook notebookObject = notebook();
List<Map<String, Object>> jobNotes = null;
if (notebookObject != null) {
jobNotes = notebook().getJobListByUnixTime(false, lastUpdateUnixTime, null);
notebookJobs = jobNotes == null ? notebookJobs : jobNotes;
noteJobs = jobNotes == null ? noteJobs : jobNotes;
}
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs != null ? notebookJobs : new LinkedList<>());
response.put("jobs", noteJobs != null ? noteJobs : new LinkedList<>());
broadcast(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(),
new Message(OP.LIST_UPDATE_NOTEBOOK_JOBS).put("notebookRunningJobs", response));
new Message(OP.LIST_UPDATE_NOTE_JOBS).put("noteRunningJobs", response));
}
public void unsubscribeNotebookJobInfo(NotebookSocket conn) {
public void unsubscribeNoteJobInfo(NotebookSocket conn) {
removeConnectionFromNote(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(), conn);
}
public void saveInterpreterBindings(NotebookSocket conn, Message fromMessage) {
String noteId = (String) fromMessage.data.get("noteID");
String noteId = (String) fromMessage.data.get("noteId");
try {
List<String> settingIdList = gson.fromJson(String.valueOf(
fromMessage.data.get("selectedSettingIds")), new TypeToken<ArrayList<String>>() {
@ -482,20 +482,20 @@ public class NotebookServer extends WebSocketServlet implements
public void getInterpreterBindings(NotebookSocket conn, Message fromMessage)
throws IOException {
String noteID = (String) fromMessage.data.get("noteID");
String noteId = (String) fromMessage.data.get("noteId");
List<InterpreterSettingsList> settingList =
InterpreterBindingUtils.getInterpreterBindings(notebook(), noteID);
InterpreterBindingUtils.getInterpreterBindings(notebook(), noteId);
conn.send(serializeMessage(new Message(OP.INTERPRETER_BINDINGS)
.put("interpreterBindings", settingList)));
}
public List<Map<String, String>> generateNotebooksInfo(boolean needsReload,
public List<Map<String, String>> generateNotesInfo(boolean needsReload,
AuthenticationInfo subject, HashSet<String> userAndRoles) {
Notebook notebook = notebook();
ZeppelinConfiguration conf = notebook.getConf();
String homescreenNotebookId = conf.getString(ConfVars.ZEPPELIN_NOTEBOOK_HOMESCREEN);
String homescreenNoteId = conf.getString(ConfVars.ZEPPELIN_NOTEBOOK_HOMESCREEN);
boolean hideHomeScreenNotebookFromList = conf
.getBoolean(ConfVars.ZEPPELIN_NOTEBOOK_HOMESCREEN_HIDE);
@ -512,7 +512,7 @@ public class NotebookServer extends WebSocketServlet implements
for (Note note : notes) {
Map<String, String> info = new HashMap<>();
if (hideHomeScreenNotebookFromList && note.getId().equals(homescreenNotebookId)) {
if (hideHomeScreenNotebookFromList && note.getId().equals(homescreenNoteId)) {
continue;
}
@ -539,21 +539,21 @@ public class NotebookServer extends WebSocketServlet implements
subject = new AuthenticationInfo(StringUtils.EMPTY);
}
//send first to requesting user
List<Map<String, String>> notesInfo = generateNotebooksInfo(false, subject, userAndRoles);
List<Map<String, String>> notesInfo = generateNotesInfo(false, subject, userAndRoles);
multicastToUser(subject.getUser(), new Message(OP.NOTES_INFO).put("notes", notesInfo));
//to others afterwards
for (String user: userConnectedSockets.keySet()) {
if (subject.getUser() == user) {
continue;
}
notesInfo = generateNotebooksInfo(false, new AuthenticationInfo(user), userAndRoles);
notesInfo = generateNotesInfo(false, new AuthenticationInfo(user), userAndRoles);
multicastToUser(user, new Message(OP.NOTES_INFO).put("notes", notesInfo));
}
}
public void unicastNoteList(NotebookSocket conn, AuthenticationInfo subject,
HashSet<String> userAndRoles) {
List<Map<String, String>> notesInfo = generateNotebooksInfo(false, subject, userAndRoles);
List<Map<String, String>> notesInfo = generateNotesInfo(false, subject, userAndRoles);
unicast(new Message(OP.NOTES_INFO).put("notes", notesInfo), conn);
}
@ -562,7 +562,7 @@ public class NotebookServer extends WebSocketServlet implements
subject = new AuthenticationInfo(StringUtils.EMPTY);
}
//reload and reply first to requesting user
List<Map<String, String>> notesInfo = generateNotebooksInfo(true, subject, userAndRoles);
List<Map<String, String>> notesInfo = generateNotesInfo(true, subject, userAndRoles);
multicastToUser(subject.getUser(), new Message(OP.NOTES_INFO).put("notes", notesInfo));
//to others afterwards
for (String user: userConnectedSockets.keySet()) {
@ -570,7 +570,7 @@ public class NotebookServer extends WebSocketServlet implements
continue;
}
//reloaded already above; parameter - false
notesInfo = generateNotebooksInfo(false, new AuthenticationInfo(user), userAndRoles);
notesInfo = generateNotesInfo(false, new AuthenticationInfo(user), userAndRoles);
multicastToUser(user, new Message(OP.NOTES_INFO).put("notes", notesInfo));
}
}
@ -785,8 +785,8 @@ public class NotebookServer extends WebSocketServlet implements
throws IOException {
Note note = null;
if (fromMessage != null) {
String noteName = (String) ((Map) fromMessage.get("notebook")).get("name");
String noteJson = gson.toJson(fromMessage.get("notebook"));
String noteName = (String) ((Map) fromMessage.get("note")).get("name");
String noteJson = gson.toJson(fromMessage.get("note"));
AuthenticationInfo subject = null;
if (fromMessage.principal != null) {
subject = new AuthenticationInfo(fromMessage.principal);
@ -1096,12 +1096,12 @@ public class NotebookServer extends WebSocketServlet implements
final AngularObject removed = registry.remove(varName, noteId, paragraphId);
if (removed != null) {
this.broadcastExcept(
noteId,
new Message(OP.ANGULAR_OBJECT_REMOVE).put("angularObject", removed)
.put("interpreterGroupId", interpreterGroupId)
.put("noteId", noteId)
.put("paragraphId", paragraphId),
conn);
noteId,
new Message(OP.ANGULAR_OBJECT_REMOVE).put("angularObject", removed)
.put("interpreterGroupId", interpreterGroupId)
.put("noteId", noteId)
.put("paragraphId", paragraphId),
conn);
}
}
@ -1132,7 +1132,7 @@ public class NotebookServer extends WebSocketServlet implements
private void insertParagraph(NotebookSocket conn, HashSet<String> userAndRoles,
Notebook notebook, Message fromMessage) throws IOException {
final int index = (int) Double.parseDouble(fromMessage.get("index")
.toString());
.toString());
String noteId = getOpenNoteId(conn);
final Note note = notebook.getNote(noteId);
NotebookAuthorization notebookAuthorization = notebook.getNotebookAuthorization();
@ -1252,7 +1252,7 @@ public class NotebookServer extends WebSocketServlet implements
.put("configurations", configurations)));
}
private void checkpointNotebook(NotebookSocket conn, Notebook notebook,
private void checkpointNote(NotebookSocket conn, Notebook notebook,
Message fromMessage) throws IOException {
String noteId = (String) fromMessage.get("noteId");
String commitMessage = (String) fromMessage.get("commitMessage");
@ -1384,7 +1384,7 @@ public class NotebookServer extends WebSocketServlet implements
@Override
public void onParagraphRemove(Paragraph p) {
try {
notebookServer.broadcastUpdateNotebookJobInfo(System.currentTimeMillis() - 5000);
notebookServer.broadcastUpdateNoteJobInfo(System.currentTimeMillis() - 5000);
} catch (IOException ioe) {
LOG.error("can not broadcast for job manager {}", ioe.getMessage());
}
@ -1393,14 +1393,14 @@ public class NotebookServer extends WebSocketServlet implements
@Override
public void onNoteRemove(Note note) {
try {
notebookServer.broadcastUpdateNotebookJobInfo(System.currentTimeMillis() - 5000);
notebookServer.broadcastUpdateNoteJobInfo(System.currentTimeMillis() - 5000);
} catch (IOException ioe) {
LOG.error("can not broadcast for job manager {}", ioe.getMessage());
}
List<Map<String, Object>> notesInfo = new LinkedList<>();
Map<String, Object> info = new HashMap<>();
info.put("notebookId", note.getId());
info.put("noteId", note.getId());
// set paragraphs
List<Map<String, Object>> paragraphsInfo = new LinkedList<>();
@ -1416,7 +1416,7 @@ public class NotebookServer extends WebSocketServlet implements
response.put("jobs", notesInfo);
notebookServer.broadcast(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(),
new Message(OP.LIST_UPDATE_NOTEBOOK_JOBS).put("notebookRunningJobs", response));
new Message(OP.LIST_UPDATE_NOTE_JOBS).put("noteRunningJobs", response));
}
@ -1424,35 +1424,35 @@ public class NotebookServer extends WebSocketServlet implements
public void onParagraphCreate(Paragraph p) {
Notebook notebook = notebookServer.notebook();
List<Map<String, Object>> notebookJobs = notebook.getJobListByParagraphId(
p.getId()
p.getId()
);
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs);
notebookServer.broadcast(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(),
new Message(OP.LIST_UPDATE_NOTEBOOK_JOBS).put("notebookRunningJobs", response));
new Message(OP.LIST_UPDATE_NOTE_JOBS).put("noteRunningJobs", response));
}
@Override
public void onNoteCreate(Note note) {
Notebook notebook = notebookServer.notebook();
List<Map<String, Object>> notebookJobs = notebook.getJobListBymNotebookId(
note.getId()
List<Map<String, Object>> notebookJobs = notebook.getJobListByNoteId(
note.getId()
);
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs);
notebookServer.broadcast(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(),
new Message(OP.LIST_UPDATE_NOTEBOOK_JOBS).put("notebookRunningJobs", response));
new Message(OP.LIST_UPDATE_NOTE_JOBS).put("noteRunningJobs", response));
}
@Override
public void onParagraphStatusChange(Paragraph p, Status status) {
Notebook notebook = notebookServer.notebook();
List<Map<String, Object>> notebookJobs = notebook.getJobListByParagraphId(
p.getId()
p.getId()
);
Map<String, Object> response = new HashMap<>();
@ -1460,21 +1460,21 @@ public class NotebookServer extends WebSocketServlet implements
response.put("jobs", notebookJobs);
notebookServer.broadcast(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(),
new Message(OP.LIST_UPDATE_NOTEBOOK_JOBS).put("notebookRunningJobs", response));
new Message(OP.LIST_UPDATE_NOTE_JOBS).put("noteRunningJobs", response));
}
@Override
public void onUnbindInterpreter(Note note, InterpreterSetting setting) {
Notebook notebook = notebookServer.notebook();
List<Map<String, Object>> notebookJobs = notebook.getJobListBymNotebookId(
note.getId()
List<Map<String, Object>> notebookJobs = notebook.getJobListByNoteId(
note.getId()
);
Map<String, Object> response = new HashMap<>();
response.put("lastResponseUnixTime", System.currentTimeMillis());
response.put("jobs", notebookJobs);
notebookServer.broadcast(JOB_MANAGER_SERVICE.JOB_MANAGER_PAGE.getKey(),
new Message(OP.LIST_UPDATE_NOTEBOOK_JOBS).put("notebookRunningJobs", response));
new Message(OP.LIST_UPDATE_NOTE_JOBS).put("noteRunningJobs", response));
}
}
@ -1523,7 +1523,7 @@ public class NotebookServer extends WebSocketServlet implements
notebookServer.broadcastNote(note);
try {
notebookServer.broadcastUpdateNotebookJobInfo(System.currentTimeMillis() - 5000);
notebookServer.broadcastUpdateNoteJobInfo(System.currentTimeMillis() - 5000);
} catch (IOException e) {
LOG.error("can not broadcast for job manager {}", e);
}
@ -1538,9 +1538,9 @@ public class NotebookServer extends WebSocketServlet implements
@Override
public void onOutputAppend(Paragraph paragraph, InterpreterOutput out, String output) {
Message msg = new Message(OP.PARAGRAPH_APPEND_OUTPUT)
.put("noteId", paragraph.getNote().getId())
.put("paragraphId", paragraph.getId())
.put("data", output);
.put("noteId", paragraph.getNote().getId())
.put("paragraphId", paragraph.getId())
.put("data", output);
notebookServer.broadcast(paragraph.getNote().getId(), msg);
}
@ -1554,9 +1554,9 @@ public class NotebookServer extends WebSocketServlet implements
@Override
public void onOutputUpdate(Paragraph paragraph, InterpreterOutput out, String output) {
Message msg = new Message(OP.PARAGRAPH_UPDATE_OUTPUT)
.put("noteId", paragraph.getNote().getId())
.put("paragraphId", paragraph.getId())
.put("data", output);
.put("noteId", paragraph.getNote().getId())
.put("paragraphId", paragraph.getId())
.put("data", output);
notebookServer.broadcast(paragraph.getNote().getId(), msg);
}

2
zeppelin-server/src/test/java/org/apache/zeppelin/AbstractZeppelinIT.java
View file

@ -122,7 +122,7 @@ abstract public class AbstractZeppelinIT {
driver.findElement(By.xpath(".//*[@id='main']//button[@ng-click='removeNote(note.id)']"))
.sendKeys(Keys.ENTER);
ZeppelinITUtils.sleep(1000, true);
driver.findElement(By.xpath("//div[@class='modal-dialog'][contains(.,'delete this notebook')]" +
driver.findElement(By.xpath("//div[@class='modal-dialog'][contains(.,'delete this note')]" +
"//div[@class='modal-footer']//button[contains(.,'OK')]")).click();
ZeppelinITUtils.sleep(100, true);
}

8
zeppelin-server/src/test/java/org/apache/zeppelin/integration/AuthenticationIT.java
View file

@ -180,7 +180,7 @@ public class AuthenticationIT extends AbstractZeppelinIT {
try {
WebElement element = pollingWait(By.xpath("//*[@id='notebook-names']//a[contains(@href, '" + noteId + "')]"),
MAX_BROWSER_TIMEOUT_SEC);
collector.checkThat("Check is user has permission to view this notebook link", false,
collector.checkThat("Check is user has permission to view this note link", false,
CoreMatchers.equalTo(element.isDisplayed()));
} catch (Exception e) {
//This should have failed, nothing to worry.
@ -191,7 +191,7 @@ public class AuthenticationIT extends AbstractZeppelinIT {
List<WebElement> privilegesModal = driver.findElements(
By.xpath("//div[@class='modal-content']//div[@class='bootstrap-dialog-header']" +
"//div[contains(.,'Insufficient privileges')]"));
collector.checkThat("Check is user has permission to view this notebook", 1,
collector.checkThat("Check is user has permission to view this note", 1,
CoreMatchers.equalTo(privilegesModal.size()));
driver.findElement(
By.xpath("//div[@class='modal-content'][contains(.,'Insufficient privileges')]" +
@ -202,7 +202,7 @@ public class AuthenticationIT extends AbstractZeppelinIT {
try {
WebElement element = pollingWait(By.xpath("//*[@id='notebook-names']//a[contains(@href, '" + noteId + "')]"),
MAX_BROWSER_TIMEOUT_SEC);
collector.checkThat("Check is user has permission to view this notebook link", true,
collector.checkThat("Check is user has permission to view this note link", true,
CoreMatchers.equalTo(element.isDisplayed()));
} catch (Exception e) {
//This should have failed, nothing to worry.
@ -213,7 +213,7 @@ public class AuthenticationIT extends AbstractZeppelinIT {
privilegesModal = driver.findElements(
By.xpath("//div[@class='modal-content']//div[@class='bootstrap-dialog-header']" +
"//div[contains(.,'Insufficient privileges')]"));
collector.checkThat("Check is user has permission to view this notebook", 0,
collector.checkThat("Check is user has permission to view this note", 0,
CoreMatchers.equalTo(privilegesModal.size()));
deleteTestNotebook(driver);
authenticationIT.logoutUser("finance2");

2
zeppelin-server/src/test/java/org/apache/zeppelin/integration/ZeppelinIT.java
View file

@ -182,7 +182,7 @@ public class ZeppelinIT extends AbstractZeppelinIT {
driver.findElement(By.xpath(".//*[@id='main']//button[@ng-click='removeNote(note.id)']"))
.sendKeys(Keys.ENTER);
ZeppelinITUtils.sleep(1000, true);
driver.findElement(By.xpath("//div[@class='modal-dialog'][contains(.,'delete this notebook')]" +
driver.findElement(By.xpath("//div[@class='modal-dialog'][contains(.,'delete this note')]" +
"//div[@class='modal-footer']//button[contains(.,'OK')]")).click();
ZeppelinITUtils.sleep(100, true);

120
zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRepoRestApiTest.java Normal file
View file

@ -0,0 +1,120 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest;
import static org.hamcrest.CoreMatchers.is;
import static org.hamcrest.CoreMatchers.not;
import static org.junit.Assert.assertThat;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PutMethod;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
/**
* NotebookRepo rest api test.
*/
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class NotebookRepoRestApiTest extends AbstractTestRestApi {
Gson gson = new Gson();
AuthenticationInfo anonymous;
@BeforeClass
public static void init() throws Exception {
AbstractTestRestApi.startUp();
}
@AfterClass
public static void destroy() throws Exception {
AbstractTestRestApi.shutDown();
}
@Before
public void setUp() {
anonymous = new AuthenticationInfo("anonymous");
}
private List<Map<String, Object>> getListOfReposotiry() throws IOException {
GetMethod get = httpGet("/notebook-repositories");
Map<String, Object> responce = gson.fromJson(get.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {}.getType());
get.releaseConnection();
return (List<Map<String, Object>>) responce.get("body");
}
private void updateNotebookRepoWithNewSetting(String payload) throws IOException {
PutMethod put = httpPut("/notebook-repositories", payload);
int status = put.getStatusCode();
put.releaseConnection();
assertThat(status, is(200));
}
@Test public void ThatCanGetNotebookRepositoiesSettings() throws IOException {
List<Map<String, Object>> listOfRepositories = getListOfReposotiry();
assertThat(listOfRepositories.size(), is(not(0)));
}
@Test public void setNewDirectoryForLocalDirectory() throws IOException {
List<Map<String, Object>> listOfRepositories = getListOfReposotiry();
String localVfs = StringUtils.EMPTY;
String className = StringUtils.EMPTY;
for (int i = 0; i < listOfRepositories.size(); i++) {
if (listOfRepositories.get(i).get("name").equals("VFSNotebookRepo")) {
localVfs = (String) ((List<Map<String, Object>>)listOfRepositories.get(i).get("settings")).get(0).get("selected");
className = (String) listOfRepositories.get(i).get("className");
break;
}
}
if (StringUtils.isBlank(localVfs)) {
// no loval VFS set...
return;
}
String payload = "{ \"name\": \"" + className + "\", \"settings\" : { \"Notebook Path\" : \"/tmp/newDir\" } }";
updateNotebookRepoWithNewSetting(payload);
// Verify
listOfRepositories = getListOfReposotiry();
String updatedPath = StringUtils.EMPTY;
for (int i = 0; i < listOfRepositories.size(); i++) {
if (listOfRepositories.get(i).get("name").equals("VFSNotebookRepo")) {
updatedPath = (String) ((List<Map<String, Object>>)listOfRepositories.get(i).get("settings")).get(0).get("selected");
break;
}
}
assertThat(updatedPath, is("/tmp/newDir"));
// go back to normal
payload = "{ \"name\": \"" + className + "\", \"settings\" : { \"Notebook Path\" : \"" + localVfs + "\" } }";
updateNotebookRepoWithNewSetting(payload);
}
}

12
zeppelin-server/src/test/java/org/apache/zeppelin/rest/NotebookRestApiTest.java
View file

@ -155,28 +155,28 @@ public class NotebookRestApiTest extends AbstractTestRestApi {
}
@Test
public void testCloneNotebook() throws IOException {
public void testCloneNote() throws IOException {
Note note1 = ZeppelinServer.notebook.createNote(anonymous);
PostMethod post = httpPost("/notebook/" + note1.getId(), "");
LOG.info("testCloneNotebook response\n" + post.getResponseBodyAsString());
LOG.info("testCloneNote response\n" + post.getResponseBodyAsString());
assertThat(post, isCreated());
Map<String, Object> resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
String clonedNotebookId = (String) resp.get("body");
String clonedNoteId = (String) resp.get("body");
post.releaseConnection();
GetMethod get = httpGet("/notebook/" + clonedNotebookId);
GetMethod get = httpGet("/notebook/" + clonedNoteId);
assertThat(get, isAllowed());
Map<String, Object> resp2 = gson.fromJson(get.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
Map<String, Object> resp2Body = (Map<String, Object>) resp2.get("body");
assertEquals((String)resp2Body.get("name"), "Note " + clonedNotebookId);
assertEquals((String)resp2Body.get("name"), "Note " + clonedNoteId);
get.releaseConnection();
//cleanup
ZeppelinServer.notebook.removeNote(note1.getId(), anonymous);
ZeppelinServer.notebook.removeNote(clonedNotebookId, anonymous);
ZeppelinServer.notebook.removeNote(clonedNoteId, anonymous);
}
}

200
zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinRestApiTest.java
View file

@ -29,10 +29,8 @@ import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.PutMethod;
import org.apache.commons.lang3.StringUtils;
import org.apache.zeppelin.interpreter.InterpreterSetting;
import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.scheduler.Job.Status;
import org.apache.zeppelin.server.ZeppelinServer;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.junit.AfterClass;
@ -84,8 +82,8 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
@Test
public void testGetNotebookInfo() throws IOException {
LOG.info("testGetNotebookInfo");
public void testGetNoteInfo() throws IOException {
LOG.info("testGetNoteInfo");
// Create note to get info
Note note = ZeppelinServer.notebook.createNote(anonymous);
assertNotNull("can't create new note", note);
@ -98,10 +96,10 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setText(paragraphText);
note.persist(anonymous);
String sourceNoteID = note.getId();
GetMethod get = httpGet("/notebook/" + sourceNoteID);
LOG.info("testGetNotebookInfo \n" + get.getResponseBodyAsString());
assertThat("test notebook get method:", get, isAllowed());
String sourceNoteId = note.getId();
GetMethod get = httpGet("/notebook/" + sourceNoteId);
LOG.info("testGetNoteInfo \n" + get.getResponseBodyAsString());
assertThat("test note get method:", get, isAllowed());
Map<String, Object> resp = gson.fromJson(get.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
@ -115,45 +113,45 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
assertTrue(paragraphs.size() > 0);
assertEquals(paragraphText, paragraphs.get(0).get("text"));
//
ZeppelinServer.notebook.removeNote(sourceNoteID, anonymous);
ZeppelinServer.notebook.removeNote(sourceNoteId, anonymous);
}
@Test
public void testNotebookCreateWithName() throws IOException {
public void testNoteCreateWithName() throws IOException {
String noteName = "Test note name";
testNotebookCreate(noteName);
testNoteCreate(noteName);
}
@Test
public void testNotebookCreateNoName() throws IOException {
testNotebookCreate("");
public void testNoteCreateNoName() throws IOException {
testNoteCreate("");
}
@Test
public void testNotebookCreateWithParagraphs() throws IOException {
// Call Create Notebook REST API
public void testNoteCreateWithParagraphs() throws IOException {
// Call Create Note REST API
String noteName = "test";
String jsonRequest = "{\"name\":\"" + noteName + "\", \"paragraphs\": [" +
"{\"title\": \"title1\", \"text\": \"text1\"}," +
"{\"title\": \"title2\", \"text\": \"text2\"}" +
"]}";
PostMethod post = httpPost("/notebook/", jsonRequest);
LOG.info("testNotebookCreate \n" + post.getResponseBodyAsString());
assertThat("test notebook create method:", post, isCreated());
LOG.info("testNoteCreate \n" + post.getResponseBodyAsString());
assertThat("test note create method:", post, isCreated());
Map<String, Object> resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
String newNotebookId = (String) resp.get("body");
LOG.info("newNotebookId:=" + newNotebookId);
Note newNote = ZeppelinServer.notebook.getNote(newNotebookId);
String newNoteId = (String) resp.get("body");
LOG.info("newNoteId:=" + newNoteId);
Note newNote = ZeppelinServer.notebook.getNote(newNoteId);
assertNotNull("Can not find new note by id", newNote);
// This is partial test as newNote is in memory but is not persistent
String newNoteName = newNote.getName();
LOG.info("new note name is: " + newNoteName);
String expectedNoteName = noteName;
if (noteName.isEmpty()) {
expectedNoteName = "Note " + newNotebookId;
expectedNoteName = "Note " + newNoteId;
}
assertEquals("compare note name", expectedNoteName, newNoteName);
assertEquals("initial paragraph check failed", 3, newNote.getParagraphs().size());
@ -165,34 +163,34 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
assertTrue("paragraph text check failed", p.getText().startsWith("text"));
}
// cleanup
ZeppelinServer.notebook.removeNote(newNotebookId, anonymous);
ZeppelinServer.notebook.removeNote(newNoteId, anonymous);
post.releaseConnection();
}
private void testNotebookCreate(String noteName) throws IOException {
// Call Create Notebook REST API
private void testNoteCreate(String noteName) throws IOException {
// Call Create Note REST API
String jsonRequest = "{\"name\":\"" + noteName + "\"}";
PostMethod post = httpPost("/notebook/", jsonRequest);
LOG.info("testNotebookCreate \n" + post.getResponseBodyAsString());
assertThat("test notebook create method:", post, isCreated());
LOG.info("testNoteCreate \n" + post.getResponseBodyAsString());
assertThat("test note create method:", post, isCreated());
Map<String, Object> resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
String newNotebookId = (String) resp.get("body");
LOG.info("newNotebookId:=" + newNotebookId);
Note newNote = ZeppelinServer.notebook.getNote(newNotebookId);
String newNoteId = (String) resp.get("body");
LOG.info("newNoteId:=" + newNoteId);
Note newNote = ZeppelinServer.notebook.getNote(newNoteId);
assertNotNull("Can not find new note by id", newNote);
// This is partial test as newNote is in memory but is not persistent
String newNoteName = newNote.getName();
LOG.info("new note name is: " + newNoteName);
String expectedNoteName = noteName;
if (noteName.isEmpty()) {
expectedNoteName = "Note " + newNotebookId;
expectedNoteName = "Note " + newNoteId;
}
assertEquals("compare note name", expectedNoteName, newNoteName);
// cleanup
ZeppelinServer.notebook.removeNote(newNotebookId, anonymous);
ZeppelinServer.notebook.removeNote(newNoteId, anonymous);
post.releaseConnection();
}
@ -203,20 +201,20 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
//Create note and get ID
Note note = ZeppelinServer.notebook.createNote(anonymous);
String noteId = note.getId();
testDeleteNotebook(noteId);
testDeleteNote(noteId);
}
@Test
public void testDeleteNoteBadId() throws IOException {
LOG.info("testDeleteNoteBadId");
testDeleteNotebook("2AZFXEX97");
testDeleteNotebook("bad_ID");
testDeleteNote("2AZFXEX97");
testDeleteNote("bad_ID");
}
@Test
public void testExportNotebook() throws IOException {
LOG.info("testExportNotebook");
public void testexportNote() throws IOException {
LOG.info("testexportNote");
Note note = ZeppelinServer.notebook.createNote(anonymous);
assertNotNull("can't create new note", note);
note.setName("source note for export");
@ -226,11 +224,11 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setConfig(config);
paragraph.setText("%md This is my new paragraph in my new note");
note.persist(anonymous);
String sourceNoteID = note.getId();
// Call export Notebook REST API
GetMethod get = httpGet("/notebook/export/" + sourceNoteID);
LOG.info("testNotebookExport \n" + get.getResponseBodyAsString());
assertThat("test notebook export method:", get, isAllowed());
String sourceNoteId = note.getId();
// Call export Note REST API
GetMethod get = httpGet("/notebook/export/" + sourceNoteId);
LOG.info("testNoteExport \n" + get.getResponseBodyAsString());
assertThat("test note export method:", get, isAllowed());
Map<String, Object> resp =
gson.fromJson(get.getResponseBodyAsString(),
@ -239,7 +237,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
String exportJSON = (String) resp.get("body");
assertNotNull("Can not find new notejson", exportJSON);
LOG.info("export JSON:=" + exportJSON);
ZeppelinServer.notebook.removeNote(sourceNoteID, anonymous);
ZeppelinServer.notebook.removeNote(sourceNoteId, anonymous);
get.releaseConnection();
}
@ -248,8 +246,8 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
public void testImportNotebook() throws IOException {
Map<String, Object> resp;
String noteName = "source note for import";
LOG.info("testImortNotebook");
// create test notebook
LOG.info("testImortNote");
// create test note
Note note = ZeppelinServer.notebook.createNote(anonymous);
assertNotNull("can't create new note", note);
note.setName(noteName);
@ -259,10 +257,10 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setConfig(config);
paragraph.setText("%md This is my new paragraph in my new note");
note.persist(anonymous);
String sourceNoteID = note.getId();
String sourceNoteId = note.getId();
// get note content as JSON
String oldJson = getNoteContent(sourceNoteID);
// call notebook post
String oldJson = getNoteContent(sourceNoteId);
// call note post
PostMethod importPost = httpPost("/notebook/import/", oldJson);
assertThat(importPost, isCreated());
resp =
@ -270,7 +268,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
new TypeToken<Map<String, Object>>() {}.getType());
String importId = (String) resp.get("body");
assertNotNull("Did not get back a notebook id in body", importId);
assertNotNull("Did not get back a note id in body", importId);
Note newNote = ZeppelinServer.notebook.getNote(importId);
assertEquals("Compare note names", noteName, newNote.getName());
assertEquals("Compare paragraphs count", note.getParagraphs().size(), newNote.getParagraphs()
@ -295,22 +293,22 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
return body;
}
private void testDeleteNotebook(String notebookId) throws IOException {
private void testDeleteNote(String noteId) throws IOException {
DeleteMethod delete = httpDelete(("/notebook/" + notebookId));
LOG.info("testDeleteNotebook delete response\n" + delete.getResponseBodyAsString());
DeleteMethod delete = httpDelete(("/notebook/" + noteId));
LOG.info("testDeleteNote delete response\n" + delete.getResponseBodyAsString());
assertThat("Test delete method:", delete, isAllowed());
delete.releaseConnection();
// make sure note is deleted
if (!notebookId.isEmpty()) {
Note deletedNote = ZeppelinServer.notebook.getNote(notebookId);
if (!noteId.isEmpty()) {
Note deletedNote = ZeppelinServer.notebook.getNote(noteId);
assertNull("Deleted note should be null", deletedNote);
}
}
@Test
public void testCloneNotebook() throws IOException, CloneNotSupportedException, IllegalArgumentException {
LOG.info("testCloneNotebook");
public void testCloneNote() throws IOException, CloneNotSupportedException, IllegalArgumentException {
LOG.info("testCloneNote");
// Create note to clone
Note note = ZeppelinServer.notebook.createNote(anonymous);
assertNotNull("can't create new note", note);
@ -321,21 +319,21 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setConfig(config);
paragraph.setText("%md This is my new paragraph in my new note");
note.persist(anonymous);
String sourceNoteID = note.getId();
String sourceNoteId = note.getId();
String noteName = "clone Note Name";
// Call Clone Notebook REST API
// Call Clone Note REST API
String jsonRequest = "{\"name\":\"" + noteName + "\"}";
PostMethod post = httpPost("/notebook/" + sourceNoteID, jsonRequest);
LOG.info("testNotebookClone \n" + post.getResponseBodyAsString());
assertThat("test notebook clone method:", post, isCreated());
PostMethod post = httpPost("/notebook/" + sourceNoteId, jsonRequest);
LOG.info("testNoteClone \n" + post.getResponseBodyAsString());
assertThat("test note clone method:", post, isCreated());
Map<String, Object> resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
String newNotebookId = (String) resp.get("body");
LOG.info("newNotebookId:=" + newNotebookId);
Note newNote = ZeppelinServer.notebook.getNote(newNotebookId);
String newNoteId = (String) resp.get("body");
LOG.info("newNoteId:=" + newNoteId);
Note newNote = ZeppelinServer.notebook.getNote(newNoteId);
assertNotNull("Can not find new note by id", newNote);
assertEquals("Compare note names", noteName, newNote.getName());
assertEquals("Compare paragraphs count", note.getParagraphs().size(), newNote.getParagraphs().size());
@ -346,16 +344,16 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
@Test
public void testListNotebooks() throws IOException {
LOG.info("testListNotebooks");
public void testListNotes() throws IOException {
LOG.info("testListNotes");
GetMethod get = httpGet("/notebook/ ");
assertThat("List notebooks method", get, isAllowed());
assertThat("List notes method", get, isAllowed());
Map<String, Object> resp = gson.fromJson(get.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
List<Map<String, String>> body = (List<Map<String, String>>) resp.get("body");
//TODO(khalid): anonymous or specific user notes?
HashSet<String> anonymous = Sets.newHashSet("anonymous");
assertEquals("List notebooks are equal", ZeppelinServer.notebook.getAllNotes(anonymous).size(), body.size());
assertEquals("List notes are equal", ZeppelinServer.notebook.getAllNotes(anonymous).size(), body.size());
get.releaseConnection();
}
@ -374,7 +372,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setText("%md This is test paragraph.");
note.persist(anonymous);
String noteID = note.getId();
String noteId = note.getId();
note.runAll();
// wait until job is finished or timeout.
@ -387,25 +385,25 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
}
// Call Run Notebook Jobs REST API
PostMethod postNoteJobs = httpPost("/notebook/job/" + noteID, "");
assertThat("test notebook jobs run:", postNoteJobs, isAllowed());
// Call Run note jobs REST API
PostMethod postNoteJobs = httpPost("/notebook/job/" + noteId, "");
assertThat("test note jobs run:", postNoteJobs, isAllowed());
postNoteJobs.releaseConnection();
// Call Stop Notebook Jobs REST API
DeleteMethod deleteNoteJobs = httpDelete("/notebook/job/" + noteID);
assertThat("test notebook stop:", deleteNoteJobs, isAllowed());
// Call Stop note jobs REST API
DeleteMethod deleteNoteJobs = httpDelete("/notebook/job/" + noteId);
assertThat("test note stop:", deleteNoteJobs, isAllowed());
deleteNoteJobs.releaseConnection();
Thread.sleep(1000);
// Call Run paragraph REST API
PostMethod postParagraph = httpPost("/notebook/job/" + noteID + "/" + paragraph.getId(), "");
PostMethod postParagraph = httpPost("/notebook/job/" + noteId + "/" + paragraph.getId(), "");
assertThat("test paragraph run:", postParagraph, isAllowed());
postParagraph.releaseConnection();
Thread.sleep(1000);
// Call Stop paragraph REST API
DeleteMethod deleteParagraph = httpDelete("/notebook/job/" + noteID + "/" + paragraph.getId());
DeleteMethod deleteParagraph = httpDelete("/notebook/job/" + noteId + "/" + paragraph.getId());
assertThat("test paragraph stop:", deleteParagraph, isAllowed());
deleteParagraph.releaseConnection();
Thread.sleep(1000);
@ -415,8 +413,8 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
@Test
public void testGetNotebookJob() throws IOException, InterruptedException {
LOG.info("testGetNotebookJob");
public void testGetNoteJob() throws IOException, InterruptedException {
LOG.info("testGetNoteJob");
// Create note to run test.
Note note = ZeppelinServer.notebook.createNote(anonymous);
assertNotNull("can't create new note", note);
@ -430,7 +428,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setText("%sh sleep 1");
paragraph.setAuthenticationInfo(anonymous);
note.persist(anonymous);
String noteID = note.getId();
String noteId = note.getId();
note.runAll();
@ -440,12 +438,12 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
// assume that status of the paragraph is running
GetMethod get = httpGet("/notebook/job/" + noteID);
assertThat("test get notebook job: ", get, isAllowed());
GetMethod get = httpGet("/notebook/job/" + noteId);
assertThat("test get note job: ", get, isAllowed());
String responseBody = get.getResponseBodyAsString();
get.releaseConnection();
LOG.info("test get notebook job: \n" + responseBody);
LOG.info("test get note job: \n" + responseBody);
Map<String, Object> resp = gson.fromJson(responseBody, new TypeToken<Map<String, Object>>() {
}.getType());
@ -460,7 +458,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
while (!paragraph.isTerminated()) {
Thread.sleep(100);
if (timeout++ > 10) {
LOG.info("testGetNotebookJob timeout job.");
LOG.info("testGetNoteJob timeout job.");
break;
}
}
@ -483,7 +481,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
paragraph.setText("%spark\nval param = z.input(\"param\").toString\nprintln(param)");
note.persist(anonymous);
String noteID = note.getId();
String noteId = note.getId();
note.runAll();
// wait until job is finished or timeout.
@ -497,13 +495,13 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
// Call Run paragraph REST API
PostMethod postParagraph = httpPost("/notebook/job/" + noteID + "/" + paragraph.getId(),
PostMethod postParagraph = httpPost("/notebook/job/" + noteId + "/" + paragraph.getId(),
"{\"params\": {\"param\": \"hello\", \"param2\": \"world\"}}");
assertThat("test paragraph run:", postParagraph, isAllowed());
postParagraph.releaseConnection();
Thread.sleep(1000);
Note retrNote = ZeppelinServer.notebook.getNote(noteID);
Note retrNote = ZeppelinServer.notebook.getNote(noteId);
Paragraph retrParagraph = retrNote.getParagraph(paragraph.getId());
Map<String, Object> params = retrParagraph.settings.getParams();
assertEquals("hello", params.get("param"));
@ -574,7 +572,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
note.persist(anonymous);
GetMethod getNoteJobs = httpGet("/notebook/job/" + note.getId());
assertThat("test notebook jobs run:", getNoteJobs, isAllowed());
assertThat("test note jobs run:", getNoteJobs, isAllowed());
Map<String, Object> resp = gson.fromJson(getNoteJobs.getResponseBodyAsString(), new TypeToken<Map<String, Object>>() {
}.getType());
List<Map<String, String>> body = (List<Map<String, String>>) resp.get("body");
@ -722,13 +720,13 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
Note note1 = ZeppelinServer.notebook.createNote(anonymous);
String jsonRequest = "{\"title\": \"title1\", \"text\": \"ThisIsToTestSearchMethodWithPermissions 1\"}";
PostMethod postNotebookText = httpPost("/notebook/" + note1.getId() + "/paragraph", jsonRequest);
postNotebookText.releaseConnection();
PostMethod postNoteText = httpPost("/notebook/" + note1.getId() + "/paragraph", jsonRequest);
postNoteText.releaseConnection();
Note note2 = ZeppelinServer.notebook.createNote(anonymous);
jsonRequest = "{\"title\": \"title1\", \"text\": \"ThisIsToTestSearchMethodWithPermissions 2\"}";
postNotebookText = httpPost("/notebook/" + note2.getId() + "/paragraph", jsonRequest);
postNotebookText.releaseConnection();
postNoteText = httpPost("/notebook/" + note2.getId() + "/paragraph", jsonRequest);
postNoteText.releaseConnection();
String jsonPermissions = "{\"owners\":[\"" + username + "\"],\"readers\":[\"" + username + "\"],\"writers\":[\"" + username + "\"]}";
PutMethod putPermission = httpPut("/notebook/" + note1.getId() + "/permissions", jsonPermissions);
@ -738,9 +736,9 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
putPermission = httpPut("/notebook/" + note2.getId() + "/permissions", jsonPermissions);
putPermission.releaseConnection();
GetMethod searchNotebook = httpGet("/notebook/search?q='ThisIsToTestSearchMethodWithPermissions'");
searchNotebook.addRequestHeader("Origin", "http://localhost");
Map<String, Object> respSearchResult = gson.fromJson(searchNotebook.getResponseBodyAsString(),
GetMethod searchNote = httpGet("/notebook/search?q='ThisIsToTestSearchMethodWithPermissions'");
searchNote.addRequestHeader("Origin", "http://localhost");
Map<String, Object> respSearchResult = gson.fromJson(searchNote.getResponseBodyAsString(),
new TypeToken<Map<String, Object>>() {
}.getType());
ArrayList searchBody = (ArrayList) respSearchResult.get("body");
@ -766,7 +764,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
getPermission.releaseConnection();
}
searchNotebook.releaseConnection();
searchNote.releaseConnection();
ZeppelinServer.notebook.removeNote(note1.getId(), anonymous);
ZeppelinServer.notebook.removeNote(note2.getId(), anonymous);
}
@ -775,12 +773,12 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
public void testTitleSearch() throws IOException {
Note note = ZeppelinServer.notebook.createNote(anonymous);
String jsonRequest = "{\"title\": \"testTitleSearchOfParagraph\", \"text\": \"ThisIsToTestSearchMethodWithTitle \"}";
PostMethod postNotebookText = httpPost("/notebook/" + note.getId() + "/paragraph", jsonRequest);
postNotebookText.releaseConnection();
PostMethod postNoteText = httpPost("/notebook/" + note.getId() + "/paragraph", jsonRequest);
postNoteText.releaseConnection();
GetMethod searchNotebook = httpGet("/notebook/search?q='testTitleSearchOfParagraph'");
searchNotebook.addRequestHeader("Origin", "http://localhost");
Map<String, Object> respSearchResult = gson.fromJson(searchNotebook.getResponseBodyAsString(),
GetMethod searchNote = httpGet("/notebook/search?q='testTitleSearchOfParagraph'");
searchNote.addRequestHeader("Origin", "http://localhost");
Map<String, Object> respSearchResult = gson.fromJson(searchNote.getResponseBodyAsString(),
new TypeToken<Map<String, Object>>() {
}.getType());
ArrayList searchBody = (ArrayList) respSearchResult.get("body");
@ -793,7 +791,7 @@ public class ZeppelinRestApiTest extends AbstractTestRestApi {
}
}
assertEquals("Paragraph title hits must be at-least one", true, numberOfTitleHits >= 1);
searchNotebook.releaseConnection();
searchNote.releaseConnection();
ZeppelinServer.notebook.removeNote(note.getId(), anonymous);
}

2
zeppelin-server/src/test/java/org/apache/zeppelin/socket/NotebookServerTest.java
View file

@ -154,7 +154,7 @@ public class NotebookServerTest extends AbstractTestRestApi {
@Test
public void testImportNotebook() throws IOException {
String msg = "{\"op\":\"IMPORT_NOTE\",\"data\":" +
"{\"notebook\":{\"paragraphs\": [{\"text\": \"Test " +
"{\"note\":{\"paragraphs\": [{\"text\": \"Test " +
"paragraphs import\",\"config\":{},\"settings\":{}}]," +
"\"name\": \"Test Zeppelin notebook import\",\"config\": " +
"{}}}}";

5
zeppelin-web/src/app/app.js
View file

@ -70,6 +70,11 @@
templateUrl: 'app/interpreter/interpreter.html',
controller: 'InterpreterCtrl'
})
.when('/notebookRepos', {
templateUrl: 'app/notebookRepos/notebookRepos.html',
controller: 'NotebookReposCtrl',
controllerAs: 'noterepo'
})
.when('/credential', {
templateUrl: 'app/credential/credential.html',
controller: 'CredentialCtrl'

4
zeppelin-web/src/app/configuration/configuration.html
View file

@ -40,11 +40,11 @@ limitations under the License.
<th>value</th>
</tr>
</thead>
<tr ng-repeat="(key, value) in configurations" >
<tr ng-repeat="key in configurations | sortByKey" >
<td>{{key}}</td>
<td>
<div class="hiding_overflow">
{{value}}
{{configurations[key]}}
</div>
</td>
</tr>

10
zeppelin-web/src/app/home/home.controller.js
View file

@ -18,17 +18,17 @@
HomeCtrl.$inject = [
'$scope',
'notebookListDataFactory',
'noteListDataFactory',
'websocketMsgSrv',
'$rootScope',
'arrayOrderingSrv',
'ngToast'
];
function HomeCtrl($scope, notebookListDataFactory, websocketMsgSrv, $rootScope, arrayOrderingSrv, ngToast) {
function HomeCtrl($scope, noteListDataFactory, websocketMsgSrv, $rootScope, arrayOrderingSrv, ngToast) {
ngToast.dismiss();
var vm = this;
vm.notes = notebookListDataFactory;
vm.notes = noteListDataFactory;
vm.websocketMsgSrv = websocketMsgSrv;
vm.arrayOrderingSrv = arrayOrderingSrv;
@ -42,12 +42,12 @@
$scope.isReloading = false;
var initHome = function() {
websocketMsgSrv.getHomeNotebook();
websocketMsgSrv.getHomeNote();
};
initHome();
$scope.reloadNotebookList = function() {
$scope.reloadNoteList = function() {
websocketMsgSrv.reloadAllNotesFromRepo();
$scope.isReloadingNotes = true;
};

8
zeppelin-web/src/app/home/home.html
View file

@ -47,7 +47,7 @@ limitations under the License.
<h4>Notebook
<i ng-class="isReloadingNotes ? 'fa fa-refresh fa-spin' : 'fa fa-refresh'"
ng-style="!isReloadingNotes && {'cursor': 'pointer'}" style="font-size: 13px;"
ng-click="reloadNotebookList();"
ng-click="reloadNoteList();"
tooltip-placement="bottom" tooltip="Reload notes from storage">
</i>
</h4>
@ -59,15 +59,15 @@ limitations under the License.
<i style="font-size: 15px;" class="icon-notebook"></i> Create new note</a></h5>
<ul id="notebook-names">
<li class="filter-names" ng-include="'components/filterNoteNames/filter-note-names.html'"></li>
<li ng-repeat="note in home.notes.list | filter:query.q | orderBy:home.arrayOrderingSrv.notebookListOrdering track by $index">
<li ng-repeat="note in home.notes.list | filter:query.q | orderBy:home.arrayOrderingSrv.noteListOrdering track by $index">
<i style="font-size: 10px;" class="icon-doc"></i>
<a style="text-decoration: none;" href="#/notebook/{{note.id}}">{{noteName(note)}}</a>
</li>
<div ng-if="!query || query.name === ''">
<li ng-repeat="node in home.notes.root.children | orderBy:home.arrayOrderingSrv.notebookListOrdering track by $index" ng-include="'notebook_folder_renderer.html'" />
<li ng-repeat="node in home.notes.root.children | orderBy:home.arrayOrderingSrv.noteListOrdering track by $index" ng-include="'notebook_folder_renderer.html'" />
</div>
<div ng-if="query && query.name !== ''">
<li ng-repeat="note in home.notes.flatList | filter:query.q | orderBy:home.arrayOrderingSrv.notebookListOrdering track by $index">
<li ng-repeat="note in home.notes.flatList | filter:query.q | orderBy:home.arrayOrderingSrv.noteListOrdering track by $index">
<i style="font-size: 10px;" class="icon-doc"></i>
<a style="text-decoration: none;" href="#/notebook/{{note.id}}">{{noteName(note)}}</a>
</li>

10
zeppelin-web/src/app/interpreter/interpreter.html
View file

@ -19,11 +19,11 @@ limitations under the License.
Interpreters
</h3>
<div class="pull-right" style="margin-top:10px;">
<span style="cursor:pointer;margin-right:4px;"
ng-click="showRepositoryInfo = !showRepositoryInfo"
tooltip-placement="bottom" tooltip="Repository information">
<i class="fa fa-cog" ng-style="{color: showRepositoryInfo ? '#3071A9' : 'black' }"></i>
</span>
<button class="btn btn-default btn-sm"
ng-click="showRepositoryInfo = !showRepositoryInfo">
<i class="fa fa-database" ng-style="{color: showRepositoryInfo ? '#3071A9' : 'black' }"></i>
Repository
</button>
<button class="btn btn-default btn-sm"
ng-click="showAddNewSetting = !showAddNewSetting">
<i class="fa fa-plus"></i>

22
zeppelin-web/src/app/jobmanager/jobmanager.controller.js
View file

@ -34,7 +34,7 @@
$scope.jobInfomations = [];
$scope.JobInfomationsByFilter = $scope.jobInfomations;
websocketMsgSrv.getNotebookJobsList();
websocketMsgSrv.getNoteJobsList();
$scope.$on('$destroy', function() {
websocketMsgSrv.unsubscribeJobManager();
@ -45,34 +45,34 @@
** $scope.$on functions below
*/
$scope.$on('setNotebookJobs', function(event, responseData) {
$scope.$on('setNoteJobs', function(event, responseData) {
$scope.lastJobServerUnixTime = responseData.lastResponseUnixTime;
$scope.jobInfomations = responseData.jobs;
$scope.jobInfomationsIndexs = $scope.jobInfomations ? _.indexBy($scope.jobInfomations, 'notebookId') : {};
$scope.jobInfomationsIndexs = $scope.jobInfomations ? _.indexBy($scope.jobInfomations, 'noteId') : {};
});
$scope.$on('setUpdateNotebookJobs', function(event, responseData) {
$scope.$on('setUpdateNoteJobs', function(event, responseData) {
var jobInfomations = $scope.jobInfomations;
var indexStore = $scope.jobInfomationsIndexs;
$scope.lastJobServerUnixTime = responseData.lastResponseUnixTime;
var notes = responseData.jobs;
notes.map(function(changedItem) {
if (indexStore[changedItem.notebookId] === undefined) {
if (indexStore[changedItem.noteId] === undefined) {
var newItem = angular.copy(changedItem);
jobInfomations.push(newItem);
indexStore[changedItem.notebookId] = newItem;
indexStore[changedItem.noteId] = newItem;
} else {
var changeOriginTarget = indexStore[changedItem.notebookId];
var changeOriginTarget = indexStore[changedItem.noteId];
if (changedItem.isRemoved !== undefined && changedItem.isRemoved === true) {
// remove Item.
var removeIndex = _.findIndex(indexStore, changedItem.notebookId);
var removeIndex = _.findIndex(indexStore, changedItem.noteId);
if (removeIndex > -1) {
indexStore.splice(removeIndex, 1);
}
removeIndex = _.findIndex(jobInfomations, {'notebookId': changedItem.notebookId});
removeIndex = _.findIndex(jobInfomations, {'noteId': changedItem.noteId});
if (removeIndex) {
jobInfomations.splice(removeIndex, 1);
}
@ -80,8 +80,8 @@
} else {
// change value for item.
changeOriginTarget.isRunningJob = changedItem.isRunningJob;
changeOriginTarget.notebookName = changedItem.notebookName;
changeOriginTarget.notebookType = changedItem.notebookType;
changeOriginTarget.noteName = changedItem.noteName;
changeOriginTarget.noteType = changedItem.noteType;
changeOriginTarget.interpreter = changedItem.interpreter;
changeOriginTarget.unixTimeLastRun = changedItem.unixTimeLastRun;
changeOriginTarget.paragraphs = changedItem.paragraphs;

21
zeppelin-web/src/app/jobmanager/jobmanager.html
View file

@ -23,7 +23,7 @@ limitations under the License.
</div>
<div class="row">
<div class="col-md-12">
You can monitor the written notebook. Check the status of the Notebook and can control the action.
You can monitor the status of notebook and navigate to note or paragraph.
</div>
</div>
</div>
@ -41,8 +41,7 @@ limitations under the License.
<span
ng-switch-when="FINISHED">
<i style="color: green; margin-right: 3px;" class="fa fa-circle"
ng-click=""
tooltip="FINISHED">
ng-click="">
</i>
{{jobStatus}}
</span>
@ -50,8 +49,7 @@ limitations under the License.
ng-switch-when="RUNNING"
style="margin-right: 3px;">
<i style="color: blue" class="fa fa-spinner"
ng-click=""
tooltip="RUNNING">
ng-click="">
</i>
{{jobStatus}}
</span>
@ -59,8 +57,7 @@ limitations under the License.
ng-switch-when="READY"
style="margin-right: 3px;">
<i style="color: green" class="fa fa-circle-o"
ng-click=""
tooltip="READY">
ng-click="">
</i>
{{jobStatus}}
</span>
@ -68,9 +65,7 @@ limitations under the License.
ng-switch-when="PENDING"
style="margin-right: 3px;">
<i style="color: gray" class="fa fa-circle"
ng-click=""
tooltip="PENDING"
>
ng-click="">
</i>
{{jobStatus}}
</span>
@ -78,8 +73,7 @@ limitations under the License.
ng-switch-when="ABORT"
style="margin-right: 3px;">
<i style="color: orange" class="fa fa-circle"
ng-click=""
tooltip="ABORT">
ng-click="">
</i>
{{jobStatus}}
</span>
@ -87,8 +81,7 @@ limitations under the License.
ng-switch-when="ERROR"
style="margin-right: 3px;">
<i style="color: red" class="fa fa-circle"
ng-click=""
tooltip="ERROR">
ng-click="">
</i>
{{jobStatus}}
</span>

6
zeppelin-web/src/app/jobmanager/jobs/job-control.html
View file

@ -12,16 +12,16 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="{{notebookJob.notebookId}}_control" class="control">
<div id="{{notebookJob.noteId}}_control" class="control">
<span>
{{lastExecuteTime(notebookJob.unixTimeLastRun)}}
</span>
<span>
<span ng-if="notebookJob.isRunningJob === true">
Notebook is RUNNING
Note is RUNNING
</span>
<span ng-if="notebookJob.isRunningJob === false">
Notebook is READY
Note is READY
</span>
</span>

4
zeppelin-web/src/app/jobmanager/jobs/job-progressBar.html
View file

@ -12,8 +12,8 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
<div id="{{notebookJob.notebookId}}_runControl" class="runControl">
<div id="{{notebookJob.notebookId}}_progress" class="progress" ng-if="notebookJob.isRunningJob === true">
<div id="{{notebookJob.noteId}}_runControl" class="runControl">
<div id="{{notebookJob.noteId}}_progress" class="progress" ng-if="notebookJob.isRunningJob === true">
<div ng-if="getProgress()>0 && getProgress()<100 && notebookJob.isRunningJob === true"
class="progress-bar" role="progressbar" ng-style="{width:getProgress()+'%'}"></div>
<div ng-if="(getProgress()<=0 || getProgress()>=100) && (notebookJob.isRunningJob === true)"

20
zeppelin-web/src/app/jobmanager/jobs/job.html
View file

@ -17,15 +17,15 @@ limitations under the License.
<div ng-include src="'app/jobmanager/jobs/job-control.html'"></div>
<span
class="job-types"
ng-switch="notebookJob.notebookType">
ng-switch="notebookJob.noteType">
<i ng-switch-when="normal" class="icon-doc"></i>
<i ng-switch-when="cron" class="icon-clock"></i>
<i ng-switch-default class="icon-question"></i>
</span>
&nbsp;
<a style="text-decoration: none !important;" ng-href="#/notebook/{{notebookJob.notebookId}}">
<a style="text-decoration: none !important;" ng-href="#/notebook/{{notebookJob.noteId}}">
<span>
{{notebookJob.notebookName}}
{{notebookJob.noteName}}
</span>
<span>
&nbsp;-&nbsp;
@ -48,7 +48,7 @@ limitations under the License.
ng-switch="paragraphJob.status">
<a ng-switch-when="READY"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i style="color: green" class="fa fa-circle-o"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is READY">
@ -56,7 +56,7 @@ limitations under the License.
</a>
<a ng-switch-when="FINISHED"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i style="color: green" class="fa fa-circle"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is FINISHED">
@ -64,7 +64,7 @@ limitations under the License.
</a>
<a ng-switch-when="ABORT"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i style="color: orange" class="fa fa-circle"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is ABORT">
@ -72,7 +72,7 @@ limitations under the License.
</a>
<a ng-switch-when="ERROR"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i style="color: red" class="fa fa-circle"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is ERROR">
@ -80,7 +80,7 @@ limitations under the License.
</a>
<a ng-switch-when="PENDING"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i style="color: gray" class="fa fa-circle"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is PENDING">
@ -88,7 +88,7 @@ limitations under the License.
</a>
<a ng-switch-when="RUNNING"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i style="color: blue" class="fa fa-spinner spinAnimation"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is RUNNING">
@ -96,7 +96,7 @@ limitations under the License.
</a>
<a ng-switch-default class="icon-question"
style="text-decoration: none !important;"
ng-href="#/notebook/{{notebookJob.notebookId}}/paragraph/{{paragraphJob.id}}">
ng-href="#/notebook/{{notebookJob.noteId}}/paragraph/{{paragraphJob.id}}">
<i class="icon-question"
tooltip-placement="top-left"
tooltip="{{paragraphJob.name}} is {{paragraphJob.status}}">

12
zeppelin-web/src/app/notebook/notebook-actionBar.html
View file

@ -53,7 +53,7 @@ limitations under the License.
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Clone the notebook" data-source-note-name="{{note.name}}"
tooltip-placement="bottom" tooltip="Clone this note" data-source-note-name="{{note.name}}"
data-toggle="modal" data-target="#noteNameModal" data-clone="true"
>
<i class="fa fa-copy"></i>
@ -61,8 +61,8 @@ limitations under the License.
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
ng-click="exportNotebook()"
tooltip-placement="bottom" tooltip="Export the notebook">
ng-click="exportNote()"
tooltip-placement="bottom" tooltip="Export this note">
<i class="fa fa-download"></i>
</button>
</span>
@ -91,9 +91,9 @@ limitations under the License.
<button type="button"
class="btn btn-default btn-xs"
ng-hide="viewOnly"
ng-click="checkpointNotebook(note.checkpoint.message)"
ng-click="checkpointNote(note.checkpoint.message)"
style="margin-left: 4px;"
tooltip-placement="bottom" tooltip="Commit the notebook">Commit
tooltip-placement="bottom" tooltip="Commit this note">Commit
</button>
</div>
</div>
@ -130,7 +130,7 @@ limitations under the License.
class="btn btn-default btn-xs"
ng-click="removeNote(note.id)"
ng-hide="viewOnly"
tooltip-placement="bottom" tooltip="Remove the notebook">
tooltip-placement="bottom" tooltip="Remove this note">
<i class="icon-trash"></i>
</button>
</span>

22
zeppelin-web/src/app/notebook/notebook.controller.js
View file

@ -86,7 +86,7 @@
/** Init the new controller */
var initNotebook = function() {
websocketMsgSrv.getNotebook($routeParams.noteId);
websocketMsgSrv.getNote($routeParams.noteId);
websocketMsgSrv.listRevisionHistory($routeParams.noteId);
var currentRoute = $route.current;
if (currentRoute) {
@ -145,10 +145,10 @@
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this notebook?',
message: 'Do you want to delete this note?',
callback: function(result) {
if (result) {
websocketMsgSrv.deleteNotebook(noteId);
websocketMsgSrv.deleteNote(noteId);
$location.path('/');
}
}
@ -156,7 +156,7 @@
};
//Export notebook
$scope.exportNotebook = function() {
$scope.exportNote = function() {
var jsonContent = JSON.stringify($scope.note);
saveAsService.saveAs(jsonContent, $scope.note.name, 'json');
};
@ -166,10 +166,10 @@
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to clone this notebook?',
message: 'Do you want to clone this note?',
callback: function(result) {
if (result) {
websocketMsgSrv.cloneNotebook(noteId);
websocketMsgSrv.cloneNote(noteId);
$location.path('/');
}
}
@ -177,14 +177,14 @@
};
// checkpoint/commit notebook
$scope.checkpointNotebook = function(commitMessage) {
$scope.checkpointNote = function(commitMessage) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Commit notebook to current repository?',
message: 'Commit note to current repository?',
callback: function(result) {
if (result) {
websocketMsgSrv.checkpointNotebook($routeParams.noteId, commitMessage);
websocketMsgSrv.checkpointNote($routeParams.noteId, commitMessage);
}
}
});
@ -336,13 +336,13 @@
if (config) {
$scope.note.config = config;
}
websocketMsgSrv.updateNotebook($scope.note.id, $scope.note.name, $scope.note.config);
websocketMsgSrv.updateNote($scope.note.id, $scope.note.name, $scope.note.config);
};
/** Update the note name */
$scope.sendNewName = function() {
if ($scope.note.name) {
websocketMsgSrv.updateNotebook($scope.note.id, $scope.note.name, $scope.note.config);
websocketMsgSrv.updateNote($scope.note.id, $scope.note.name, $scope.note.config);
}
};

91
zeppelin-web/src/app/notebookRepos/notebookRepos.controller.js Normal file
View file

@ -0,0 +1,91 @@
/*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
'use strict';
(function() {
angular.module('zeppelinWebApp').controller('NotebookReposCtrl', NotebookReposCtrl);
NotebookReposCtrl.$inject = ['$http', 'baseUrlSrv', 'ngToast'];
function NotebookReposCtrl($http, baseUrlSrv, ngToast) {
var vm = this;
vm.notebookRepos = [];
vm.showDropdownSelected = showDropdownSelected;
vm.saveNotebookRepo = saveNotebookRepo;
_init();
// Public functions
function saveNotebookRepo(valueform, repo, data) {
console.log('data %o', data);
$http.put(baseUrlSrv.getRestApiBase() + '/notebook-repositories', {
'name': repo.className,
'settings': data
}).success(function(data) {
var index = _.findIndex(vm.notebookRepos, {'className': repo.className});
if (index >= 0) {
vm.notebookRepos[index] = data.body;
console.log('repos %o, data %o', vm.notebookRepos, data.body);
}
valueform.$show();
}).error(function() {
ngToast.danger({
content: 'We couldn\'t save that NotebookRepo\'s settings',
verticalPosition: 'bottom',
timeout: '3000'
});
valueform.$show();
});
return 'manual';
}
function showDropdownSelected(setting) {
var index = _.findIndex(setting.value, {'value': setting.selected});
if (index < 0) {
return 'No value';
} else {
return setting.value[index].name;
}
}
// Private functions
function _getInterpreterSettings() {
$http.get(baseUrlSrv.getRestApiBase() + '/notebook-repositories')
.success(function(data, status, headers, config) {
vm.notebookRepos = data.body;
console.log('ya notebookRepos %o', vm.notebookRepos);
}).error(function(data, status, headers, config) {
if (status === 401) {
ngToast.danger({
content: 'You don\'t have permission on this page',
verticalPosition: 'bottom',
timeout: '3000'
});
setTimeout(function() {
window.location.replace('/');
}, 3000);
}
console.log('Error %o %o', status, data.message);
});
}
function _init() {
_getInterpreterSettings();
};
}
})();

98
zeppelin-web/src/app/notebookRepos/notebookRepos.html Normal file
View file

@ -0,0 +1,98 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<div class="interpreterHead">
<div class="header">
<div class="row">
<div class="col-md-12">
<h3 class="new_h3">
Notebook Repos
</h3>
</div>
</div>
<div class="row">
<div class="col-md-12">
Manage your Notebook Repositories' settings.
</div>
</div>
</div>
</div>
<div class="box width-full"
ng-repeat="repo in noterepo.notebookRepos | orderBy: 'name'">
<div id="{{repo.name | lowercase}}">
<div class="row interpreter">
<div class="col-md-12">
<h3 class="interpreter-title">{{repo.name}}</h3>
<span style="float:right" ng-show="repo.settings.length > 0">
<button class="btn btn-default btn-xs"
ng-click="valueform.$show();">
<span class="fa fa-pencil"></span> edit</button>
</span>
</div>
</div>
<div class="row interpreter">
<div class="col-md-12" ng-show="repo.settings.length > 0">
<h5>Settings</h5>
<table class="table table-striped">
<thead>
<tr>
<th style="width:40%">name</th>
<th style="width:60%">value</th>
</tr>
</thead>
<tr ng-repeat="setting in repo.settings" >
<td ng-bind="setting.name"></td>
<td>
<span class="btn-group">
<span ng-show="setting.type === 'DROPDOWN'">
<span editable-select="setting.selected"
e-name="{{setting.name}}"
e-ng-options="s.value as s.name for s in setting.value"
class="selectpicker" ng-disabled="!valueform.$visible" e-form="valueform">
{{noterepo.showDropdownSelected(setting)}}
</span>
</span>
<span ng-show="setting.type === 'INPUT'">
<span editable-textarea="setting.selected" e-name="{{setting.name}}" e-form="valueform" e-msd-elastic e-cols="100">
{{setting.selected | breakFilter}}
</span>
</span>
</span>
</td>
</tr>
</table>
</div>
</div>
<span style="float:right" ng-show="valueform.$visible">
<form editable-form name="valueform"
onbeforesave="noterepo.saveNotebookRepo(valueform, repo, $data)"
ng-show="valueform.$visible">
<button type="submit" class="btn btn-primary btn-xs">
<span class="fa fa-check"></span> Save
</button>
<button type="button" class="btn btn-default btn-xs"
ng-disabled="valueform.$waiting"
ng-click="valueform.$cancel();">
<span class="fa fa-remove"></span> Cancel
</button>
</form>
</span>
<div class="row interpreter">
<div ng-show="repo.settings.length === 0 || valueform.$hidden" class="col-md-12 gray40-message">
<em>Currently there are no settings for this Notebook Repository</em>
</div>
</div>
</div>
</div>

2
zeppelin-web/src/components/arrayOrderingSrv/arrayOrdering.service.js
View file

@ -19,7 +19,7 @@
function arrayOrderingSrv() {
var arrayOrderingSrv = this;
this.notebookListOrdering = function(note) {
this.noteListOrdering = function(note) {
return arrayOrderingSrv.getNoteName(note);
};

4
zeppelin-web/src/components/navbar/navbar-notebookList-elem.html → zeppelin-web/src/components/navbar/navbar-noteList-elem.html
View file

@ -26,9 +26,9 @@ limitations under the License.
</div>
<div class="expandable" style="color: black;">
<ul>
<li ng-repeat="note in note.children | orderBy:navbar.arrayOrderingSrv.notebookListOrdering track by $index"
<li ng-repeat="note in note.children | orderBy:navbar.arrayOrderingSrv.noteListOrdering track by $index"
ng-class="{'active' : navbar.isActive(note.id)}"
ng-include="'components/navbar/navbar-notebookList-elem.html'">
ng-include="'components/navbar/navbar-noteList-elem.html'">
</li>
</ul>
</div>

16
zeppelin-web/src/components/navbar/navbar.controller.js
View file

@ -22,7 +22,7 @@
'$http',
'$routeParams',
'$location',
'notebookListDataFactory',
'noteListDataFactory',
'baseUrlSrv',
'websocketMsgSrv',
'arrayOrderingSrv',
@ -30,14 +30,14 @@
];
function NavCtrl($scope, $rootScope, $http, $routeParams, $location,
notebookListDataFactory, baseUrlSrv, websocketMsgSrv,
noteListDataFactory, baseUrlSrv, websocketMsgSrv,
arrayOrderingSrv, searchService) {
var vm = this;
vm.arrayOrderingSrv = arrayOrderingSrv;
vm.connected = websocketMsgSrv.isConnected();
vm.isActive = isActive;
vm.logout = logout;
vm.notes = notebookListDataFactory;
vm.notes = noteListDataFactory;
vm.search = search;
vm.searchForm = searchService;
vm.showLoginWindow = showLoginWindow;
@ -57,7 +57,7 @@
}
function initController() {
$scope.isDrawNavbarNotebookList = false;
$scope.isDrawNavbarNoteList = false;
angular.element('#notebook-list').perfectScrollbar({suppressScrollX: true});
angular.element(document).click(function() {
@ -73,7 +73,7 @@
}
function loadNotes() {
websocketMsgSrv.getNotebookList();
websocketMsgSrv.getNoteList();
}
function logout() {
@ -113,7 +113,7 @@
*/
$scope.$on('setNoteMenu', function(event, notes) {
notebookListDataFactory.setNotes(notes);
noteListDataFactory.setNotes(notes);
});
$scope.$on('setConnectedStatus', function(event, param) {
@ -129,11 +129,11 @@
*/
angular.element(document).ready(function() {
angular.element('.notebook-list-dropdown').on('show.bs.dropdown', function() {
$scope.isDrawNavbarNotebookList = true;
$scope.isDrawNavbarNoteList = true;
});
angular.element('.notebook-list-dropdown').on('hide.bs.dropdown', function() {
$scope.isDrawNavbarNotebookList = false;
$scope.isDrawNavbarNoteList = false;
});
});
}

9
zeppelin-web/src/components/navbar/navbar.html
View file

@ -30,10 +30,10 @@ limitations under the License.
<ul class="dropdown-menu navbar-dropdown-maxHeight" role="menu">
<li><a href="" data-toggle="modal" data-target="#noteNameModal"><i class="fa fa-plus"></i> Create new note</a></li>
<li class="divider"></li>
<div id="notebook-list" class="scrollbar-container" ng-if="isDrawNavbarNotebookList">
<div id="notebook-list" class="scrollbar-container" ng-if="isDrawNavbarNoteList">
<li class="filter-names" ng-include="'components/filterNoteNames/filter-note-names.html'"></li>
<li ng-repeat="note in navbar.notes.root.children | filter:query.q | orderBy:navbar.arrayOrderingSrv.notebookListOrdering track by $index"
ng-class="{'active' : navbar.isActive(note.id)}" ng-include="'components/navbar/navbar-notebookList-elem.html'">
<li ng-repeat="note in navbar.notes.root.children | filter:query.q | orderBy:navbar.arrayOrderingSrv.noteListOrdering track by $index"
ng-class="{'active' : navbar.isActive(note.id)}" ng-include="'components/navbar/navbar-noteList-elem.html'">
</li>
</div>
</ul>
@ -57,7 +57,7 @@ limitations under the License.
id="searchTermId"
ng-disabled="!navbar.connected"
class="form-control"
placeholder="Search your Notebooks"
placeholder="Search your Notes"
/>
<span class="input-group-btn">
<button
@ -86,6 +86,7 @@ limitations under the License.
<li><a href="" data-toggle="modal" data-target="#aboutModal">About Zeppelin</a></li>
<li role="separator" style="margin: 5px 0;" class="divider"></li>
<li><a href="#/interpreter">Interpreter</a></li>
<li><a href="#/notebookRepos">Notebook Repos</a></li>
<li><a href="#/credential">Credential</a></li>
<li><a href="#/configuration">Configuration</a></li>
<li ng-if="ticket.principal && ticket.principal !== 'anonymous'" role="separator" style="margin: 5px 0;" class="divider"></li>

4
zeppelin-web/src/components/notebookListDataFactory/notebookList.datafactory.js → zeppelin-web/src/components/noteListDataFactory/noteList.datafactory.js
View file

@ -14,9 +14,9 @@
'use strict';
(function() {
angular.module('zeppelinWebApp').factory('notebookListDataFactory', notebookListDataFactory);
angular.module('zeppelinWebApp').factory('noteListDataFactory', noteListDataFactory);
function notebookListDataFactory() {
function noteListDataFactory() {
var notes = {
root: {children: []},
flatList: [],

10
zeppelin-web/src/components/noteName-create/notename.controller.js
View file

@ -18,24 +18,24 @@
NotenameCtrl.$inject = [
'$scope',
'notebookListDataFactory',
'noteListDataFactory',
'$routeParams',
'websocketMsgSrv'
];
function NotenameCtrl($scope, notebookListDataFactory, $routeParams, websocketMsgSrv) {
function NotenameCtrl($scope, noteListDataFactory, $routeParams, websocketMsgSrv) {
var vm = this;
vm.clone = false;
vm.notes = notebookListDataFactory;
vm.notes = noteListDataFactory;
vm.websocketMsgSrv = websocketMsgSrv;
$scope.note = {};
vm.createNote = function() {
if (!vm.clone) {
vm.websocketMsgSrv.createNotebook($scope.note.notename);
vm.websocketMsgSrv.createNote($scope.note.notename);
} else {
var noteId = $routeParams.noteId;
vm.websocketMsgSrv.cloneNotebook(noteId, $scope.note.notename);
vm.websocketMsgSrv.cloneNote(noteId, $scope.note.notename);
}
};

2
zeppelin-web/src/components/noteName-import/notenameImport.controller.js
View file

@ -112,7 +112,7 @@
} else {
result.name = $scope.note.noteImportName;
}
websocketMsgSrv.importNotebook(result);
websocketMsgSrv.importNote(result);
//angular.element('#noteImportModal').modal('hide');
} else {
$scope.note.errorText = 'Invalid JSON';

8
zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
View file

@ -64,10 +64,10 @@
$location.path('/notebook/' + data.note.id);
} else if (op === 'NOTES_INFO') {
$rootScope.$broadcast('setNoteMenu', data.notes);
} else if (op === 'LIST_NOTEBOOK_JOBS') {
$rootScope.$broadcast('setNotebookJobs', data.notebookJobs);
} else if (op === 'LIST_UPDATE_NOTEBOOK_JOBS') {
$rootScope.$broadcast('setUpdateNotebookJobs', data.notebookRunningJobs);
} else if (op === 'LIST_NOTE_JOBS') {
$rootScope.$broadcast('setNoteJobs', data.noteJobs);
} else if (op === 'LIST_UPDATE_NOTE_JOBS') {
$rootScope.$broadcast('setUpdateNoteJobs', data.noteRunningJobs);
} else if (op === 'AUTH_INFO') {
BootstrapDialog.show({
closable: false,

40
zeppelin-web/src/components/websocketEvents/websocketMsg.service.js
View file

@ -21,23 +21,23 @@
function websocketMsgSrv($rootScope, websocketEvents) {
return {
getHomeNotebook: function() {
getHomeNote: function() {
websocketEvents.sendNewEvent({op: 'GET_HOME_NOTE'});
},
createNotebook: function(noteName) {
createNote: function(noteName) {
websocketEvents.sendNewEvent({op: 'NEW_NOTE',data: {name: noteName}});
},
deleteNotebook: function(noteId) {
deleteNote: function(noteId) {
websocketEvents.sendNewEvent({op: 'DEL_NOTE', data: {id: noteId}});
},
cloneNotebook: function(noteIdToClone, newNoteName) {
cloneNote: function(noteIdToClone, newNoteName) {
websocketEvents.sendNewEvent({op: 'CLONE_NOTE', data: {id: noteIdToClone, name: newNoteName}});
},
getNotebookList: function() {
getNoteList: function() {
websocketEvents.sendNewEvent({op: 'LIST_NOTES'});
},
@ -45,11 +45,11 @@
websocketEvents.sendNewEvent({op: 'RELOAD_NOTES_FROM_REPO'});
},
getNotebook: function(noteId) {
getNote: function(noteId) {
websocketEvents.sendNewEvent({op: 'GET_NOTE', data: {id: noteId}});
},
updateNotebook: function(noteId, noteName, noteConfig) {
updateNote: function(noteId, noteName, noteConfig) {
websocketEvents.sendNewEvent({op: 'NOTE_UPDATE', data: {id: noteId, name: noteName, config: noteConfig}});
},
@ -146,18 +146,18 @@
});
},
importNotebook: function(notebook) {
importNote: function(note) {
websocketEvents.sendNewEvent({
op: 'IMPORT_NOTE',
data: {
notebook: notebook
note: note
}
});
},
checkpointNotebook: function(noteId, commitMessage) {
checkpointNote: function(noteId, commitMessage) {
websocketEvents.sendNewEvent({
op: 'CHECKPOINT_NOTEBOOK',
op: 'CHECKPOINT_NOTE',
data: {
noteId: noteId,
commitMessage: commitMessage
@ -198,27 +198,27 @@
return websocketEvents.isConnected();
},
getNotebookJobsList: function() {
websocketEvents.sendNewEvent({op: 'LIST_NOTEBOOK_JOBS'});
getNoteJobsList: function() {
websocketEvents.sendNewEvent({op: 'LIST_NOTE_JOBS'});
},
getUpdateNotebookJobsList: function(lastUpdateServerUnixTime) {
getUpdateNoteJobsList: function(lastUpdateServerUnixTime) {
websocketEvents.sendNewEvent(
{op: 'LIST_UPDATE_NOTEBOOK_JOBS', data: {lastUpdateUnixTime: lastUpdateServerUnixTime * 1}}
{op: 'LIST_UPDATE_NOTE_JOBS', data: {lastUpdateUnixTime: lastUpdateServerUnixTime * 1}}
);
},
unsubscribeJobManager: function() {
websocketEvents.sendNewEvent({op: 'UNSUBSCRIBE_UPDATE_NOTEBOOK_JOBS'});
websocketEvents.sendNewEvent({op: 'UNSUBSCRIBE_UPDATE_NOTE_JOBS'});
},
getInterpreterBindings: function(noteID) {
websocketEvents.sendNewEvent({op: 'GET_INTERPRETER_BINDINGS', data: {noteID: noteID}});
getInterpreterBindings: function(noteId) {
websocketEvents.sendNewEvent({op: 'GET_INTERPRETER_BINDINGS', data: {noteId: noteId}});
},
saveInterpreterBindings: function(noteID, selectedSettingIds) {
saveInterpreterBindings: function(noteId, selectedSettingIds) {
websocketEvents.sendNewEvent({op: 'SAVE_INTERPRETER_BINDINGS',
data: {noteID: noteID, selectedSettingIds: selectedSettingIds}});
data: {noteId: noteId, selectedSettingIds: selectedSettingIds}});
},
listConfigurations: function() {

3
zeppelin-web/src/index.html
View file

@ -162,6 +162,7 @@ limitations under the License.
<script src="app/configuration/configuration.controller.js"></script>
<script src="app/notebook/paragraph/paragraph.controller.js"></script>
<script src="app/search/result-list.controller.js"></script>
<script src="app/notebookRepos/notebookRepos.controller.js"></script>
<script src="components/arrayOrderingSrv/arrayOrdering.service.js"></script>
<script src="components/navbar/navbar.controller.js"></script>
<script src="components/ngescape/ngescape.directive.js"></script>
@ -177,7 +178,7 @@ limitations under the License.
<script src="components/noteName-create/visible.directive.js"></script>
<script src="components/websocketEvents/websocketMsg.service.js"></script>
<script src="components/websocketEvents/websocketEvents.factory.js"></script>
<script src="components/notebookListDataFactory/notebookList.datafactory.js"></script>
<script src="components/noteListDataFactory/noteList.datafactory.js"></script>
<script src="components/baseUrl/baseUrl.service.js"></script>
<script src="components/browser-detect/browserDetect.service.js"></script>
<script src="components/saveAs/saveAs.service.js"></script>

4
zeppelin-web/test/spec/controllers/notebook.js
View file

@ -6,7 +6,7 @@ describe('Controller: NotebookCtrl', function() {
var scope;
var websocketMsgSrvMock = {
getNotebook: function() {},
getNote: function() {},
listRevisionHistory: function() {},
getInterpreterBindings: function() {}
};
@ -19,7 +19,7 @@ describe('Controller: NotebookCtrl', function() {
var noteMock = {
id: 1,
name: 'my notebook',
name: 'my note',
config: {},
};

8
zeppelin-web/test/spec/controllers/notename.js
View file

@ -5,14 +5,14 @@ describe('Controller: NotenameCtrl', function() {
var scope;
var ctrl;
var notebookList;
var noteList;
beforeEach(inject(function($injector, $rootScope, $controller) {
notebookList = $injector.get('notebookListDataFactory');
noteList = $injector.get('noteListDataFactory');
scope = $rootScope.$new();
ctrl = $controller('NotenameCtrl', {
$scope: scope,
notebookListDataFactory: notebookList
noteListDataFactory: noteList
});
}));
@ -25,7 +25,7 @@ describe('Controller: NotenameCtrl', function() {
{name: 'Untitled Note 6', id: '4'}
];
notebookList.setNotes(notesList);
noteList.setNotes(notesList);
ctrl.sourceNoteName = 'test name';
expect(ctrl.cloneNoteName()).toEqual('test name 1');

14
zeppelin-web/test/spec/factory/notebookList.js → zeppelin-web/test/spec/factory/noteList.js
View file

@ -1,14 +1,14 @@
'use strict';
describe('Factory: NotebookList', function() {
describe('Factory: NoteList', function() {
var notebookList;
var noteList;
beforeEach(function() {
module('zeppelinWebApp');
inject(function($injector) {
notebookList = $injector.get('notebookListDataFactory');
noteList = $injector.get('noteListDataFactory');
});
});
@ -16,7 +16,7 @@ describe('Factory: NotebookList', function() {
var notesList = [
{name: 'A', id: '000001'},
{name: 'B', id: '000002'},
{id: '000003'}, // notebook without name
{id: '000003'}, // note without name
{name: '/C/CA', id: '000004'},
{name: '/C/CB', id: '000005'},
{name: '/C/CB/CBA', id: '000006'}, // same name with a dir
@ -24,9 +24,9 @@ describe('Factory: NotebookList', function() {
{name: 'C///CB//CBB', id: '000008'},
{name: 'D/D[A/DA]B', id: '000009'} // check if '[' and ']' considered as folder seperator
];
notebookList.setNotes(notesList);
noteList.setNotes(notesList);
var flatList = notebookList.flatList;
var flatList = noteList.flatList;
expect(flatList.length).toBe(9);
expect(flatList[0].name).toBe('A');
expect(flatList[0].id).toBe('000001');
@ -39,7 +39,7 @@ describe('Factory: NotebookList', function() {
expect(flatList[7].name).toBe('C///CB//CBB');
expect(flatList[8].name).toBe('D/D[A/DA]B');
var folderList = notebookList.root.children;
var folderList = noteList.root.children;
expect(folderList.length).toBe(5);
expect(folderList[0].name).toBe('A');
expect(folderList[0].id).toBe('000001');

127
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
View file

@ -89,7 +89,7 @@ public class Notebook implements NoteEventListener {
private org.quartz.Scheduler quartzSched;
private JobListenerFactory jobListenerFactory;
private NotebookRepo notebookRepo;
private SearchService notebookIndex;
private SearchService noteSearchService;
private NotebookAuthorization notebookAuthorization;
private final List<NotebookEventListener> notebookEventListeners =
Collections.synchronizedList(new LinkedList<NotebookEventListener>());
@ -98,13 +98,13 @@ public class Notebook implements NoteEventListener {
/**
* Main constructor \w manual Dependency Injection
*
* @param notebookIndex - (nullable) for indexing all notebooks on creating.
* @param noteSearchService - (nullable) for indexing all notebooks on creating.
* @throws IOException
* @throws SchedulerException
*/
public Notebook(ZeppelinConfiguration conf, NotebookRepo notebookRepo,
SchedulerFactory schedulerFactory, InterpreterFactory replFactory,
JobListenerFactory jobListenerFactory, SearchService notebookIndex,
JobListenerFactory jobListenerFactory, SearchService noteSearchService,
NotebookAuthorization notebookAuthorization, Credentials credentials)
throws IOException, SchedulerException {
this.conf = conf;
@ -112,7 +112,7 @@ public class Notebook implements NoteEventListener {
this.schedulerFactory = schedulerFactory;
this.replFactory = replFactory;
this.jobListenerFactory = jobListenerFactory;
this.notebookIndex = notebookIndex;
this.noteSearchService = noteSearchService;
this.notebookAuthorization = notebookAuthorization;
this.credentials = credentials;
quertzSchedFact = new org.quartz.impl.StdSchedulerFactory();
@ -121,10 +121,10 @@ public class Notebook implements NoteEventListener {
CronJob.notebook = this;
loadAllNotes();
if (this.notebookIndex != null) {
if (this.noteSearchService != null) {
long start = System.nanoTime();
logger.info("Notebook indexing started...");
notebookIndex.addIndexDocs(notes.values());
noteSearchService.addIndexDocs(notes.values());
logger.info("Notebook indexing finished: {} indexed in {}s", notes.size(),
TimeUnit.NANOSECONDS.toSeconds(start - System.nanoTime()));
}
@ -144,7 +144,7 @@ public class Notebook implements NoteEventListener {
} else {
note = createNote(null, subject);
}
notebookIndex.addIndexDoc(note);
noteSearchService.addIndexDoc(note);
return note;
}
@ -156,7 +156,8 @@ public class Notebook implements NoteEventListener {
public Note createNote(List<String> interpreterIds, AuthenticationInfo subject)
throws IOException {
Note note =
new Note(notebookRepo, replFactory, jobListenerFactory, notebookIndex, credentials, this);
new Note(notebookRepo, replFactory, jobListenerFactory,
noteSearchService, credentials, this);
synchronized (notes) {
notes.put(note.getId(), note);
}
@ -169,7 +170,7 @@ public class Notebook implements NoteEventListener {
owners.add(subject.getUser());
notebookAuthorization.setOwners(note.getId(), owners);
}
notebookIndex.addIndexDoc(note);
noteSearchService.addIndexDoc(note);
note.persist(subject);
fireNoteCreateEvent(note);
return note;
@ -198,7 +199,7 @@ public class Notebook implements NoteEventListener {
*
* @param sourceJson - the note JSON to import
* @param noteName - the name of the new note
* @return notebook ID
* @return note ID
* @throws IOException
*/
public Note importNote(String sourceJson, String noteName, AuthenticationInfo subject)
@ -235,17 +236,17 @@ public class Notebook implements NoteEventListener {
/**
* Clone existing note.
*
* @param sourceNoteID - the note ID to clone
* @param sourceNoteId - the note ID to clone
* @param newNoteName - the name of the new note
* @return noteId
* @throws IOException, CloneNotSupportedException, IllegalArgumentException
*/
public Note cloneNote(String sourceNoteID, String newNoteName, AuthenticationInfo subject)
public Note cloneNote(String sourceNoteId, String newNoteName, AuthenticationInfo subject)
throws IOException, CloneNotSupportedException, IllegalArgumentException {
Note sourceNote = getNote(sourceNoteID);
Note sourceNote = getNote(sourceNoteId);
if (sourceNote == null) {
throw new IllegalArgumentException(sourceNoteID + "not found");
throw new IllegalArgumentException(sourceNoteId + "not found");
}
Note newNote = createNote(subject);
if (newNoteName != null) {
@ -262,7 +263,7 @@ public class Notebook implements NoteEventListener {
newNote.addCloneParagraph(p);
}
notebookIndex.addIndexDoc(newNote);
noteSearchService.addIndexDoc(newNote);
newNote.persist(subject);
return newNote;
}
@ -317,7 +318,7 @@ public class Notebook implements NoteEventListener {
note = notes.remove(id);
}
replFactory.removeNoteInterpreterSettingBinding(subject.getUser(), id);
notebookIndex.deleteIndexDocs(note);
noteSearchService.deleteIndexDocs(note);
notebookAuthorization.removeNote(id);
// remove from all interpreter instance's angular object registry
@ -338,7 +339,7 @@ public class Notebook implements NoteEventListener {
}
}
}
// remove notebook scope object
// remove note scope object
((RemoteAngularObjectRegistry) registry).removeAllAndNotifyRemoteProcess(id, null);
} else {
// remove paragraph scope object
@ -353,7 +354,7 @@ public class Notebook implements NoteEventListener {
}
}
}
// remove notebook scope object
// remove note scope object
registry.removeAll(id, null);
}
}
@ -397,7 +398,7 @@ public class Notebook implements NoteEventListener {
}
//Manually inject ALL dependencies, as DI constructor was NOT used
note.setIndex(this.notebookIndex);
note.setIndex(this.noteSearchService);
note.setCredentials(this.credentials);
note.setInterpreterFactory(replFactory);
@ -471,7 +472,7 @@ public class Notebook implements NoteEventListener {
/**
* Reload all notes from repository after clearing `notes`
* to reflect the changes of added/deleted/modified notebooks on file system level.
* to reflect the changes of added/deleted/modified notes on file system level.
*
* @throws IOException
*/
@ -618,23 +619,23 @@ public class Notebook implements NoteEventListener {
return lastRunningUnixTime;
}
public List<Map<String, Object>> getJobListByParagraphId(String paragraphID) {
public List<Map<String, Object>> getJobListByParagraphId(String paragraphId) {
String gotNoteId = null;
List<Note> notes = getAllNotes();
for (Note note : notes) {
Paragraph p = note.getParagraph(paragraphID);
Paragraph p = note.getParagraph(paragraphId);
if (p != null) {
gotNoteId = note.getId();
}
}
return getJobListBymNotebookId(gotNoteId);
return getJobListByNoteId(gotNoteId);
}
public List<Map<String, Object>> getJobListBymNotebookId(String notebookID) {
final String CRON_TYPE_NOTEBOOK_KEYWORD = "cron";
public List<Map<String, Object>> getJobListByNoteId(String noteId) {
final String CRON_TYPE_NOTE_KEYWORD = "cron";
long lastRunningUnixTime = 0;
boolean isNotebookRunning = false;
Note jobNote = getNote(notebookID);
boolean isNoteRunning = false;
Note jobNote = getNote(noteId);
List<Map<String, Object>> notesInfo = new LinkedList<>();
if (jobNote == null) {
return notesInfo;
@ -642,19 +643,19 @@ public class Notebook implements NoteEventListener {
Map<String, Object> info = new HashMap<>();
info.put("notebookId", jobNote.getId());
String notebookName = jobNote.getName();
if (notebookName != null && !notebookName.equals("")) {
info.put("notebookName", jobNote.getName());
info.put("noteId", jobNote.getId());
String noteName = jobNote.getName();
if (noteName != null && !noteName.equals("")) {
info.put("noteName", jobNote.getName());
} else {
info.put("notebookName", "Note " + jobNote.getId());
info.put("noteName", "Note " + jobNote.getId());
}
// set notebook type ( cron or normal )
if (jobNote.getConfig().containsKey(CRON_TYPE_NOTEBOOK_KEYWORD) && !jobNote.getConfig()
.get(CRON_TYPE_NOTEBOOK_KEYWORD).equals("")) {
info.put("notebookType", "cron");
// set note type ( cron or normal )
if (jobNote.getConfig().containsKey(CRON_TYPE_NOTE_KEYWORD) && !jobNote.getConfig()
.get(CRON_TYPE_NOTE_KEYWORD).equals("")) {
info.put("noteType", "cron");
} else {
info.put("notebookType", "normal");
info.put("noteType", "normal");
}
// set paragraphs
@ -662,7 +663,7 @@ public class Notebook implements NoteEventListener {
for (Paragraph paragraph : jobNote.getParagraphs()) {
// check paragraph's status.
if (paragraph.getStatus().isRunning()) {
isNotebookRunning = true;
isNoteRunning = true;
}
// get data for the job manager.
@ -679,9 +680,9 @@ public class Notebook implements NoteEventListener {
interpreterGroupName = replFactory.getInterpreterSettings(jobNote.getId()).get(0).getName();
}
// notebook json object root information.
// note json object root information.
info.put("interpreter", interpreterGroupName);
info.put("isRunningJob", isNotebookRunning);
info.put("isRunningJob", isNoteRunning);
info.put("unixTimeLastRun", lastRunningUnixTime);
info.put("paragraphs", paragraphsInfo);
notesInfo.add(info);
@ -691,7 +692,7 @@ public class Notebook implements NoteEventListener {
public List<Map<String, Object>> getJobListByUnixTime(boolean needsReload,
long lastUpdateServerUnixTime, AuthenticationInfo subject) {
final String CRON_TYPE_NOTEBOOK_KEYWORD = "cron";
final String CRON_TYPE_NOTE_KEYWORD = "cron";
if (needsReload) {
try {
@ -704,28 +705,28 @@ public class Notebook implements NoteEventListener {
List<Note> notes = getAllNotes();
List<Map<String, Object>> notesInfo = new LinkedList<>();
for (Note note : notes) {
boolean isNotebookRunning = false;
boolean isUpdateNotebook = false;
boolean isNoteRunning = false;
boolean isUpdateNote = false;
long lastRunningUnixTime = 0;
Map<String, Object> info = new HashMap<>();
// set notebook ID
info.put("notebookId", note.getId());
// set note ID
info.put("noteId", note.getId());
// set notebook Name
String notebookName = note.getName();
if (notebookName != null && !notebookName.equals("")) {
info.put("notebookName", note.getName());
// set note Name
String noteName = note.getName();
if (noteName != null && !noteName.equals("")) {
info.put("noteName", note.getName());
} else {
info.put("notebookName", "Note " + note.getId());
info.put("noteName", "Note " + note.getId());
}
// set notebook type ( cron or normal )
if (note.getConfig().containsKey(CRON_TYPE_NOTEBOOK_KEYWORD) && !note.getConfig()
.get(CRON_TYPE_NOTEBOOK_KEYWORD).equals("")) {
info.put("notebookType", "cron");
// set note type ( cron or normal )
if (note.getConfig().containsKey(CRON_TYPE_NOTE_KEYWORD) && !note.getConfig()
.get(CRON_TYPE_NOTE_KEYWORD).equals("")) {
info.put("noteType", "cron");
} else {
info.put("notebookType", "normal");
info.put("noteType", "normal");
}
// set paragraphs
@ -733,17 +734,17 @@ public class Notebook implements NoteEventListener {
for (Paragraph paragraph : note.getParagraphs()) {
// check paragraph's status.
if (paragraph.getStatus().isRunning()) {
isNotebookRunning = true;
isUpdateNotebook = true;
isNoteRunning = true;
isUpdateNote = true;
}
// get data for the job manager.
Map<String, Object> paragraphItem = getParagraphForJobManagerItem(paragraph);
lastRunningUnixTime = getUnixTimeLastRunParagraph(paragraph);
// is update notebook for last server update time.
// is update note for last server update time.
if (lastRunningUnixTime > lastUpdateServerUnixTime) {
isUpdateNotebook = true;
isUpdateNote = true;
}
paragraphsInfo.add(paragraphItem);
}
@ -756,13 +757,13 @@ public class Notebook implements NoteEventListener {
}
// not update and not running -> pass
if (!isUpdateNotebook && !isNotebookRunning) {
if (!isUpdateNote && !isNoteRunning) {
continue;
}
// notebook json object root information.
// note json object root information.
info.put("interpreter", interpreterGroupName);
info.put("isRunningJob", isNotebookRunning);
info.put("isRunningJob", isNoteRunning);
info.put("unixTimeLastRun", lastRunningUnixTime);
info.put("paragraphs", paragraphsInfo);
notesInfo.add(info);
@ -879,7 +880,7 @@ public class Notebook implements NoteEventListener {
public void close() {
this.notebookRepo.close();
this.notebookIndex.close();
this.noteSearchService.close();
}
public void addNotebookEventListener(NotebookEventListener listener) {

45
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/AzureNotebookRepo.java
View file

@ -17,11 +17,19 @@
package org.apache.zeppelin.notebook.repo;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.microsoft.azure.storage.CloudStorageAccount;
import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.file.*;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.net.URISyntaxException;
import java.security.InvalidKeyException;
import java.util.Collections;
import java.util.Date;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
@ -33,12 +41,16 @@ import org.apache.zeppelin.scheduler.Job;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.*;
import java.net.URISyntaxException;
import java.security.InvalidKeyException;
import java.util.Date;
import java.util.LinkedList;
import java.util.List;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.microsoft.azure.storage.CloudStorageAccount;
import com.microsoft.azure.storage.StorageException;
import com.microsoft.azure.storage.file.CloudFile;
import com.microsoft.azure.storage.file.CloudFileClient;
import com.microsoft.azure.storage.file.CloudFileDirectory;
import com.microsoft.azure.storage.file.CloudFileShare;
import com.microsoft.azure.storage.file.ListFileItem;
/**
* Azure storage backend for notebooks
@ -227,4 +239,15 @@ public class AzureNotebookRepo implements NotebookRepo {
// Auto-generated method stub
return null;
}
@Override
public List<NotebookRepoSettingsInfo> getSettings(AuthenticationInfo subject) {
LOG.warn("Method not implemented");
return Collections.emptyList();
}
@Override
public void updateSettings(Map<String, String> settings, AuthenticationInfo subject) {
LOG.warn("Method not implemented");
}
}

19
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java
View file

@ -19,6 +19,7 @@ package org.apache.zeppelin.notebook.repo;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import org.apache.zeppelin.annotation.ZeppelinApi;
import org.apache.zeppelin.notebook.Note;
@ -39,7 +40,7 @@ public interface NotebookRepo {
/**
* Get the notebook with the given id.
* @param noteId is notebook id.
* @param noteId is note id.
* @param subject contains user information.
* @return
* @throws IOException
@ -100,6 +101,22 @@ public interface NotebookRepo {
*/
@ZeppelinApi public List<Revision> revisionHistory(String noteId, AuthenticationInfo subject);
/**
* Get NotebookRepo settings got the given user.
*
* @param subject
* @return
*/
@ZeppelinApi public List<NotebookRepoSettingsInfo> getSettings(AuthenticationInfo subject);
/**
* update notebook repo settings.
*
* @param settings
* @param subject
*/
@ZeppelinApi public void updateSettings(Map<String, String> settings, AuthenticationInfo subject);
/**
* Represents the 'Revision' a point in life of the notebook
*/

44
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSettingsInfo.java Normal file
View file

@ -0,0 +1,44 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.notebook.repo;
import java.util.List;
import java.util.Map;
/**
* Notebook repo settings. This represent a structure of a notebook repo settings that will mostly
* used in the frontend.
*
*/
public class NotebookRepoSettingsInfo {
/**
* Type of value, It can be text or list.
*/
public enum Type {
INPUT, DROPDOWN
}
public static NotebookRepoSettingsInfo newInstance() {
return new NotebookRepoSettingsInfo();
}
public Type type;
public List<Map<String, String>> value;
public String selected;
public String name;
}

139
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoSync.java
View file

@ -26,6 +26,7 @@ import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.conf.ZeppelinConfiguration.ConfVars;
@ -37,15 +38,17 @@ import org.apache.zeppelin.user.AuthenticationInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.collect.Lists;
/**
* Notebook repository sync with remote storage
*/
public class NotebookRepoSync implements NotebookRepo {
private static final Logger LOG = LoggerFactory.getLogger(NotebookRepoSync.class);
private static final int maxRepoNum = 2;
private static final String pushKey = "pushNoteIDs";
private static final String pullKey = "pullNoteIDs";
private static final String delDstKey = "delDstNoteIDs";
private static final String pushKey = "pushNoteIds";
private static final String pullKey = "pullNoteIds";
private static final String delDstKey = "delDstNoteIds";
private static ZeppelinConfiguration config;
private static final String defaultStorage = "org.apache.zeppelin.notebook.repo.VFSNotebookRepo";
@ -54,9 +57,7 @@ public class NotebookRepoSync implements NotebookRepo {
private final boolean oneWaySync;
/**
* @param noteIndex
* @param (conf)
* @throws - Exception
* @param conf
*/
@SuppressWarnings("static-access")
public NotebookRepoSync(ZeppelinConfiguration conf) {
@ -70,7 +71,7 @@ public class NotebookRepoSync implements NotebookRepo {
String[] storageClassNames = allStorageClassNames.split(",");
if (storageClassNames.length > getMaxRepoNum()) {
LOG.warn("Unsupported number {} of storage classes in ZEPPELIN_NOTEBOOK_STORAGE : {}\n" +
"first {} will be used", storageClassNames.length, allStorageClassNames, getMaxRepoNum());
"first {} will be used", storageClassNames.length, allStorageClassNames, getMaxRepoNum());
}
for (int i = 0; i < Math.min(storageClassNames.length, getMaxRepoNum()); i++) {
@ -79,7 +80,7 @@ public class NotebookRepoSync implements NotebookRepo {
try {
notebookStorageClass = getClass().forName(storageClassNames[i].trim());
Constructor<?> constructor = notebookStorageClass.getConstructor(
ZeppelinConfiguration.class);
ZeppelinConfiguration.class);
repos.add((NotebookRepo) constructor.newInstance(conf));
} catch (ClassNotFoundException | NoSuchMethodException | SecurityException |
InstantiationException | IllegalAccessException | IllegalArgumentException |
@ -89,17 +90,9 @@ public class NotebookRepoSync implements NotebookRepo {
}
// couldn't initialize any storage, use default
if (getRepoCount() == 0) {
LOG.info("No storages could be initialized, using default {} storage", defaultStorage);
LOG.info("No storage could be initialized, using default {} storage", defaultStorage);
initializeDefaultStorage(conf);
}
if (getRepoCount() > 1) {
try {
AuthenticationInfo subject = new AuthenticationInfo("anonymous");
sync(0, 1, subject);
} catch (IOException e) {
LOG.warn("Failed to sync with secondary storage on start {}", e);
}
}
}
@SuppressWarnings("static-access")
@ -117,6 +110,39 @@ public class NotebookRepoSync implements NotebookRepo {
}
}
public List<NotebookRepoWithSettings> getNotebookRepos(AuthenticationInfo subject) {
List<NotebookRepoWithSettings> reposSetting = Lists.newArrayList();
NotebookRepoWithSettings repoWithSettings;
for (NotebookRepo repo : repos) {
repoWithSettings = NotebookRepoWithSettings
.builder(repo.getClass().getSimpleName())
.className(repo.getClass().getName())
.settings(repo.getSettings(subject))
.build();
reposSetting.add(repoWithSettings);
}
return reposSetting;
}
public NotebookRepoWithSettings updateNotebookRepo(String name, Map<String, String> settings,
AuthenticationInfo subject) {
NotebookRepoWithSettings updatedSettings = NotebookRepoWithSettings.EMPTY;
for (NotebookRepo repo : repos) {
if (repo.getClass().getName().equals(name)) {
repo.updateSettings(settings, subject);
updatedSettings = NotebookRepoWithSettings
.builder(repo.getClass().getSimpleName())
.className(repo.getClass().getName())
.settings(repo.getSettings(subject))
.build();
break;
}
}
return updatedSettings;
}
/**
* Lists Notebooks from the first repository
*/
@ -172,6 +198,10 @@ public class NotebookRepoSync implements NotebookRepo {
/* TODO(khalid): handle case when removing from secondary storage fails */
}
void remove(int repoIndex, String noteId, AuthenticationInfo subject) throws IOException {
getRepo(repoIndex).remove(noteId, subject);
}
/**
* Copies new/updated notes from source to destination storage
*
@ -186,38 +216,38 @@ public class NotebookRepoSync implements NotebookRepo {
List <NoteInfo> srcNotes = auth.filterByUser(allSrcNotes, subject);
List <NoteInfo> dstNotes = dstRepo.list(subject);
Map<String, List<String>> noteIDs = notesCheckDiff(srcNotes, srcRepo, dstNotes, dstRepo,
Map<String, List<String>> noteIds = notesCheckDiff(srcNotes, srcRepo, dstNotes, dstRepo,
subject);
List<String> pushNoteIDs = noteIDs.get(pushKey);
List<String> pullNoteIDs = noteIDs.get(pullKey);
List<String> delDstNoteIDs = noteIDs.get(delDstKey);
List<String> pushNoteIds = noteIds.get(pushKey);
List<String> pullNoteIds = noteIds.get(pullKey);
List<String> delDstNoteIds = noteIds.get(delDstKey);
if (!pushNoteIDs.isEmpty()) {
if (!pushNoteIds.isEmpty()) {
LOG.info("Notes with the following IDs will be pushed");
for (String id : pushNoteIDs) {
for (String id : pushNoteIds) {
LOG.info("ID : " + id);
}
pushNotes(subject, pushNoteIDs, srcRepo, dstRepo);
pushNotes(subject, pushNoteIds, srcRepo, dstRepo, false);
} else {
LOG.info("Nothing to push");
}
if (!pullNoteIDs.isEmpty()) {
if (!pullNoteIds.isEmpty()) {
LOG.info("Notes with the following IDs will be pulled");
for (String id : pullNoteIDs) {
for (String id : pullNoteIds) {
LOG.info("ID : " + id);
}
pushNotes(subject, pullNoteIDs, dstRepo, srcRepo);
pushNotes(subject, pullNoteIds, dstRepo, srcRepo, true);
} else {
LOG.info("Nothing to pull");
}
if (!delDstNoteIDs.isEmpty()) {
if (!delDstNoteIds.isEmpty()) {
LOG.info("Notes with the following IDs will be deleted from dest");
for (String id : delDstNoteIDs) {
for (String id : delDstNoteIds) {
LOG.info("ID : " + id);
}
deleteNotes(subject, delDstNoteIDs, dstRepo);
deleteNotes(subject, delDstNoteIds, dstRepo);
} else {
LOG.info("Nothing to delete from dest");
}
@ -230,16 +260,43 @@ public class NotebookRepoSync implements NotebookRepo {
}
private void pushNotes(AuthenticationInfo subject, List<String> ids, NotebookRepo localRepo,
NotebookRepo remoteRepo) {
NotebookRepo remoteRepo, boolean setPermissions) {
for (String id : ids) {
try {
remoteRepo.save(localRepo.get(id, subject), subject);
if (setPermissions && emptyNoteAcl(id)) {
makePrivate(id, subject);
}
} catch (IOException e) {
LOG.error("Failed to push note to storage, moving onto next one", e);
}
}
}
private boolean emptyNoteAcl(String noteId) {
NotebookAuthorization notebookAuthorization = NotebookAuthorization.getInstance();
return notebookAuthorization.getOwners(noteId).isEmpty()
&& notebookAuthorization.getReaders(noteId).isEmpty()
&& notebookAuthorization.getWriters(noteId).isEmpty();
}
private void makePrivate(String noteId, AuthenticationInfo subject) {
if (AuthenticationInfo.isAnonymous(subject)) {
LOG.info("User is anonymous, permissions are not set for pulled notes");
return;
}
NotebookAuthorization notebookAuthorization = NotebookAuthorization.getInstance();
Set<String> users = notebookAuthorization.getOwners(noteId);
users.add(subject.getUser());
notebookAuthorization.setOwners(noteId, users);
users = notebookAuthorization.getReaders(noteId);
users.add(subject.getUser());
notebookAuthorization.setReaders(noteId, users);
users = notebookAuthorization.getWriters(noteId);
users.add(subject.getUser());
notebookAuthorization.setWriters(noteId, users);
}
private void deleteNotes(AuthenticationInfo subject, List<String> ids, NotebookRepo repo)
throws IOException {
for (String id : ids) {
@ -421,4 +478,24 @@ public class NotebookRepoSync implements NotebookRepo {
}
return revisions;
}
@Override
public List<NotebookRepoSettingsInfo> getSettings(AuthenticationInfo subject) {
List<NotebookRepoSettingsInfo> repoSettings = Collections.emptyList();
try {
repoSettings = getRepo(0).getSettings(subject);
} catch (IOException e) {
LOG.error("Cannot get notebook repo settings", e);
}
return repoSettings;
}
@Override
public void updateSettings(Map<String, String> settings, AuthenticationInfo subject) {
try {
getRepo(0).updateSettings(settings, subject);
} catch (IOException e) {
LOG.error("Cannot update notebook repo settings", e);
}
}
}

79
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepoWithSettings.java Normal file
View file

@ -0,0 +1,79 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.notebook.repo;
import java.util.Collections;
import java.util.List;
import org.apache.commons.lang.StringUtils;
/**
* Representation of a notebook repo with settings. This is mostly a Wrapper around notebook repo
* information plus settings.
*/
public class NotebookRepoWithSettings {
public static final NotebookRepoWithSettings EMPTY =
NotebookRepoWithSettings.builder(StringUtils.EMPTY).build();
public String name;
public String className;
public List<NotebookRepoSettingsInfo> settings;
private NotebookRepoWithSettings() {}
public static Builder builder(String name) {
return new Builder(name);
}
private NotebookRepoWithSettings(Builder builder) {
name = builder.name;
className = builder.className;
settings = builder.settings;
}
public boolean isEmpty() {
return this.equals(EMPTY);
}
/**
* Simple builder :).
*/
public static class Builder {
private final String name;
private String className = StringUtils.EMPTY;
private List<NotebookRepoSettingsInfo> settings = Collections.emptyList();
public Builder(String name) {
this.name = name;
}
public NotebookRepoWithSettings build() {
return new NotebookRepoWithSettings(this);
}
public Builder className(String className) {
this.className = className;
return this;
}
public Builder settings(List<NotebookRepoSettingsInfo> settings) {
this.settings = settings;
return this;
}
}
}

21
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/S3NotebookRepo.java
View file

@ -23,14 +23,12 @@ import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.Collections;
import java.util.Date;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3EncryptionClient;
import com.amazonaws.services.s3.model.EncryptionMaterialsProvider;
import com.amazonaws.services.s3.model.KMSEncryptionMaterialsProvider;
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
@ -45,10 +43,14 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.amazonaws.AmazonClientException;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.AmazonS3EncryptionClient;
import com.amazonaws.services.s3.model.EncryptionMaterialsProvider;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.KMSEncryptionMaterialsProvider;
import com.amazonaws.services.s3.model.ListObjectsRequest;
import com.amazonaws.services.s3.model.ObjectListing;
import com.amazonaws.services.s3.model.PutObjectRequest;
@ -270,4 +272,15 @@ public class S3NotebookRepo implements NotebookRepo {
// Auto-generated method stub
return null;
}
@Override
public List<NotebookRepoSettingsInfo> getSettings(AuthenticationInfo subject) {
LOG.warn("Method not implemented");
return Collections.emptyList();
}
@Override
public void updateSettings(Map<String, String> settings, AuthenticationInfo subject) {
LOG.warn("Method not implemented");
}
}

65
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/VFSNotebookRepo.java
View file

@ -23,11 +23,14 @@ import java.io.InputStream;
import java.io.OutputStream;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Collections;
import java.util.Date;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.commons.vfs2.FileContent;
import org.apache.commons.vfs2.FileObject;
import org.apache.commons.vfs2.FileSystemManager;
@ -40,13 +43,14 @@ import org.apache.zeppelin.conf.ZeppelinConfiguration.ConfVars;
import org.apache.zeppelin.notebook.ApplicationState;
import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.NoteInfo;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.notebook.NotebookImportDeserializer;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.scheduler.Job.Status;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.collect.Lists;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
@ -54,7 +58,7 @@ import com.google.gson.GsonBuilder;
*
*/
public class VFSNotebookRepo implements NotebookRepo {
Logger logger = LoggerFactory.getLogger(VFSNotebookRepo.class);
private static final Logger LOG = LoggerFactory.getLogger(VFSNotebookRepo.class);
private FileSystemManager fsManager;
private URI filesystemRoot;
@ -62,12 +66,15 @@ public class VFSNotebookRepo implements NotebookRepo {
public VFSNotebookRepo(ZeppelinConfiguration conf) throws IOException {
this.conf = conf;
setNotebookDirectory(conf.getNotebookDir());
}
private void setNotebookDirectory(String notebookDirPath) throws IOException {
try {
if (conf.isWindowsPath(conf.getNotebookDir())) {
filesystemRoot = new File(conf.getNotebookDir()).toURI();
if (conf.isWindowsPath(notebookDirPath)) {
filesystemRoot = new File(notebookDirPath).toURI();
} else {
filesystemRoot = new URI(conf.getNotebookDir());
filesystemRoot = new URI(notebookDirPath);
}
} catch (URISyntaxException e1) {
throw new IOException(e1);
@ -76,7 +83,7 @@ public class VFSNotebookRepo implements NotebookRepo {
if (filesystemRoot.getScheme() == null) { // it is local path
try {
this.filesystemRoot = new URI(new File(
conf.getRelativeDir(filesystemRoot.getPath())).getAbsolutePath());
conf.getRelativeDir(filesystemRoot.getPath())).getAbsolutePath());
} catch (URISyntaxException e) {
throw new IOException(e);
}
@ -85,11 +92,15 @@ public class VFSNotebookRepo implements NotebookRepo {
fsManager = VFS.getManager();
FileObject file = fsManager.resolveFile(filesystemRoot.getPath());
if (!file.exists()) {
logger.info("Notebook dir doesn't exist, create.");
LOG.info("Notebook dir doesn't exist, create on is {}.", file.getName());
file.createFolder();
}
}
private String getNotebookDirPath() {
return filesystemRoot.getPath().toString();
}
private String getPath(String path) {
if (path == null || path.trim().length() == 0) {
return filesystemRoot.toString();
@ -141,7 +152,7 @@ public class VFSNotebookRepo implements NotebookRepo {
infos.add(info);
}
} catch (Exception e) {
logger.error("Can't read note " + f.getName().toString(), e);
LOG.error("Can't read note " + f.getName().toString(), e);
}
}
@ -285,4 +296,42 @@ public class VFSNotebookRepo implements NotebookRepo {
return null;
}
@Override
public List<NotebookRepoSettingsInfo> getSettings(AuthenticationInfo subject) {
NotebookRepoSettingsInfo repoSetting = NotebookRepoSettingsInfo.newInstance();
List<NotebookRepoSettingsInfo> settings = Lists.newArrayList();
repoSetting.name = "Notebook Path";
repoSetting.type = NotebookRepoSettingsInfo.Type.INPUT;
repoSetting.value = Collections.emptyList();
repoSetting.selected = getNotebookDirPath();
settings.add(repoSetting);
return settings;
}
@Override
public void updateSettings(Map<String, String> settings, AuthenticationInfo subject) {
if (settings == null || settings.isEmpty()) {
LOG.error("Cannot update {} with empty settings", this.getClass().getName());
return;
}
String newNotebookDirectotyPath = StringUtils.EMPTY;
if (settings.containsKey("Notebook Path")) {
newNotebookDirectotyPath = settings.get("Notebook Path");
}
if (StringUtils.isBlank(newNotebookDirectotyPath)) {
LOG.error("Notebook path is invalid");
return;
}
LOG.warn("{} will change notebook dir from {} to {}",
subject.getUser(), getNotebookDirPath(), newNotebookDirectotyPath);
try {
setNotebookDirectory(newNotebookDirectotyPath);
} catch (IOException e) {
LOG.error("Cannot update notebook directory", e);
}
}
}

13
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/zeppelinhub/ZeppelinHubRepo.java
View file

@ -21,12 +21,14 @@ import java.net.URI;
import java.net.URISyntaxException;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import org.apache.commons.lang.StringUtils;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.NoteInfo;
import org.apache.zeppelin.notebook.repo.NotebookRepo;
import org.apache.zeppelin.notebook.repo.NotebookRepoSettingsInfo;
import org.apache.zeppelin.notebook.repo.zeppelinhub.rest.ZeppelinhubRestApiHandler;
import org.apache.zeppelin.notebook.repo.zeppelinhub.websocket.Client;
import org.apache.zeppelin.user.AuthenticationInfo;
@ -234,4 +236,15 @@ public class ZeppelinHubRepo implements NotebookRepo {
return history;
}
@Override
public List<NotebookRepoSettingsInfo> getSettings(AuthenticationInfo subject) {
LOG.warn("Method not implemented");
return Collections.emptyList();
}
@Override
public void updateSettings(Map<String, String> settings, AuthenticationInfo subject) {
LOG.warn("Method not implemented");
}
}

14
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/socket/Message.java
View file

@ -34,7 +34,7 @@ public class Message {
// @param id note id
NOTE, // [s-c] note info
// @param note serlialized Note object
// @param note serialized Note object
PARAGRAPH, // [s-c] paragraph info
// @param paragraph serialized paragraph object
@ -115,7 +115,7 @@ public class Message {
CONFIGURATIONS_INFO, // [s-c] all key/value pairs of configurations
// @param settings serialized Map<String, String> object
CHECKPOINT_NOTEBOOK, // [c-s] checkpoint notebook to storage repository
CHECKPOINT_NOTE, // [c-s] checkpoint note to storage repository
// @param noteId
// @param checkpointName
@ -130,14 +130,14 @@ public class Message {
APP_LOAD, // [s-c] on app load
APP_STATUS_CHANGE, // [s-c] on app status change
LIST_NOTEBOOK_JOBS, // [c-s] get notebook job management infomations
LIST_UPDATE_NOTEBOOK_JOBS, // [c-s] get job management informations for until unixtime
UNSUBSCRIBE_UPDATE_NOTEBOOK_JOBS, // [c-s] unsubscribe job information for job management
LIST_NOTE_JOBS, // [c-s] get note job management information
LIST_UPDATE_NOTE_JOBS, // [c-s] get job management information for until unixtime
UNSUBSCRIBE_UPDATE_NOTE_JOBS, // [c-s] unsubscribe job information for job management
// @param unixTime
GET_INTERPRETER_BINDINGS, // [c-s] get interpreter bindings
// @param noteID
// @param noteId
SAVE_INTERPRETER_BINDINGS, // [c-s] save interpreter bindings
// @param noteID
// @param noteId
// @param selectedSettingIds
INTERPRETER_BINDINGS, // [s-c] interpreter bindings
ERROR_INFO // [s-c] error information to be sent

6
zeppelin-zengine/src/main/java/org/apache/zeppelin/search/SearchService.java
View file

@ -25,7 +25,7 @@ import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.Paragraph;
/**
* Search (both, indexing and query) the notebooks.
* Search (both, indexing and query) the notes.
*
* Intended to have multiple implementation, i.e:
* - local Lucene (in-memory, on-disk)
@ -34,7 +34,7 @@ import org.apache.zeppelin.notebook.Paragraph;
public interface SearchService {
/**
* Full-text search in all the notebooks
* Full-text search in all the notes
*
* @param queryStr a query
* @return A list of matching paragraphs (id, text, snippet w/ highlight)
@ -59,7 +59,7 @@ public interface SearchService {
public void addIndexDocs(Collection<Note> collection);
/**
* Indexes the given notebook.
* Indexes the given note.
*
* @throws IOException If there is a low-level I/O error
*/

4
zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java
View file

@ -428,7 +428,7 @@ public class NotebookTest implements JobListenerFactory{
Paragraph cp = cloneNote.paragraphs.get(0);
assertEquals(cp.getStatus(), Status.READY);
// Keep same ParagraphID
// Keep same ParagraphId
assertEquals(cp.getId(), p.getId());
assertEquals(cp.text, p.text);
assertEquals(cp.getResult().message(), p.getResult().message());
@ -476,7 +476,7 @@ public class NotebookTest implements JobListenerFactory{
Note cloneNote = notebook.cloneNote(note.getId(), "clone note with Exception result", anonymous);
Paragraph cp = cloneNote.paragraphs.get(0);
// Keep same ParagraphID
// Keep same ParagraphId
assertEquals(cp.getId(), p.getId());
assertEquals(cp.text, p.text);
assertNull(cp.getResult());

68
zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/NotebookRepoSyncTest.java
View file

@ -24,7 +24,9 @@ import static org.mockito.Mockito.mock;
import java.io.File;
import java.io.IOException;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import org.apache.commons.io.FileUtils;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
@ -314,6 +316,72 @@ public class NotebookRepoSyncTest implements JobListenerFactory {
notebookRepoSync.remove(note.getId(), anonymous);
}
@Test
public void testSyncWithAcl() throws IOException {
/* scenario 1 - note exists with acl on main storage */
AuthenticationInfo user1 = new AuthenticationInfo("user1");
Note note = notebookSync.createNote(user1);
assertEquals(0, note.getParagraphs().size());
// saved on both storages
assertEquals(1, notebookRepoSync.list(0, null).size());
assertEquals(1, notebookRepoSync.list(1, null).size());
/* check that user1 is the only owner */
NotebookAuthorization authInfo = NotebookAuthorization.getInstance();
Set<String> entity = new HashSet<String>();
entity.add(user1.getUser());
assertEquals(true, authInfo.isOwner(note.getId(), entity));
assertEquals(1, authInfo.getOwners(note.getId()).size());
assertEquals(0, authInfo.getReaders(note.getId()).size());
assertEquals(0, authInfo.getWriters(note.getId()).size());
/* update note and save on secondary storage */
Paragraph p1 = note.addParagraph();
p1.setText("hello world");
assertEquals(1, note.getParagraphs().size());
notebookRepoSync.save(1, note, null);
/* check paragraph isn't saved into first storage */
assertEquals(0, notebookRepoSync.get(0,
notebookRepoSync.list(0, null).get(0).getId(), null).getParagraphs().size());
/* check paragraph is saved into second storage */
assertEquals(1, notebookRepoSync.get(1,
notebookRepoSync.list(1, null).get(0).getId(), null).getParagraphs().size());
/* now sync by user1 */
notebookRepoSync.sync(user1);
/* check that note updated and acl are same on main storage*/
assertEquals(1, notebookRepoSync.get(0,
notebookRepoSync.list(0, null).get(0).getId(), null).getParagraphs().size());
assertEquals(true, authInfo.isOwner(note.getId(), entity));
assertEquals(1, authInfo.getOwners(note.getId()).size());
assertEquals(0, authInfo.getReaders(note.getId()).size());
assertEquals(0, authInfo.getWriters(note.getId()).size());
/* scenario 2 - note doesn't exist on main storage */
/* remove from main storage */
notebookRepoSync.remove(0, note.getId(), user1);
assertEquals(0, notebookRepoSync.list(0, null).size());
assertEquals(1, notebookRepoSync.list(1, null).size());
authInfo.removeNote(note.getId());
assertEquals(0, authInfo.getOwners(note.getId()).size());
assertEquals(0, authInfo.getReaders(note.getId()).size());
assertEquals(0, authInfo.getWriters(note.getId()).size());
/* now sync - should bring note from secondary storage with added acl */
notebookRepoSync.sync(user1);
assertEquals(1, notebookRepoSync.list(0, null).size());
assertEquals(1, notebookRepoSync.list(1, null).size());
assertEquals(1, authInfo.getOwners(note.getId()).size());
assertEquals(1, authInfo.getReaders(note.getId()).size());
assertEquals(1, authInfo.getWriters(note.getId()).size());
assertEquals(true, authInfo.isOwner(note.getId(), entity));
assertEquals(true, authInfo.isReader(note.getId(), entity));
assertEquals(true, authInfo.isWriter(note.getId(), entity));
}
static void delete(File file){
if(file.isFile()) file.delete();
else if(file.isDirectory()){

30
zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/VFSNotebookRepoTest.java
View file

@ -22,6 +22,7 @@ import static org.mockito.Mockito.mock;
import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import org.apache.commons.io.FileUtils;
@ -31,12 +32,13 @@ import org.apache.zeppelin.dep.DependencyResolver;
import org.apache.zeppelin.interpreter.InterpreterFactory;
import org.apache.zeppelin.interpreter.InterpreterOption;
import org.apache.zeppelin.interpreter.mock.MockInterpreter1;
import org.apache.zeppelin.notebook.*;
import org.apache.zeppelin.notebook.repo.zeppelinhub.security.Authentication;
import org.apache.zeppelin.scheduler.JobListener;
import org.apache.zeppelin.notebook.JobListenerFactory;
import org.apache.zeppelin.notebook.Note;
import org.apache.zeppelin.notebook.Notebook;
import org.apache.zeppelin.notebook.Paragraph;
import org.apache.zeppelin.notebook.ParagraphJobListener;
import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.apache.zeppelin.search.SearchService;
import org.apache.zeppelin.search.LuceneSearch;
import org.apache.zeppelin.user.AuthenticationInfo;
import org.junit.After;
import org.junit.Before;
@ -44,6 +46,8 @@ import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.common.collect.ImmutableMap;
public class VFSNotebookRepoTest implements JobListenerFactory {
private static final Logger LOG = LoggerFactory.getLogger(VFSNotebookRepoTest.class);
private ZeppelinConfiguration conf;
@ -141,6 +145,24 @@ public class VFSNotebookRepoTest implements JobListenerFactory {
assertEquals(note.getName(), "SaveTest");
notebookRepo.remove(note.getId(), null);
}
@Test
public void testUpdateSettings() throws IOException {
AuthenticationInfo subject = new AuthenticationInfo("anonymous");
File tmpDir = File.createTempFile("temp", Long.toString(System.nanoTime()));
Map<String, String> settings = ImmutableMap.of("Notebook Path", tmpDir.getAbsolutePath());
List<NotebookRepoSettingsInfo> repoSettings = notebookRepo.getSettings(subject);
String originalDir = repoSettings.get(0).selected;
notebookRepo.updateSettings(settings, subject);
repoSettings = notebookRepo.getSettings(subject);
assertEquals(repoSettings.get(0).selected, tmpDir.getAbsolutePath());
// restaure
notebookRepo.updateSettings(ImmutableMap.of("Notebook Path", originalDir), subject);
FileUtils.deleteQuietly(tmpDir);
}
class NotebookWriter implements Runnable {
Note note;

51
zeppelin-zengine/src/test/java/org/apache/zeppelin/search/LuceneSearchTest.java
View file

@ -41,7 +41,8 @@ public class LuceneSearchTest {
private static NotebookRepo notebookRepoMock;
private static InterpreterFactory interpreterFactory;
private SearchService notebookIndex;
private SearchService noteSearchService;
private AuthenticationInfo anonymous;
@BeforeClass
@ -55,13 +56,13 @@ public class LuceneSearchTest {
@Before
public void startUp() {
notebookIndex = new LuceneSearch();
noteSearchService = new LuceneSearch();
anonymous = new AuthenticationInfo("anonymous");
}
@After
public void shutDown() {
notebookIndex.close();
noteSearchService.close();
}
@Test public void canIndexNotebook() {
@ -71,17 +72,17 @@ public class LuceneSearchTest {
List<Note> notebook = Arrays.asList(note1, note2);
//when
notebookIndex.addIndexDocs(notebook);
noteSearchService.addIndexDocs(notebook);
}
@Test public void canIndexAndQuery() {
//given
Note note1 = newNoteWithParagraph("Notebook1", "test");
Note note2 = newNoteWithParagraphs("Notebook2", "not test", "not test at all");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
//when
List<Map<String, String>> results = notebookIndex.query("all");
List<Map<String, String>> results = noteSearchService.query("all");
//then
assertThat(results).isNotEmpty();
@ -94,10 +95,10 @@ public class LuceneSearchTest {
//given
Note note1 = newNoteWithParagraph("Notebook1", "test");
Note note2 = newNoteWithParagraphs("Notebook2", "not test", "not test at all");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
//when
List<Map<String, String>> results = notebookIndex.query("Notebook1");
List<Map<String, String>> results = noteSearchService.query("Notebook1");
//then
assertThat(results).isNotEmpty();
@ -110,10 +111,10 @@ public class LuceneSearchTest {
//given
Note note1 = newNoteWithParagraph("Notebook1", "test", "testingTitleSearch");
Note note2 = newNoteWithParagraph("Notebook2", "not test", "notTestingTitleSearch");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
//when
List<Map<String, String>> results = notebookIndex.query("testingTitleSearch");
List<Map<String, String>> results = noteSearchService.query("testingTitleSearch");
//then
assertThat(results).isNotEmpty();
@ -131,7 +132,7 @@ public class LuceneSearchTest {
//give
Note note1 = newNoteWithParagraph("Notebook1", "test");
//when
notebookIndex.addIndexDoc(note1);
noteSearchService.addIndexDoc(note1);
//then
String id = resultForQuery("test").get(0).get(LuceneSearch.ID_FIELD);
@ -141,9 +142,9 @@ public class LuceneSearchTest {
@Test //(expected=IllegalStateException.class)
public void canNotSearchBeforeIndexing() {
//given NO notebookIndex.index() was called
//given NO noteSearchService.index() was called
//when
List<Map<String, String>> result = notebookIndex.query("anything");
List<Map<String, String>> result = noteSearchService.query("anything");
//then
assertThat(result).isEmpty();
//assert logs were printed
@ -154,18 +155,18 @@ public class LuceneSearchTest {
//given
Note note1 = newNoteWithParagraph("Notebook1", "test");
Note note2 = newNoteWithParagraphs("Notebook2", "not test", "not test at all");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
//when
Paragraph p2 = note2.getLastParagraph();
p2.setText("test indeed");
notebookIndex.updateIndexDoc(note2);
noteSearchService.updateIndexDoc(note2);
//then
List<Map<String, String>> results = notebookIndex.query("all");
List<Map<String, String>> results = noteSearchService.query("all");
assertThat(results).isEmpty();
results = notebookIndex.query("indeed");
results = noteSearchService.query("indeed");
assertThat(results).isNotEmpty();
}
@ -173,21 +174,21 @@ public class LuceneSearchTest {
//give
// looks like a bug in web UI: it tries to delete a note twice (after it has just been deleted)
//when
notebookIndex.deleteIndexDocs(null);
noteSearchService.deleteIndexDocs(null);
}
@Test public void canDeleteFromIndex() throws IOException {
//given
Note note1 = newNoteWithParagraph("Notebook1", "test");
Note note2 = newNoteWithParagraphs("Notebook2", "not test", "not test at all");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
assertThat(resultForQuery("Notebook2")).isNotEmpty();
//when
notebookIndex.deleteIndexDocs(note2);
noteSearchService.deleteIndexDocs(note2);
//then
assertThat(notebookIndex.query("all")).isEmpty();
assertThat(noteSearchService.query("all")).isEmpty();
assertThat(resultForQuery("Notebook2")).isEmpty();
List<Map<String, String>> results = resultForQuery("test");
@ -199,7 +200,7 @@ public class LuceneSearchTest {
//given: total 2 notebooks, 3 paragraphs
Note note1 = newNoteWithParagraph("Notebook1", "test");
Note note2 = newNoteWithParagraphs("Notebook2", "not test", "not test at all");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
assertThat(resultForQuery("test").size()).isEqualTo(3);
//when
@ -224,7 +225,7 @@ public class LuceneSearchTest {
//given: total 2 notebooks, 3 paragraphs
Note note1 = newNoteWithParagraph("Notebook1", "test");
Note note2 = newNoteWithParagraphs("Notebook2", "not test", "not test at all");
notebookIndex.addIndexDocs(Arrays.asList(note1, note2));
noteSearchService.addIndexDocs(Arrays.asList(note1, note2));
assertThat(resultForQuery("test").size()).isEqualTo(3);
//when
@ -238,7 +239,7 @@ public class LuceneSearchTest {
}
private List<Map<String, String>> resultForQuery(String q) {
return notebookIndex.query(q);
return noteSearchService.query(q);
}
/**
@ -287,7 +288,7 @@ public class LuceneSearchTest {
}
private Note newNote(String name) {
Note note = new Note(notebookRepoMock, interpreterFactory, null, notebookIndex, null, null);
Note note = new Note(notebookRepoMock, interpreterFactory, null, noteSearchService, null, null);
note.setName(name);
return note;
}