Merge branch 'master' into notebook-search

Conflicts:
	docs/rest-api/rest-notebook.md
This commit is contained in:
Alexander Bezzubov 2015-12-21 15:56:13 +09:00
commit eb7878ab57
12 changed files with 235 additions and 40 deletions

View file

@ -70,6 +70,15 @@ function addEachJarInDir(){
fi
}
function addEachJarInDirRecursive(){
if [[ -d "${1}" ]]; then
for jar in $(find -L "${1}" -type f -name '*jar'); do
ZEPPELIN_CLASSPATH="$jar:$ZEPPELIN_CLASSPATH"
done
fi
}
function addJarInDir(){
if [[ -d "${1}" ]]; then
ZEPPELIN_CLASSPATH="${1}/*:${ZEPPELIN_CLASSPATH}"

View file

@ -87,7 +87,7 @@ if [[ "${INTERPRETER_ID}" == "spark" ]]; then
# add Hadoop jars into classpath
if [[ -n "${HADOOP_HOME}" ]]; then
# Apache
addEachJarInDir "${HADOOP_HOME}/share"
addEachJarInDirRecursive "${HADOOP_HOME}/share"
# CDH
addJarInDir "${HADOOP_HOME}"

View file

@ -361,3 +361,37 @@ limitations under the License.
</td>
</tr>
</table>
<br/>
<table class="table-configuration">
<col width="200">
<tr>
<th>Restart an interpreter</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```PUT``` method restart the given interpreter id.</td>
</tr>
<tr>
<td>URL</td>
<td>```http://[zeppelin-server]:[zeppelin-port]/api/interpreter/setting/restart/[interpreter ID]```</td>
</tr>
<tr>
<td>Success code</td>
<td>200</td>
</tr>
<tr>
<td> Fail code</td>
<td> 500 </td>
</tr>
<tr>
<td> sample JSON response
</td>
<td>
<pre>{"status":"OK"}</pre>
</td>
</tr>
</table>

View file

@ -20,12 +20,12 @@ limitations under the License.
{% include JB/setup %}
## Zeppelin REST API
Zeppelin provides several REST API's for interaction and remote activation of zeppelin functionality.
Zeppelin provides several REST APIs for interaction and remote activation of zeppelin functionality.
All REST API are available starting with the following endpoint ```http://[zeppelin-server]:[zeppelin-port]/api```
All REST APIs are available starting with the following endpoint ```http://[zeppelin-server]:[zeppelin-port]/api```
Note that zeppelin REST API receive or return JSON objects, it it recommended you install some JSON view such as
[JSONView](https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc)
Note that zeppelin REST APIs receive or return JSON objects, it is recommended for you to install some JSON viewer
such as [JSONView](https://chrome.google.com/webstore/detail/jsonview/chklaanhfefbnpoihckbnefhakgolnmc)
If you work with zeppelin and find a need for an additional REST API please [file an issue or send us mail](../../community.html)
@ -43,7 +43,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method list the available notebooks on your server.
<td>This ```GET``` method lists the available notebooks on your server.
Notebook JSON contains the ```name``` and ```id``` of all notebooks.
</td>
</tr>
@ -75,8 +75,8 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method create a new notebook using the given name or default name if none given.
The body field of the returned JSON contain the new notebook id.
<td>This ```POST``` method creates a new notebook using the given name or default name if none given.
The body field of the returned JSON contains the new notebook id.
</td>
</tr>
<tr>
@ -129,7 +129,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method delete a notebook by the given notebook id.
<td>This ```DELETE``` method deletes a notebook by the given notebook id.
</td>
</tr>
<tr>
@ -160,9 +160,9 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method clone a notebook by the given id and create a new notebook using the given name
<td>This ```POST``` method clones a notebook by the given id and create a new notebook using the given name
or default name if none given.
The body field of the returned JSON contain the new notebook id.
The body field of the returned JSON contains the new notebook id.
</td>
</tr>
<tr>
@ -197,7 +197,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method run all paragraph in the given notebook id.
<td>This ```POST``` method runs all paragraph in the given notebook id.
</td>
</tr>
<tr>
@ -228,7 +228,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method stop all paragraph in the given notebook id.
<td>This ```DELETE``` method stops all paragraph in the given notebook id.
</td>
</tr>
<tr>
@ -261,7 +261,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method get all paragraph status by the given notebook id.
<td>This ```GET``` method gets all paragraph status by the given notebook id.
The body field of the returned JSON contains of the array that compose of the paragraph id, paragraph status, paragraph finish date, paragraph started date.
</td>
</tr>
@ -293,7 +293,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method run the paragraph by given notebook and paragraph id.
<td>This ```POST``` method runs the paragraph by given notebook and paragraph id.
</td>
</tr>
<tr>
@ -324,7 +324,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method stop the paragraph by given notebook and paragraph id.
<td>This ```DELETE``` method stops the paragraph by given notebook and paragraph id.
</td>
</tr>
<tr>
@ -355,7 +355,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```POST``` method add cron job by the given notebook id.
<td>This ```POST``` method adds cron job by the given notebook id.
</td>
</tr>
<tr>
@ -390,7 +390,7 @@ limitations under the License.
</tr>
<tr>
<td>Description</td>
<td>This ```DELETE``` method remove cron job by the given notebook id.
<td>This ```DELETE``` method removes cron job by the given notebook id.
</td>
</tr>
<tr>
@ -416,13 +416,13 @@ limitations under the License.
<table class="table-configuration">
<col width="200">
<tr>
<th>Get clone job</th>
<th>Get cron job</th>
<th></th>
</tr>
<tr>
<td>Description</td>
<td>This ```GET``` method get cron job expression of given notebook id.
The body field of the returned JSON contain the cron expression.
<td>This ```GET``` method gets cron job expression of given notebook id.
The body field of the returned JSON contains the cron expression.
</td>
</tr>
<tr>

View file

@ -58,6 +58,11 @@ import org.apache.zeppelin.spark.dep.DependencyContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.Gson;
import com.google.gson.JsonElement;
import com.google.gson.JsonParseException;
import com.google.gson.JsonParser;
import py4j.GatewayServer;
/**
@ -368,12 +373,96 @@ public class PySparkInterpreter extends Interpreter implements ExecuteResultHand
return sparkInterpreter.getProgress(context);
}
@Override
public List<String> completion(String buf, int cursor) {
// not supported
return new LinkedList<String>();
if (buf.length() < cursor) {
cursor = buf.length();
}
String completionString = getCompletionTargetString(buf, cursor);
String completionCommand = "completion.getCompletion('" + completionString + "')";
//start code for completion
SparkInterpreter sparkInterpreter = getSparkInterpreter();
if (sparkInterpreter.getSparkVersion().isUnsupportedVersion() == false
&& pythonscriptRunning == false) {
return new LinkedList<String>();
}
outputStream.reset();
pythonInterpretRequest = new PythonInterpretRequest(completionCommand, "");
statementOutput = null;
synchronized (statementSetNotifier) {
statementSetNotifier.notify();
}
synchronized (statementFinishedNotifier) {
while (statementOutput == null) {
try {
statementFinishedNotifier.wait(1000);
} catch (InterruptedException e) {
// not working
logger.info("wait drop");
return new LinkedList<String>();
}
}
}
if (statementError) {
return new LinkedList<String>();
}
InterpreterResult completionResult = new InterpreterResult(Code.SUCCESS, statementOutput);
//end code for completion
Gson gson = new Gson();
return gson.fromJson(completionResult.message(), LinkedList.class);
}
private String getCompletionTargetString(String text, int cursor) {
String[] completionSeqCharaters = {" ", "\n", "\t"};
int completionEndPosition = cursor;
int completionStartPosition = cursor;
int indexOfReverseSeqPostion = cursor;
String resultCompletionText = "";
String completionScriptText = "";
try {
completionScriptText = text.substring(0, cursor);
}
catch (Exception e) {
logger.error(e.toString());
return null;
}
completionEndPosition = completionScriptText.length();
String tempReverseCompletionText = new StringBuilder(completionScriptText).reverse().toString();
for (String seqCharacter : completionSeqCharaters) {
indexOfReverseSeqPostion = tempReverseCompletionText.indexOf(seqCharacter);
if (indexOfReverseSeqPostion < completionStartPosition && indexOfReverseSeqPostion > 0) {
completionStartPosition = indexOfReverseSeqPostion;
}
}
if (completionStartPosition == completionEndPosition) {
completionStartPosition = 0;
}
else
{
completionStartPosition = completionEndPosition - completionStartPosition;
}
resultCompletionText = completionScriptText.substring(
completionStartPosition , completionEndPosition);
return resultCompletionText;
}
private SparkInterpreter getSparkInterpreter() {
InterpreterGroup intpGroup = getInterpreterGroup();
LazyOpenInterpreter lazy = null;

View file

@ -15,7 +15,7 @@
# limitations under the License.
#
import sys, getopt, traceback
import sys, getopt, traceback, json, re
from py4j.java_gateway import java_import, JavaGateway, GatewayClient
from py4j.protocol import Py4JJavaError
@ -107,6 +107,50 @@ class SparkVersion(object):
def isImportAllPackageUnderSparkSql(self):
return self.version >= self.SPARK_1_3_0
class PySparkCompletion:
def getGlobalCompletion(self):
objectDefList = []
try:
for completionItem in list(globals().iterkeys()):
objectDefList.append(completionItem)
except:
return None
else:
return objectDefList
def getMethodCompletion(self, text_value):
objectDefList = []
completion_target = text_value
try:
if len(completion_target) <= 0:
return None
if text_value[-1] == ".":
completion_target = text_value[:-1]
exec("%s = %s(%s)" % ("objectDefList", "dir", completion_target))
except:
return None
else:
return objectDefList
def getCompletion(self, text_value):
completionList = set()
globalCompletionList = self.getGlobalCompletion()
if globalCompletionList != None:
for completionItem in list(globalCompletionList):
completionList.add(completionItem)
if text_value != None:
objectCompletionList = self.getMethodCompletion(text_value)
if objectCompletionList != None:
for completionItem in list(objectCompletionList):
completionList.add(completionItem)
if len(completionList) <= 0:
print ""
else:
print json.dumps(filter(lambda x : not re.match("^__.*", x), list(completionList)))
output = Logger()
sys.stdout = output
@ -149,6 +193,7 @@ sc = SparkContext(jsc=jsc, gateway=gateway, conf=conf)
sqlc = SQLContext(sc, intp.getSQLContext())
sqlContext = sqlc
completion = PySparkCompletion()
z = PyZeppelinContext(intp.getZeppelinContext())
while True :

View file

@ -20,7 +20,7 @@ angular.module('zeppelinWebApp').controller('MainCtrl', function($scope, $rootSc
var init = function() {
$scope.asIframe = (($window.location.href.indexOf('asIframe') > -1) ? true : false);
};
init();
$rootScope.$on('setIframe', function(event, data) {
@ -36,10 +36,13 @@ angular.module('zeppelinWebApp').controller('MainCtrl', function($scope, $rootSc
event.preventDefault();
}
});
// Set The lookAndFeel to default on every page
$rootScope.$on('$routeChangeStart', function(event, next, current) {
$rootScope.$broadcast('setLookAndFeel', 'default');
});
BootstrapDialog.defaultOptions.onshown = function() {
angular.element('#' + this.id).find('.btn:last').focus();
};
});

View file

@ -57,6 +57,7 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
$scope.updateInterpreterSetting = function(settingId) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to update this interpreter and restart with new settings?',
callback: function(result) {
@ -89,6 +90,7 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
$scope.removeInterpreterSetting = function(settingId) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this interpreter setting?',
callback: function(result) {
@ -126,6 +128,7 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
$scope.restartInterpreterSetting = function(settingId) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to restart this interpreter?',
callback: function(result) {
@ -146,6 +149,7 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
$scope.addNewInterpreterSetting = function() {
if (!$scope.newInterpreterSetting.name || !$scope.newInterpreterSetting.group) {
BootstrapDialog.alert({
closable: true,
title: 'Add interpreter',
message: 'Please determine name and interpreter'
});
@ -154,6 +158,7 @@ angular.module('zeppelinWebApp').controller('InterpreterCtrl', function($scope,
if (_.findIndex($scope.interpreterSettings, { 'name': $scope.newInterpreterSetting.name }) >= 0) {
BootstrapDialog.alert({
closable: true,
title: 'Add interpreter',
message: 'Name ' + $scope.newInterpreterSetting.name + ' already exists'
});

View file

@ -14,7 +14,7 @@ limitations under the License.
<div class="noteAction" ng-show="note.id && !paragraphUrl">
<h3>
<input type="text" pu-elastic-input class="form-control2" placeholder="{{note.name || 'Note ' + note.id}}" style="min-width: 200px; max-width: 600px;"
ng-show="showEditor" ng-model="note.name" ng-blur="sendNewName()" ng-enter="sendNewName()" ng-escape="note.name = oldName; showEditor = false" focus-if="showEditor" />
ng-show="showEditor" ng-model="note.name" ng-blur="sendNewName();showEditor = false;" ng-enter="sendNewName();showEditor = false;" ng-escape="note.name = oldName; showEditor = false" focus-if="showEditor" />
<p class="form-control-static2" ng-click="showEditor = true; oldName = note.name" ng-show="!showEditor">{{note.name || 'Note ' + note.id}}</p>
<span class="labelBtn btn-group">
<button type="button"

View file

@ -95,6 +95,7 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl',
/** TODO(anthony): In the nearly future, go back to the main page and telle to the dude that the note have been remove */
$scope.removeNote = function(noteId) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this notebook?',
callback: function(result) {
@ -115,6 +116,7 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl',
//Clone note
$scope.cloneNote = function(noteId) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to clone this notebook?',
callback: function(result) {
@ -128,6 +130,7 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl',
$scope.runNote = function() {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Run all paragraphs?',
callback: function(result) {
@ -151,6 +154,7 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl',
$scope.clearAllParagraphOutput = function() {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to clear all output?',
callback: function(result) {
@ -257,7 +261,6 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl',
/** Update the note name */
$scope.sendNewName = function() {
$scope.showEditor = false;
if ($scope.note.name) {
websocketMsgSrv.updateNotebook($scope.note.id, $scope.note.name, $scope.note.config);
}
@ -519,16 +522,19 @@ angular.module('zeppelinWebApp').controller('NotebookCtrl',
$scope.closeSetting = function() {
if (isSettingDirty()) {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Changes will be discarded',
message: 'Changes will be discarded.',
callback: function(result) {
if (result) {
$scope.$apply(function () {
$scope.$apply(function() {
$scope.showSetting = false;
});
}
}
});
} else {
$scope.showSetting = false;
}
};

View file

@ -298,6 +298,7 @@ angular.module('zeppelinWebApp')
$scope.removeParagraph = function() {
BootstrapDialog.confirm({
closable: true,
title: '',
message: 'Do you want to delete this paragraph?',
callback: function(result) {
@ -941,14 +942,18 @@ angular.module('zeppelinWebApp')
var renderTable = function() {
var html = '';
html += '<table class="table table-hover table-condensed">';
html += '<table class="table table-hover table-condensed" style="top: 0; position: absolute;">';
html += ' <thead>';
html += ' <tr style="background-color: #F6F6F6; font-weight: bold;">';
for (var c in $scope.paragraph.result.columnNames) {
html += '<th>'+$scope.paragraph.result.columnNames[c].name+'</th>';
for (var titleIndex in $scope.paragraph.result.columnNames) {
html += '<th>'+$scope.paragraph.result.columnNames[titleIndex].name+'</th>';
}
html += ' </tr>';
html += ' </thead>';
html += '</table>';
html += '<table class="table table-hover table-condensed" style="margin-top: 31px;">';
for (var r in $scope.paragraph.result.msgTable) {
var row = $scope.paragraph.result.msgTable[r];

View file

@ -32,17 +32,16 @@
*/
.paragraph .text {
display: block;
unicode-bidi: embed;
display: block !important;
margin: 0 0 0 !important;
font-family: "Monaco","Menlo","Ubuntu Mono","Consolas","source-code-pro",monospace;
font-size: 12px !important;
line-height: 1.42857143 !important;
margin: 0 0 5px !important;
padding-top: 2px;
unicode-bidi: embed;
white-space: pre-wrap;
word-break: break-all !important;
word-wrap: break-word !important;
font-family: 'Monaco', 'Menlo', 'Ubuntu Mono', 'Consolas', 'source-code-pro', monospace;
font-size: 12px !important;
margin-bottom: 5px !important;
padding-top: 2px;
}
.paragraph table {