From 0d1dab2ab25e85b49b6a392e1bb5995ca691215c Mon Sep 17 00:00:00 2001 From: Gabe Fernando Date: Fri, 24 Nov 2017 14:00:02 -0500 Subject: [PATCH 01/14] ZEPPELIN-2984: allow enter from username ### What is this PR for? Allow an "Enter" keypress in the username field during login ### What type of PR is it? Improvement ### Todos * N/A ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-2984 ### How should this be tested? * Pull up login screen. Fill in username and password. Click in username field. Press enter. We would expect the login handler to be called. * Pull up login screen. Fill in username and password. Press enter (while still in password field). We would expect the login handler to be called just as before. * Alternatively, use a password manager (e.g. LastPass). Save the password. Pull up the login screen. Press Enter. We would expect the login handler to be called. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Gabe Fernando Closes #2616 from gef756/dev-login-enter and squashes the following commits: 301d398 [Gabe Fernando] MRG: Merge remote-tracking branch 'origin/master' into dev-login-enter 2a1865a [Gabe Fernando] (ZEPPELIN-2984) ENH: allow enter from username --- zeppelin-web/src/components/login/login.html | 1 + 1 file changed, 1 insertion(+) diff --git a/zeppelin-web/src/components/login/login.html b/zeppelin-web/src/components/login/login.html index d4a94a6f85..fd98bc8c4e 100644 --- a/zeppelin-web/src/components/login/login.html +++ b/zeppelin-web/src/components/login/login.html @@ -29,6 +29,7 @@ limitations under the License.
From 14427442ee9db0248ec9aa26b23c82cca450d2e1 Mon Sep 17 00:00:00 2001 From: Ondrej Kokes Date: Wed, 8 Nov 2017 16:08:21 +0100 Subject: [PATCH 02/14] [ZEPPELIN-3042] updating jgit to support post-commit hooks ### What is this PR for? Git functionality in Zeppelin is provided by `jgit`, but it's fixed to an old version. A newer version of said library supports new things, primarily post-commit hooks. ### What type of PR is it? Improvement ### Todos * [ x ] - CI ([two failed, but due to yarn](https://travis-ci.org/kokes/zeppelin)) ### What is the Jira issue? [ZEPPELIN-3042](https://issues.apache.org/jira/browse/ZEPPELIN-3042) ### Questions: * Does the licenses files need update? -- Yes, only because the license file notes the exact version of included libraries. `jgit` hasn't changed, license-wise, so it's only a matter of updating the version information * Is there breaking changes for older versions? -- nope * Does this needs documentation? -- possibly, to let people know they get use post-commit hooks now? Author: Ondrej Kokes Closes #2658 from kokes/jgit-update and squashes the following commits: a2cde16 [Ondrej Kokes] [ZEPPELIN-3042] newer jgit supports post-commit hooks a4fc816 [Ondrej Kokes] [ZEPPELIN-3042] newer jgit supports post-commit hooks --- zeppelin-distribution/src/bin_license/LICENSE | 2 +- zeppelin-zengine/pom.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/zeppelin-distribution/src/bin_license/LICENSE b/zeppelin-distribution/src/bin_license/LICENSE index ab120f235f..7b23313b21 100644 --- a/zeppelin-distribution/src/bin_license/LICENSE +++ b/zeppelin-distribution/src/bin_license/LICENSE @@ -311,7 +311,7 @@ The text of each license is also included at licenses/LICENSE-[project]-[version The following components are provided under the BSD-style License. - (New BSD License) JGit (org.eclipse.jgit:org.eclipse.jgit:jar:4.1.1.201511131810-r - https://eclipse.org/jgit/) + (New BSD License) JGit (org.eclipse.jgit:org.eclipse.jgit:jar:4.9.0.201710071750-r - https://eclipse.org/jgit/) (New BSD License) Kryo (com.esotericsoftware.kryo:kryo:3.0.3 - http://code.google.com/p/kryo/) (New BSD License) MinLog (com.esotericsoftware.minlog:minlog:1.3 - http://code.google.com/p/minlog/) (New BSD License) ReflectASM (com.esotericsoftware.reflectasm:reflectasm:1.07 - http://code.google.com/p/reflectasm/) diff --git a/zeppelin-zengine/pom.xml b/zeppelin-zengine/pom.xml index d1a227052c..a29d4a0955 100644 --- a/zeppelin-zengine/pom.xml +++ b/zeppelin-zengine/pom.xml @@ -46,7 +46,7 @@ 5.3.1 0.9.8 1.4.01 - 4.1.1.201511131810-r + 4.9.0.201710071750-r 1.3 From 4dc6bf57080e11ebf0533bbedcdba780c5e71398 Mon Sep 17 00:00:00 2001 From: liguohui Date: Tue, 28 Nov 2017 22:53:48 +0800 Subject: [PATCH 03/14] [ZEPPELIN-3076]Chart field is also draggable and sortable in the 'keys', 'groups' and 'values' ### What is this PR for? Current the `keys`, `groups` and `values` is only droppable. It is a little inconvenient if I want to drag it to other place or sort it. This feature let `keys`, `groups` and `values` not only `droppable`, but also `draggable` and `sorttable`. ### What type of PR is it? [Feature] ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3076 [ZEPPELIN-3076] ### How should this be tested? I have Add the Screenshots for the test. Just as the Screenshots showing, all the element in `keys`, `groups` and `values` is draggable and sorttable. ![untitled project4](https://user-images.githubusercontent.com/5969176/33252922-ed1ab0b4-d37b-11e7-8a5c-b3dbb6765d18.gif) ### Questions: * Does the licenses files need update? NO * Is there breaking changes for older versions? NO * Does this needs documentation? NO Author: liguohui Closes #2686 from liguohuicmss/draggable-field-keys-groups-values2 and squashes the following commits: 08b86b8 [liguohui] add a new line ebb8436 [liguohui] Chart field is also draggable and sortable in the 'keys', 'groups' and 'values' --- .../src/app/tabledata/pivot_settings.html | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/zeppelin-web/src/app/tabledata/pivot_settings.html b/zeppelin-web/src/app/tabledata/pivot_settings.html index abfe73069b..68de019c27 100644 --- a/zeppelin-web/src/app/tabledata/pivot_settings.html +++ b/zeppelin-web/src/app/tabledata/pivot_settings.html @@ -49,7 +49,11 @@ limitations under the License. data-drop="true" jqyoui-droppable="{multiple:true, onDrop:'save()'}" class="list-unstyled" style="border-radius: 6px; margin-top: 7px;"> -
  • +
  • {{item.name}} @@ -69,7 +73,11 @@ limitations under the License. jqyoui-droppable="{multiple:true, onDrop:'save()'}" class="list-unstyled" style="border-radius: 6px; margin-top: 7px;"> -
  • +
  • {{item.name}} @@ -89,7 +97,11 @@ limitations under the License. jqyoui-droppable="{multiple:true, onDrop:'save()'}" class="list-unstyled" style="border-radius: 6px; margin-top: 7px;"> -
  • +
  • + From 625b268553efe199710b772d4c589d8d526e366f Mon Sep 17 00:00:00 2001 From: Keiji Yoshida Date: Sat, 2 Dec 2017 00:55:06 +0900 Subject: [PATCH 04/14] [hotfix] Downgrade JGit from 4.9.0 to 4.5.4 ### What is this PR for? The version of JGit was updated to 4.9.0 at https://github.com/apache/zeppelin/pull/2658. However, this version does not support Java 7 and Travis CI test always fails now: https://travis-ci.org/kjmrknsn/zeppelin/builds/310104872 To fix this issue, downgrade JGit from 4.9.0 to 4.5.4 which is the latest JGit version which supports Java 7: https://projects.eclipse.org/projects/technology.jgit. I confirmed that Travis CI test was passed with JGit 4.5.4: https://travis-ci.org/kjmrknsn/zeppelin/builds/310107611 Now, all PRs to Zeppelin don't pass Travis CI tests, so it's preferable that this PR is merged soon. ### What type of PR is it? [Bug Fix] ### Todos ### What is the Jira issue? ### How should this be tested? I confirmed that Travis CI test passed with JGit 4.5.4: https://travis-ci.org/kjmrknsn/zeppelin/builds/310107611 ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? Yes. zeppelin-distribution/src/bin_license/LICENSE was updated at this PR. * Is there breaking changes for older versions? No. * Does this needs documentation? No. Author: Keiji Yoshida Closes #2693 from kjmrknsn/ci-test-against-jgit-4.5.4 and squashes the following commits: 27b08c9 [Keiji Yoshida] CI Test against JGit 4.5.4 --- zeppelin-distribution/src/bin_license/LICENSE | 2 +- zeppelin-zengine/pom.xml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/zeppelin-distribution/src/bin_license/LICENSE b/zeppelin-distribution/src/bin_license/LICENSE index 7b23313b21..37fbce1b1f 100644 --- a/zeppelin-distribution/src/bin_license/LICENSE +++ b/zeppelin-distribution/src/bin_license/LICENSE @@ -311,7 +311,7 @@ The text of each license is also included at licenses/LICENSE-[project]-[version The following components are provided under the BSD-style License. - (New BSD License) JGit (org.eclipse.jgit:org.eclipse.jgit:jar:4.9.0.201710071750-r - https://eclipse.org/jgit/) + (New BSD License) JGit (org.eclipse.jgit:org.eclipse.jgit:jar:4.5.4.201711221230-r - https://eclipse.org/jgit/) (New BSD License) Kryo (com.esotericsoftware.kryo:kryo:3.0.3 - http://code.google.com/p/kryo/) (New BSD License) MinLog (com.esotericsoftware.minlog:minlog:1.3 - http://code.google.com/p/minlog/) (New BSD License) ReflectASM (com.esotericsoftware.reflectasm:reflectasm:1.07 - http://code.google.com/p/reflectasm/) diff --git a/zeppelin-zengine/pom.xml b/zeppelin-zengine/pom.xml index a29d4a0955..fd2d1dca96 100644 --- a/zeppelin-zengine/pom.xml +++ b/zeppelin-zengine/pom.xml @@ -46,7 +46,7 @@ 5.3.1 0.9.8 1.4.01 - 4.9.0.201710071750-r + 4.5.4.201711221230-r 1.3 From 38ba2d47569300dd2a31bc9141d7439b0d370091 Mon Sep 17 00:00:00 2001 From: tinkoff-dwh Date: Thu, 30 Nov 2017 10:55:14 +0300 Subject: [PATCH 05/14] [Zeppelin-2571] & [Zeppelin-465] Run paragraphs: from first/current to current/last ### What is this PR for? This pr add the ability to run all paragraphs from the first to the current and from the current to the last. This makes it easier to update the data if changes are made in one of the related paragraphs. ### What type of PR is it? Feature ### What is the Jira issue? [ZEPPELIN-2571](https://issues.apache.org/jira/browse/ZEPPELIN-2571) - (Add a "Run to here" option on paragraphs) [ZEPPELIN-465](https://issues.apache.org/jira/browse/ZEPPELIN-465) - (Capability to run all cells below the current cell) ### How should this be tested? 1. Click on the "Run: from first to this" or "Run: from this to last" in the dropdown menu. Paragraphs from the first/current to the current/last will be started in order. 2. Press the key combination: CTRL + SHIFT + ENTER. A window will appear with the choice of the desired action. ![capture2](https://user-images.githubusercontent.com/25951039/33269914-58eff828-d393-11e7-9ebf-6437ec11c8f2.PNG) Choose one of two actions. The selected action will be performed. ### Screenshots (if appropriate) ![capture1_edit](https://user-images.githubusercontent.com/25951039/33269915-5951b19e-d393-11e7-831b-42d4523908ae.png) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: tinkoff-dwh Closes #2688 from tinkoff-dwh/ZEPPELIN-2571&465 and squashes the following commits: 304c196 [tinkoff-dwh] [ZEPPELIN-2517]&[ZEPPELIN-465] fix shortcut description 3d57b30 [tinkoff-dwh] [ZEPPELIN-2517]&[ZEPPELIN-465] save paragraph's focus 70f130c [tinkoff-dwh] [Zeppelin-2517]&[Zeppelin-465] some text change 8ca1df2 [tinkoff-dwh] [Zeppelin-2517]&[Zeppelin-465] line reduction a7a4158 [tinkoff-dwh] [ZEPPELIN-2517]&[ZEPPELIN-465] logic change 9fbcdb6 [tinkoff-dwh] [ZEPPELIN-2517]&[ZEPPELIN-465] add keyboard shortcut bfc3891 [tinkoff-dwh] [ZEPPELIN-2517]&[ZEPPELIN-465] Run paragraphs from 1st to this. From this to last --- .../src/app/notebook/notebook.controller.js | 90 ++++++++++++++++++- .../notebook/paragraph/paragraph-control.html | 18 ++++ .../paragraph/paragraph.controller.js | 56 ++++++++++-- zeppelin-web/src/app/notebook/shortcut.html | 11 +++ 4 files changed, 166 insertions(+), 9 deletions(-) diff --git a/zeppelin-web/src/app/notebook/notebook.controller.js b/zeppelin-web/src/app/notebook/notebook.controller.js index 48fc6e7154..1fa63231ea 100644 --- a/zeppelin-web/src/app/notebook/notebook.controller.js +++ b/zeppelin-web/src/app/notebook/notebook.controller.js @@ -164,7 +164,7 @@ function NotebookCtrl ($scope, $route, $routeParams, $location, $rootScope, for (let i = 0; i < $scope.note.paragraphs.length; i++) { let paragraphId = $scope.note.paragraphs[i].id if (jQuery.contains(angular.element('#' + paragraphId + '_container')[0], clickEvent.target)) { - $scope.$broadcast('focusParagraph', paragraphId, 0, true) + $scope.$broadcast('focusParagraph', paragraphId, 0, null, true) break } } @@ -512,7 +512,7 @@ function NotebookCtrl ($scope, $route, $routeParams, $location, $rootScope, para.focus = true // we need `$timeout` since angular DOM might not be initialized - $timeout(() => { $scope.$broadcast('focusParagraph', para.id, 0, false) }) + $timeout(() => { $scope.$broadcast('focusParagraph', para.id, 0, null, false) }) } }) } @@ -1188,6 +1188,92 @@ function NotebookCtrl ($scope, $route, $routeParams, $location, $rootScope, ** $scope.$on functions below */ + $scope.$on('runAllAbove', function (event, paragraph, isNeedConfirm) { + let allParagraphs = $scope.note.paragraphs + let toRunParagraphs = [] + + for (let i = 0; allParagraphs[i] !== paragraph; i++) { + if (i === allParagraphs.length - 1) { return } // if paragraph not in array of all paragraphs + toRunParagraphs.push(allParagraphs[i]) + } + + const paragraphs = toRunParagraphs.map(p => { + return { + id: p.id, + title: p.title, + paragraph: p.text, + config: p.config, + params: p.settings.params + } + }) + + if (!isNeedConfirm) { + websocketMsgSrv.runAllParagraphs($scope.note.id, paragraphs) + } else { + BootstrapDialog.confirm({ + closable: true, + title: '', + message: 'Run all above?', + callback: function (result) { + if (result) { + websocketMsgSrv.runAllParagraphs($scope.note.id, paragraphs) + } + } + }) + } + + $scope.saveCursorPosition(paragraph) + }) + + $scope.$on('runAllBelowAndCurrent', function (event, paragraph, isNeedConfirm) { + let allParagraphs = $scope.note.paragraphs + let toRunParagraphs = [] + + for (let i = allParagraphs.length - 1; allParagraphs[i] !== paragraph; i--) { + if (i < 0) { return } // if paragraph not in array of all paragraphs + toRunParagraphs.push(allParagraphs[i]) + } + + toRunParagraphs.push(paragraph) + toRunParagraphs.reverse() + + const paragraphs = toRunParagraphs.map(p => { + return { + id: p.id, + title: p.title, + paragraph: p.text, + config: p.config, + params: p.settings.params + } + }) + + if (!isNeedConfirm) { + websocketMsgSrv.runAllParagraphs($scope.note.id, paragraphs) + } else { + BootstrapDialog.confirm({ + closable: true, + title: '', + message: 'Run current and all below?', + callback: function (result) { + if (result) { + websocketMsgSrv.runAllParagraphs($scope.note.id, paragraphs) + } + } + }) + } + + $scope.saveCursorPosition(paragraph) + }) + + $scope.saveCursorPosition = function (paragraph) { + let angParagEditor = angular + .element('#' + paragraph.id + '_paragraphColumn_main') + .scope().editor + let col = angParagEditor.selection.lead.column + let row = angParagEditor.selection.lead.row + $scope.$broadcast('focusParagraph', paragraph.id, row + 1, col) + } + $scope.$on('setConnectedStatus', function (event, param) { if (connectedOnce && param) { initNotebook() diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html b/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html index d6599725cc..0b4ca1e4d6 100644 --- a/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html +++ b/zeppelin-web/src/app/notebook/paragraph/paragraph-control.html @@ -140,6 +140,24 @@ limitations under the License. Insert new
    • +
  • + + + + Ctrl+Shift+Enter + Run all above + +
    • +
  • + + + + Ctrl+Shift+Enter + Run all below + +
  • Ctrl+Shift+C diff --git a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js index c5788416ad..d3ed3466e7 100644 --- a/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js +++ b/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js @@ -471,6 +471,43 @@ function ParagraphCtrl ($scope, $rootScope, $route, $window, $routeParams, $loca $scope.runParagraph($scope.getEditorValue(), false, false) } + $scope.runAllToThis = function(paragraph) { + $scope.$emit('runAllAbove', paragraph, true) + } + + $scope.runAllFromThis = function(paragraph) { + $scope.$emit('runAllBelowAndCurrent', paragraph, true) + } + + $scope.runAllToOrFromThis = function (paragraph) { + BootstrapDialog.show({ + message: 'Run paragraphs:', + title: '', + buttons: [{ + label: 'Close', + action: function(dialog) { + dialog.close() + } + }, + { + label: 'Run all above', + cssClass: 'btn-primary', + action: function(dialog) { + $scope.$emit('runAllAbove', paragraph, false) + dialog.close() + } + }, + { + label: 'Run current and all below', + cssClass: 'btn-primary', + action: function(dialog) { + $scope.$emit('runAllBelowAndCurrent', paragraph, false) + dialog.close() + } + }] + }) + } + $scope.turnOnAutoRun = function (paragraph) { paragraph.config.runOnSelectionChange = !paragraph.config.runOnSelectionChange commitParagraph(paragraph) @@ -1446,8 +1483,10 @@ function ParagraphCtrl ($scope, $rootScope, $route, $window, $routeParams, $loca // move focus to next paragraph // $timeout stops chaining effect of focus propogation $timeout(() => $scope.$emit('moveFocusToNextParagraph', paragraphId)) - } else if (keyEvent.shiftKey && keyCode === 13) { // Shift + Enter + } else if (!keyEvent.ctrlKey && keyEvent.shiftKey && keyCode === 13) { // Shift + Enter $scope.runParagraphFromShortcut($scope.getEditorValue()) + } else if (keyEvent.ctrlKey && keyEvent.shiftKey && keyCode === 13) { // Ctrl + Shift + Enter + $scope.runAllToOrFromThis($scope.paragraph) } else if (keyEvent.ctrlKey && keyEvent.altKey && keyCode === 67) { // Ctrl + Alt + c $scope.cancelParagraph($scope.paragraph) } else if (keyEvent.ctrlKey && keyEvent.altKey && keyCode === 68) { // Ctrl + Alt + d @@ -1500,7 +1539,10 @@ function ParagraphCtrl ($scope, $rootScope, $route, $window, $routeParams, $loca } }) - $scope.$on('focusParagraph', function (event, paragraphId, cursorPos, mouseEvent) { + $scope.$on('focusParagraph', function (event, paragraphId, cursorPosRow, cursorPosCol, mouseEvent) { + if (cursorPosCol === null || cursorPosCol === undefined) { + cursorPosCol = 0 + } if ($scope.paragraph.id === paragraphId) { // focus editor if (!$scope.paragraph.config.editorHide) { @@ -1508,14 +1550,14 @@ function ParagraphCtrl ($scope, $rootScope, $route, $window, $routeParams, $loca $scope.editor.focus() // move cursor to the first row (or the last row) let row - if (cursorPos >= 0) { - row = cursorPos - $scope.editor.gotoLine(row, 0) + if (cursorPosRow >= 0) { + row = cursorPosRow + $scope.editor.gotoLine(row, cursorPosCol) } else { row = $scope.editor.session.getLength() - $scope.editor.gotoLine(row, 0) + $scope.editor.gotoLine(row, cursorPosCol) } - $scope.scrollToCursor($scope.paragraph.id, 0) + $scope.scrollToCursor($scope.paragraph.id, cursorPosCol) } } handleFocus(true) diff --git a/zeppelin-web/src/app/notebook/shortcut.html b/zeppelin-web/src/app/notebook/shortcut.html index c4b4009ee1..9bc55973a5 100644 --- a/zeppelin-web/src/app/notebook/shortcut.html +++ b/zeppelin-web/src/app/notebook/shortcut.html @@ -37,6 +37,17 @@ limitations under the License. + + +
    Run all above/below paragraphs
    + + +
    + Ctrl + Shift + Enter +
    + + +
    Cancel
    From a82e3ec3a335863a30959cabbc3371169e2c1ce9 Mon Sep 17 00:00:00 2001 From: liguohui Date: Thu, 30 Nov 2017 15:01:27 +0800 Subject: [PATCH 06/14] [ZEPPELIN-3075]Fix unqiue algo for the web side in pivot.js file. ### What is this PR for? unique() algorithm is not correct in pivot.js file. If the input is `[2, 3, 3, 3, 4, 5]` and the output will be `[2, 3, 3, 4, 5]`. The number `3` is still duplicated. ### What type of PR is it? [Bug Fix] ### Todos * [ ] - Task ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3075 [ZEPPELIN-3075] ### How should this be tested? This is very easy and no need test. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: liguohui Closes #2685 from liguohuicmss/pivot-unqiue-algo and squashes the following commits: 2063175d5 [liguohui] delete the multi empty line ea582d95e [liguohui] delete some spaces at the end of the line 79c763a6b [liguohui] add a empty line 99cf93da9 [liguohui] Revert "Chart field is also draggable and sortable in the 'keys', 'groups' and 'values'" fdde39f52 [liguohui] add unit test for unique algo in pivot.js f99674724 [liguohui] Chart field is also draggable and sortable in the 'keys', 'groups' and 'values' 943e80a96 [liguohui] Fix unqiue algo for the web side in pivot.js file. --- zeppelin-web/src/app/tabledata/pivot.js | 1 + .../src/app/tabledata/tabledata.test.js | 45 +++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/zeppelin-web/src/app/tabledata/pivot.js b/zeppelin-web/src/app/tabledata/pivot.js index 1c938ea828..a0f61b219a 100644 --- a/zeppelin-web/src/app/tabledata/pivot.js +++ b/zeppelin-web/src/app/tabledata/pivot.js @@ -89,6 +89,7 @@ export default class PivotTransformation extends Transformation { for (let j = i + 1; j < list.length; j++) { if (angular.equals(list[i], list[j])) { list.splice(j, 1) + j-- } } } diff --git a/zeppelin-web/src/app/tabledata/tabledata.test.js b/zeppelin-web/src/app/tabledata/tabledata.test.js index 7e41de4bec..3de2fa3fb7 100644 --- a/zeppelin-web/src/app/tabledata/tabledata.test.js +++ b/zeppelin-web/src/app/tabledata/tabledata.test.js @@ -13,6 +13,7 @@ */ import TableData from './tabledata.js' +import PivotTransformation from './pivot.js' describe('TableData build', function () { let td @@ -39,3 +40,47 @@ describe('TableData build', function () { expect(td.comment).toBe('hello') }) }) + +describe('PivotTransformation build', function() { + let pt + + beforeEach(function () { + console.log(PivotTransformation) + pt = new PivotTransformation() + }) + + it('check the result of keys, groups and values unique', function() { + // set inited mock data + let config = { + common: { + pivot: { + keys: [{index: 4, name: '4'}, + {index: 3, name: '3'}, + {index: 4, name: '4'}, + {index: 3, name: '3'}, + {index: 3, name: '3'}, + {index: 3, name: '3'}, + {index: 3, name: '3'}, + {index: 5, name: '5'}], + groups: [], + values: [] + } + } + } + pt.tableDataColumns = [ + {index: 1, name: '1'}, + {index: 2, name: '2'}, + {index: 3, name: '3'}, + {index: 4, name: '4'}, + {index: 5, name: '5'}] + + pt.setConfig(config) + + pt.removeUnknown() + + expect(config.common.pivot.keys.length).toBe(3) + expect(config.common.pivot.keys[0].index).toBe(4) + expect(config.common.pivot.keys[1].index).toBe(3) + expect(config.common.pivot.keys[2].index).toBe(5) + }) +}) From 4a679fc055a7e6b03faa3c215562cc295714bc99 Mon Sep 17 00:00:00 2001 From: tinkoff-dwh Date: Thu, 7 Dec 2017 03:25:16 +0500 Subject: [PATCH 07/14] [FIX] fix autocomplete ### What is this PR for? After refactoring of Interpreter autocomplete (from server side) not works without first Run of interpreter. This PR fix it. ### What type of PR is it? [Fix] ### How should this be tested? * Create new Note (JDBC interpreter), try to use autocomplete (schema, tables) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: tinkoff-dwh Closes #2691 from tinkoff-dwh/fix_autocomplete and squashes the following commits: e9bad01 [tinkoff-dwh] remove trim from completion 141dff5 [tinkoff-dwh] [FIX] fix autocomplete --- .../interpreter/remote/RemoteInterpreter.java | 3 +- .../apache/zeppelin/notebook/Paragraph.java | 34 ++++++++----------- .../zeppelin/notebook/ParagraphTest.java | 4 +-- 3 files changed, 17 insertions(+), 24 deletions(-) diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java index 8964210c1c..4ad36cf1b2 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java @@ -346,8 +346,7 @@ public class RemoteInterpreter extends Interpreter { final InterpreterContext interpreterContext) throws InterpreterException { if (!isOpened) { - LOGGER.warn("completion is called when RemoterInterpreter is not opened for " + className); - return new ArrayList<>(); + open(); } RemoteInterpreterProcess interpreterProcess = null; try { diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java index 10a8548798..5ec132931f 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java @@ -191,7 +191,7 @@ public class Paragraph extends Job implements Cloneable, JsonSerializable { this.scriptText = this.text.substring(headingSpace.length() + intpText.length() + 1).trim(); } else { this.intpText = ""; - this.scriptText = this.text; + this.scriptText = this.text.trim(); } } } @@ -250,14 +250,17 @@ public class Paragraph extends Job implements Cloneable, JsonSerializable { return note.getInterpreterCompletion(); } } - String trimmedBuffer = buffer != null ? buffer.trim() : null; - cursor = calculateCursorPosition(buffer, trimmedBuffer, cursor); + this.interpreter = getBindedInterpreter(); + + setText(buffer); + + cursor = calculateCursorPosition(buffer, cursor); InterpreterContext interpreterContext = getInterpreterContextWithoutRunner(null); try { if (this.interpreter != null) { - return this.interpreter.completion(scriptText, cursor, interpreterContext); + return this.interpreter.completion(this.scriptText, cursor, interpreterContext); } else { return null; } @@ -266,24 +269,15 @@ public class Paragraph extends Job implements Cloneable, JsonSerializable { } } - public int calculateCursorPosition(String buffer, String trimmedBuffer, int cursor) { - int countWhitespacesAtStart = buffer.indexOf(trimmedBuffer); - if (countWhitespacesAtStart > 0) { - cursor -= countWhitespacesAtStart; - } + public int calculateCursorPosition(String buffer, int cursor) { + // scriptText trimmed - // parse text to get interpreter component - String repl = null; - if (trimmedBuffer != null) { - Matcher matcher = REPL_PATTERN.matcher(trimmedBuffer); - if (matcher.matches()) { - repl = matcher.group(2); - } + if (this.scriptText.isEmpty()) { + return 0; } - - if (repl != null && cursor > repl.length()) { - String body = trimmedBuffer.substring(repl.length() + 1); - cursor -= repl.length() + 1 + body.indexOf(body.trim()); + int countCharactersBeforeScript = buffer.indexOf(this.scriptText); + if (countCharactersBeforeScript > 0) { + cursor -= countCharactersBeforeScript; } return cursor; diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/ParagraphTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/ParagraphTest.java index 9e9ce27ea9..e46b7393ee 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/ParagraphTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/ParagraphTest.java @@ -271,7 +271,6 @@ public class ParagraphTest extends AbstractInterpreterTest { @Test public void testCursorPosition() { Paragraph paragraph = spy(new Paragraph()); - doReturn(null).when(paragraph).getIntpText(); // left = buffer, middle = cursor position into source code, right = cursor position after parse List> dataSet = Arrays.asList( Triple.of("%jdbc schema.", 13, 7), @@ -294,7 +293,8 @@ public class ParagraphTest extends AbstractInterpreterTest { ); for (Triple data : dataSet) { - Integer actual = paragraph.calculateCursorPosition(data.getLeft(), data.getLeft().trim(), data.getMiddle()); + paragraph.setText(data.getLeft()); + Integer actual = paragraph.calculateCursorPosition(data.getLeft(), data.getMiddle()); assertEquals(data.getRight(), actual); } } From 3505625c26a45df14d6412b73da0b33aac68e908 Mon Sep 17 00:00:00 2001 From: Jeff Zhang Date: Mon, 11 Dec 2017 20:53:08 +0800 Subject: [PATCH 08/14] [HOTFIX]: Fix IPythonInterpreter unit test ### What is this PR for? This is for hotfix of `IPythonInterpreter` unit test failure. Just specify the version of ipython in `install_external_dependencies.sh`, otherwise latest ipython version will be installed, and the behavior may change. ### What type of PR is it? [Hot Fix] ### Todos * [ ] - Task ### What is the Jira issue? * ### How should this be tested? * CI pass ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang Closes #2703 from zjffdu/ipython_version and squashes the following commits: b4c7b42 [Jeff Zhang] HotFix: Fix IPythonInterpreter unit test --- testing/install_external_dependencies.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/testing/install_external_dependencies.sh b/testing/install_external_dependencies.sh index daa670bc7d..e34296e3ab 100755 --- a/testing/install_external_dependencies.sh +++ b/testing/install_external_dependencies.sh @@ -44,6 +44,6 @@ if [[ -n "$PYTHON" ]] ; then conda update -q conda conda info -a conda config --add channels conda-forge - conda install -q matplotlib pandasql ipython jupyter_client ipykernel matplotlib bokeh=0.12.6 + conda install -q matplotlib pandasql ipython=5.4.1 jupyter_client ipykernel matplotlib bokeh=0.12.6 pip install -q grpcio ggplot fi From 549bce6738ffd7f460867d3f5ee00a9e2ec14125 Mon Sep 17 00:00:00 2001 From: Liu Date: Tue, 5 Dec 2017 15:24:20 +0800 Subject: [PATCH 09/14] [ZEPPELIN-3014] NPE bug fix and Error message enhancement with Kylin Interpreter ### What is this PR for? A few sentences describing the overall goals of the pull request's commits. First time? Check out the contributing guide - https://zeppelin.apache.org/contribution/contributions.html ### What type of PR is it? Bug Fix ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3014 ### How should this be tested? * Setup Travis CI as described on https://zeppelin.apache.org/contribution/contributions.html#continuous-integration * Use existing unit tests in kylin module. ### Screenshots (if appropriate) #### before: NPE when result set is empty ![image](https://user-images.githubusercontent.com/18542573/32154048-f1b8ba58-bcfb-11e7-98cc-98cdf484f2d5.png) #### after: no NPE when result set is empty, just an empty table ![image](https://user-images.githubusercontent.com/18542573/32154069-110215d0-bcfc-11e7-87e9-cc049001f1c7.png) #### before: when query fails, only error code is returned, no error message ![image](https://user-images.githubusercontent.com/18542573/32154088-29651938-bcfc-11e7-9e66-cd2cfccba054.png) #### after: when query fails, both error code and error message are displayed to users ![image](https://user-images.githubusercontent.com/18542573/32154096-3d3ab01c-bcfc-11e7-8cf3-d710d96b8c5a.png) ### Questions: * Does the licenses files need update? No. * Is there breaking changes for older versions? No. * Does this needs documentation? No. Author: Liu Closes #2645 from jinxliu/kylin-intp-new and squashes the following commits: d5692bf [Liu] refactor 85b6424 [Liu] add test for empty result set 4596470 [Liu] ZEPPELIN-3014: NPE bug fix and Error message enhancement with Kylin Interpreter --- .../zeppelin/kylin/KylinErrorResponse.java | 63 ++++++++++++++++ .../zeppelin/kylin/KylinInterpreter.java | 71 +++++++++++-------- .../zeppelin/kylin/KylinInterpreterTest.java | 24 +++++++ 3 files changed, 130 insertions(+), 28 deletions(-) create mode 100644 kylin/src/main/java/org/apache/zeppelin/kylin/KylinErrorResponse.java diff --git a/kylin/src/main/java/org/apache/zeppelin/kylin/KylinErrorResponse.java b/kylin/src/main/java/org/apache/zeppelin/kylin/KylinErrorResponse.java new file mode 100644 index 0000000000..00439e8c62 --- /dev/null +++ b/kylin/src/main/java/org/apache/zeppelin/kylin/KylinErrorResponse.java @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.kylin; + +import com.google.gson.Gson; +import com.google.gson.JsonSyntaxException; +import org.apache.zeppelin.common.JsonSerializable; + +/** + * class for Kylin Error Response. + */ +class KylinErrorResponse implements JsonSerializable { + private static final Gson gson = new Gson(); + + private String stacktrace; + private String exception; + private String url; + private String code; + private Object data; + private String msg; + + public KylinErrorResponse(String stacktrace, String exception, String url, + String code, Object data, String msg) { + this.stacktrace = stacktrace; + this.exception = exception; + this.url = url; + this.code = code; + this.data = data; + this.msg = msg; + } + + public String getException() { + return exception; + } + + public String toJson() { + return gson.toJson(this); + } + + public static KylinErrorResponse fromJson(String json) { + try { + return gson.fromJson(json, KylinErrorResponse.class); + } catch (JsonSyntaxException ex) { + return null; + } + } + +} diff --git a/kylin/src/main/java/org/apache/zeppelin/kylin/KylinInterpreter.java b/kylin/src/main/java/org/apache/zeppelin/kylin/KylinInterpreter.java index 6b68d288e4..c7cd689a74 100755 --- a/kylin/src/main/java/org/apache/zeppelin/kylin/KylinInterpreter.java +++ b/kylin/src/main/java/org/apache/zeppelin/kylin/KylinInterpreter.java @@ -18,6 +18,7 @@ package org.apache.zeppelin.kylin; import org.apache.commons.codec.binary.Base64; +import org.apache.commons.io.IOUtils; import org.apache.http.HttpResponse; import org.apache.http.client.HttpClient; import org.apache.http.client.methods.HttpPost; @@ -30,9 +31,7 @@ import org.apache.zeppelin.interpreter.thrift.InterpreterCompletion; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import java.io.BufferedReader; import java.io.IOException; -import java.io.InputStreamReader; import java.util.List; import java.util.Properties; import java.util.regex.Matcher; @@ -166,28 +165,42 @@ public class KylinInterpreter extends Interpreter { } private InterpreterResult executeQuery(String sql) throws IOException { - HttpResponse response = prepareRequest(sql); + String result; - if (response.getStatusLine().getStatusCode() != 200) { - logger.error("failed to execute query: " + response.getEntity().getContent().toString()); - return new InterpreterResult(InterpreterResult.Code.ERROR, - "Failed : HTTP error code " + response.getStatusLine().getStatusCode()); + try { + int code = response.getStatusLine().getStatusCode(); + result = IOUtils.toString(response.getEntity().getContent(), "UTF-8"); + + if (code != 200) { + StringBuilder errorMessage = new StringBuilder("Failed : HTTP error code " + code + " ."); + logger.error("Failed to execute query: " + result); + + KylinErrorResponse kylinErrorResponse = KylinErrorResponse.fromJson(result); + if (kylinErrorResponse == null) { + logger.error("Cannot get json from string: " + result); + // when code is 401, the response is html, not json + if (code == 401) { + errorMessage.append(" Error message: Unauthorized. This request requires " + + "HTTP authentication. Please make sure your have set your credentials " + + "correctly."); + } else { + errorMessage.append(" Error message: " + result + " ."); + } + } else { + String exception = kylinErrorResponse.getException(); + logger.error("The exception is " + exception); + errorMessage.append(" Error message: " + exception + " ."); + } + + return new InterpreterResult(InterpreterResult.Code.ERROR, errorMessage.toString()); + } + } catch (NullPointerException | IOException e) { + throw new IOException(e); } - BufferedReader br = new BufferedReader( - new InputStreamReader((response.getEntity().getContent()))); - StringBuilder sb = new StringBuilder(); - - String output; - logger.info("Output from Server .... \n"); - while ((output = br.readLine()) != null) { - logger.info(output); - sb.append(output).append('\n'); - } - InterpreterResult rett = new InterpreterResult(InterpreterResult.Code.SUCCESS, - formatResult(sb.toString())); - return rett; + return new InterpreterResult(InterpreterResult.Code.SUCCESS, + formatResult(result)); } String formatResult(String msg) { @@ -205,16 +218,18 @@ public class KylinInterpreter extends Interpreter { table = mr.group(1); } - String[] row = table.split("],\\["); - for (int i = 0; i < row.length; i++) { - String[] col = row[i].split(",(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)", -1); - for (int j = 0; j < col.length; j++) { - if (col[j] != null) { - col[j] = col[j].replaceAll("^\"|\"$", ""); + if (table != null && !table.isEmpty()) { + String[] row = table.split("],\\["); + for (int i = 0; i < row.length; i++) { + String[] col = row[i].split(",(?=(?:[^\"]*\"[^\"]*\")*[^\"]*$)", -1); + for (int j = 0; j < col.length; j++) { + if (col[j] != null) { + col[j] = col[j].replaceAll("^\"|\"$", ""); + } + res.append(col[j] + " \t"); } - res.append(col[j] + " \t"); + res.append(" \n"); } - res.append(" \n"); } return res.toString(); } diff --git a/kylin/src/test/java/org/apache/zeppelin/kylin/KylinInterpreterTest.java b/kylin/src/test/java/org/apache/zeppelin/kylin/KylinInterpreterTest.java index 4471a07689..35f0f3c2eb 100755 --- a/kylin/src/test/java/org/apache/zeppelin/kylin/KylinInterpreterTest.java +++ b/kylin/src/test/java/org/apache/zeppelin/kylin/KylinInterpreterTest.java @@ -108,6 +108,30 @@ public class KylinInterpreterTest { Assert.assertEquals(expected, actual); } + @Test + public void testParseEmptyResult() { + String msg = "{\"columnMetas\":[{\"isNullable\":1,\"displaySize\":256,\"label\":\"COUNTRY\",\"name\":\"COUNTRY\"," + + "\"schemaName\":\"DEFAULT\",\"catelogName\":null,\"tableName\":\"SALES_TABLE\",\"precision\":256," + + "\"scale\":0,\"columnType\":12,\"columnTypeName\":\"VARCHAR\",\"writable\":false,\"readOnly\":true," + + "\"definitelyWritable\":false,\"autoIncrement\":false,\"caseSensitive\":true,\"searchable\":false," + + "\"currency\":false,\"signed\":true},{\"isNullable\":1,\"displaySize\":256,\"label\":\"CURRENCY\"," + + "\"name\":\"CURRENCY\",\"schemaName\":\"DEFAULT\",\"catelogName\":null,\"tableName\":\"SALES_TABLE\"," + + "\"precision\":256,\"scale\":0,\"columnType\":12,\"columnTypeName\":\"VARCHAR\",\"writable\":false," + + "\"readOnly\":true,\"definitelyWritable\":false,\"autoIncrement\":false,\"caseSensitive\":true," + + "\"searchable\":false,\"currency\":false,\"signed\":true},{\"isNullable\":0,\"displaySize\":19," + + "\"label\":\"COUNT__\",\"name\":\"COUNT__\",\"schemaName\":\"DEFAULT\",\"catelogName\":null," + + "\"tableName\":\"SALES_TABLE\",\"precision\":19,\"scale\":0,\"columnType\":-5,\"columnTypeName\":" + + "\"BIGINT\",\"writable\":false,\"readOnly\":true,\"definitelyWritable\":false,\"autoIncrement\":false," + + "\"caseSensitive\":true,\"searchable\":false,\"currency\":false,\"signed\":true}],\"results\":" + + "[]," + "\"cube\":\"Sample_Cube\",\"affectedRowCount\":0,\"isException\":false,\"exceptionMessage\":null," + + "\"duration\":134,\"totalScanCount\":1,\"hitExceptionCache\":false,\"storageCacheUsed\":false," + + "\"partial\":false}"; + String expected="%table COUNTRY \tCURRENCY \tCOUNT__ \t \n"; + KylinInterpreter t = new MockKylinInterpreter(getDefaultProperties()); + String actual = t.formatResult(msg); + Assert.assertEquals(expected, actual); + } + private Properties getDefaultProperties() { Properties prop = new Properties(); prop.put("kylin.api.username", "ADMIN"); From 13f8e6cc65ffa629e24b6947cbd2ee63ba8159f2 Mon Sep 17 00:00:00 2001 From: Jeff Zhang Date: Sun, 12 Nov 2017 09:18:41 +0800 Subject: [PATCH 10/14] ZEPPELIN-3085 Introduce generic ConfInterpreter for more fine-grained control of interpreter setting ### What is this PR for? Zeppelin's interpreter setting is shared by all the users and notes, if you want to have different setting you have to create new interpreter, e.g. you can create `spark_jar1` for running spark with dependency jar1 and `spark_jar2` for running spark with dependency jar2. This approach works, but not so convenient. `ConfInterpreter` can provide more fine-grained control on interpreter setting and more flexibility. `ConfInterpreter` is a generic interpreter that could be used by any interpreters. The input format should be property file format. In the first paragraph, we use ConfInterpreter to make custom configuration of spark interpreter (set app name, yarn-client mode & add spark-csv dependencies). Then you can run the second paragraph which use spark-csv. ![conf_interpreter](https://user-images.githubusercontent.com/164491/33419465-74a3fae8-d5e5-11e7-8b25-76407804d979.png) It can be used to make custom setting for any interpreter. `ConfInterpreter` would run in the zeppelin server side, it would update the interpreter properties before you launch the interpreter process, so it needs to run before interpreter process launched. And when interpreter process is launched is determined by interpreter mode setting. So users needs to understand the interpreter mode setting of zeppelin and be aware when interpreter process is launched. E.g. If we set spark interpreter setting as isolated per note. Under this setting, each note will launch one interpreter process. In this scenario, user need to put `ConfInterpreter` as the first paragraph as the above example. Otherwise the customized setting can not be applied (The paragraph using ConfInterpreter will report ERROR). ### What type of PR is it? [Feature | Documentation] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3085 ### How should this be tested? * Unit test, System test is added, also manually verified it. ### Screenshots (if appropriate) ![conf_interpreter](https://user-images.githubusercontent.com/164491/33419465-74a3fae8-d5e5-11e7-8b25-76407804d979.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang Closes #2692 from zjffdu/ZEPPELIN-3085 and squashes the following commits: 87ce20f [Jeff Zhang] ZEPPELIN-3085. Introduce generic ConfInterpreter for more fine-grained control of interpreter setting --- .../img/screenshots/conf_interpreter.png | Bin 0 -> 39388 bytes docs/usage/interpreter/overview.md | 13 +++ .../rest/ZeppelinSparkClusterTest.java | 30 ++++++ .../zeppelin/interpreter/ConfInterpreter.java | 92 ++++++++++++++++ .../interpreter/InterpreterSetting.java | 43 ++++++-- .../interpreter/ManagedInterpreterGroup.java | 8 +- .../interpreter/remote/RemoteInterpreter.java | 7 +- .../apache/zeppelin/notebook/Paragraph.java | 1 + .../interpreter/ConfInterpreterTest.java | 102 ++++++++++++++++++ .../ManagedInterpreterGroupTest.java | 4 +- 10 files changed, 287 insertions(+), 13 deletions(-) create mode 100644 docs/assets/themes/zeppelin/img/screenshots/conf_interpreter.png create mode 100644 zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ConfInterpreter.java create mode 100644 zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ConfInterpreterTest.java diff --git a/docs/assets/themes/zeppelin/img/screenshots/conf_interpreter.png b/docs/assets/themes/zeppelin/img/screenshots/conf_interpreter.png new file mode 100644 index 0000000000000000000000000000000000000000..156c3575c9ba289702b2e0cf05725f5615f730a0 GIT binary patch literal 39388 zcmeFZ=R2I=_coje@C#?I0D!i9%XQ9yIsJ3wZ| zdJhoy{XTt1$xohzO7uMR@rIJwRfyoJDQV{!au z%)gs!-*fR388M1gVu`do6T@8midFleI;&svaVc_u6gx!4v^74Ekg=H=fUg5>QmxiO zUnRvdU%Z)GCc+{y?-3UBMM3D>$FM9G>TDxjb|Ld34GL*(I-hFOnP+VQmg6H+Cy>%w zW%e6fZk%u7%Y}4bVlHkK0DHVI`X9z}Dq&4HNN-nb#UK9`^Zz+#h5qHC7!+k^vnV`2 z^o0tjZ~rCvYD{nYzVa2ac=@PhmZm)py1Mk^ZJO6qR)y>4n-hprtL-O()2{+l3Pkba z<{LAzkQC3#obGRRXdFcO=%0B9CveT3Sxiz|>@_9oZ5PA&&^6Om!VCN+|ufZ zyFn`-r3u3GB41E4E8Y67W5v+;_ViZo-?dUEJ3DfeE#U~$)a_1pvXg%1&dL&2DzzuT z@A=VTgWpRM*c~Ffu@<;W_?RSe`6w?rA@s)Y&@aK+%1=lu^CuOPYF97awr5>Z4Hk{d zVK5b2Fn7vdMX6t5ks0Pxzll2x(aSo04*%-@9%f8ktQ#4nJt0zu#U1Fj_pt;*0zXL? zhXgm#!)d7azds0Tq}6DoK~$J8@7$83v28K@u&uPn?>wqXLAExv zC7k^EV;Oqzv#(0 z%DnNY!?MC`O;|>^X8v{t!l>`S@&I=vK@vK$&z%ikG}^9p?=N4DAf-LLot@yey=cU!9F)|RvI5&Rmp671Wl%0jraH7gFubi5^}J*^bT_O@xRDRc zf85cC1b2gx;68AxIYq2oPsT#ioTw|Ohh+!Jp1^7{6L8b-RxM~P$)QB;F1OhwD=^7XHsY;>d0cr53fGH zaFO!dAKHe;t|6Nj+5h^|RaeS?j<&eC-d~{j=e(a!&)>Ib-@W}35%QOp{<*q8cl)nH z(S|>E!^Ci}**!{6;zbYSe9ATTy!4m(U*$KWc+%P`$iWJIdpA2s&QSii|L!Ns%SDq^ zEd0aomj61~)rVxt!QT6s3IU`);TsA3bMTN>6{I7C&?gs@6{F1|dhc&528F%PXtA<- zBwe$x`Hy#<87&RXc5A zlkVT(+FD5dO!>EQvtH9&-4}SZ@WSER-we`BK^b&3`0a0t{)bukB83u_LxZiz`LXa1 z*R0X|mKaOc>*vPZzqa_m^LV!4brehKi@!za5$)CeZS`jqe;ZDx{47A3OSXSh|IMQB zEzhPNWcTuKCZMEEx)1{SAZ1Z?^KVhNCG>cyGWTbf1BY2iURHTx0MP| zNvY?@1-}G^Dw#px&1OhEesk_uSYlFA63(CD&Od>uvwgEV+Zx(gidwGlBe<_mH=2fr zhx-ZmclV?Z-d6H+Fh|Ufx){D3K|pUM)w1qb2%3w3y}9aIItn#ayuUe;{yA=~9=-Ne zb;b@e%>}d{HuRYsdkHe4(|%S^bp1SoWVO)B69?{X963NtWEx(9<)jT>Zol=zs;j9* z_k$y%E5sSEu4_tNlz< zXrIE~b66NIS`OduR6CXbRTS~k8BxHi(RsVN#rW>) zOA;>4$2BS2x*DymDFhK#X~)%G<%5G2h}Y@yo{*FjaI=YAHc{hLyo^i>z%;onD8Eoq z!JU=A1BXQ1QkM%TcP!<8^D!x{t*tG6wv#8>?5#~=l@FfS#Rn-gEH$y8s4A)zMJZ5U zxixD>7%BVm`3v-Fo+zNob$@{8^Y33JHx~f{go-%ekUev zcXA5pHCInZlDG3-j;23M_nGz^zoW3$WDy@9pOH8ASl}p=Yyn zE|#}Gkbt!48|FbG^!9Awso%r$(^+L)vgzWI*AnGS>)Hv^i7xa zy2@YD5$5(a$*!%1Exj^35kFFC;4|>-$+Y18c%Rv*sF6{9Y4teRP%gctje`qjT7S_X z;J%?1xqj1{?1mY#^aHK`4%{x+KON*$DU4EuZ?NyqK_jvHH#k{xkaF?BI%D>{qiY33 zQ)!`t$llL8BUWX1&&xIYOshi$F&qKw;+vNB-W!27=1qZh@Z9gH-@+`x2TLsD#||7p z`)~MkhH70b`E=lK+arYQqJvZ6=F+PRvSkRYU_oPy{LU9+lhXLNR#wD~+8O^i!jTC+ zU5|BoMQv&ducxq!ztbq=P#XQytgZ}2!chNy`b_;H+`aN2z$t}NUK%!}=f}w5VJ6lS^}s0u@JdX09k$ANnleyyKMLnr(DY@s zz-zhHq}Dw;^Jf!OzL>;QcKQ^YI--vj>vkC26}SJn&u$)!e#O^+U2fscNK-bP1&tQLeRxgGW7SSfqCPM|Yx-+@>~B z$(uSVL-QR7KE^Yu)$Z1w$70tFVD}p99jsgWv}RIi2P%VBgR29!t853J`pkf9?{RZC zE;%FCcndP5kBgu#t(;vy0V5r$^wmJZ;ka5=;R&g)psD< zM4=T6IPR-hldBEW_|m?7Pd~6w=SfeJs0W2XT@k@xxvM#Enz^Xpri*??IAPakiz;>M z_(fh?qa#Abr0C}H^nm)FQEUd-!xzKrG{rqN_A>)(k(5`w@{$L8WiVgr8MC!0I*`Dv z8SGg-A5%7@&aMJo=7tj&*wAuRZEUr}Js#?Uzngl#+)1>Td5&4Rq9+SqoJ8J8Mht1V z(=g2)cx)IDZ3lq*8pp+k^t$E+S^9~P;6dWjY^I>!$!jFX45smpDbmC~u&(O_eT6aNnQ3ilQD z&CW*f#0)OiI)1uyZv8;@qy^7zb?HjxFFbi3XFHf1K3Zm8EMi(yXoE~`s(gsXqG7Pp zlfCA?AW84FawxDEdbUZ&cQ7e}4x=U87V7Higv2?7h|hJBfk z)j0-TYq1-84c`yFZ3|6JMs*-_9>m`pr#fFJSZq)U3mNp@{+*r-0l z7Q>it+*qm|l5QD*gS7d|t&*?7d&0)Sw(D_@nX_T$O}zlXAH#Qh&0T(rj>+`y`*cTR zm6>3B_fZdt%~Qp1Wx4=Ep@w&bShK+CE`!8^%9>`MNw+Y^*TMp_s=@f9p`w*GIx&Nd ziE232dbCgIV4k|1mD|BmvfM&>@M#dplGC)WFQGDkhv)wNhklOJM~XW#TUP!og z^tL}euGs*h?5$UvL{C&svhR)YAdNI>WLl|JXY@|*^^}LIXI^ zA)29{C@X#DS{dVQQ^W6Jt|<&=lWM3Js$zwe19PE!g|D;Wc zX4{0mmA>^Z*{KKl=8}_lM~t~*UtiqL9XTKT@Cz36T#QSz!tA<`b`Z+J(Xj-+JADSM zGL{Yu@{0UAqO+06T>{L?W0{89K#H~Rnc6c?QEQ+3=omMTj_qr>SPjJDd70cqAr#`XL zr9PWH2|Hv~C%{rp?YmPhAcEv2ayvjr1mQEtoi$o5A%R^G@JvSz9?Ima82A?JyM)c|f z&+4+lGpqn{I*>Qj4ZqIgIRZ3V`>(`4IhHpYfe*dkR3<`EM8_%{I@U{i1U(F zWYg(peNAgnK;JZOOD$jl?JzPW4 zBks1vh&F!9bFiwZla!F^(io7?cW&bbmX*NkW8!~_c8vz(85*Y^Jzcx|W}JdAc~btV zT^CB=K*ugo$fZZV@Y3Ep1INAHuG==d(*Ql~cI)uUbRVR+pOBZY`nGtCP=kF!ZNf^f zU+tXx6d8J*Nn^Iq)7K^dd|G_ce`4XmrKYvsNnQbG_$nhsO#0jF&==loHB7R~(RWfr ze6MqGlvyo~J4WH!-bNTx4eZQrXkFMEqV6Sg-iK&ScNbR2wxlE({y7J=|S` z>STWa{IdJ?nZDAAbjMqpLr#6=5C=m6w^q)QlHu--?J*JUyl^Y~3~I?exh)6ioak(C zU)yZnOXbp+t#&~7d8ECEdB@~OvB`9cl5Iea3nnK?^CC& zy(}v%ER<(fsQJ>4@;^J~_;8r`huVXLCoA@w>CYQCdUs^Qq5Ki&GHH)5UN7%S-I%IJ z(;O~(GiCz@8dd3}9e$CTclSj!J+)rhW~RMP_Qo`U1Ij3Lyp#IQPKP{%7j79FRPVDn zTH}Oy2a-7>5qif#T^%p~r3cIs_Y%9Pzcrm4I)JGAl}-;(@u?z9#X**6$8zYq+cSU+ z+LyC@K)Q~=!d(ZB{t=-10hSy6SH|q#-K+dt@iFR+8IfM!P4@mwl2_i@{}4X!nEQ9J zI;eYwF>T3n(Z_KkV*`6^~%oEvpR_E^CvzPwgm;Rrf{(nb71j|IG z-y`pb7IZAXq29%Yf-LiOe{d;GFO7PCP(3m$uEr)QRa{h^Vb7s*n4i*Qgq4gU?Ys1! zWmN$N&`eI+ZKHW6O$0OZAwjq{=*QF~)481CYdYq`+e)g3JxbUy$N}V_)@^l&blJp}vmJH-Zr^db>|JEFp@s zShMGe$HMg&d&qVf@<~zFT&`a43DQ|H8;jj0PoUkB#NGeS&;&7TG zN+S!=fNyKJ`Dng|;X3R`q+ajb)Yf~CrqtvZG}pAJEP7SVYd?f&>`*yFY^tmQC=pg7 z!C#Xj5Iz)>n#|bW_VHJzDxIrPn&$%(1=Cc2Y2p4Bl5=PcWAq?{i}=ijs=C+pxEgdD(MaUOT?(?K>Z$|0$^S`*u1st!WH2 zqz@_m;}W1@vOX`4oye~s`vtjI7_`<&ot~R4xxRry0vV3(+nE)}5nnu4D*YNYf=Kk( z{Y?bkuup{~9oF9z_5+sUEOMMq_o{g5arr4O-iD`<7c|!+!+_+^O%qeQcF0q|D(9l| z$1}m~Vq~Vki8)FnMD7)m&8t7#J6_6aamqvqm9ZR4!x(=(=qN_eY+5@!V|q}i7QK7_ zz)y2z3?sPOR}iGrd6%#|bouy3lWS5&E@Br*BWgSmcEwWqnEB8PZ%Tky!M~)Z|Z6_%JjqSthFa^7?J$r1f$}m{jk7%c4 zSu0~-<|5>!d&ucgJx!T2dc3Dty?TNuOKH$ZGXz7Aa}aACEK?Vw-*3ojhFgmzS=a)# zPg!-9<88DDkAt;G%wvM_k$ESn+1_6G;SPIk2lGi2`ucS?rosRIZ4G7MxaY})uF*ZVl1U-BHDNF&m%uq)K6{e(mr{}3OAL>m(PoR1AdJ?89a0NW5 zVq!_5I$PO@ARkBRy`}kmu2n%Et^@TiB#^Kg0Aw~w~(w5_6Eq$3_V5 zVO_LiJ#w#mQQHlr-i?qz+czD=<|`pgwh_;E(s)O95*W88NL`!6UqUreMTaQ3n7|v$ zRK3wa=5mE6&@wx~j8Qe?sQu)8usvy2g-EK+Hn{@_nudXYlozWMxpbP)`iu-RU1D1u zog*adPu>t@$zm@62<(11F=0EcSCFFMu1~9f zh0`YNcOwv~A(pIojDDGjmonYlYv`^c6a$)uhR+s=jq?H4R@XoA7HCy%7ByP2Ut^W4 zj$i=(qogIoB)M|}Vh$od7r7lALr6zJRu-7KZ=ZMOUkF+FXOR#Ic}f5M;1ga%(Ig|) z07wHD9(_OXRm1lec#N~3Ossyw8E{8WmEfWy0IfPXI~*s=W0^E1+>=2e?ZTCYa#MeIB!IB?RvUUIhaUjis!Ss5t) zBvh|&o#$-V=>kX9$s?~&idLbDqz7~5FYQR$6`gjdyNM5*zfOFA_E-Jc^?@x(!HV2vU5{$j6^)Yag&J&5?-fT@vc1#e^(<*eX%SFL6~7m`0d z1Wzh)r>^z6tv8dU?UmDaH>cTAW4G>L_{+{cDVeGNJt)6FRL%&t>!nR5{T3P63Xvje z!3_PuAu*gpM(05@rgGa?&w)osyd258T$M8@ug4K`E^E>1XIg>B2a_T!%S62=vtkUHWVN ztg2p#i+%39zcjc8wezUrhoDV_FiNfY1^7?;%+^2)}3e?0UR z7L+6OXzEAslwYz<EY3kH`8 zUogDClaQYLNFek)K%$dNb;gl-JTUohtjn9E7l6(XF6~@yW4APK{x@Vj`i7>w6yyRkJLY%%tjZ#(lG_!2pIF|+AKD=!251&o9@qkLU_AgLIzi+b%=b` z_xFd+%k&O^OIGwqG9;Mdukc8&O?LKrutM=LFl=$xlV{VcfacUsUC84%b&PGn+xdBB zP-b4bN?%phSJSmR{3j7)lJn~ZT#wb#61Ne6+4%qg^0e9Y>qE(`b6ODe-R3(%JLF_$Qe^?-=lz!Cj79*sKa%pu% zb`AB6G-Fb-l;7Ays-^Bh_6Cy9nkF@>X+}=z%Nl_?Wu=!hE@;`lRE77N*foc)<)#$L z#foH(*1$N?NqdiQWj;ZqW$goSR85M{fNAe;bR>7BDkJ@--V50Dv_`eqX0bzl?{GPr zX66WuR&*RNmxfVN@7;Je;-E{4^`T#0rppRmB}>wWSTs?!WeQS-N7c)FM9BN}o!BSW z*B8Dkj7Dc>y-XVd8$`OACOioFNO$ko2>w8pb!wp;Ap?W<_qzhiFi&RiS9ba!I|)57 zBeq*T{vVSXbp%&3DVT%o7)y_j+%(yJtM5J2Y!-vU@Z2>*!-|)ub3QU9I9QBOI-Q8C zjXzUr8Wddnv_$agfrJ84>&;9-Z`d>)Wp$K-kKY?#7t=en9eB)R@1+O!h{@{S*nOU0 zP+^|{zhb%&#=Dji#KKyymQSDPH6zz96}V<#o+y4%KvKoBL31~A-&VWA-+wy|WTFn1YUdd}4E3VHYbAYnVoL|0O$KMHX zAEc+%{Yc}#wB)+Jqxax($~V1bKhYapmsHQ^57M(*Zqpp?+gq&_l2}jd!RGUPs|&@w zDfNa=jbb>mcd_I77p*1vbS6?9#C?>{gGy)GqBb#Cj4<0H|ge3XX zDJsvB2$5SnU}n{s9@3g3>vaOC9jug%unY9Du#vC8;o11%|JRVJ z>S-|XQ+(M7{o&f6Z0N}Hj**w~DX!XaUX3a10s2aHu}6WlVSov4e$;-+u zvI&S%6^W2RTVGi*dsq5I5x zxBTPK{}Gb1$abCOS^JZ)sp0wVCd5vT9qV*+l|?{S+RfDcnbhstVtWHR`j~j-nHdcr z@+YQ(qk2+9cK-adapB?(lu_t2y)h+aI}+x!zx3A(kX&nnBc|~>^vbCu)Yv5WYb1Ua z4j;{0w$_B}dA&#sAo?g@8e-TO(q$j)D5bF@Si~50#vr-R}?gzrAN|i`g z44x-xiLlE(FCJF5R0{YisQeLn9BMx7GVkaF2>Hk#z~(_&6wZ5$Fv}>;b66i1d}Po{ znb6mw+w{Wnc%?F#xOou*B{y7V-a*&;w|+n3LoB#~<&hhQ(`w*JoJ3yfiKZ9ZM>+NK zM0pk3#;oSij^_FfeixAE5j4*(PW!$z&LBF_ zqS8F#lr!qE3ie7j9flek83i~m?@2u;mLYVB(aKOSbb88lN$?mSZM}cOp?681tC5I; zczK8)$Fjh7oltu383Mi5lQU8uemhI9ZFf{o`_aV`!_c5O2c5V~n130A=F8!J!vc4j zy7Y}jh4W~&3lvn^T$Y!V?7}`6tU;4#>9j^f2@5AZd+3CkRNHUiN<1e`s@berqFVhc zul_8bo~Y&ckPKt`N_TTroe3?btg?>pu{KdVcT9WD1gt2OXQ1Hfmo89xVaExCkK^8YpM%FimQ>geQj-k(guKZ~>Dm2-% z#V!vatfI+O>{e`Xr+aU1yZR_1&L|{>6H)TwL_VM)vA4RuhI2mG$^atA#t+ zf28$O5s7=2Zf% zc*?qN2^7%`FfALNk@9JSPwcgF+n@aL^hsTrp|!F3x+1?Iwb8GY_@FiZK0xSC;hxPIPBWc1JNP@q; z8KZ;iIh2+4ajtz5Mi~Y6%~B-$!Q;$tcoH)dJk=f6RyXAu$ix3uvUX`Vv7Jbp+N@?P zDVJq6zEaf{MVpYH+1QhbJvA+Dqn$pl)GbJD)u1rZ+nZx)WF@CFfrNZ6GFr;&>Y-VX zLX?}l*|H)@%zds-#Jg4GlbW8mASbdG8^a?Gtn-^pYPd1<{}aowqGVFE{C7SqN1Ww zKpN9zd&6kbyPjUQ^UIWoZ;zX(gPtNaqCDY*Im1o)0D59tT194CBa+s zTzc(VOXru*36koK;}3c&!T`^Bd#u^+>&ph~zps2Jnd3n&VOYU8<~)t0R@KQ00NYjp5AB) z!M?lEmlZi~V$E#O=-5%|V(h%25M{7;+L%q3zh^kI3_A<{@CH7_zkf0^nJb#^*4#kZ@8&t83A(l#bw)RRujaG`arH+ zvf{dHyx(=oO8D+4=yuKB=i$v2ykF9A@1DdD0a=`q`ZS-gpCL=tk%480%ZR3I4Vq@v zgB1g5l6gWS@Pszzi_Y`0qHT34z?V2ZjaR71B^o0;$fu9YpT0MSYXJkaxH*B`=V>Lm zT)Qdfdj-vSdeRr*P4kMd%+{~hmIM!)173dbPVBkX-xVv@MtLoeqa9h2{q-K0H-nu; zU&`S(#-veBOS|!b;YOIK+!@n@x!u1a>3#nU0sw^r>|L%Jj#38hm=u6-3Q`B%)N*nt zqCekgaLd^ex})^tcLlIdSNa@4i7f|dTHS5mdKevzn0GEd|mrlO;5q6vwuk`Y@vFK{7Fqi zDWyUcJy^uOJ3ctYr{6VhOiI{K&oCEyS*kXMFo5559#vJy4|#OtQ?iUS&0l=%N0HBC zyMyDd7M^=<^1bZ)3qsY&$8Ji;-?!B@UNt|aL-S--kRhv4L;CT|wbmJ;2#3UpJGBdxN}Q@%bwtzogjB|j#hd@}tWOuA_@uZMzuRM5-Grzf%^VgB zZpKDo4xtzo2UL=qob|>-DaC>k*%viGG#4J>mPa|41s}wptWZP+f`80w&0apvC326H znR}FurK!vx7khOwG2(`#`1D$XqFhF_th5U?I50k`U!78YW6DaE1dSFvQq$bivHDWk zpc|h)LyDo**kX%g=p&h`BHpr|F*INAN znzBp|;gWj7(zE~Snl&w6>sujcJ= zGc$)*6oaHw?xYG>1E?aj+LS`a3`xIt*5WfSR>rjqKJgF6Z`)Y#t|g9&5dG@!Mrevt z%dM<-x5J>@3i?{E9p6xR*K7Z2M}@#GxMnUX{ZBPFt;8u`?Y}PbU8!yPPrbew8ms@0 znA@9ptGaQSO2kx3Utt1)b4t~PfhgshWBG(R%Gip5==WWw@~T0WMv(|pDOMR>wg8x^ z%MhH?@lTSB%Z4iYgPV4ts`6>+6WE@Sbuk*97QbCVsN{dFV1udZ%gIY zrw;$4S7G1r!z&;vhSbaPazoQ&C5le%8fcn_xL%DF2916)Zv30J{Xt;J7pGxC@NYC1SvgJ zWgV}Mwf~NKT!6!mP;QQaYJfKKDx~ICxlAcB&Me|>z&Zn!HIk(?L`K>W6NxHq-lT8| z=f(}g+`id+`^yo=E5a{kLT+%^r=f;W5eIJH0XC#@)<;4(&{T6w!Ft4P{B1E(!1AB~>(|E_A8S;>rMz zky)uSut(rG^}*oj;ON7mx1atvdL0BZnQ4ZOc3}V1d}5t8IplW8e{7c-7}QZb5%nzb z^#>h@ya(JYjltiY^R(KR#~f@W1QcdlUev@aadaX^xG9n}F%Ms8r@aZVwhexf@&jiR zR5+ghyA`E4s)sT%YGHl({6DcLS?}q7cXXtlrW_96!J%P3*2i)+qY7oG2C+ix`L$-a zUNE{eeSwucCvOzd^%HcUF8XZks5;m&-C?lYxz7)+nwDW-1D@*io9W0E+4n2MDb;oX zcaB=iKHfQNf=Q15+uIx;?2Y6^4@xFGpc%#Fyd=SkLMxe_HDcsMbKJ^LUNAAwVR=2q zy!LyI6xcmh=9vY?!}gbL0sXO(*=j`8)S}5-EC;#W-=*a9zQf6Y!H7#GQdWsz)lgi%OZ>HAV>$p)bJ3(ApttiXeUOPbq zO~5YiY)5*a_Ezf7I*^&RI`V@@i!4uZjVOz(&jFe3 zyo?RjnJZL%1T!c}$0$xc?6i`$qwia}>C!Q`B(55VB2 zN=3_YrDt|!)dU+vQ~DNX!O`r2_d1*dW2XJwINsDeo~6XY$wX;JYCgrtOg)3N?xp}5 zq!v;2s$&rLp0dI_ZOGf5G|@Fwa~=)&IXkA@TG)`KP*76;T;(}o*EydkVEL$wbdXuy zlSJCI(g*ZNyG1>KknzAJXySH?r9^I1vB7vPQXjIMItv zP(N2poGtfY?!jTz$bAMH1839@s25Zb?htao*fX#Jd1ysE`V%aNUY12!XAS;m2%z(T z($NvLf^QBy5oo@oL<*bjKU6Ru`e6k>e!q4wcQBfbpM8IvJ#_NN+@obr6g^kn7tgMsb*5$2O$Ibf0T&}%3Gtd)SxrLgoH=F!DoJt|D;QjyyxAv6dZIy-(@T`k+gwmb z*JjA_oGZq)88u2-rp@U}Z!fdL-lJkl0o+?y9ANP=r|~-F8+Ym;5-1^Ut2kb$Q^~oR z!;1GOf(?GwQqg1?=*W=FAK^?$? zltRB~ML?W|ho<*o+%oG>UPLhtgWqIMyQx=EjH^2_dPs~hOTB$TUEK_qDvPB20z!J8 zG^I-qRo~Mii+8ONgc zXqg}top;CCr9P;AQY#AX60P+1S-C0Qy7H#S8|R-Ae{*8XO$4&{&G}t+*V<9 z#p{zrza@7jvPV%N>>UgLqs}7Kk;nB7no{rP9|elV8NU!k7J06pq#KNwk~v&Zw0lv6 z<@j8Bx;%fdi-D{*8Qyrfr@HSH|5WO+d1^A!u&qf|^>Awaw!S&q;w(`}cG{lt0}e?h zY8hzL#I}1kZgUhdz{+%8e^vHzNFz^=6P5Rb3i#w|^$uL&k`tjW4q*2h`<-p%c3|#A zthEG2zoAN0e6z+$DxmPYSFfP?h~Ar4y9sMy{~1gD2@ie%tQoKX;MO|y*^KU7Ut4(O zdIXxnn_9=DV~CWjVres*1S=-w+$rGQ^`W7vnXbfR$e8tj1i*dxr#lIk>BED(B%ZHg z6eY#-S<7G5t1*h<$b43S)37YA`-Lx^Z1N>VrOSIl4d{OqUNtAg%ka-X1xkSOdQm5x zgjfMn#0n1UcrU=P_P^T8GshV}u3INGS$(hw1jX9#-YK3&O6)zFXkNIifLU`>An(Ha zW)?CshvJm<)5(`fG_Q3kKT*>=XKhk9z}WNisa)g>^s1)^&2JyO3+|D%KPXqh^6v@- zU{?}iMpnmSU^8)iX|oOuTF^cvP@xx*?1|^LwA&6im0wQTGxnLdOUP)p(&UF zz&_IG(IC^Gz`a1zj5G~+I=r96a$`enApZpfYP3i>H8bC|u#TC_v8yA68xRc#54D?7 zHI{ldQ77P_v)1_0d;iCHNV#S?i0G@(ICb;|@5nLFpy(SEu1WSnoPautt4jYj^WrIO zEfcG^B4v%g;PasD70@dfRAW}u?129Ws@@D$89s_#OVR8`!x#4Ns|JIdV*Dt5QubIz zdwoS2bFFiefObrft%HATL?FaT!b2g88R6a&dCtx8N;2`IO-va7malbVh3ww;(L>p2 zOvt)S=~!WA^eUf|lvY!8=(3;R37P6J#u&G`O`Wj|eri)@7R>VQrW=IHH)yao^C^op z%`lJf{!=g>wbzmy4iNojNm&+gIo2B`LoQD{A z94(pU%N zeviL)7wJ;tlw{-oYU3X$kvRUmlnW=_FFkqnOgR|FzI=I0U9|bxo>#qk4VbJzxqR~k z6VC#zHi#OWt_gHHCO7;yH;U?jK3Y) zav1H-ghmt8DF8)zO{B!fO!cNtiFNsu0uz`bIdjy-GFwK@XmkkPOGTDnnIDg6$W0L) z3IrcaQRi`J4IaLryOAtN#VY0#dL*xe|GjYH5bFP5tedn)$vs^45OG=d=P~x8iLD~K zE4;#Fe?+)Iq2?(-KrJjnYD8a27@d(?gcF>^F}QQ=)weJ!Ue5`N3N;|)?FBuMvy{&NBd>T!_;FCR`Q@wr5$RStk2>3FByf_`EkgYPqftv$q+nola>@ZHGaTtA z*X!d`c~Ayec-trA65d1d+Z)NOnpweCKV*8)08f&J^{$4kp$^$=6!5>$KGywIXxs_g z66F!h(Z090B%8xrQMZWfzbJckxsDU|AL$6@e?kJM7CP5Iy_M0Bc`)6~n(_JPTjWiS zOa*0mySwxVgiZT9d}sL@L2+8*?3T$&Dq#D`W~R9JW^^w-9uE6`CbM|+q(hMf(-bsW zv@%+o><2hGsjhdPNlU>v;=v-$Y|Z1Qys(Y&=GI2lx&jEP8kAHs#e?TBlpF~j z!+|!9q29|UFO4@U$?NX)GURmMiAS#88t8JiC3m?zUzxhvwPO|Sv+6>*tVZm}(FGK3 zUM8-<>fbXV@5FJ~yFQt)eCR&AG+u@wYrhI;V~hRi;t_E|T(RmrSqL)n#W*BA^YA12 z81DpLG8$?v)}TvDJ9c5MUrsrWU`RQBp_U1$3u%N^RB29=kbvn&4wDB-4!yHIjaJuJ z$K8Eo+NBl_a1Bct+a4sf9VLr^r20LFok;6Glu;Jyn+GaX=}|!6*v=>y?&K-CHfRpP z)3fr3;3ok|i&V%?f8-s!f@n{u4LJTyy)WvzH*n;Dw|o0R6rNy~)Oc3QoYn8|5D!u5 z2W4OO3|D75Z7LRwJzDO$;7j<_rzHa-tOIS*L2dFFZ?S9|ykvxuqFuMPDtI&_;0hUqmKyW5x_JlMPo}^uw800Fxtsv`NW)8{1-;Q zgMeL!-#nB1z);)nb<2#-vtUC7uQO$CP5CBiG0;ZA8XlxgCI{AeqkN{~mr|-wW|0S* zS1$iY*6-5`u6_=sC(`1Ag55{?-aDP2>DelQdIixLXxH`F%+%b)cZ6W4v_{MBM0yX2 z`!AYH2&q~_z3g-&qCb8 zOnFR?qsfcEeFaMNYP=`JHDt(pA9GXPuF(CD_P#T!$!%L(QB-6jA|M+Pu+XbWFA0eB z-g{Az7MfC}iwX!Ry$1*gBtRetp?6S<^peo4^jEOwKSC`@WIgR@@jR%fF3^iNIqYy6RFR)cy*v>7wP)ysoMl5 zZ?N8@v>F1eqHSYctyRZVNm;VukatWN?kHzGVxUk7XCc&WaWqFwwH~ZXv;D7S$@?=tt>W z>5%J~XC&dj0h#5o2CMfU1-m(pMXhYFfvd7m5Zsh@p>2T|$Z`~)w>sOpe|KVrn+PWR ztTFwbo5G2if4%>_Z7xLVZ^L>5ev5o^Ph)zqQ-;qcCpH{ZI8!({#BAMh8c(lfL}NbJ zPcF$b=!prk=?*@2!_Z#G86c2dtUEhTlJTzvRZktWZ-aeJ#OnA}As*F=nuHooi3yW- z-hlQY-z?z>osPn>*NkW6TBh?UMeMAOQrsR3T>Co!@6XRfNQrkGS|aKJzTMK4$shhs zwts#`B^EF(J@ioQ{D(9NNV%-*^s><38rDC2H5HKXP@t2_@|m3qgfXf9^QQmE=|5UP zFoge))Tdqej0_Gd7R-To=-6fOeGL}> zz{fT=H|s;G>BX0OZ0E9+rN|N(th&E6rn8^-vW6c7 zhB#4LK>L}sjK4;;jv{cx#s4=)oVDhF&GmCU;=Q+T{WMa#>Cs`neo@ zgPlo4VVo4oD0RP{&?E-E9{QT(}t;)VQbFh=J}&4 zPQ;g}^FY+M4E5<>wdH?kkW>_f!l;J=i8pzQt3Np-Fu$av#AL{2T58&RgJ3sT$r&UZ zeT5>rzzG)EjR_?cl}_b8ZL@{g?t>r3t2gXg%jmCH=7hE7=d|dUj}T z#%L^vE#&NBDQMoTWXg^n4gpIY-xT58?Q#huC6@i|gK!rf!~&A{wr*-yBNqHrGb6K3w#4 ztjxj1ER32HT7PdSf@xpJ@4Y@(sR@mT#uJ^ZKvVxd$7I|(j&Xudg{^n$&xFRE3i0$%aYSLUWS7` zN|53@bH5zB?M!yR^pD$0((H;ES`YB?zUEeIcRGGc;6h`c3u9uJ>fm zIDULlw8yji5Lm9hxp*SJD04bws){&?xno=%(2_5nHm(@{w|LYabtI89MD9$X^qSG} znw+#F!xpD0{d0BXZ2Ip0fSvt_(m@f)%IP(MHZN58grZ0aQdq47CHix87J%$ zH#nZatmkk?*fHfcmqEI0*zM&(V<+9f>*b&ygq*zhx+oo+<#+dGxufqJy&p;J(EFuB zx-ZZgWOwT|J?f%OkL4WoQekHL^Cq-?zU*IfTp!rAjDxGs+$u~CiG(5#mog(dKSprv zY>SFd{wy1Fbdk&CGOshC7kAf@k@-Nd0jL*dC71AbGTdph#t^Pophe&tD=O z_lU3co2v7jbRW~?o9{^QSV8($CUS>GD63Bzo2lBxqm0dY&_larOBi|hD$cI|_Adsx z3AZ87Z;-VS7m-XiiN&=xdOF-M>Yk9hfW;VZa#B(k=nGB)BcW0pM>c^KL9BU=zgWA2 z`0CY;sw}s$ug~A#k?b|Ny3hD(^f~1H#F|3Z(vr={Vu@APoj5kl2L-xN<4R?kqZFKc zaGW-?^4uWzjBK1km-}bB3X|535)^;=qPrMXu~n$q5?Y^f@v109xg4j`0dv($mllLh zP=;IG9rX4xPh{Ph&M<~q>^kNxo>Wv$r3B~|cdrjpzB)vD-q3|hp+VUEKs zcfV@m_miq4f1juHlgEaBfn#=!WxP7v!LFlm4gIm}18a)h)re~7m>p%x3z|Q7%fA%i z2B0oid2DW679i;uzQ?*;<~*+y$89{AM1bo(&8X+TJx%`3Lr<{8obvs_zY|u9GQ;gw&Q1b0z zZMt{wN}glxV(Q8(B#`^`lOn1Ug!Uu6EwO-Qi_JvYo16^?aXE^Gm2^m&W)j|ONdc`d z=zqv2$a?@=IH^$YUVGigTIx2z1u(3xs^Y0p<@cxkl6Xu zxOUxZ@u@U~Nj)Ose|YzQ^}owZ&*bq(1QL(Mg0Aq%A~2cnT0FP0kf487C8d_g)&9fg z@bse0kTsjz;>|{0_8ya33fks&Uw-FTf*peSOI~&~3h3J5K|?=tH8X=4eD^fxyTC)< z6Hmzjb0q_5Jqf%J>>H4{5MeDd3EYl$a<>HcZr2hh+fs1r{6$Z+jD+ZfMktJgq>9H% zYzI>MWZcDoJp!)8q_w z*Irp!2y#Fbyf(AmQ|&}>kG@u^s&~E=2Tx3gTWM$v7W3M5R6XIax{+`)?b7weuVBB~ zv3E>02)^fn6m)88`dB&Kb-Dis z_3Q9o9CVbdYOaHsMcTpCyk;hWq%>+eli_-X<&LRX109`Mk11qKiK%-@SgI0tCAU7p zWYTti?Ih6g$LGcE-L5YT(yT3gR@KbF;j{k4#Qiy z-LYo4J#Qto{RM5=sSWT7cYN1nrOfIQ51_xuyn{4j;cwz+N|@%sSBEhnr}Srq%IpV4 z87%~MmP|&!gC~TIUU~br(Dc%1DV%a`ZM0NbIiBnFV1_M$8PuaBB$dF6_L#tnHdL=z z#q5wkCaQJaZ>==V4HP}N>jZ>|P#38C8^Pb&pKy53mYzcM!m0-M%LzMSaif8aLCy_& zi8+)Df!_wMlJq!)sCsweo;5)|kU6^W_}#l|X%fr%s}_aGhuLdJiudo|SEDuF*K$e_ zI^e4v6ZTJJapcKhDc>AudON0@E&W>X2M!*;A(CNKE}7rw-@KeUp)6?o9qa`)c9^ea zRykf}G7M`~`C0a?|D`26zk%n^=3=j)eVeel6RRSfttW-^iq^c1@XMB31h?&sFGqW}GoPCJOB^{PxDHsSGi4rGl+~{}2q%Xe(&y^9j85mHU zEa^Dx(+K`YgzYZ`u&}h`U95Is!{N%q+fzNq9=s*Li~~ z53sqU<#tWHbbB(hV*=^5-Y+{3LCE7o0yAH^K@5xP9Bsf$yp#OsaDnrDdv}%Crx zsFYqpneJVLZ9ANNDPx8CR;v2?d$Qiq65H9!8iY7(V0i#m+q_N%5~4yrk4!D&<_43E zu6<+__f4T^CiHtSNaRQdUe6^&G`Zn*JF(cBtHi{j+MCVazl#iDM#@Gd#hIs)_^ngk z+7e=$w4Vuk$G5@{NCw(~<{i58d;p`zVXoiaSFHxCqh^_rX6h4Vd7AgswCh$7?U$;& zFDePrXE~zRn^ZE|^>mVZbE6OWNez>1r*S^DM#pr+E{A(F9uvCILrM<*$t+uX$$=Es z_?#auQC23ABSg-56~0)+%B;^;-LTYh38%?p?FL`ZIsMdjj|kAXnS!xU)zGb-+b^%4 z9YwfOeVqxSx7@*R*0YMQw1!_Zs&Wl+teCkn(%b6zi$2dZbFqGSe{E#;6I-59X&&W? z$r_;PVJ7AT>u>9_rCEDoKsKSpv?;4g@O)f^sE)U9&$H}Hz9BNt?--XXd?+98DpxmruQ5p0eWi?aB#4|b zjZK2&+BJ=G#|`tv#qL!2;qIdNlL#^m` z>Fcv6)*l9-s^FJnYa?sP67I3xwb)58bZc>=dUG%@4o*0OQOy3H6V0qFmhp)`$zuKe z`}b}?zrUt@Jg@)$3YkPX!BHXlN<4S1Mr^0nK5|!YE-o&nPIg^!6|{po7**>yxl*v2 zT8U_xfWqWW6;f|Y&ghldk9=S}ZT8E-RU`VgMT0R>{Oyb8I?dTf5qc13{@CuZ^!Ej3&_evp+{u&AZa2`&&hG0Fe{IU4 zGY><*a36j6+aB~NA1?4;!?}MK!%Lxy?;xG~-aS;>O9d7=a`~Fy z<1xx}NyT2Fuirc}DUFCf=cxKt%8K&d!4G@&l#_G5UxO~E)RqexXe4%a=j#QcB-OXm ztVqCX+<9oFe!+#t9$mc6^6%%!;6H*QXw?5tLDAki`hM30HWkD(7y^&SXt{D38Pl8~ z(A8?k9>U@jSyTSz(63>2Pe0cAIaS05%D`%a5`B#xc?8@rZ(z19JkpAHm4tkz>BlOM zNNk*`gp`}4Suzy7 zZp!TnD)2ycOC94RtV&RAPV()+q*}PMD!x%eEFW5tU zFg_58$$-OvUXCFzT(x#bYhY9H#CD4H9+)7}lX{Rvi69d&KSVUk6L3YWb-hy4HzB0} zj}lU4!%KFVf0ZTms_v*sa>{W}1j`5A?VFkKruM9*s1Xu#_V$@<4u2qh><8jAsgD%6j(YTZw)he>l zkDGqxFjlA9i0@gEUq8`49vjkB9Dr)Yt>igonilB;;)x3`0+KKvV)v>om>l`#MFgxd(lbA%(njG7KeYjoZWH+{Tw{}&i-vPK#Gz!`tY?2WS zLeLj*P*-_EtiQzg`~^VyGbXB7xy<9RH48RJPdeu2ksquqCEREEC{o=lbJdnmOoOY| zA|~)^$!LR45vytJIkOh;QXkDj$h=Fw;;!N_V98bu> z(j;>&`Vf0X3}{mfTq)}}_3UJ)pYT!eM;+?8EK+L@Kvp?6Y(757%&LWu4Q#!5ShAj8 zf*gQZodP~r9Rj*cJx^}3fQD>LwGl}|2|k+2zBmpG0^+9El&SNmJ>(yFC# z=}AOH>8ih~TdTGtSX;i-757N68tXDK0!w0%@GjqAjFVMIP2_G<3a2zHxbqrKVe@jX znjIOCnI~r}QnJ8P**zZD+4rjV{-pU=olC~1aN>F%ot&7Om)iLYt20VuCENS@ zg7e3$0=w_yy@Y@tLU7hGasXXbU2tdruB1;PYv?Abzuo~9ukFUjm~&rUpg%uw3S?fv z+VXnpv1Ro^cVhY5yIO5E0UQQ!*bU)hjZ-__(Ys`5Yt=Cuqi5<4*RJ;mpFGD#L-wn2U|uzdt+M* z6Qw2e3fnwyy+Kha6Jc4HM{cFxE6(VymyXv7EzCWXabPH{Qm=!J&eE{4*@VS+o=EGG z8U@tkY4Sp#7=KmL8uO`ZzIY{l2X;SLJDs}FSz$>4+`IuB?cR4;k~(=oUnQg<8J`!s zv=9{ycYf$bXJiB+n@WF6o~9jkmBwb8_4S5xW!E7%Z)v%UK6Xj~SB0gqYa;cg@^US> zeY&Dm>DiO#<%VB%rvL!mB7hFtF=aOpd0=Fv_fi$SEDby+UD;zl&~Nwtr8O_-1eUwD z51R$0UV-z+OzDMHd5#D6;6c)f!c`o`b;1JPYXfTDIye?xge8vzF6K&4yjjX=oYCa- z)oR5lzjC?kWb(mx!c)#5Kl?QgT+ZZgWZu@vEZv9Fmi0yjR^D3mwkX6l$f^kplw2R0 zZV6PGxyGHx#$BF}6eUwAOpEC6=^CuLj}@aP)|?YuVGoY;5Lk z8@ph%>#J*q#xw=^s>^Iz`Ib-G$#lAZ&$0{?JB}bJQnT9)zlpZ@fhD{19q-vMRL7OB zb&M7}MusL?WJ?~H@TZw`60RgrLbsYe*|`E$i`n`&Bx~Fg)}=hb%Hg+CK~Go4QS;nV zw;pSy>mmGfo^icEP9233hZJhATXS-NHicJS)P;Q;5{&Rq+a~sW`p<;rzqzOi5HZjC zN64F&fi!b0<=3MzA?!y)-L|whj-_vQN6qA3N4ET0pw*L%J6?yMjWKqGQKL@46VW+0%lhS~r3H}_qFtnC zg~1JI-pKon>Y@2jeA}?%b}P~}_5EkAU+kIWjO(vQE6r=x(Wyvf`Ik7slB&7=!o(KL z(pxBxE2aPtiR|t=eDlch?u}*xHrW%TQYR$>>qRRjmjYcIopUV=l8~#bDi&gNw!aqQ^D}#Iw8yp?-D(|PYWbOL_86|ZcPvU8tsT~I! zlk*)tS#vBtx7OMnpf*O$g>7Od=}$5y8Ornl!R5RA4@;|^Z~VD^|C`ZBO|RT+YnWzL zx%!TLdCRYwrL)qw|JnB-epYvSjqlqJjYPLD9Rnw_$AGo+qK;gJ#)S!bm?Jgsr)r!# z(_*TH^>Tel^jaeJ0xZWa!;W-4B*haRPTOwts=Cn!72r2alm9WKYf`drCCvlyRYClAl+ z$ab7v7h%9IY+B1Vt)V{g(tuXmc~M_==#DMbE&1JWhvKw76@4)!Zdff$u1DCUsYCb3 zNvqR|Cqf?;Pn=xNtwO>2ZV%syX3_YO`yjg$C-?r!a`5Ld0hd?&fru=uamQxViN{x8yc5jqtEtA3u#gjw- z^E6ZL`k3oTwzgD`G~NMP_aKAMIK1b!Gx*G1@jWeeb)REJ z0v^TBo#gz|Lw$Rp0L>mf%?w*nEr4@-oaD`1tQTwDCEq}Vq6<|u7RDzYU~6E^77ui1 zf5ayr2H<`6AHR1?w`HuYbgpxsAS7x@>t<5g^pFvHC&P-O0#k?f{M3??{O?$&Xct}j z6SitsOFHhFlmbxR9Mgj^!R$quQlW(5;-sdwHo43)YPQj99%($=%XlGnbwo$g3Wfg(SyAYzMVKg-M)# z{jBAZ=@@bK+!~6K=h}D<*%bGKiBU8DvT&X314GQwL`2J2%bfdKsN_>)y@5ykmg7#( zId!KbieuOFM0G_ThO00nmJ|AJF&2qR0=Btj!n^}021BYnT@PJ`-U)BGmg?19V+?Xe z&e`2PwBksVW_bp*=~cL^Y3f^7SK!G~^e5zroT`gn*~Ne`5#%abySWZf4dt$18e7AHPa<+w zvM?(rL6^FZ~b!a)oX zm4#e=3fLg2G(cJ{d3ok2w)*X7I-XZy;i!;Eg~Nre+`FRP1%T$!FArTsz8_UU_@xea zzq3A2o3RO`zoG4#0@?}7hk+Z_wKV{xml{= z!=<})H_oc1qGi8%_iny@WvBG+4|{!;E&eXFzGi)0!QghLPxLZr@Xd;A0U8(&)67|y z8>^X217A#KEKS~>W#Mld{COgcwp=bgH9pyOhdN&}kE0*a;LAp4YXxavGx;Q?Hvw3o zlub~vx`uAnX%+l;K<~e2`7*tM`qt4)?IeWiisgF ziWFkPfQzuFW9O9!2aw*pcTjnOEA&j%@0Uyykc0u}k6dl&&K2rK0>n@AR#x+@^Zp-l zaK23#Fa>hM;mqURufUZ@YFd)S?+ZCkZ=1T+(iXA6hcAl6UrK%ZObSSVaeFd2Um!(R zr9aoub{K+BMth1eWBz`hZQ*^Z`r9W(8o|8N(IxtR?8;S;o)*$&e6~FU_RL!n+G2@L z6}ONx;33T5X_f7_X4dP=2ag_mZ) zQp+B9YccKmn1I$ggF{|D&%S7?3Cw!R-*_rt-JAG1r#yT-7z3GIzJ-vCdANq}qV^x-qs9hznrLWtQ zG1ozlRL@lAiWTC1;e!wpXt-c|?vwXdK=^i)%>C~x6g|24tFsuNZ|^wz4;;8n#PrAz zyjk8}r#Uu(w&Sw}g3gC>s=n|zLM30;jz?`yV47`7_hs90Qu1H$Zdu{!o3f2Iv==TM zdZ+wNB}?53kM;qy+sIKaR^~1+eHviwCgGp*;C&g#8?3v-RGn$?fXFD5eTS=?y}_XO zR1ly(#>G}?T@?Am*SIE~(eGFXtvV&&;4F-St7px0P!YHofQ2jbh5D5{ z%$SWduImb~&_dcybYl$K=42M*B^_^OZ)RL7GjBBw?V!%WRKHIJUfGl!`>_Nd-LryK z>vosq`e1@U2Mu9Hv!e6_#L&G{ck0T9pV0bo>X1984Yt>rNNohzC0jJ|p6X^S6y&}> z@KFt|Q}_H5pg#-A!ZJ`!wf+-NNqx#tJf{uNL8O=+Sp~Z=y_^sZ>EtZIkB_AtC`bE? zm#Q)XeD}q*a-NZW_84_b+lP5Uo{p$W^ zQlH~)5&fp#*u^kP^zUUG{OBB9=g4Bx5*Oo{HvSkag}WkIC%ZSy`E>71=9-bH1&q+; zYK+RsA;$T&!Hvu_QbUgfrMba=Ib#e3d8*n&iN7~ye=}ZKE`XAuS6h$n2;m&!rjFu_ zJbt%+@%h?k2R+qeAhz%8=Hlwmk3!p_L`2@iEDaVJ3e0uzPFO9KPhOdqEeEa9)KYu+ z#58qXfM8}Bb-Px`?UWKDptVPlpurqC;Tj~+oVk2AJ|)xL7g~&g3=ZDv0pX4glQ9z- zf^0UyEm{3(A2qnmMGVye``IA3yv&hPZ=HzG9bcf-H�eg;EyN`fxMNpp~OZs)-4^ znwAJ!sGRjrDh7Yu-cGYT?7;>4+@t6^0`pGtZ39@G3!${@hb~C37U89ilYcF04Y&N9 ztj5SlU)16|-#6uEUaEJe_T)Ah&bte720)m>D6qb7u=HF1@%FI(8{Kk=9r9pS+vlZmb19Qo>%glf)St! zK8YWde`wt`^z#rY@M1cKL%(7CyFI&qdFKxldzXydg$vZqiZV}hd^(q7$?UtLJBDOj zgtoZ6>gx_NQ?AqvX)e+&#V4zy;(1XkrM#|-;@YKee)y;|o+9$7z7hrG=`9*{(T$pi z)$p&%E>-*amTtBfaV&1+4X~Jip1W0X><*jQZQB0Sz|b;RCEo zQJ~7Mr2gciWij;1W0|lk0$K(3uhnRLIDjxT(oE}wAFeCh;|%Kt)t>eSOg{HQ+SAH@ zo=gw$fVwF?rfmA{jUmtwf+o+b;;65Ey4;^<@U4b1HR*--?S1bF+YaANmA*(ZN!y(R zNx?l>5eI)|-spy#TK2)M(p6G`WfM3YQqbRkFBG1CP*QQgZ)92c=wZm@%3kSC>8Xj! zr0pDvLZc`JXcnt#=rTCi|Nd&pT5TTaM3K{B{8;PD^z%j-ubs<*FV3x*mBSkGTaRg9 zeh0GEVI$OaB@a8Uqw19#KjM^as$Dmv-+@)+w!dj1L_lc2S}N};6xn<}1{SK(lvqR_ zb$-mp>i~rcfqt(Owb#~^IozR18R)y}|AbF&l6l z^LfYJG$l7yj`qvuby1_m4`l>53bTXLa^95rC{1x||I`2h)9yqIMiwU(TC}y;df_r? zFm_ShIJe~sapF!&ZVZqEe;arEap+bx=W@8*x~seofrruaNqK$0&Qq4eP5%gE>e5Yd zjPVS7Xi-2_TV!GR#^#y9i+-ULF8m|m79~(h-vt@Es zW5P*%O+-gmzFp3Uphxell|*^9VU6AFOtfXYP|m#JhA$s{-E~5zLGmuH@2qB~al0Dl zN`r>o5`}^)3ze6v=dD?y#0%9qt{A$sEaUxnZM|+M;8Y$B%%lqM6|E~ywARa9?YcWD z8_ZIZ7gV00H6t#z3eVnth$_|{2%8AAIqb-EYe1DuXI|W4-wGX#?bz60`}_X;Ur}6= z*{_&NO9wy?xz@a(#k2dcGS#6I$!x^ioMO0#;i_z>?j;NC5}~G~${%xU*VdWI*f8G% z97aL%e&N2|mQ8u7KH(M&H? z?+gsTT6m@r84ejPWSn$>P3|6}+!W-AZ|7zs0*dYoT7MJ@N~HF`!QD&8&C&+b($__1 zdad#9E{?2`pF;cv3>zsZu8$O{AQKoQdZ_)X;9ZoV$M~miOUJ42A{d7P*6sGbk#oAP z1jUYN10t4X?v#CXgXE550{1#;z%|*&N62JNl~#dWxa??|^Ouc!WWxS1Vnm{?$DDZf z=idAQ`)HkY5>6<;pvMC&@8Nzk_Q=H?RsjgxTP&jt6(NM&$e~0jW~sn1NM9etCa49v zo{Vwi1PJ-nn-p1938_oZ;p7{Q$vM6cj?FzOknm0LD(75v7&>&f|KNJGSs-@P=N_;@ zaip8ynYA11G!Gv!l=Po^YVEfP2Lb9`u3ARHF>0q#Mo9eEBgcH`eyWj3d4$T%6=rK| z7L$PrGhfVl@wym;z7)n7kQ>|q(HsYQfVFUr^P1rV3TC2b_4=BIc{x2jlB*0;mLUB+ zq;UFc9&w~<&rpRLb#g@ib0Qs zkP|yUI9EQ^xA3;iYiHy801eCLY^~_o^>6x>QaHr+Y;!xbLSg__(jst3<)cQYU(e(p z6Eb(bB-7DHTXJvV;q zdbN4Q%rjrf=@JINf?$xEE5r@A)2pmh3Gf#qcHm9D2OFlv1}d6;!#Zl@kdWy)ML@zj zQ^<3ARiBQj^&zc3bTI8!cD;R`2gt6zJ%WBytc&)3XaqOI>gx{u_JV!C;hE}Mg~_`4 z=RHJ{cJ8-lrMQ{^F*N7wcoz)SG5W?_wiLhRqFQk9p-M1^bLa{p_XCy0T@M@}7 zrO2#P6ExwD{%gc*YELPP1m7>AH}X@g7g7}FnHz@HR}LdtlGtbKM$2S8k{gT-R&G-J~C(Hl-vy~h@lJCoa^Mzwn{OaH)7waiTEzUrt0 zjw-QYvQv5YcSjR974o(4j@b*^9V!`C_2ujiLK>mYN#Tjh#H~)rIfL_~xv$rUN^Ii1 zQ#qRN_KWeqTN!SdzaAo_P0S&-q4OEd9y=P@sZf)7bMDLa^iU@jIeI7F4hV`}k}yvpl0ez+ZQJ`8nOiLF6pd(xio`v9Xa*+t^t0jAYbnMJPh4SI?b-0D?Z+o~XJe#B*lzVTC=h7OCG!X6_^ps>$4g+|RU+Ifiz@ zIYU>IAwY)6r~KCIt&3;DCewMhD^%o;OwcW^XJ*lUK`RQv0VuhngRC*roHO13ANb|H zrvwbQK2seJZ5mA?R=$2x>6%cqBx zqUVn1M;LF}T2b}PIF#@MsUSjq=u0+9;d7gy`_&2K9dqT9oTUuE7O+i&0PX!YzqoMj zcsP=qM_CYN!E>QRS`NY@XB>^+J#Rb(Ny2!0$Sjs~7x_O*MNs1Z{YrIEVZsv_7|71T z!g37CK363!1h`UvH4N`^wJ(e5Us#=8+@vZd9{DdPFx^h|@$=`}$8>7&znJO0E+QaM z02C|4ZP% z+vkz-ML*jG)_vXsdaAYgbw*8bT52I)YV1(kC zBGEnXN$e-Xq1JBF->vH&6{T&p@)NaPkydk!t>jwf{)b=FPky^tzg5vcs_OmM#aJf_ zx#Ey@{8-5>gNR4`37EwHR`}L7Uq(d7(ba~cA|A?8LB)*`WWKjvBx-#8lX0I%`{FLI?_YBg745_asV5x%4J1`Gr^XSKxZ$nVPjf}u zwUyr(7%d5dP5kU!O64*f#}mH5r-Km9@Vlb1!umXxJ!#xe3f9ie6I?UlA%dyNa_m9; zsVFF(TfOGLsNu6TOSpqRVs$Gj^YtO3cEQ<}fBoM}Q~ua2|M`!C?6krXA3dq~%w@^C zy1H(KbT0Rok%mXDtdG<;ba>{e&f6m1Hq0~KOyn$o@!|!QUHoncz+&h~be3ysHhbPzAN?WxA14h{kxj|t3kZgztdsc))A&LLEF_(62v+~yY zsVqJsp`GpTfY&?&YX@7w6q5R#53@4`o)CI uRou~@jre&=6DdUq1} + diff --git a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java index 56056d65cc..615675544e 100644 --- a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java +++ b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java @@ -535,6 +535,8 @@ public class ZeppelinSparkClusterTest extends AbstractTestRestApi { assertEquals("1", result[1]); assertEquals("items: Seq[Object] = Buffer(2)", result[2]); assertEquals("2", result[3]); + + ZeppelinServer.notebook.removeNote(note.getId(), anonymous); } @Test @@ -568,5 +570,33 @@ public class ZeppelinSparkClusterTest extends AbstractTestRestApi { assertEquals("default_name", result[0]); assertEquals("1", result[1]); assertEquals("2", result[2]); + + ZeppelinServer.notebook.removeNote(note.getId(), anonymous); + } + + @Test + public void testConfInterpreter() throws IOException { + Note note = ZeppelinServer.notebook.createNote(AuthenticationInfo.ANONYMOUS); + Paragraph p = note.addNewParagraph(AuthenticationInfo.ANONYMOUS); + Map config = p.getConfig(); + config.put("enabled", true); + p.setConfig(config); + p.setText("%spark.conf spark.jars.packages\tcom.databricks:spark-csv_2.11:1.2.0"); + p.setAuthenticationInfo(anonymous); + note.run(p.getId()); + waitForFinish(p); + assertEquals(Status.FINISHED, p.getStatus()); + + Paragraph p1 = note.addNewParagraph(AuthenticationInfo.ANONYMOUS); + p1.setConfig(config); + p1.setText("%spark\nimport com.databricks.spark.csv._"); + p1.setAuthenticationInfo(anonymous); + note.run(p1.getId()); + + waitForFinish(p1); + assertEquals(Status.FINISHED, p1.getStatus()); + + ZeppelinServer.notebook.removeNote(note.getId(), anonymous); + } } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ConfInterpreter.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ConfInterpreter.java new file mode 100644 index 0000000000..d50c57b4da --- /dev/null +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ConfInterpreter.java @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.interpreter; + +import org.apache.commons.lang.exception.ExceptionUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.io.StringReader; +import java.util.Properties; + +/** + * Special Interpreter for Interpreter Configuration customization. It is attached to each + * InterpreterGroup implicitly by Zeppelin. + */ +public class ConfInterpreter extends Interpreter { + + private static Logger LOGGER = LoggerFactory.getLogger(ConfInterpreter.class); + + private String interpreterGroupId; + private InterpreterSetting interpreterSetting; + + + public ConfInterpreter(Properties properties, + String interpreterGroupId, + InterpreterSetting interpreterSetting) { + super(properties); + this.interpreterGroupId = interpreterGroupId; + this.interpreterSetting = interpreterSetting; + } + + @Override + public void open() throws InterpreterException { + + } + + @Override + public void close() throws InterpreterException { + + } + + @Override + public InterpreterResult interpret(String st, InterpreterContext context) + throws InterpreterException { + + try { + Properties finalProperties = new Properties(); + finalProperties.putAll(getProperties()); + Properties newProperties = new Properties(); + newProperties.load(new StringReader(st)); + finalProperties.putAll(newProperties); + LOGGER.debug("Properties for InterpreterGroup: " + interpreterGroupId + " is " + + finalProperties); + interpreterSetting.setInterpreterGroupProperties(interpreterGroupId, finalProperties); + return new InterpreterResult(InterpreterResult.Code.SUCCESS); + } catch (IOException e) { + LOGGER.error("Fail to update interpreter setting", e); + return new InterpreterResult(InterpreterResult.Code.ERROR, ExceptionUtils.getStackTrace(e)); + } + } + + @Override + public void cancel(InterpreterContext context) throws InterpreterException { + + } + + @Override + public FormType getFormType() throws InterpreterException { + return null; + } + + @Override + public int getProgress(InterpreterContext context) throws InterpreterException { + return 0; + } +} diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java index 26fcd8e93d..d5ff947ad0 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java @@ -138,9 +138,11 @@ public class InterpreterSetting { // launcher in future when we have other launcher implementation. e.g. third party launcher // service like livy private transient InterpreterLauncher launcher; - /////////////////////////////////////////////////////////////////////////////////////////// private transient LifecycleManager lifecycleManager; + /////////////////////////////////////////////////////////////////////////////////////////// + + /** * Builder class for InterpreterSetting @@ -648,12 +650,11 @@ public class InterpreterSetting { /////////////////////////////////////////////////////////////////////////////////////// // This is the only place to create interpreters. For now we always create multiple interpreter // together (one session). We don't support to create single interpreter yet. - List createInterpreters(String user, String sessionId) { + List createInterpreters(String user, String interpreterGroupId, String sessionId) { List interpreters = new ArrayList<>(); List interpreterInfos = getInterpreterInfos(); for (InterpreterInfo info : interpreterInfos) { - Interpreter interpreter = null; - interpreter = new RemoteInterpreter(getJavaProperties(), sessionId, + Interpreter interpreter = new RemoteInterpreter(getJavaProperties(), sessionId, info.getClassName(), user, lifecycleManager); if (info.isDefaultInterpreter()) { interpreters.add(0, interpreter); @@ -663,15 +664,17 @@ public class InterpreterSetting { LOGGER.info("Interpreter {} created for user: {}, sessionId: {}", interpreter.getClassName(), user, sessionId); } + interpreters.add(new ConfInterpreter(getJavaProperties(), interpreterGroupId, this)); return interpreters; } - synchronized RemoteInterpreterProcess createInterpreterProcess() throws IOException { + synchronized RemoteInterpreterProcess createInterpreterProcess(Properties properties) + throws IOException { if (launcher == null) { createLauncher(); } InterpreterLaunchContext launchContext = new - InterpreterLaunchContext(getJavaProperties(), option, interpreterRunner, id, group, name); + InterpreterLaunchContext(properties, option, interpreterRunner, id, group, name); RemoteInterpreterProcess process = (RemoteInterpreterProcess) launcher.launch(launchContext); process.setRemoteInterpreterEventPoller( new RemoteInterpreterEventPoller(remoteInterpreterProcessListener, appEventListener)); @@ -716,6 +719,11 @@ public class InterpreterSetting { return info.getClassName(); } } + //TODO(zjffdu) It requires user can not create interpreter with name `conf`, + // conf is a reserved word of interpreter name + if (replName.equals("conf")) { + return ConfInterpreter.class.getName(); + } return null; } @@ -728,6 +736,29 @@ public class InterpreterSetting { return interpreterGroup; } + /** + * Throw exception when interpreter process has already launched + * + * @param interpreterGroupId + * @param properties + * @throws IOException + */ + public void setInterpreterGroupProperties(String interpreterGroupId, Properties properties) + throws IOException { + ManagedInterpreterGroup interpreterGroup = this.interpreterGroups.get(interpreterGroupId); + for (List session : interpreterGroup.sessions.values()) { + for (Interpreter intp : session) { + if (!intp.getProperties().equals(properties) && + interpreterGroup.getRemoteInterpreterProcess() != null && + interpreterGroup.getRemoteInterpreterProcess().isRunning()) { + throw new IOException("Can not change interpreter properties when interpreter process " + + "has already been launched"); + } + intp.setProperties(properties); + } + } + } + private void loadInterpreterDependencies() { setStatus(Status.DOWNLOADING_DEPENDENCIES); setErrorReason(null); diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java index 219204f041..2378f140da 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java @@ -28,6 +28,7 @@ import org.slf4j.LoggerFactory; import java.io.IOException; import java.util.Collection; import java.util.List; +import java.util.Properties; /** * ManagedInterpreterGroup runs under zeppelin server @@ -54,10 +55,11 @@ public class ManagedInterpreterGroup extends InterpreterGroup { return interpreterSetting; } - public synchronized RemoteInterpreterProcess getOrCreateInterpreterProcess() throws IOException { + public synchronized RemoteInterpreterProcess getOrCreateInterpreterProcess(Properties properties) + throws IOException { if (remoteInterpreterProcess == null) { LOGGER.info("Create InterpreterProcess for InterpreterGroup: " + getId()); - remoteInterpreterProcess = interpreterSetting.createInterpreterProcess(); + remoteInterpreterProcess = interpreterSetting.createInterpreterProcess(properties); } return remoteInterpreterProcess; } @@ -131,7 +133,7 @@ public class ManagedInterpreterGroup extends InterpreterGroup { if (sessions.containsKey(sessionId)) { return sessions.get(sessionId); } else { - List interpreters = interpreterSetting.createInterpreters(user, sessionId); + List interpreters = interpreterSetting.createInterpreters(user, id, sessionId); for (Interpreter interpreter : interpreters) { interpreter.setInterpreterGroup(this); } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java index 4ad36cf1b2..6defd9ba82 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java @@ -25,6 +25,7 @@ import org.apache.zeppelin.display.AngularObject; import org.apache.zeppelin.display.AngularObjectRegistry; import org.apache.zeppelin.display.GUI; import org.apache.zeppelin.display.Input; +import org.apache.zeppelin.interpreter.ConfInterpreter; import org.apache.zeppelin.interpreter.Interpreter; import org.apache.zeppelin.interpreter.InterpreterContext; import org.apache.zeppelin.interpreter.InterpreterContextRunner; @@ -101,7 +102,7 @@ public class RemoteInterpreter extends Interpreter { return this.interpreterProcess; } ManagedInterpreterGroup intpGroup = getInterpreterGroup(); - this.interpreterProcess = intpGroup.getOrCreateInterpreterProcess(); + this.interpreterProcess = intpGroup.getOrCreateInterpreterProcess(properties); synchronized (interpreterProcess) { if (!interpreterProcess.isRunning()) { interpreterProcess.start(this.getUserName(), false); @@ -130,7 +131,9 @@ public class RemoteInterpreter extends Interpreter { for (Interpreter interpreter : getInterpreterGroup() .getOrCreateSession(this.getUserName(), sessionId)) { try { - ((RemoteInterpreter) interpreter).internal_create(); + if (!(interpreter instanceof ConfInterpreter)) { + ((RemoteInterpreter) interpreter).internal_create(); + } } catch (IOException e) { throw new InterpreterException(e); } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java index 5ec132931f..32b9b73263 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Paragraph.java @@ -351,6 +351,7 @@ public class Paragraph extends Job implements Cloneable, JsonSerializable { setStatus(Job.Status.ERROR); throw intpException; } + setStatus(Status.READY); if (getConfig().get("enabled") == null || (Boolean) getConfig().get("enabled")) { setAuthenticationInfo(getAuthenticationInfo()); interpreter.getScheduler().submit(this); diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ConfInterpreterTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ConfInterpreterTest.java new file mode 100644 index 0000000000..4d74c7cbfb --- /dev/null +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ConfInterpreterTest.java @@ -0,0 +1,102 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.interpreter; + +import com.sun.net.httpserver.Authenticator; +import org.apache.zeppelin.display.GUI; +import org.apache.zeppelin.interpreter.remote.RemoteInterpreter; +import org.apache.zeppelin.user.AuthenticationInfo; +import org.junit.Test; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; + +public class ConfInterpreterTest extends AbstractInterpreterTest { + + @Test + public void testCorrectConf() throws IOException, InterpreterException { + interpreterSettingManager.setInterpreterBinding("user1", "note1", interpreterSettingManager.getSettingIds()); + assertTrue(interpreterFactory.getInterpreter("user1", "note1", "test.conf") instanceof ConfInterpreter); + ConfInterpreter confInterpreter = (ConfInterpreter) interpreterFactory.getInterpreter("user1", "note1", "test.conf"); + + InterpreterContext context = new InterpreterContext("noteId", "paragraphId", "repl", + "title", "text", AuthenticationInfo.ANONYMOUS, new HashMap(), new GUI(), new GUI(), + null, null, new ArrayList(), null); + InterpreterResult result = confInterpreter.interpret("property_1\tnew_value\nnew_property\tdummy_value", context); + assertEquals(InterpreterResult.Code.SUCCESS, result.code); + + assertTrue(interpreterFactory.getInterpreter("user1", "note1", "test") instanceof RemoteInterpreter); + RemoteInterpreter remoteInterpreter = (RemoteInterpreter) interpreterFactory.getInterpreter("user1", "note1", "test"); + remoteInterpreter.interpret("hello world", context); + assertEquals(7, remoteInterpreter.getProperties().size()); + assertEquals("new_value", remoteInterpreter.getProperty("property_1")); + assertEquals("dummy_value", remoteInterpreter.getProperty("new_property")); + assertEquals("value_3", remoteInterpreter.getProperty("property_3")); + + // rerun the paragraph with the same properties would result in SUCCESS + result = confInterpreter.interpret("property_1\tnew_value\nnew_property\tdummy_value", context); + assertEquals(InterpreterResult.Code.SUCCESS, result.code); + + // run the paragraph with the same properties would result in ERROR + result = confInterpreter.interpret("property_1\tnew_value_2\nnew_property\tdummy_value", context); + assertEquals(InterpreterResult.Code.ERROR, result.code); + } + + @Test + public void testEmptyConf() throws IOException, InterpreterException { + interpreterSettingManager.setInterpreterBinding("user1", "note1", interpreterSettingManager.getSettingIds()); + assertTrue(interpreterFactory.getInterpreter("user1", "note1", "test.conf") instanceof ConfInterpreter); + ConfInterpreter confInterpreter = (ConfInterpreter) interpreterFactory.getInterpreter("user1", "note1", "test.conf"); + + InterpreterContext context = new InterpreterContext("noteId", "paragraphId", "repl", + "title", "text", AuthenticationInfo.ANONYMOUS, new HashMap(), new GUI(), new GUI(), + null, null, new ArrayList(), null); + InterpreterResult result = confInterpreter.interpret("", context); + assertEquals(InterpreterResult.Code.SUCCESS, result.code); + + assertTrue(interpreterFactory.getInterpreter("user1", "note1", "test") instanceof RemoteInterpreter); + RemoteInterpreter remoteInterpreter = (RemoteInterpreter) interpreterFactory.getInterpreter("user1", "note1", "test"); + assertEquals(6, remoteInterpreter.getProperties().size()); + assertEquals("value_1", remoteInterpreter.getProperty("property_1")); + assertEquals("value_3", remoteInterpreter.getProperty("property_3")); + } + + + @Test + public void testRunningAfterOtherInterpreter() throws IOException, InterpreterException { + interpreterSettingManager.setInterpreterBinding("user1", "note1", interpreterSettingManager.getSettingIds()); + assertTrue(interpreterFactory.getInterpreter("user1", "note1", "test.conf") instanceof ConfInterpreter); + ConfInterpreter confInterpreter = (ConfInterpreter) interpreterFactory.getInterpreter("user1", "note1", "test.conf"); + + InterpreterContext context = new InterpreterContext("noteId", "paragraphId", "repl", + "title", "text", AuthenticationInfo.ANONYMOUS, new HashMap(), new GUI(), new GUI(), + null, null, new ArrayList(), null); + RemoteInterpreter remoteInterpreter = (RemoteInterpreter) interpreterFactory.getInterpreter("user1", "note1", "test"); + InterpreterResult result = remoteInterpreter.interpret("hello world", context); + assertEquals(InterpreterResult.Code.SUCCESS, result.code); + + result = confInterpreter.interpret("property_1\tnew_value\nnew_property\tdummy_value", context); + assertEquals(InterpreterResult.Code.ERROR, result.code); + } + +} diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroupTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroupTest.java index 74bd201085..aa7374991b 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroupTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroupTest.java @@ -62,7 +62,7 @@ public class ManagedInterpreterGroupTest { // create session_1 List interpreters = interpreterGroup.getOrCreateSession("user1", "session_1"); - assertEquals(2, interpreters.size()); + assertEquals(3, interpreters.size()); assertEquals(EchoInterpreter.class.getName(), interpreters.get(0).getClassName()); assertEquals(DoubleEchoInterpreter.class.getName(), interpreters.get(1).getClassName()); assertEquals(1, interpreterGroup.getSessionNum()); @@ -73,7 +73,7 @@ public class ManagedInterpreterGroupTest { // create session_2 List interpreters2 = interpreterGroup.getOrCreateSession("user1", "session_2"); - assertEquals(2, interpreters2.size()); + assertEquals(3, interpreters2.size()); assertEquals(EchoInterpreter.class.getName(), interpreters2.get(0).getClassName()); assertEquals(DoubleEchoInterpreter.class.getName(), interpreters2.get(1).getClassName()); assertEquals(2, interpreterGroup.getSessionNum()); From 4c8f20ae33ceb47209402c0469791d7a19571471 Mon Sep 17 00:00:00 2001 From: Jeff Zhang Date: Tue, 5 Dec 2017 16:27:12 +0800 Subject: [PATCH 11/14] ZEPPELIN-3051. Support Interpreter Process Recovery ### What is this PR for? This PR is for the purpose of recover running interpreter process when zeppelin server is restarted. This would be useful when restarting zeppelin without interrupt current running interpreter processes, should be useful when admin do maintenance or upgrading. Interface `RecoveryStorage` is used for storing the information of running interpreter process. Currently it only has one implementation `FileSystemRecoveryStorage`, other implementation could be done later (such as zookeeper based). `InterpreterLauncher` is the component where to recover the running interpreter process. Test: * RecoveryTest.java * FileSystemRecoveryStorageTest.java Design Doc: https://docs.google.com/document/d/1Plm3Hd40aGdNaXmjdsoY4ek3f-gTijTMGMkNjAZN39Y/edit?usp=sharing ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3051 ### How should this be tested? Unit test & Integration Test is added. Also manually verified. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang Closes #2668 from zjffdu/ZEPPELIN-3051 and squashes the following commits: a4c9b9c [Jeff Zhang] address comments 575b7b9 [Jeff Zhang] fix the pid of interpreter process id 02b118f [Jeff Zhang] address comments da7cbb9 [Jeff Zhang] ZEPPELIN-3051. Support Interpreter Process Recovery --- bin/interpreter.sh | 2 +- bin/stop-interpreter.sh | 47 +++++ bin/zeppelin-daemon.sh | 12 -- conf/zeppelin-site.xml.template | 41 +++++ docs/usage/interpreter/overview.md | 8 + .../main/resources/interpreter-setting.json | 2 +- .../zeppelin/conf/ZeppelinConfiguration.java | 17 ++ .../launcher/InterpreterClient.java | 14 +- .../launcher/InterpreterLaunchContext.java | 7 + .../launcher/InterpreterLauncher.java | 5 +- .../interpreter/recovery/RecoveryStorage.java | 80 +++++++++ zeppelin-server/notebook/.python.recovery.crc | Bin 0 -> 12 bytes zeppelin-server/notebook/python.recovery | 1 + zeppelin-server/pom.xml | 15 ++ .../zeppelin/server/ZeppelinServer.java | 10 +- .../zeppelin/recovery/RecoveryTest.java | 162 +++++++++++++++++ .../zeppelin/rest/AbstractTestRestApi.java | 13 +- .../interpreter/InterpreterSetting.java | 45 ++++- .../InterpreterSettingManager.java | 38 +++- .../interpreter/ManagedInterpreterGroup.java | 25 ++- .../launcher/ShellScriptLauncher.java | 30 +++- .../launcher/SparkInterpreterLauncher.java | 5 +- .../recovery/FileSystemRecoveryStorage.java | 139 +++++++++++++++ .../recovery/NullRecoveryStorage.java | 54 ++++++ .../interpreter/recovery/StopInterpreter.java | 40 +++++ .../interpreter/remote/RemoteInterpreter.java | 11 +- .../RemoteInterpreterManagedProcess.java | 3 +- .../remote/RemoteInterpreterProcess.java | 6 - .../RemoteInterpreterRunningProcess.java | 28 ++- .../zeppelin/notebook/FileSystemStorage.java | 168 ++++++++++++++++++ .../notebook/repo/FileSystemNotebookRepo.java | 124 +++---------- .../apache/zeppelin/util/ReflectionUtils.java | 99 +++++++++++ .../interpreter/AbstractInterpreterTest.java | 2 +- .../launcher/ShellScriptLauncherTest.java | 7 +- .../SparkInterpreterLauncherTest.java | 31 ++-- .../FileSystemRecoveryStorageTest.java | 92 ++++++++++ 36 files changed, 1195 insertions(+), 188 deletions(-) create mode 100755 bin/stop-interpreter.sh create mode 100644 zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/recovery/RecoveryStorage.java create mode 100644 zeppelin-server/notebook/.python.recovery.crc create mode 100644 zeppelin-server/notebook/python.recovery create mode 100644 zeppelin-server/src/test/java/org/apache/zeppelin/recovery/RecoveryTest.java create mode 100644 zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorage.java create mode 100644 zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/NullRecoveryStorage.java create mode 100644 zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/StopInterpreter.java create mode 100644 zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java create mode 100644 zeppelin-zengine/src/main/java/org/apache/zeppelin/util/ReflectionUtils.java create mode 100644 zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorageTest.java diff --git a/bin/interpreter.sh b/bin/interpreter.sh index 458ffc00d4..f23ca823e6 100755 --- a/bin/interpreter.sh +++ b/bin/interpreter.sh @@ -220,8 +220,8 @@ if [[ ! -z "$ZEPPELIN_IMPERSONATE_USER" ]] && [[ -n "${suid}" || -z "${SPARK_SUB fi eval $INTERPRETER_RUN_COMMAND & - pid=$! + if [[ -z "${pid}" ]]; then exit 1; else diff --git a/bin/stop-interpreter.sh b/bin/stop-interpreter.sh new file mode 100755 index 0000000000..e6ff16e9e9 --- /dev/null +++ b/bin/stop-interpreter.sh @@ -0,0 +1,47 @@ +#!/bin/bash +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Stop Zeppelin Interpreter Processes +# + +bin=$(dirname "${BASH_SOURCE-$0}") +bin=$(cd "${bin}">/dev/null; pwd) + +. "${bin}/common.sh" + +export ZEPPELIN_FORCE_STOP=1 + +ZEPPELIN_STOP_INTERPRETER_MAIN=org.apache.zeppelin.interpreter.recovery.StopInterpreter +ZEPPELIN_LOGFILE="${ZEPPELIN_LOG_DIR}/stop-interpreter.log" +JAVA_OPTS+=" -Dzeppelin.log.file=${ZEPPELIN_LOGFILE}" + +if [[ -d "${ZEPPELIN_HOME}/zeppelin-zengine/target/classes" ]]; then + ZEPPELIN_CLASSPATH+=":${ZEPPELIN_HOME}/zeppelin-zengine/target/classes" +fi + +if [[ -d "${ZEPPELIN_HOME}/zeppelin-interpreter/target/classes" ]]; then + ZEPPELIN_CLASSPATH+=":${ZEPPELIN_HOME}/zeppelin-interpreter/target/classes" +fi + +addJarInDir "${ZEPPELIN_HOME}/zeppelin-interpreter/target/lib" +addJarInDir "${ZEPPELIN_HOME}/zeppelin-server/target/lib" +addJarInDir "${ZEPPELIN_HOME}/lib" +addJarInDir "${ZEPPELIN_HOME}/lib/interpreter" + +CLASSPATH+=":${ZEPPELIN_CLASSPATH}" +$ZEPPELIN_RUNNER $JAVA_OPTS -cp $CLASSPATH $ZEPPELIN_STOP_INTERPRETER_MAIN ${@} diff --git a/bin/zeppelin-daemon.sh b/bin/zeppelin-daemon.sh index 5982aee2e0..e898849751 100755 --- a/bin/zeppelin-daemon.sh +++ b/bin/zeppelin-daemon.sh @@ -217,18 +217,6 @@ function stop() { action_msg "${ZEPPELIN_NAME} stop" "${SET_OK}" fi fi - - # list all pid that used in remote interpreter and kill them - for f in ${ZEPPELIN_PID_DIR}/*.pid; do - if [[ ! -f ${f} ]]; then - continue; - fi - - pid=$(cat ${f}) - wait_for_zeppelin_to_die $pid 20 - $(rm -f ${f}) - done - } function find_zeppelin_process() { diff --git a/conf/zeppelin-site.xml.template b/conf/zeppelin-site.xml.template index 3c5bbeae59..d566a71788 100755 --- a/conf/zeppelin-site.xml.template +++ b/conf/zeppelin-site.xml.template @@ -480,4 +480,45 @@ 10000:10010 --> + + + + + + + + + + + + diff --git a/docs/usage/interpreter/overview.md b/docs/usage/interpreter/overview.md index dd5ed220c8..035c381b8a 100644 --- a/docs/usage/interpreter/overview.md +++ b/docs/usage/interpreter/overview.md @@ -144,3 +144,11 @@ So users needs to understand the ([interpreter mode setting ](../usage/interpret In this scenario, user need to put `ConfInterpreter` as the first paragraph as the below example. Otherwise the customized setting can not be applied (Actually it would report ERROR) + +## Interpreter Process Recovery + +Before 0.8.0, shutting down Zeppelin also mean to shutdown all the running interpreter processes. Usually admin will shutdown Zeppelin server for maintenance or upgrade, but don't want to shut down the running interpreter processes. +In such cases, interpreter process recovery is necessary. Starting from 0.8.0, user can enable interpreter process recovering via setting `zeppelin.recovery.storage.class` as +`org.apache.zeppelin.interpreter.recovery.FileSystemRecoveryStorage` or other implementations if available in future, by default it is `org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage` + which means recovery is not enabled. Enable recover means shutting down Zeppelin would not terminating interpreter process, +and when Zeppelin is restarted, it would try to reconnect to the existing running interpreter processes. If you want to kill all the interpreter processes after terminating Zeppelin even when recovery is enabled, you can run `bin/stop-interpreter.sh` diff --git a/spark/src/main/resources/interpreter-setting.json b/spark/src/main/resources/interpreter-setting.json index 485f6950df..d656532eb0 100644 --- a/spark/src/main/resources/interpreter-setting.json +++ b/spark/src/main/resources/interpreter-setting.json @@ -61,7 +61,7 @@ "description": "Spark master uri. ex) spark://masterhost:7077", "type": "string" }, - "zeppelin.spark.unSupportedVersionCheck": { + "zeppelin.spark.enableSupportedVersionCheck": { "envName": null, "propertyName": "zeppelin.spark.enableSupportedVersionCheck", "defaultValue": true, diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java index 438c661f8b..77279edcd3 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java @@ -355,6 +355,19 @@ public class ZeppelinConfiguration extends XMLConfiguration { return getString(ConfVars.ZEPPELIN_NOTEBOOK_DIR); } + public String getRecoveryDir() { + return getRelativeDir(ConfVars.ZEPPELIN_RECOVERY_DIR); + } + + public String getRecoveryStorageClass() { + return getString(ConfVars.ZEPPELIN_RECOVERY_STORAGE_CLASS); + } + + public boolean isRecoveryEnabled() { + return !getString(ConfVars.ZEPPELIN_RECOVERY_STORAGE_CLASS).equals( + "org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage"); + } + public String getUser() { return getString(ConfVars.ZEPPELIN_NOTEBOOK_S3_USER); } @@ -658,6 +671,10 @@ public class ZeppelinConfiguration extends XMLConfiguration { ZEPPELIN_INTERPRETER_OUTPUT_LIMIT("zeppelin.interpreter.output.limit", 1024 * 100), ZEPPELIN_ENCODING("zeppelin.encoding", "UTF-8"), ZEPPELIN_NOTEBOOK_DIR("zeppelin.notebook.dir", "notebook"), + ZEPPELIN_RECOVERY_DIR("zeppelin.recovery.dir", "recovery"), + ZEPPELIN_RECOVERY_STORAGE_CLASS("zeppelin.recovery.storage.class", + "org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage"), + // use specified notebook (id) as homescreen ZEPPELIN_NOTEBOOK_HOMESCREEN("zeppelin.notebook.homescreen", null), // whether homescreen notebook will be hidden from notebook list or not diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterClient.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterClient.java index b991079fec..813dad8688 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterClient.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterClient.java @@ -19,8 +19,20 @@ package org.apache.zeppelin.interpreter.launcher; /** * Interface to InterpreterClient which is created by InterpreterLauncher. This is the component - * that is used to for the communication fromzeppelin-server process to zeppelin interpreter process + * that is used to for the communication from zeppelin-server process to zeppelin interpreter + * process. */ public interface InterpreterClient { + String getInterpreterSettingName(); + + void start(String userName, Boolean isUserImpersonate); + + void stop(); + + String getHost(); + + int getPort(); + + boolean isRunning(); } diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLaunchContext.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLaunchContext.java index 9e253555a9..6901e2c7a6 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLaunchContext.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLaunchContext.java @@ -30,6 +30,7 @@ public class InterpreterLaunchContext { private Properties properties; private InterpreterOption option; private InterpreterRunner runner; + private String interpreterGroupId; private String interpreterSettingId; private String interpreterSettingGroup; private String interpreterSettingName; @@ -37,12 +38,14 @@ public class InterpreterLaunchContext { public InterpreterLaunchContext(Properties properties, InterpreterOption option, InterpreterRunner runner, + String interpreterGroupId, String interpreterSettingId, String interpreterSettingGroup, String interpreterSettingName) { this.properties = properties; this.option = option; this.runner = runner; + this.interpreterGroupId = interpreterGroupId; this.interpreterSettingId = interpreterSettingId; this.interpreterSettingGroup = interpreterSettingGroup; this.interpreterSettingName = interpreterSettingName; @@ -60,6 +63,10 @@ public class InterpreterLaunchContext { return runner; } + public String getInterpreterGroupId() { + return interpreterGroupId; + } + public String getInterpreterSettingId() { return interpreterSettingId; } diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLauncher.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLauncher.java index 5d0acf3515..1cee20e7a0 100644 --- a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLauncher.java +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/launcher/InterpreterLauncher.java @@ -18,6 +18,7 @@ package org.apache.zeppelin.interpreter.launcher; import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.recovery.RecoveryStorage; import java.io.IOException; import java.util.Properties; @@ -29,9 +30,11 @@ public abstract class InterpreterLauncher { protected ZeppelinConfiguration zConf; protected Properties properties; + protected RecoveryStorage recoveryStorage; - public InterpreterLauncher(ZeppelinConfiguration zConf) { + public InterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage recoveryStorage) { this.zConf = zConf; + this.recoveryStorage = recoveryStorage; } public abstract InterpreterClient launch(InterpreterLaunchContext context) throws IOException; diff --git a/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/recovery/RecoveryStorage.java b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/recovery/RecoveryStorage.java new file mode 100644 index 0000000000..8bbe8302fc --- /dev/null +++ b/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/recovery/RecoveryStorage.java @@ -0,0 +1,80 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.interpreter.recovery; + +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.launcher.InterpreterClient; + +import java.io.IOException; +import java.util.Map; + + +/** + * Interface for storing interpreter process recovery metadata. + * + */ +public abstract class RecoveryStorage { + + protected ZeppelinConfiguration zConf; + protected Map restoredClients; + + public RecoveryStorage(ZeppelinConfiguration zConf) throws IOException { + this.zConf = zConf; + } + + /** + * Update RecoveryStorage when new InterpreterClient is started + * @param client + * @throws IOException + */ + public abstract void onInterpreterClientStart(InterpreterClient client) throws IOException; + + /** + * Update RecoveryStorage when InterpreterClient is stopped + * @param client + * @throws IOException + */ + public abstract void onInterpreterClientStop(InterpreterClient client) throws IOException; + + /** + * + * It is only called when Zeppelin Server is started. + * + * @return + * @throws IOException + */ + public abstract Map restore() throws IOException; + + + /** + * It is called after constructor + * + * @throws IOException + */ + public void init() throws IOException { + this.restoredClients = restore(); + } + + public InterpreterClient getInterpreterClient(String interpreterGroupId) { + if (restoredClients.containsKey(interpreterGroupId)) { + return restoredClients.get(interpreterGroupId); + } else { + return null; + } + } +} diff --git a/zeppelin-server/notebook/.python.recovery.crc b/zeppelin-server/notebook/.python.recovery.crc new file mode 100644 index 0000000000000000000000000000000000000000..6bd3e7ae43b861a0504394ec70405b321f2d9b0a GIT binary patch literal 12 TcmYc;N@ieSU}E^);Is??6cz)X literal 0 HcmV?d00001 diff --git a/zeppelin-server/notebook/python.recovery b/zeppelin-server/notebook/python.recovery new file mode 100644 index 0000000000..eaf4938fda --- /dev/null +++ b/zeppelin-server/notebook/python.recovery @@ -0,0 +1 @@ +2CZA1DVUG:shared_process 192.168.3.2:55410 \ No newline at end of file diff --git a/zeppelin-server/pom.xml b/zeppelin-server/pom.xml index 08ede293e4..925c637fcf 100644 --- a/zeppelin-server/pom.xml +++ b/zeppelin-server/pom.xml @@ -349,6 +349,21 @@ + + maven-surefire-plugin + ${plugin.surefire.version} + + -Xmx2g -Xms1g -Dfile.encoding=UTF-8 + + ${tests.to.exclude} + + + 1 + + + + + org.scala-tools maven-scala-plugin diff --git a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java index 0b66a437d5..f8625c2357 100644 --- a/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java +++ b/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java @@ -162,7 +162,7 @@ public class ZeppelinServer extends Application { public static void main(String[] args) throws InterruptedException { - ZeppelinConfiguration conf = ZeppelinConfiguration.create(); + final ZeppelinConfiguration conf = ZeppelinConfiguration.create(); conf.setProperty("args", args); jettyWebServer = setupJettyServer(conf); @@ -199,7 +199,9 @@ public class ZeppelinServer extends Application { LOG.info("Shutting down Zeppelin Server ... "); try { jettyWebServer.stop(); - notebook.getInterpreterSettingManager().close(); + if (!conf.isRecoveryEnabled()) { + ZeppelinServer.notebook.getInterpreterSettingManager().close(); + } notebook.close(); Thread.sleep(3000); } catch (Exception e) { @@ -222,7 +224,9 @@ public class ZeppelinServer extends Application { } jettyWebServer.join(); - ZeppelinServer.notebook.getInterpreterSettingManager().close(); + if (!conf.isRecoveryEnabled()) { + ZeppelinServer.notebook.getInterpreterSettingManager().close(); + } } private static Server setupJettyServer(ZeppelinConfiguration conf) { diff --git a/zeppelin-server/src/test/java/org/apache/zeppelin/recovery/RecoveryTest.java b/zeppelin-server/src/test/java/org/apache/zeppelin/recovery/RecoveryTest.java new file mode 100644 index 0000000000..37277ee0c3 --- /dev/null +++ b/zeppelin-server/src/test/java/org/apache/zeppelin/recovery/RecoveryTest.java @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.recovery; + +import com.google.common.io.Files; +import com.google.gson.Gson; +import com.google.gson.reflect.TypeToken; +import org.apache.commons.httpclient.methods.PostMethod; +import org.apache.commons.io.FileUtils; +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.ManagedInterpreterGroup; +import org.apache.zeppelin.interpreter.recovery.FileSystemRecoveryStorage; +import org.apache.zeppelin.interpreter.recovery.StopInterpreter; +import org.apache.zeppelin.notebook.Note; +import org.apache.zeppelin.notebook.Paragraph; +import org.apache.zeppelin.rest.AbstractTestRestApi; +import org.apache.zeppelin.scheduler.Job; +import org.apache.zeppelin.server.ZeppelinServer; +import org.apache.zeppelin.user.AuthenticationInfo; +import org.junit.AfterClass; +import org.junit.BeforeClass; +import org.junit.Test; + +import java.io.File; +import java.util.Map; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertThat; + +public class RecoveryTest extends AbstractTestRestApi { + + private Gson gson = new Gson(); + private static File recoveryDir = null; + + @BeforeClass + public static void init() throws Exception { + System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_RECOVERY_STORAGE_CLASS.getVarName(), + FileSystemRecoveryStorage.class.getName()); + recoveryDir = Files.createTempDir(); + System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_RECOVERY_DIR.getVarName(), recoveryDir.getAbsolutePath()); + startUp(RecoveryTest.class.getSimpleName()); + } + + @AfterClass + public static void destroy() throws Exception { + shutDown(); + FileUtils.deleteDirectory(recoveryDir); + } + + @Test + public void testRecovery() throws Exception { + Note note1 = ZeppelinServer.notebook.createNote(AuthenticationInfo.ANONYMOUS); + + // run python interpreter and create new variable `user` + Paragraph p1 = note1.addNewParagraph(AuthenticationInfo.ANONYMOUS); + p1.setText("%python user='abc'"); + PostMethod post = httpPost("/notebook/job/" + note1.getId(), ""); + assertThat(post, isAllowed()); + Map resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken>() { + }.getType()); + assertEquals(resp.get("status"), "OK"); + post.releaseConnection(); + assertEquals(Job.Status.FINISHED, p1.getStatus()); + + // shutdown zeppelin and restart it + shutDown(); + startUp(RecoveryTest.class.getSimpleName()); + + // run the paragraph again, but change the text to print variable `user` + note1 = ZeppelinServer.notebook.getNote(note1.getId()); + p1 = note1.getParagraph(p1.getId()); + p1.setText("%python print(user)"); + post = httpPost("/notebook/job/" + note1.getId(), ""); + assertEquals(resp.get("status"), "OK"); + post.releaseConnection(); + assertEquals(Job.Status.FINISHED, p1.getStatus()); + assertEquals("abc\n", p1.getResult().message().get(0).getData()); + } + + @Test + public void testRecovery_2() throws Exception { + Note note1 = ZeppelinServer.notebook.createNote(AuthenticationInfo.ANONYMOUS); + + // run python interpreter and create new variable `user` + Paragraph p1 = note1.addNewParagraph(AuthenticationInfo.ANONYMOUS); + p1.setText("%python user='abc'"); + PostMethod post = httpPost("/notebook/job/" + note1.getId(), ""); + assertThat(post, isAllowed()); + Map resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken>() { + }.getType()); + assertEquals(resp.get("status"), "OK"); + post.releaseConnection(); + assertEquals(Job.Status.FINISHED, p1.getStatus()); + + // restart the python interpreter + ZeppelinServer.notebook.getInterpreterSettingManager().restart( + ((ManagedInterpreterGroup) p1.getBindedInterpreter().getInterpreterGroup()) + .getInterpreterSetting().getId() + ); + + // shutdown zeppelin and restart it + shutDown(); + startUp(RecoveryTest.class.getSimpleName()); + + // run the paragraph again, but change the text to print variable `user`. + // can not recover the python interpreter, because it has been shutdown. + note1 = ZeppelinServer.notebook.getNote(note1.getId()); + p1 = note1.getParagraph(p1.getId()); + p1.setText("%python print(user)"); + post = httpPost("/notebook/job/" + note1.getId(), ""); + assertEquals(resp.get("status"), "OK"); + post.releaseConnection(); + assertEquals(Job.Status.ERROR, p1.getStatus()); + } + + @Test + public void testRecovery_3() throws Exception { + Note note1 = ZeppelinServer.notebook.createNote(AuthenticationInfo.ANONYMOUS); + + // run python interpreter and create new variable `user` + Paragraph p1 = note1.addNewParagraph(AuthenticationInfo.ANONYMOUS); + p1.setText("%python user='abc'"); + PostMethod post = httpPost("/notebook/job/" + note1.getId(), ""); + assertThat(post, isAllowed()); + Map resp = gson.fromJson(post.getResponseBodyAsString(), new TypeToken>() { + }.getType()); + assertEquals(resp.get("status"), "OK"); + post.releaseConnection(); + assertEquals(Job.Status.FINISHED, p1.getStatus()); + + // shutdown zeppelin and restart it + shutDown(); + StopInterpreter.main(new String[]{}); + + startUp(RecoveryTest.class.getSimpleName()); + + // run the paragraph again, but change the text to print variable `user`. + // can not recover the python interpreter, because it has been shutdown. + note1 = ZeppelinServer.notebook.getNote(note1.getId()); + p1 = note1.getParagraph(p1.getId()); + p1.setText("%python print(user)"); + post = httpPost("/notebook/job/" + note1.getId(), ""); + assertEquals(resp.get("status"), "OK"); + post.releaseConnection(); + assertEquals(Job.Status.ERROR, p1.getStatus()); + } +} diff --git a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java index 431e3647b2..7c08365081 100644 --- a/zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java +++ b/zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java @@ -318,8 +318,10 @@ public abstract class AbstractTestRestApi { if (!wasRunning) { // restart interpreter to stop all interpreter processes List settingList = ZeppelinServer.notebook.getInterpreterSettingManager().get(); - for (InterpreterSetting setting : settingList) { - ZeppelinServer.notebook.getInterpreterSettingManager().restart(setting.getId()); + if (!ZeppelinServer.notebook.getConf().isRecoveryEnabled()) { + for (InterpreterSetting setting : settingList) { + ZeppelinServer.notebook.getInterpreterSettingManager().restart(setting.getId()); + } } if (shiroIni != null) { FileUtils.deleteQuietly(shiroIni); @@ -350,7 +352,12 @@ public abstract class AbstractTestRestApi { .clearProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_ANONYMOUS_ALLOWED.getVarName()); } - FileUtils.deleteDirectory(confDir); + if (!ZeppelinServer.notebook.getConf().isRecoveryEnabled()) { + // don't delete interpreter.json when recovery is enabled. otherwise the interpreter setting + // id will change after zeppelin restart, then we can not recover interpreter process + // properly + FileUtils.deleteDirectory(confDir); + } } } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java index d5ff947ad0..424aa27a16 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java @@ -38,6 +38,8 @@ import org.apache.zeppelin.interpreter.launcher.InterpreterLauncher; import org.apache.zeppelin.interpreter.launcher.ShellScriptLauncher; import org.apache.zeppelin.interpreter.launcher.SparkInterpreterLauncher; import org.apache.zeppelin.interpreter.lifecycle.NullLifecycleManager; +import org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage; +import org.apache.zeppelin.interpreter.recovery.RecoveryStorage; import org.apache.zeppelin.interpreter.remote.RemoteAngularObjectRegistry; import org.apache.zeppelin.interpreter.remote.RemoteInterpreter; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller; @@ -144,6 +146,9 @@ public class InterpreterSetting { + private transient RecoveryStorage recoveryStorage; + /////////////////////////////////////////////////////////////////////////////////////////// + /** * Builder class for InterpreterSetting */ @@ -242,6 +247,11 @@ public class InterpreterSetting { return this; } + public Builder setRecoveryStorage(RecoveryStorage recoveryStorage) { + interpreterSetting.recoveryStorage = recoveryStorage; + return this; + } + public InterpreterSetting create() { // post processing interpreterSetting.postProcessing(); @@ -261,6 +271,13 @@ public class InterpreterSetting { if (this.lifecycleManager == null) { this.lifecycleManager = new NullLifecycleManager(conf); } + if (this.recoveryStorage == null) { + try { + this.recoveryStorage = new NullRecoveryStorage(conf, interpreterSettingManager); + } catch (IOException e) { + // ignore this exception as NullRecoveryStorage will do nothing. + } + } } /** @@ -285,9 +302,9 @@ public class InterpreterSetting { private void createLauncher() { if (group.equals("spark")) { - this.launcher = new SparkInterpreterLauncher(this.conf); + this.launcher = new SparkInterpreterLauncher(this.conf, this.recoveryStorage); } else { - this.launcher = new ShellScriptLauncher(this.conf); + this.launcher = new ShellScriptLauncher(this.conf, this.recoveryStorage); } } @@ -344,6 +361,15 @@ public class InterpreterSetting { return this; } + public InterpreterSetting setRecoveryStorage(RecoveryStorage recoveryStorage) { + this.recoveryStorage = recoveryStorage; + return this; + } + + public RecoveryStorage getRecoveryStorage() { + return recoveryStorage; + } + public LifecycleManager getLifecycleManager() { return lifecycleManager; } @@ -408,7 +434,12 @@ public class InterpreterSetting { } void removeInterpreterGroup(String groupId) { - this.interpreterGroups.remove(groupId); + try { + interpreterGroupWriteLock.lock(); + this.interpreterGroups.remove(groupId); + } finally { + interpreterGroupWriteLock.unlock(); + } } public ManagedInterpreterGroup getInterpreterGroup(String user, String noteId) { @@ -425,7 +456,6 @@ public class InterpreterSetting { return interpreterGroups.get(groupId); } - @VisibleForTesting public ArrayList getAllInterpreterGroups() { try { interpreterGroupReadLock.lock(); @@ -668,16 +698,19 @@ public class InterpreterSetting { return interpreters; } - synchronized RemoteInterpreterProcess createInterpreterProcess(Properties properties) + synchronized RemoteInterpreterProcess createInterpreterProcess(String interpreterGroupId, + Properties properties) throws IOException { if (launcher == null) { createLauncher(); } InterpreterLaunchContext launchContext = new - InterpreterLaunchContext(properties, option, interpreterRunner, id, group, name); + InterpreterLaunchContext(properties, option, interpreterRunner, + interpreterGroupId, id, group, name); RemoteInterpreterProcess process = (RemoteInterpreterProcess) launcher.launch(launchContext); process.setRemoteInterpreterEventPoller( new RemoteInterpreterEventPoller(remoteInterpreterProcessListener, appEventListener)); + recoveryStorage.onInterpreterClientStart(process); return process; } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java index 0b7efd5db6..42f82fad21 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java @@ -34,12 +34,16 @@ import org.apache.zeppelin.dep.DependencyResolver; import org.apache.zeppelin.display.AngularObjectRegistryListener; import org.apache.zeppelin.helium.ApplicationEventListener; import org.apache.zeppelin.interpreter.Interpreter.RegisteredInterpreter; +import org.apache.zeppelin.interpreter.recovery.FileSystemRecoveryStorage; +import org.apache.zeppelin.interpreter.recovery.NullRecoveryStorage; +import org.apache.zeppelin.interpreter.recovery.RecoveryStorage; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcessListener; import org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService; import org.apache.zeppelin.resource.Resource; import org.apache.zeppelin.resource.ResourcePool; import org.apache.zeppelin.resource.ResourceSet; +import org.apache.zeppelin.util.ReflectionUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.sonatype.aether.repository.Authentication; @@ -118,6 +122,7 @@ public class InterpreterSettingManager { private ApplicationEventListener appEventListener; private DependencyResolver dependencyResolver; private LifecycleManager lifecycleManager; + private RecoveryStorage recoveryStorage; public InterpreterSettingManager(ZeppelinConfiguration zeppelinConfiguration, AngularObjectRegistryListener angularObjectRegistryListener, @@ -154,13 +159,17 @@ public class InterpreterSettingManager { this.angularObjectRegistryListener = angularObjectRegistryListener; this.remoteInterpreterProcessListener = remoteInterpreterProcessListener; this.appEventListener = appEventListener; - try { - this.lifecycleManager = (LifecycleManager) - Class.forName(conf.getLifecycleManagerClass()).getConstructor(ZeppelinConfiguration.class) - .newInstance(conf); - } catch (Exception e) { - throw new IOException("Fail to create LifecycleManager", e); - } + + this.recoveryStorage = ReflectionUtils.createClazzInstance(conf.getRecoveryStorageClass(), + new Class[] {ZeppelinConfiguration.class, InterpreterSettingManager.class}, + new Object[] {conf, this}); + this.recoveryStorage.init(); + LOGGER.info("Using RecoveryStorage: " + this.recoveryStorage.getClass().getName()); + + this.lifecycleManager = ReflectionUtils.createClazzInstance(conf.getLifecycleManagerClass(), + new Class[] {ZeppelinConfiguration.class}, + new Object[] {conf}); + LOGGER.info("Using LifecycleManager: " + this.lifecycleManager.getClass().getName()); init(); } @@ -174,6 +183,7 @@ public class InterpreterSettingManager { .setAppEventListener(appEventListener) .setDependencyResolver(dependencyResolver) .setLifecycleManager(lifecycleManager) + .setRecoveryStorage(recoveryStorage) .postProcessing(); } @@ -307,8 +317,16 @@ public class InterpreterSettingManager { saveToFile(); } + public RemoteInterpreterProcessListener getRemoteInterpreterProcessListener() { + return remoteInterpreterProcessListener; + } + + public ApplicationEventListener getAppEventListener() { + return appEventListener; + } + private boolean registerInterpreterFromResource(ClassLoader cl, String interpreterDir, - String interpreterJson) throws IOException { + String interpreterJson) throws IOException { URL[] urls = recursiveBuildLibList(new File(interpreterDir)); ClassLoader tempClassLoader = new URLClassLoader(urls, null); @@ -507,6 +525,10 @@ public class InterpreterSettingManager { return resourceSet; } + public RecoveryStorage getRecoveryStorage() { + return recoveryStorage; + } + public void removeResourcesBelongsToParagraph(String noteId, String paragraphId) { for (ManagedInterpreterGroup intpGroup : getAllInterpreterGroup()) { ResourceSet resourceSet = new ResourceSet(); diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java index 2378f140da..641c0ac23e 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/ManagedInterpreterGroup.java @@ -55,15 +55,31 @@ public class ManagedInterpreterGroup extends InterpreterGroup { return interpreterSetting; } - public synchronized RemoteInterpreterProcess getOrCreateInterpreterProcess(Properties properties) + public synchronized RemoteInterpreterProcess getOrCreateInterpreterProcess(String userName, + Properties properties) throws IOException { if (remoteInterpreterProcess == null) { LOGGER.info("Create InterpreterProcess for InterpreterGroup: " + getId()); - remoteInterpreterProcess = interpreterSetting.createInterpreterProcess(properties); + remoteInterpreterProcess = interpreterSetting.createInterpreterProcess(id, properties); + synchronized (remoteInterpreterProcess) { + if (!remoteInterpreterProcess.isRunning()) { + remoteInterpreterProcess.start(userName, false); + remoteInterpreterProcess.getRemoteInterpreterEventPoller() + .setInterpreterProcess(remoteInterpreterProcess); + remoteInterpreterProcess.getRemoteInterpreterEventPoller().setInterpreterGroup(this); + remoteInterpreterProcess.getRemoteInterpreterEventPoller().start(); + getInterpreterSetting().getRecoveryStorage() + .onInterpreterClientStart(remoteInterpreterProcess); + } + } } return remoteInterpreterProcess; } + public RemoteInterpreterProcess getInterpreterProcess() { + return remoteInterpreterProcess; + } + public RemoteInterpreterProcess getRemoteInterpreterProcess() { return remoteInterpreterProcess; } @@ -94,6 +110,11 @@ public class ManagedInterpreterGroup extends InterpreterGroup { if (remoteInterpreterProcess != null) { LOGGER.info("Kill RemoteInterpreterProcess"); remoteInterpreterProcess.stop(); + try { + interpreterSetting.getRecoveryStorage().onInterpreterClientStop(remoteInterpreterProcess); + } catch (IOException e) { + LOGGER.error("Fail to store recovery data", e); + } remoteInterpreterProcess = null; } } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncher.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncher.java index 8c86129f64..6ddcacf275 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncher.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncher.java @@ -21,50 +21,68 @@ package org.apache.zeppelin.interpreter.launcher; import org.apache.zeppelin.conf.ZeppelinConfiguration; import org.apache.zeppelin.interpreter.InterpreterOption; import org.apache.zeppelin.interpreter.InterpreterRunner; +import org.apache.zeppelin.interpreter.recovery.RecoveryStorage; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterRunningProcess; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.io.IOException; import java.util.HashMap; import java.util.Map; /** * Interpreter Launcher which use shell script to launch the interpreter process. - * */ public class ShellScriptLauncher extends InterpreterLauncher { private static final Logger LOGGER = LoggerFactory.getLogger(ShellScriptLauncher.class); - public ShellScriptLauncher(ZeppelinConfiguration zConf) { - super(zConf); + public ShellScriptLauncher(ZeppelinConfiguration zConf, RecoveryStorage recoveryStorage) { + super(zConf, recoveryStorage); } @Override - public InterpreterClient launch(InterpreterLaunchContext context) { + public InterpreterClient launch(InterpreterLaunchContext context) throws IOException { LOGGER.info("Launching Interpreter: " + context.getInterpreterSettingGroup()); this.properties = context.getProperties(); InterpreterOption option = context.getOption(); InterpreterRunner runner = context.getRunner(); String groupName = context.getInterpreterSettingGroup(); String name = context.getInterpreterSettingName(); - int connectTimeout = zConf.getInt(ZeppelinConfiguration.ConfVars.ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT); + if (option.isExistingProcess()) { return new RemoteInterpreterRunningProcess( + context.getInterpreterSettingName(), connectTimeout, option.getHost(), option.getPort()); } else { + // try to recover it first + if (zConf.isRecoveryEnabled()) { + InterpreterClient recoveredClient = + recoveryStorage.getInterpreterClient(context.getInterpreterGroupId()); + if (recoveredClient != null) { + if (recoveredClient.isRunning()) { + LOGGER.info("Recover interpreter process: " + recoveredClient.getHost() + ":" + + recoveredClient.getPort()); + return recoveredClient; + } else { + LOGGER.warn("Cannot recover interpreter process: " + recoveredClient.getHost() + ":" + + recoveredClient.getPort() + ", as it is already terminated."); + } + } + } + // create new remote process String localRepoPath = zConf.getInterpreterLocalRepoPath() + "/" + context.getInterpreterSettingId(); return new RemoteInterpreterManagedProcess( runner != null ? runner.getPath() : zConf.getInterpreterRemoteRunnerPath(), - zConf.getCallbackPortRange(), zConf.getInterpreterPortRange(), + zConf.getCallbackPortRange(), zConf.getInterpreterPortRange(), zConf.getInterpreterDir() + "/" + groupName, localRepoPath, buildEnvFromProperties(), connectTimeout, name); } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java index 32a0530af1..e8a9cdf881 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncher.java @@ -19,6 +19,7 @@ package org.apache.zeppelin.interpreter.launcher; import org.apache.commons.lang3.StringUtils; import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.recovery.RecoveryStorage; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -35,8 +36,8 @@ public class SparkInterpreterLauncher extends ShellScriptLauncher { private static final Logger LOGGER = LoggerFactory.getLogger(SparkInterpreterLauncher.class); - public SparkInterpreterLauncher(ZeppelinConfiguration zConf) { - super(zConf); + public SparkInterpreterLauncher(ZeppelinConfiguration zConf, RecoveryStorage recoveryStorage) { + super(zConf, recoveryStorage); } @Override diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorage.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorage.java new file mode 100644 index 0000000000..5a0c8adf6c --- /dev/null +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorage.java @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.interpreter.recovery; + +import org.apache.commons.lang.StringUtils; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.FileStatus; +import org.apache.hadoop.fs.FileSystem; +import org.apache.hadoop.fs.Path; +import org.apache.hadoop.fs.PathFilter; +import org.apache.hadoop.io.IOUtils; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.InterpreterSetting; +import org.apache.zeppelin.interpreter.InterpreterSettingManager; +import org.apache.zeppelin.interpreter.ManagedInterpreterGroup; +import org.apache.zeppelin.interpreter.launcher.InterpreterClient; +import org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller; +import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess; +import org.apache.zeppelin.interpreter.remote.RemoteInterpreterRunningProcess; +import org.apache.zeppelin.notebook.FileSystemStorage; +import org.apache.zeppelin.notebook.repo.FileSystemNotebookRepo; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.net.URI; +import java.net.URISyntaxException; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + + +/** + * Hadoop compatible FileSystem based RecoveryStorage implementation. + * + * Save InterpreterProcess in the format of: + * InterpreterGroupId host:port + */ +public class FileSystemRecoveryStorage extends RecoveryStorage { + + private static final Logger LOGGER = LoggerFactory.getLogger(FileSystemRecoveryStorage.class); + + private InterpreterSettingManager interpreterSettingManager; + private FileSystemStorage fs; + private Path recoveryDir; + + public FileSystemRecoveryStorage(ZeppelinConfiguration zConf, + InterpreterSettingManager interpreterSettingManager) + throws IOException { + super(zConf); + this.interpreterSettingManager = interpreterSettingManager; + this.zConf = zConf; + this.fs = FileSystemStorage.get(zConf); + this.recoveryDir = this.fs.makeQualified(new Path(zConf.getRecoveryDir())); + LOGGER.info("Using folder {} to store recovery data", recoveryDir); + this.fs.tryMkDir(recoveryDir); + } + + @Override + public void onInterpreterClientStart(InterpreterClient client) throws IOException { + save(client.getInterpreterSettingName()); + } + + @Override + public void onInterpreterClientStop(InterpreterClient client) throws IOException { + save(client.getInterpreterSettingName()); + } + + private void save(String interpreterSettingName) throws IOException { + InterpreterSetting interpreterSetting = + interpreterSettingManager.getInterpreterSettingByName(interpreterSettingName); + List recoveryContent = new ArrayList<>(); + for (ManagedInterpreterGroup interpreterGroup : interpreterSetting.getAllInterpreterGroups()) { + RemoteInterpreterProcess interpreterProcess = interpreterGroup.getInterpreterProcess(); + if (interpreterProcess != null) { + recoveryContent.add(interpreterGroup.getId() + "\t" + interpreterProcess.getHost() + ":" + + interpreterProcess.getPort()); + } + } + LOGGER.debug("Updating recovery data for interpreterSetting: " + interpreterSettingName); + LOGGER.debug("Recovery Data: " + StringUtils.join(recoveryContent, System.lineSeparator())); + Path recoveryFile = new Path(recoveryDir, interpreterSettingName + ".recovery"); + fs.writeFile(StringUtils.join(recoveryContent, System.lineSeparator()), recoveryFile, true); + } + + @Override + public Map restore() throws IOException { + Map clients = new HashMap<>(); + List paths = fs.list(new Path(recoveryDir + "/*.recovery")); + + for (Path path : paths) { + String fileName = path.getName(); + String interpreterSettingName = fileName.substring(0, + fileName.length() - ".recovery".length()); + String recoveryContent = fs.readFile(path); + if (!StringUtils.isBlank(recoveryContent)) { + for (String line : recoveryContent.split(System.lineSeparator())) { + String[] tokens = line.split("\t"); + String groupId = tokens[0]; + String[] hostPort = tokens[1].split(":"); + int connectTimeout = + zConf.getInt(ZeppelinConfiguration.ConfVars.ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT); + RemoteInterpreterRunningProcess client = new RemoteInterpreterRunningProcess( + interpreterSettingName, connectTimeout, hostPort[0], Integer.parseInt(hostPort[1])); + // interpreterSettingManager may be null when this class is used when it is used + // stop-interpreter.sh + if (interpreterSettingManager != null) { + client.setRemoteInterpreterEventPoller(new RemoteInterpreterEventPoller( + interpreterSettingManager.getRemoteInterpreterProcessListener(), + interpreterSettingManager.getAppEventListener())); + } + clients.put(groupId, client); + LOGGER.info("Recovering Interpreter Process: " + hostPort[0] + ":" + hostPort[1]); + } + } + } + + return clients; + } +} diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/NullRecoveryStorage.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/NullRecoveryStorage.java new file mode 100644 index 0000000000..3a7d12c70f --- /dev/null +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/NullRecoveryStorage.java @@ -0,0 +1,54 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.zeppelin.interpreter.recovery; + +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.InterpreterSettingManager; +import org.apache.zeppelin.interpreter.launcher.InterpreterClient; + +import java.io.IOException; +import java.util.Map; + + +/** + * RecoveryStorage that do nothing, used when recovery is not enabled. + * + */ +public class NullRecoveryStorage extends RecoveryStorage { + + public NullRecoveryStorage(ZeppelinConfiguration zConf, + InterpreterSettingManager interpreterSettingManager) + throws IOException { + super(zConf); + } + + @Override + public void onInterpreterClientStart(InterpreterClient client) throws IOException { + + } + + @Override + public void onInterpreterClientStop(InterpreterClient client) throws IOException { + + } + + @Override + public Map restore() throws IOException { + return null; + } +} diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/StopInterpreter.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/StopInterpreter.java new file mode 100644 index 0000000000..d74b1621e7 --- /dev/null +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/recovery/StopInterpreter.java @@ -0,0 +1,40 @@ +package org.apache.zeppelin.interpreter.recovery; + +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.interpreter.InterpreterSettingManager; +import org.apache.zeppelin.interpreter.launcher.InterpreterClient; +import org.apache.zeppelin.util.ReflectionUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.IOException; +import java.util.Map; + + +/** + * Utility class for stopping interpreter in the case that you want to stop all the + * interpreter process even when you enable recovery, or you want to kill interpreter process + * to avoid orphan process. + */ +public class StopInterpreter { + + private static Logger LOGGER = LoggerFactory.getLogger(StopInterpreter.class); + + public static void main(String[] args) throws IOException { + ZeppelinConfiguration zConf = ZeppelinConfiguration.create(); + RecoveryStorage recoveryStorage = null; + + recoveryStorage = ReflectionUtils.createClazzInstance(zConf.getRecoveryStorageClass(), + new Class[] {ZeppelinConfiguration.class, InterpreterSettingManager.class}, + new Object[] {zConf, null}); + + LOGGER.info("Using RecoveryStorage: " + recoveryStorage.getClass().getName()); + Map restoredClients = recoveryStorage.restore(); + if (restoredClients != null) { + for (InterpreterClient client : restoredClients.values()) { + LOGGER.info("Stop Interpreter Process: " + client.getHost() + ":" + client.getPort()); + client.stop(); + } + } + } +} diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java index 6defd9ba82..bda8010d93 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java @@ -102,16 +102,7 @@ public class RemoteInterpreter extends Interpreter { return this.interpreterProcess; } ManagedInterpreterGroup intpGroup = getInterpreterGroup(); - this.interpreterProcess = intpGroup.getOrCreateInterpreterProcess(properties); - synchronized (interpreterProcess) { - if (!interpreterProcess.isRunning()) { - interpreterProcess.start(this.getUserName(), false); - interpreterProcess.getRemoteInterpreterEventPoller() - .setInterpreterProcess(interpreterProcess); - interpreterProcess.getRemoteInterpreterEventPoller().setInterpreterGroup(intpGroup); - interpreterProcess.getRemoteInterpreterEventPoller().start(); - } - } + this.interpreterProcess = intpGroup.getOrCreateInterpreterProcess(getUserName(), properties); return interpreterProcess; } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterManagedProcess.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterManagedProcess.java index 27e826c70a..3dd5bfa349 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterManagedProcess.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterManagedProcess.java @@ -214,7 +214,7 @@ public class RemoteInterpreterManagedProcess extends RemoteInterpreterProcess callbackServer.stop(); } if (isRunning()) { - logger.info("kill interpreter process"); + logger.info("Kill interpreter process"); try { callRemoteFunction(new RemoteFunction() { @Override @@ -263,7 +263,6 @@ public class RemoteInterpreterManagedProcess extends RemoteInterpreterProcess return interpreterDir; } - @VisibleForTesting public String getInterpreterSettingName() { return interpreterSettingName; } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterProcess.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterProcess.java index 88cc4894be..08653ae390 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterProcess.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterProcess.java @@ -51,12 +51,6 @@ public abstract class RemoteInterpreterProcess implements InterpreterClient { this.remoteInterpreterEventPoller = eventPoller; } - public abstract String getHost(); - public abstract int getPort(); - public abstract void start(String userName, Boolean isUserImpersonate); - public abstract void stop(); - public abstract boolean isRunning(); - public int getConnectTimeout() { return connectTimeout; } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterRunningProcess.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterRunningProcess.java index d8715a0d49..0e87e4f7d4 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterRunningProcess.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterRunningProcess.java @@ -17,6 +17,7 @@ package org.apache.zeppelin.interpreter.remote; import org.apache.zeppelin.helium.ApplicationEventListener; +import org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -27,13 +28,16 @@ public class RemoteInterpreterRunningProcess extends RemoteInterpreterProcess { private final Logger logger = LoggerFactory.getLogger(RemoteInterpreterRunningProcess.class); private final String host; private final int port; + private final String interpreterSettingName; public RemoteInterpreterRunningProcess( + String interpreterSettingName, int connectTimeout, String host, int port ) { super(connectTimeout); + this.interpreterSettingName = interpreterSettingName; this.host = host; this.port = port; } @@ -48,6 +52,11 @@ public class RemoteInterpreterRunningProcess extends RemoteInterpreterProcess { return port; } + @Override + public String getInterpreterSettingName() { + return interpreterSettingName; + } + @Override public void start(String userName, Boolean isUserImpersonate) { // assume process is externally managed. nothing to do @@ -55,7 +64,24 @@ public class RemoteInterpreterRunningProcess extends RemoteInterpreterProcess { @Override public void stop() { - // assume process is externally managed. nothing to do + // assume process is externally managed. nothing to do. But will kill it + // when you want to force stop it. ENV ZEPPELIN_FORCE_STOP control that. + if (System.getenv("ZEPPELIN_FORCE_STOP") != null) { + if (isRunning()) { + logger.info("Kill interpreter process"); + try { + callRemoteFunction(new RemoteFunction() { + @Override + public Void call(RemoteInterpreterService.Client client) throws Exception { + client.shutdown(); + return null; + } + }); + } catch (Exception e) { + logger.warn("ignore the exception when shutting down interpreter process.", e); + } + } + } } @Override diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java new file mode 100644 index 0000000000..6f3d3f97f5 --- /dev/null +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/FileSystemStorage.java @@ -0,0 +1,168 @@ +package org.apache.zeppelin.notebook; + +import org.apache.commons.lang.StringUtils; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.FileStatus; +import org.apache.hadoop.fs.FileSystem; +import org.apache.hadoop.fs.Path; +import org.apache.hadoop.fs.RawLocalFileSystem; +import org.apache.hadoop.io.IOUtils; +import org.apache.hadoop.security.UserGroupInformation; +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.net.URI; +import java.net.URISyntaxException; +import java.security.PrivilegedExceptionAction; +import java.util.ArrayList; +import java.util.List; + + +/** + * Hadoop FileSystem wrapper. Support both secure and no-secure mode + */ +public class FileSystemStorage { + + private static Logger LOGGER = LoggerFactory.getLogger(FileSystemStorage.class); + + private static FileSystemStorage instance; + + private ZeppelinConfiguration zConf; + private Configuration hadoopConf; + private boolean isSecurityEnabled = false; + private FileSystem fs; + + private FileSystemStorage(ZeppelinConfiguration zConf) throws IOException { + this.zConf = zConf; + this.hadoopConf = new Configuration(); + this.hadoopConf.set("fs.file.impl", RawLocalFileSystem.class.getName()); + this.isSecurityEnabled = UserGroupInformation.isSecurityEnabled(); + + if (isSecurityEnabled) { + String keytab = zConf.getString( + ZeppelinConfiguration.ConfVars.ZEPPELIN_SERVER_KERBEROS_KEYTAB); + String principal = zConf.getString( + ZeppelinConfiguration.ConfVars.ZEPPELIN_SERVER_KERBEROS_PRINCIPAL); + if (StringUtils.isBlank(keytab) || StringUtils.isBlank(principal)) { + throw new IOException("keytab and principal can not be empty, keytab: " + keytab + + ", principal: " + principal); + } + UserGroupInformation.loginUserFromKeytab(principal, keytab); + } + + try { + this.fs = FileSystem.get(new URI(zConf.getNotebookDir()), this.hadoopConf); + LOGGER.info("Creating FileSystem: " + this.fs.getClass().getCanonicalName()); + } catch (URISyntaxException e) { + throw new IOException(e); + } + } + + public static synchronized FileSystemStorage get(ZeppelinConfiguration zConf) throws IOException { + if (instance == null) { + instance = new FileSystemStorage(zConf); + } + return instance; + } + + public Path makeQualified(Path path) { + return fs.makeQualified(path); + } + + public void tryMkDir(final Path dir) throws IOException { + callHdfsOperation(new HdfsOperation() { + @Override + public Void call() throws IOException { + if (!fs.exists(dir)) { + fs.mkdirs(dir); + LOGGER.info("Create dir {} in hdfs", dir.toString()); + } + if (fs.isFile(dir)) { + throw new IOException(dir.toString() + " is file instead of directory, please remove " + + "it or specify another directory"); + } + fs.mkdirs(dir); + return null; + } + }); + } + + public List list(final Path path) throws IOException { + return callHdfsOperation(new HdfsOperation>() { + @Override + public List call() throws IOException { + List paths = new ArrayList<>(); + for (FileStatus status : fs.globStatus(path)) { + paths.add(status.getPath()); + } + return paths; + } + }); + } + + public boolean delete(final Path path) throws IOException { + return callHdfsOperation(new HdfsOperation() { + @Override + public Boolean call() throws IOException { + return fs.delete(path, true); + } + }); + } + + public String readFile(final Path file) throws IOException { + return callHdfsOperation(new HdfsOperation() { + @Override + public String call() throws IOException { + LOGGER.debug("Read from file: " + file); + ByteArrayOutputStream noteBytes = new ByteArrayOutputStream(); + IOUtils.copyBytes(fs.open(file), noteBytes, hadoopConf); + return new String(noteBytes.toString( + zConf.getString(ZeppelinConfiguration.ConfVars.ZEPPELIN_ENCODING))); + } + }); + } + + public void writeFile(final String content, final Path file, boolean writeTempFileFirst) + throws IOException { + callHdfsOperation(new HdfsOperation() { + @Override + public Void call() throws IOException { + InputStream in = new ByteArrayInputStream(content.getBytes( + zConf.getString(ZeppelinConfiguration.ConfVars.ZEPPELIN_ENCODING))); + Path tmpFile = new Path(file.toString() + ".tmp"); + IOUtils.copyBytes(in, fs.create(tmpFile), hadoopConf); + fs.delete(file, true); + fs.rename(tmpFile, file); + return null; + } + }); + } + + private interface HdfsOperation { + T call() throws IOException; + } + + public synchronized T callHdfsOperation(final HdfsOperation func) throws IOException { + if (isSecurityEnabled) { + UserGroupInformation.getLoginUser().reloginFromKeytab(); + try { + return UserGroupInformation.getCurrentUser().doAs(new PrivilegedExceptionAction() { + @Override + public T run() throws Exception { + return func.call(); + } + }); + } catch (InterruptedException e) { + throw new IOException(e); + } + } else { + return func.call(); + } + } + +} diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/FileSystemNotebookRepo.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/FileSystemNotebookRepo.java index ba858e6925..d8ec0e5400 100644 --- a/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/FileSystemNotebookRepo.java +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/FileSystemNotebookRepo.java @@ -8,6 +8,7 @@ import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IOUtils; import org.apache.hadoop.security.UserGroupInformation; import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.notebook.FileSystemStorage; import org.apache.zeppelin.notebook.Note; import org.apache.zeppelin.notebook.NoteInfo; import org.apache.zeppelin.user.AuthenticationInfo; @@ -37,108 +38,45 @@ import java.util.Map; public class FileSystemNotebookRepo implements NotebookRepo { private static final Logger LOGGER = LoggerFactory.getLogger(FileSystemNotebookRepo.class); - private Configuration hadoopConf; - private ZeppelinConfiguration zConf; - private boolean isSecurityEnabled = false; - private FileSystem fs; + private FileSystemStorage fs; private Path notebookDir; public FileSystemNotebookRepo(ZeppelinConfiguration zConf) throws IOException { - this.zConf = zConf; - this.hadoopConf = new Configuration(); + this.fs = FileSystemStorage.get(zConf); + this.notebookDir = this.fs.makeQualified(new Path(zConf.getNotebookDir())); + LOGGER.info("Using folder {} to store notebook", notebookDir); + this.fs.tryMkDir(notebookDir); - this.isSecurityEnabled = UserGroupInformation.isSecurityEnabled(); - if (isSecurityEnabled) { - String keytab = zConf.getString( - ZeppelinConfiguration.ConfVars.ZEPPELIN_SERVER_KERBEROS_KEYTAB); - String principal = zConf.getString( - ZeppelinConfiguration.ConfVars.ZEPPELIN_SERVER_KERBEROS_PRINCIPAL); - if (StringUtils.isBlank(keytab) || StringUtils.isBlank(principal)) { - throw new IOException("keytab and principal can not be empty, keytab: " + keytab - + ", principal: " + principal); - } - UserGroupInformation.loginUserFromKeytab(principal, keytab); - } - - try { - this.fs = FileSystem.get(new URI(zConf.getNotebookDir()), new Configuration()); - LOGGER.info("Creating FileSystem: " + this.fs.getClass().getCanonicalName()); - this.notebookDir = fs.makeQualified(new Path(zConf.getNotebookDir())); - LOGGER.info("Using folder {} to store notebook", notebookDir); - } catch (URISyntaxException e) { - throw new IOException(e); - } - if (!fs.exists(notebookDir)) { - fs.mkdirs(notebookDir); - LOGGER.info("Create notebook dir {} in hdfs", notebookDir.toString()); - } - if (fs.isFile(notebookDir)) { - throw new IOException("notebookDir {} is file instead of directory, please remove it or " + - "specify another directory"); - } } @Override public List list(AuthenticationInfo subject) throws IOException { - return callHdfsOperation(new HdfsOperation>() { - @Override - public List call() throws IOException { - List noteInfos = new ArrayList<>(); - for (FileStatus status : fs.globStatus(new Path(notebookDir, "*/note.json"))) { - NoteInfo noteInfo = new NoteInfo(status.getPath().getParent().getName(), "", null); - noteInfos.add(noteInfo); - } - return noteInfos; - } - }); + List notePaths = fs.list(new Path(notebookDir, "*/note.json")); + List noteInfos = new ArrayList<>(); + for (Path path : notePaths) { + NoteInfo noteInfo = new NoteInfo(path.getParent().getName(), "", null); + noteInfos.add(noteInfo); + } + return noteInfos; } @Override public Note get(final String noteId, AuthenticationInfo subject) throws IOException { - return callHdfsOperation(new HdfsOperation() { - @Override - public Note call() throws IOException { - Path notePath = new Path(notebookDir.toString() + "/" + noteId + "/note.json"); - LOGGER.debug("Read note from file: " + notePath); - ByteArrayOutputStream noteBytes = new ByteArrayOutputStream(); - IOUtils.copyBytes(fs.open(notePath), noteBytes, hadoopConf); - return Note.fromJson(new String(noteBytes.toString( - zConf.getString(ZeppelinConfiguration.ConfVars.ZEPPELIN_ENCODING)))); - } - }); + String content = this.fs.readFile( + new Path(notebookDir.toString() + "/" + noteId + "/note.json")); + return Note.fromJson(content); } @Override public void save(final Note note, AuthenticationInfo subject) throws IOException { - callHdfsOperation(new HdfsOperation() { - @Override - public Void call() throws IOException { - Path notePath = new Path(notebookDir.toString() + "/" + note.getId() + "/note.json"); - Path tmpNotePath = new Path(notebookDir.toString() + "/" + note.getId() + "/.note.json"); - LOGGER.debug("Saving note to file: " + notePath); - if (fs.exists(tmpNotePath)) { - fs.delete(tmpNotePath, true); - } - InputStream in = new ByteArrayInputStream(note.toJson().getBytes( - zConf.getString(ZeppelinConfiguration.ConfVars.ZEPPELIN_ENCODING))); - IOUtils.copyBytes(in, fs.create(tmpNotePath), hadoopConf); - fs.delete(notePath, true); - fs.rename(tmpNotePath, notePath); - return null; - } - }); + this.fs.writeFile(note.toJson(), + new Path(notebookDir.toString() + "/" + note.getId() + "/note.json"), + true); } @Override public void remove(final String noteId, AuthenticationInfo subject) throws IOException { - callHdfsOperation(new HdfsOperation() { - @Override - public Void call() throws IOException { - Path noteFolder = new Path(notebookDir.toString() + "/" + noteId); - fs.delete(noteFolder, true); - return null; - } - }); + this.fs.delete(new Path(notebookDir.toString() + "/" + noteId)); } @Override @@ -182,26 +120,4 @@ public class FileSystemNotebookRepo implements NotebookRepo { public void updateSettings(Map settings, AuthenticationInfo subject) { LOGGER.warn("updateSettings is not implemented for HdfsNotebookRepo"); } - - private interface HdfsOperation { - T call() throws IOException; - } - - public synchronized T callHdfsOperation(final HdfsOperation func) throws IOException { - if (isSecurityEnabled) { - UserGroupInformation.getLoginUser().reloginFromKeytab(); - try { - return UserGroupInformation.getCurrentUser().doAs(new PrivilegedExceptionAction() { - @Override - public T run() throws Exception { - return func.call(); - } - }); - } catch (InterruptedException e) { - throw new IOException(e); - } - } else { - return func.call(); - } - } } diff --git a/zeppelin-zengine/src/main/java/org/apache/zeppelin/util/ReflectionUtils.java b/zeppelin-zengine/src/main/java/org/apache/zeppelin/util/ReflectionUtils.java new file mode 100644 index 0000000000..ca09992a7d --- /dev/null +++ b/zeppelin-zengine/src/main/java/org/apache/zeppelin/util/ReflectionUtils.java @@ -0,0 +1,99 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.zeppelin.util; + +import java.io.IOException; +import java.lang.reflect.Constructor; +import java.lang.reflect.InvocationTargetException; + + +/** + * Utility class for creating instances via java reflection. + * + */ +public class ReflectionUtils { + + public static Class getClazz(String className) throws IOException { + Class clazz = null; + try { + clazz = Class.forName(className, true, Thread.currentThread().getContextClassLoader()); + } catch (ClassNotFoundException e) { + throw new IOException("Unable to load class: " + className, e); + } + + return clazz; + } + + private static T getNewInstance(Class clazz) throws IOException { + T instance; + try { + instance = clazz.newInstance(); + } catch (InstantiationException e) { + throw new IOException( + "Unable to instantiate class with 0 arguments: " + clazz.getName(), e); + } catch (IllegalAccessException e) { + throw new IOException( + "Unable to instantiate class with 0 arguments: " + clazz.getName(), e); + } + return instance; + } + + private static T getNewInstance(Class clazz, + Class[] parameterTypes, + Object[] parameters) + throws IOException { + T instance; + try { + Constructor constructor = clazz.getConstructor(parameterTypes); + instance = constructor.newInstance(parameters); + } catch (InstantiationException e) { + throw new IOException( + "Unable to instantiate class with " + parameters.length + " arguments: " + + clazz.getName(), e); + } catch (IllegalAccessException e) { + throw new IOException( + "Unable to instantiate class with " + parameters.length + " arguments: " + + clazz.getName(), e); + } catch (NoSuchMethodException e) { + throw new IOException( + "Unable to instantiate class with " + parameters.length + " arguments: " + + clazz.getName(), e); + } catch (InvocationTargetException e) { + throw new IOException( + "Unable to instantiate class with " + parameters.length + " arguments: " + + clazz.getName(), e); + } + return instance; + } + + public static T createClazzInstance(String className) throws IOException { + Class clazz = getClazz(className); + @SuppressWarnings("unchecked") + T instance = (T) getNewInstance(clazz); + return instance; + } + + public static T createClazzInstance(String className, + Class[] parameterTypes, + Object[] parameters) throws IOException { + Class clazz = getClazz(className); + T instance = (T) getNewInstance(clazz, parameterTypes, parameters); + return instance; + } + + +} diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/AbstractInterpreterTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/AbstractInterpreterTest.java index 9df402d35e..16c8c1d8ce 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/AbstractInterpreterTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/AbstractInterpreterTest.java @@ -33,7 +33,7 @@ public abstract class AbstractInterpreterTest { protected File interpreterDir; protected File confDir; protected File notebookDir; - protected ZeppelinConfiguration conf = new ZeppelinConfiguration(); + protected ZeppelinConfiguration conf; @Before public void setUp() throws Exception { diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncherTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncherTest.java index 0c7f4baacf..f7988e3570 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncherTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/ShellScriptLauncherTest.java @@ -22,6 +22,7 @@ import org.apache.zeppelin.interpreter.InterpreterOption; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess; import org.junit.Test; +import java.io.IOException; import java.util.Properties; import static org.junit.Assert.assertEquals; @@ -30,14 +31,14 @@ import static org.junit.Assert.assertTrue; public class ShellScriptLauncherTest { @Test - public void testLauncher() { + public void testLauncher() throws IOException { ZeppelinConfiguration zConf = new ZeppelinConfiguration(); - ShellScriptLauncher launcher = new ShellScriptLauncher(zConf); + ShellScriptLauncher launcher = new ShellScriptLauncher(zConf, null); Properties properties = new Properties(); properties.setProperty("ENV_1", "VALUE_1"); properties.setProperty("property_1", "value_1"); InterpreterOption option = new InterpreterOption(); - InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "groupId", "groupName", "name"); + InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "intpGroupId", "groupId", "groupName", "name"); InterpreterClient client = launcher.launch(context); assertTrue( client instanceof RemoteInterpreterManagedProcess); RemoteInterpreterManagedProcess interpreterProcess = (RemoteInterpreterManagedProcess) client; diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncherTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncherTest.java index b788ebdeee..3d7e251b07 100644 --- a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncherTest.java +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/launcher/SparkInterpreterLauncherTest.java @@ -22,6 +22,7 @@ import org.apache.zeppelin.interpreter.InterpreterOption; import org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess; import org.junit.Test; +import java.io.IOException; import java.util.Properties; import static org.junit.Assert.assertEquals; @@ -30,9 +31,9 @@ import static org.junit.Assert.assertTrue; public class SparkInterpreterLauncherTest { @Test - public void testLocalMode() { + public void testLocalMode() throws IOException { ZeppelinConfiguration zConf = new ZeppelinConfiguration(); - SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf); + SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf, null); Properties properties = new Properties(); properties.setProperty("SPARK_HOME", "/user/spark"); properties.setProperty("property_1", "value_1"); @@ -41,7 +42,7 @@ public class SparkInterpreterLauncherTest { properties.setProperty("spark.jars", "jar_1"); InterpreterOption option = new InterpreterOption(); - InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "groupId", "spark", "spark"); + InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "intpGroupId", "groupId", "spark", "spark"); InterpreterClient client = launcher.launch(context); assertTrue( client instanceof RemoteInterpreterManagedProcess); RemoteInterpreterManagedProcess interpreterProcess = (RemoteInterpreterManagedProcess) client; @@ -55,9 +56,9 @@ public class SparkInterpreterLauncherTest { } @Test - public void testYarnClientMode_1() { + public void testYarnClientMode_1() throws IOException { ZeppelinConfiguration zConf = new ZeppelinConfiguration(); - SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf); + SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf, null); Properties properties = new Properties(); properties.setProperty("SPARK_HOME", "/user/spark"); properties.setProperty("property_1", "value_1"); @@ -66,7 +67,7 @@ public class SparkInterpreterLauncherTest { properties.setProperty("spark.jars", "jar_1"); InterpreterOption option = new InterpreterOption(); - InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "groupId", "spark", "spark"); + InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "intpGroupId", "groupId", "spark", "spark"); InterpreterClient client = launcher.launch(context); assertTrue( client instanceof RemoteInterpreterManagedProcess); RemoteInterpreterManagedProcess interpreterProcess = (RemoteInterpreterManagedProcess) client; @@ -80,9 +81,9 @@ public class SparkInterpreterLauncherTest { } @Test - public void testYarnClientMode_2() { + public void testYarnClientMode_2() throws IOException { ZeppelinConfiguration zConf = new ZeppelinConfiguration(); - SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf); + SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf, null); Properties properties = new Properties(); properties.setProperty("SPARK_HOME", "/user/spark"); properties.setProperty("property_1", "value_1"); @@ -92,7 +93,7 @@ public class SparkInterpreterLauncherTest { properties.setProperty("spark.jars", "jar_1"); InterpreterOption option = new InterpreterOption(); - InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "groupId", "spark", "spark"); + InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "intpGroupId", "groupId", "spark", "spark"); InterpreterClient client = launcher.launch(context); assertTrue( client instanceof RemoteInterpreterManagedProcess); RemoteInterpreterManagedProcess interpreterProcess = (RemoteInterpreterManagedProcess) client; @@ -106,9 +107,9 @@ public class SparkInterpreterLauncherTest { } @Test - public void testYarnClusterMode_1() { + public void testYarnClusterMode_1() throws IOException { ZeppelinConfiguration zConf = new ZeppelinConfiguration(); - SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf); + SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf, null); Properties properties = new Properties(); properties.setProperty("SPARK_HOME", "/user/spark"); properties.setProperty("property_1", "value_1"); @@ -117,7 +118,7 @@ public class SparkInterpreterLauncherTest { properties.setProperty("spark.jars", "jar_1"); InterpreterOption option = new InterpreterOption(); - InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "groupId", "spark", "spark"); + InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "intpGroupId", "groupId", "spark", "spark"); InterpreterClient client = launcher.launch(context); assertTrue( client instanceof RemoteInterpreterManagedProcess); RemoteInterpreterManagedProcess interpreterProcess = (RemoteInterpreterManagedProcess) client; @@ -132,9 +133,9 @@ public class SparkInterpreterLauncherTest { } @Test - public void testYarnClusterMode_2() { + public void testYarnClusterMode_2() throws IOException { ZeppelinConfiguration zConf = new ZeppelinConfiguration(); - SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf); + SparkInterpreterLauncher launcher = new SparkInterpreterLauncher(zConf, null); Properties properties = new Properties(); properties.setProperty("SPARK_HOME", "/user/spark"); properties.setProperty("property_1", "value_1"); @@ -144,7 +145,7 @@ public class SparkInterpreterLauncherTest { properties.setProperty("spark.jars", "jar_1"); InterpreterOption option = new InterpreterOption(); - InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "groupId", "spark", "spark"); + InterpreterLaunchContext context = new InterpreterLaunchContext(properties, option, null, "intpGroupId", "groupId", "spark", "spark"); InterpreterClient client = launcher.launch(context); assertTrue( client instanceof RemoteInterpreterManagedProcess); RemoteInterpreterManagedProcess interpreterProcess = (RemoteInterpreterManagedProcess) client; diff --git a/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorageTest.java b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorageTest.java new file mode 100644 index 0000000000..cf1899c13e --- /dev/null +++ b/zeppelin-zengine/src/test/java/org/apache/zeppelin/interpreter/recovery/FileSystemRecoveryStorageTest.java @@ -0,0 +1,92 @@ +package org.apache.zeppelin.interpreter.recovery; + +import com.google.common.io.Files; +import org.apache.commons.io.FileUtils; +import org.apache.zeppelin.conf.ZeppelinConfiguration; +import org.apache.zeppelin.display.GUI; +import org.apache.zeppelin.interpreter.AbstractInterpreterTest; +import org.apache.zeppelin.interpreter.Interpreter; +import org.apache.zeppelin.interpreter.InterpreterContext; +import org.apache.zeppelin.interpreter.InterpreterContextRunner; +import org.apache.zeppelin.interpreter.InterpreterException; +import org.apache.zeppelin.interpreter.InterpreterOption; +import org.apache.zeppelin.interpreter.InterpreterSetting; +import org.apache.zeppelin.interpreter.remote.RemoteInterpreter; +import org.apache.zeppelin.user.AuthenticationInfo; +import org.junit.Before; +import org.junit.Test; + +import java.io.File; +import java.io.IOException; +import java.util.ArrayList; +import java.util.HashMap; + +import static org.junit.Assert.assertEquals; + +public class FileSystemRecoveryStorageTest extends AbstractInterpreterTest { + + private File recoveryDir = null; + + @Before + public void setUp() throws Exception { + System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_RECOVERY_STORAGE_CLASS.getVarName(), + FileSystemRecoveryStorage.class.getName()); + recoveryDir = Files.createTempDir(); + System.setProperty(ZeppelinConfiguration.ConfVars.ZEPPELIN_RECOVERY_DIR.getVarName(), recoveryDir.getAbsolutePath()); + super.setUp(); + } + + @Override + public void tearDown() throws Exception { + super.tearDown(); + FileUtils.deleteDirectory(recoveryDir); + } + + @Test + public void testSingleInterpreterProcess() throws InterpreterException, IOException { + InterpreterSetting interpreterSetting = interpreterSettingManager.getByName("test"); + interpreterSetting.getOption().setPerUser(InterpreterOption.SHARED); + + Interpreter interpreter1 = interpreterSetting.getDefaultInterpreter("user1", "note1"); + RemoteInterpreter remoteInterpreter1 = (RemoteInterpreter) interpreter1; + InterpreterContext context1 = new InterpreterContext("noteId", "paragraphId", "repl", + "title", "text", AuthenticationInfo.ANONYMOUS, new HashMap(), new GUI(), + new GUI(), null, null, new ArrayList(), null); + remoteInterpreter1.interpret("hello", context1); + + assertEquals(1, interpreterSettingManager.getRecoveryStorage().restore().size()); + + interpreterSetting.close(); + assertEquals(0, interpreterSettingManager.getRecoveryStorage().restore().size()); + } + + @Test + public void testMultipleInterpreterProcess() throws InterpreterException, IOException { + InterpreterSetting interpreterSetting = interpreterSettingManager.getByName("test"); + interpreterSetting.getOption().setPerUser(InterpreterOption.ISOLATED); + + Interpreter interpreter1 = interpreterSetting.getDefaultInterpreter("user1", "note1"); + RemoteInterpreter remoteInterpreter1 = (RemoteInterpreter) interpreter1; + InterpreterContext context1 = new InterpreterContext("noteId", "paragraphId", "repl", + "title", "text", AuthenticationInfo.ANONYMOUS, new HashMap(), new GUI(), + new GUI(), null, null, new ArrayList(), null); + remoteInterpreter1.interpret("hello", context1); + assertEquals(1, interpreterSettingManager.getRecoveryStorage().restore().size()); + + Interpreter interpreter2 = interpreterSetting.getDefaultInterpreter("user2", "note2"); + RemoteInterpreter remoteInterpreter2 = (RemoteInterpreter) interpreter2; + InterpreterContext context2 = new InterpreterContext("noteId", "paragraphId", "repl", + "title", "text", AuthenticationInfo.ANONYMOUS, new HashMap(), new GUI(), + new GUI(), null, null, new ArrayList(), null); + remoteInterpreter2.interpret("hello", context2); + + assertEquals(2, interpreterSettingManager.getRecoveryStorage().restore().size()); + + interpreterSettingManager.restart(interpreterSetting.getId(), "note1", "user1"); + assertEquals(1, interpreterSettingManager.getRecoveryStorage().restore().size()); + + interpreterSetting.close(); + assertEquals(0, interpreterSettingManager.getRecoveryStorage().restore().size()); + } + +} From 971631c1a19edd12e38f61deb8ca6440f29d5148 Mon Sep 17 00:00:00 2001 From: Lee moon soo Date: Thu, 7 Dec 2017 14:50:42 -0800 Subject: [PATCH 12/14] [MINOR] Fix notebook title bar margin ### What is this PR for? Notebook title bar location is little bit shifted to left. This minor fix notebook title bar margin. See screenshots below. ### What type of PR is it? Bug Fix ### Todos * [x] - fix margin ### Screenshots (if appropriate) Before ![image](https://user-images.githubusercontent.com/1540981/33742650-3469c2fe-db5e-11e7-8aa6-936d0a28de3a.png) After ![image](https://user-images.githubusercontent.com/1540981/33742640-250c3dc8-db5e-11e7-8af3-e8a9d7105963.png) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Lee moon soo Closes #2699 from Leemoonsoo/minor_noteaction_margin and squashes the following commits: 703f45830 [Lee moon soo] fix noteAction magin --- zeppelin-web/src/app/notebook/notebook.css | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/zeppelin-web/src/app/notebook/notebook.css b/zeppelin-web/src/app/notebook/notebook.css index a7508bce54..262ae8e6cc 100644 --- a/zeppelin-web/src/app/notebook/notebook.css +++ b/zeppelin-web/src/app/notebook/notebook.css @@ -130,8 +130,8 @@ } .noteAction { - margin-left: -10px; - margin-right: -10px; + margin-left: 0px; + margin-right: 0px; font-family: 'Roboto', sans-serif; background: white; position: fixed; From f8cd64cb50028031ab321144697d98822a60c63f Mon Sep 17 00:00:00 2001 From: Naman Mishra Date: Tue, 12 Dec 2017 10:45:34 +0530 Subject: [PATCH 13/14] [ZEPPELIN-3091] Correct aggregation functionality in charts ### What is this PR for? The aggregation functions interpret NaN columns as 1 which leads to incorrect output being shown in charts. This PR fixes this by correcting the sum, min, max and average aggregation method. ### What type of PR is it? Bug Fix ### Todos ### What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-3091 ### How should this be tested? * Update the data field of a paragraph results with %table to contain null values, e.g. `"data": "age\tvalue\n19\t4\n20\t3\n21\t7\n22\t9\n23\t20\n24\t24\n25\t44\n26\t77\n27\t94\n28\t103\n29\t97\n20\t5\n20\tnull\n"` The "null" values should be ignored for sum, min, max but included for count (and hence average). ### Screenshots (if appropriate) ![correct_sum](https://user-images.githubusercontent.com/6438072/33609178-14e05988-d9ed-11e7-9f1b-99e0141c5153.png) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Naman Mishra Closes #2696 from namanmishra91/ZEPPELIN-3091 and squashes the following commits: d8a57c238 [Naman Mishra] Add test 38ad39c65 [Naman Mishra] Merge branch 'master' into ZEPPELIN-3091 568ae3f2a [Naman Mishra] Correct aggregation functionality in charts --- zeppelin-web/src/app/tabledata/pivot.js | 36 ++++++++++----- .../src/app/tabledata/tabledata.test.js | 45 +++++++++++++++++++ 2 files changed, 71 insertions(+), 10 deletions(-) diff --git a/zeppelin-web/src/app/tabledata/pivot.js b/zeppelin-web/src/app/tabledata/pivot.js index a0f61b219a..da2990043b 100644 --- a/zeppelin-web/src/app/tabledata/pivot.js +++ b/zeppelin-web/src/app/tabledata/pivot.js @@ -138,8 +138,8 @@ export default class PivotTransformation extends Transformation { pivot (data, keys, groups, values) { let aggrFunc = { sum: function (a, b) { - let varA = (a !== undefined) ? (isNaN(a) ? 1 : parseFloat(a)) : 0 - let varB = (b !== undefined) ? (isNaN(b) ? 1 : parseFloat(b)) : 0 + let varA = (a !== undefined) ? (isNaN(a) ? 0 : parseFloat(a)) : 0 + let varB = (b !== undefined) ? (isNaN(b) ? 0 : parseFloat(b)) : 0 return varA + varB }, count: function (a, b) { @@ -148,22 +148,38 @@ export default class PivotTransformation extends Transformation { return varA + varB }, min: function (a, b) { - let varA = (a !== undefined) ? (isNaN(a) ? 1 : parseFloat(a)) : 0 - let varB = (b !== undefined) ? (isNaN(b) ? 1 : parseFloat(b)) : 0 - return Math.min(varA, varB) + let aIsValid = isValidNumber(a) + let bIsValid = isValidNumber(b) + if (!aIsValid) { + return parseFloat(b) + } else if (!bIsValid) { + return parseFloat(a) + } else { + return Math.min(parseFloat(a), parseFloat(b)) + } }, max: function (a, b) { - let varA = (a !== undefined) ? (isNaN(a) ? 1 : parseFloat(a)) : 0 - let varB = (b !== undefined) ? (isNaN(b) ? 1 : parseFloat(b)) : 0 - return Math.max(varA, varB) + let aIsValid = isValidNumber(a) + let bIsValid = isValidNumber(b) + if (!aIsValid) { + return parseFloat(b) + } else if (!bIsValid) { + return parseFloat(a) + } else { + return Math.max(parseFloat(a), parseFloat(b)) + } }, avg: function (a, b, c) { - let varA = (a !== undefined) ? (isNaN(a) ? 1 : parseFloat(a)) : 0 - let varB = (b !== undefined) ? (isNaN(b) ? 1 : parseFloat(b)) : 0 + let varA = (a !== undefined) ? (isNaN(a) ? 0 : parseFloat(a)) : 0 + let varB = (b !== undefined) ? (isNaN(b) ? 0 : parseFloat(b)) : 0 return varA + varB } } + let isValidNumber = function(num) { + return num !== undefined && !isNaN(num) + } + let aggrFuncDiv = { sum: false, count: false, diff --git a/zeppelin-web/src/app/tabledata/tabledata.test.js b/zeppelin-web/src/app/tabledata/tabledata.test.js index 3de2fa3fb7..e24b073392 100644 --- a/zeppelin-web/src/app/tabledata/tabledata.test.js +++ b/zeppelin-web/src/app/tabledata/tabledata.test.js @@ -83,4 +83,49 @@ describe('PivotTransformation build', function() { expect(config.common.pivot.keys[1].index).toBe(3) expect(config.common.pivot.keys[2].index).toBe(5) }) + + it('should aggregate values correctly', function() { + let td = new TableData() + td.loadParagraphResult({ + type: 'TABLE', + msg: 'key\tvalue\na\t10\na\tnull\na\t0\na\t1\n' + }) + + let config = { + common: { + pivot: { + keys: [ + { + 'name': 'key', + 'index': 0.0, + } + ], + groups: [], + values: [ + { + 'name': 'value', + 'index': 1.0, + 'aggr': 'sum' + } + ] + } + } + } + + pt.setConfig(config) + let transformed = pt.transform(td) + expect(transformed.rows['a']['value(sum)'].value).toBe(11) + + pt.config.common.pivot.values[0].aggr = 'max' + transformed = pt.transform(td) + expect(transformed.rows['a']['value(max)'].value).toBe(10) + + pt.config.common.pivot.values[0].aggr = 'min' + transformed = pt.transform(td) + expect(transformed.rows['a']['value(min)'].value).toBe(0) + + pt.config.common.pivot.values[0].aggr = 'count' + transformed = pt.transform(td) + expect(transformed.rows['a']['value(count)'].value).toBe(4) + }) }) From 246f696719193dc8a3274f3f9e7ebc912ab3141a Mon Sep 17 00:00:00 2001 From: Andrea Santurbano Date: Tue, 5 Dec 2017 18:43:47 +0100 Subject: [PATCH 14/14] [ZEPPELIN-3101] updated network label, added link to network display in index.md ### What is this PR for? The docs index must show the reference to the network visualization as for the the other types ### What type of PR is it? [Improvement] ### Todos * [x] - Add missing link ### What is the Jira issue? [ZEPPELIN-3101](https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-3101) ### How should this be tested? 1. cd `docs/` 2. build: `bundle exec jekyll build --safe` 3. check the link is present ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? no * Is there breaking changes for older versions? no * Does this needs documentation? no Author: Andrea Santurbano Closes #2702 from conker84/graph-docs and squashes the following commits: 58d58cc4c [Andrea Santurbano] updated network label, added link to network display in index.md --- docs/_includes/themes/zeppelin/_navigation.html | 2 +- docs/index.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/_includes/themes/zeppelin/_navigation.html b/docs/_includes/themes/zeppelin/_navigation.html index bccb5b4691..95d83ea27c 100644 --- a/docs/_includes/themes/zeppelin/_navigation.html +++ b/docs/_includes/themes/zeppelin/_navigation.html @@ -44,7 +44,7 @@
  • Text Display
  • HTML Display
  • Table Display
    • -
  • Network
    • +
  • Network Display
  • Angular Display using Backend API
  • Angular Display using Frontend API
    • diff --git a/docs/index.md b/docs/index.md index 8f3b551c6a..587ae93ed5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -59,6 +59,7 @@ limitations under the License. * [Text Display (`%text`)](./usage/display_system/basic.html#text) * [HTML Display (`%html`)](./usage/display_system/basic.html#html) * [Table Display (`%table`)](./usage/display_system/basic.html#table) + * [Network Display (`%network`)](./usage/display_system/basic.html#network) * [Angular Display using Backend API (`%angular`)](./usage/display_system/angular_backend.html) * [Angular Display using Frontend API (`%angular`)](./usage/display_system/angular_frontend.html) * Interpreter