Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge pull request #1 from apache/master

Browse source 
Merge
This commit is contained in:
swakrish 2016-01-12 09:51:35 -08:00
commit b080d7d37e
70 changed files with 818 additions and 244 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
CONTRIBUTING.md
View file

@ -68,7 +68,8 @@ Here are some things you will need to build and test Zeppelin.
### Software Configuration Management (SCM)
Zeppelin uses Git for its SCM system. Hosted by github.com. `https://github.com/apache/incubator-zeppelin` you'll need git client installed in your development machine.
Zeppelin uses Git for its SCM system. `http://git.apache.org/incubator-zeppelin.git` you'll need git client installed in your development machine.
For write access, `https://git-wip-us.apache.org/repos/asf/incubator-zeppelin.git`
### Integrated Development Environment (IDE)
@ -114,26 +115,31 @@ To build the code, install
* Apache Maven
## Getting the source code
First of all, you need the Zeppelin source code. The official location for Zeppelin is [https://github.com/apache/incubator-zeppelin](https://github.com/apache/incubator-zeppelin)
First of all, you need the Zeppelin source code. The official location for Zeppelin is [http://git.apache.org/incubator-zeppelin.git](http://git.apache.org/incubator-zeppelin.git).
### git access
Get the source code on your development machine using git.
```
git clone git@github.com:apache/incubator-zeppelin.git zeppelin
git clone http://git.apache.org/incubator-zeppelin.git zeppelin
```
You may also want to develop against a specific release. For example, for branch-0.1
```
git clone -b branch-0.1 git@github.com:apache/incubator-zeppelin.git zeppelin
git clone -b branch-0.1 http://git.apache.org/incubator-zeppelin.git zeppelin
```
or with write access
```
git clone https://git-wip-us.apache.org/repos/asf/incubator-zeppelin.git
```
### Fork repository
If you want not only build Zeppelin but also make change, then you need fork Zeppelin repository and make pull request.
If you want not only build Zeppelin but also make change, then you need fork Zeppelin github mirror repository (https://github.com/apache/incubator-zeppelin) and make pull request.
## Build

43
SECURITY-README.md Normal file
View file

@ -0,0 +1,43 @@
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
# Shiro Authentication
To connect to Zeppelin, users will be asked to enter their credentials. Once logged, a user has access to all notes including other users notes.
This a a first step toward full security as implemented by this pull request (https://github.com/apache/incubator-zeppelin/pull/53).
# Security setup
1. Secure the HTTP channel: Comment the line "/** = anon" and uncomment the line "/** = authcBasic" in the file conf/shiro.ini. Read more about he shiro.ini file format at the following URL http://shiro.apache.org/configuration.html#Configuration-INISections.
2. Secure the Websocket channel : Set to property "zeppelin.anonymous.allowed" to "false" in the file conf/zeppelin-site.xml. You can start by renaming conf/zeppelin-site.xml.template to conf/zeppelin-site.xml
3. Start Zeppelin : bin/zeppelin.sh
4. point your browser to http://localhost:8080
5. Login using one of the user/password combinations defined in the conf/shiro.ini file.
# Implementation notes
## Vocabulary
username, owner and principal are used interchangeably to designate the currently authenticated user
## What are we securing ?
Zeppelin is basically a web application that spawn remote interpreters to run commands and return HTML fragments to be displayed on the user browser.
The scope of this PR is to require credentials to access Zeppelin. To achieve this, we use Apache Shiro.
## HTTP Endpoint security
Apache Shiro sits as a servlet filter between the browser and the exposed services and handles the required authentication without any programming required. (See Apache Shiro for more info).
## Websocket security
Securing the HTTP endpoints is not enough, since Zeppelin also communicates with the browser through websockets. To secure this channel, we take the following approach:
1. The browser on startup requests a ticket through HTTP
2. The Apache Shiro Servlet filter handles the user auth
3. Once the user is authenticated, a ticket is assigned to this user and the ticket is returned to the browser
All websockets communications require the username and ticket to be submitted by the browser. Upon receiving a websocket message, the server checks that the ticket received is the one assigned to the username through the HTTP request (step 3 above).

33
conf/shiro.ini Normal file
View file

@ -0,0 +1,33 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
[users]
# List of users with their password allowed to access Zeppelin.
# To use a different strategy (LDAP / Database / ...) check the shiro doc at http://shiro.apache.org/configuration.html#Configuration-INISections
admin = password1
user1 = password2
user2 = password3
[urls]
# anon means the access is anonymous.
# authcBasic means Basic Auth Security
# To enfore security, comment the line below and uncomment the next one
/** = anon
#/** = authcBasic

6
conf/zeppelin-site.xml.template
View file

@ -180,5 +180,11 @@
<description>Allowed sources for REST and WebSocket requests (i.e. http://onehost:8080,http://otherhost.com). If you leave * you are vulnerable to https://issues.apache.org/jira/browse/ZEPPELIN-173</description>
</property>
<property>
<name>zeppelin.anonymous.allowed</name>
<value>true</value>
<description>Anonymous user allowed by default</description>
</property>
</configuration>

2
dev/create_release.sh
View file

@ -125,7 +125,7 @@ function make_binary_release() {
rm -rf ${WORKING_DIR}/zeppelin-${RELEASE_NAME}-bin-${BIN_RELEASE_NAME}
}
make_binary_release all "-Pspark-1.5 -Phadoop-2.4 -Pyarn -Ppyspark"
make_binary_release all "-Pspark-1.6 -Phadoop-2.4 -Pyarn -Ppyspark"
# remove non release files and dirs
rm -rf ${WORKING_DIR}/zeppelin

2
docs/index.md
View file

@ -117,7 +117,7 @@ This way, you can easily embed it as an iframe inside of your website.</p>
<br />
### 100% Opensource
Apache Zeppelin (incubating) is Apache2 Licensed software. Please check out the [source repository](https://github.com/apache/incubator-zeppelin) and [How to contribute](./development/howtocontribute.html)
Apache Zeppelin (incubating) is Apache2 Licensed software. Please check out the [source repository](http://git.apache.org/incubator-zeppelin.git) and [How to contribute](./development/howtocontribute.html)
Zeppelin has a very active development community.
Join the [Mailing list](./community.html) and report issues on our [Issue tracker](https://issues.apache.org/jira/browse/ZEPPELIN).

1
ignite/src/main/java/org/apache/zeppelin/ignite/IgniteInterpreter.java
View file

@ -211,6 +211,7 @@ public class IgniteInterpreter extends Interpreter {
initEx = null;
} catch (Exception e) {
logger.error("Error in IgniteInterpreter while getIgnite: " , e);
initEx = e;
}
}

2
ignite/src/main/java/org/apache/zeppelin/ignite/IgniteSqlInterpreter.java
View file

@ -154,6 +154,7 @@ public class IgniteSqlInterpreter extends Interpreter {
}
}
} catch (Exception e) {
logger.error("Exception in IgniteSqlInterpreter while InterpreterResult interpret: ", e);
return IgniteInterpreterUtils.buildErrorResult(e);
} finally {
curStmt = null;
@ -169,6 +170,7 @@ public class IgniteSqlInterpreter extends Interpreter {
curStmt.cancel();
} catch (SQLException e) {
// No-op.
logger.info("No-op while cancel in IgniteSqlInterpreter", e);
} finally {
curStmt = null;
}

3
lens/src/main/java/org/apache/zeppelin/lens/LensInterpreter.java
View file

@ -128,7 +128,7 @@ public class LensInterpreter extends Interpreter {
s_logger.info("LensInterpreter created");
}
catch (Exception e) {
e.printStackTrace();
s_logger.error(e.toString(), e);
s_logger.error("unable to create lens interpreter", e);
}
}
@ -375,6 +375,7 @@ public class LensInterpreter extends Interpreter {
closeShell(s_paraToQH.get(context.getParagraphId()).getShell());
} catch (Exception e) {
// ignore
s_logger.info("Exception in LensInterpreter while cancel finally, ignore", e);
}
s_paraToQH.remove(context.getParagraphId());
closeShell(shell);

7
lens/src/main/java/org/apache/zeppelin/lens/LensJLineShellComponent.java
View file

@ -17,6 +17,9 @@ package org.apache.zeppelin.lens;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanFactoryUtils;
import org.springframework.beans.factory.InitializingBean;
@ -56,6 +59,8 @@ public class LensJLineShellComponent extends JLineShell
private ExecutionStrategy executionStrategy = new LensSimpleExecutionStrategy();
private SimpleParser parser = new SimpleParser();
private static final Logger LOGGER = LoggerFactory.getLogger(LensJLineShellComponent.class);
public SimpleParser getSimpleParser() {
return parser;
}
@ -123,7 +128,7 @@ public class LensJLineShellComponent extends JLineShell
try {
shellThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
LOGGER.error(e.toString(), e);
}
}

4
markdown/src/main/java/org/apache/zeppelin/markdown/Markdown.java
View file

@ -29,12 +29,15 @@ import org.apache.zeppelin.interpreter.InterpreterUtils;
import org.apache.zeppelin.scheduler.Scheduler;
import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.markdown4j.Markdown4jProcessor;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Markdown interpreter for Zeppelin.
*/
public class Markdown extends Interpreter {
private Markdown4jProcessor md;
static final Logger LOGGER = LoggerFactory.getLogger(Markdown.class);
static {
Interpreter.register("md", Markdown.class.getName());
@ -58,6 +61,7 @@ public class Markdown extends Interpreter {
try {
html = md.process(st);
} catch (IOException | java.lang.RuntimeException e) {
LOGGER.error("Exception in Markdown while interpret ", e);
return new InterpreterResult(Code.ERROR, InterpreterUtils.getMostRelevantMessage(e));
}
return new InterpreterResult(Code.SUCCESS, "%html " + html);

26
pom.xml
View file

@ -208,6 +208,18 @@
<version>4.11</version>
<scope>test</scope>
</dependency>
<!-- Apache Shiro -->
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-core</artifactId>
<version>1.2.3</version>
</dependency>
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-web</artifactId>
<version>1.2.3</version>
</dependency>
</dependencies>
</dependencyManagement>
@ -694,7 +706,7 @@
<plugins>
<plugin>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.4</version>
<version>1.6</version>
<executions>
<execution>
<id>sign-artifacts</id>
@ -709,15 +721,5 @@
</build>
</profile>
</profiles>
<distributionManagement>
<site>
<id>Website</id>
<url>${site_url}</url>
</site>
<repository>
<id>${repoid}</id>
<name>${reponame}</name>
<url>${repourl}</url>
</repository>
</distributionManagement>
</project>

1
postgresql/src/main/java/org/apache/zeppelin/postgresql/PostgreSqlInterpreter.java
View file

@ -297,6 +297,7 @@ public class PostgreSqlInterpreter extends Interpreter {
try {
currentStatement.cancel();
} catch (SQLException ex) {
logger.error("SQLException in PostgreSqlInterpreter while cancel ", ex);
} finally {
currentStatement = null;
}

4
spark/src/main/java/org/apache/zeppelin/spark/DepInterpreter.java
View file

@ -41,6 +41,8 @@ import org.apache.zeppelin.interpreter.InterpreterResult.Code;
import org.apache.zeppelin.interpreter.WrappedInterpreter;
import org.apache.zeppelin.scheduler.Scheduler;
import org.apache.zeppelin.spark.dep.DependencyContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.sonatype.aether.resolution.ArtifactResolutionException;
import org.sonatype.aether.resolution.DependencyResolutionException;
@ -80,6 +82,7 @@ public class DepInterpreter extends Interpreter {
private DependencyContext depc;
private SparkJLineCompletion completor;
private SparkILoop interpreter;
static final Logger LOGGER = LoggerFactory.getLogger(DepInterpreter.class);
public DepInterpreter(Properties property) {
super(property);
@ -195,6 +198,7 @@ public class DepInterpreter extends Interpreter {
depc.fetch();
} catch (MalformedURLException | DependencyResolutionException
| ArtifactResolutionException e) {
LOGGER.error("Exception in DepInterpreter while interpret ", e);
return new InterpreterResult(Code.ERROR, e.toString());
}

8
spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java
View file

@ -82,7 +82,7 @@ import scala.tools.nsc.settings.MutableSettings.PathSetting;
*
*/
public class SparkInterpreter extends Interpreter {
Logger logger = LoggerFactory.getLogger(SparkInterpreter.class);
public static Logger logger = LoggerFactory.getLogger(SparkInterpreter.class);
static {
Interpreter.register(
@ -186,7 +186,7 @@ public class SparkInterpreter extends Interpreter {
}
} catch (NoSuchMethodException | SecurityException | IllegalAccessException
| IllegalArgumentException | InvocationTargetException e) {
e.printStackTrace();
logger.error(e.toString(), e);
return null;
}
return pl;
@ -335,6 +335,10 @@ public class SparkInterpreter extends Interpreter {
conf.set("spark.submit.pyArchives", Joiner.on(":").join(pythonLibs));
}
// Distributes needed libraries to workers.
if (getProperty("master").equals("yarn-client")) {
conf.set("spark.yarn.isPython", "true");
}
SparkContext sparkContext = new SparkContext(conf);
return sparkContext;

5
spark/src/test/java/org/apache/zeppelin/spark/SparkInterpreterTest.java
View file

@ -38,12 +38,15 @@ import org.junit.Before;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runners.MethodSorters;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class SparkInterpreterTest {
public static SparkInterpreter repl;
private InterpreterContext context;
private File tmpDir;
public static Logger LOGGER = LoggerFactory.getLogger(SparkInterpreterTest.class);
/**
@ -177,7 +180,7 @@ public class SparkInterpreterTest {
for (Object oKey : intpProperty.keySet()) {
String key = (String) oKey;
String value = (String) intpProperty.get(key);
repl.logger.debug(String.format("[%s]: [%s]", key, value));
LOGGER.debug(String.format("[%s]: [%s]", key, value));
if (key.startsWith("spark.") && value.isEmpty()) {
assertTrue(String.format("configuration starting from 'spark.' should not be empty. [%s]", key), !sparkConf.contains(key) || !sparkConf.get(key).isEmpty());
}

5
spark/src/test/java/org/apache/zeppelin/spark/SparkSqlInterpreterTest.java
View file

@ -33,6 +33,8 @@ import org.apache.zeppelin.interpreter.InterpreterResult.Type;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class SparkSqlInterpreterTest {
@ -41,6 +43,8 @@ public class SparkSqlInterpreterTest {
private InterpreterContext context;
private InterpreterGroup intpGroup;
Logger LOGGER = LoggerFactory.getLogger(SparkSqlInterpreterTest.class);
@Before
public void setUp() throws Exception {
Properties p = new Properties();
@ -96,6 +100,7 @@ public class SparkSqlInterpreterTest {
fail("Exception not catched");
} catch (Exception e) {
// okay
LOGGER.info("Exception in SparkSqlInterpreterTest while test ", e);
}
assertEquals(InterpreterResult.Code.SUCCESS, sql.interpret("select case when name==\"aa\" then name else name end from test", context).code());
}

14
tajo/src/test/java/org/apache/zeppelin/tajo/TesterResultSet.java
View file

@ -17,13 +17,16 @@
*/
package org.apache.zeppelin.tajo;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.InputStream;
import java.io.Reader;
import java.io.UnsupportedEncodingException;
import java.math.BigDecimal;
import java.sql.*;
import java.util.Calendar;
import java.util.Map;
import java.io.InputStream;
import java.io.Reader;
import java.io.UnsupportedEncodingException;
/**
* This is borrowed from Apache Commons DBCP2.
@ -31,6 +34,9 @@ import java.io.UnsupportedEncodingException;
* A dummy {@link java.sql.ResultSet}, for testing purposes.
*/
public class TesterResultSet implements ResultSet {
Logger LOGGER = LoggerFactory.getLogger(TesterResultSet.class);
public TesterResultSet(Statement stmt) {
_statement = stmt;
}
@ -262,6 +268,8 @@ public class TesterResultSet implements ResultSet {
return columnName.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
// Impossible. JVMs are required to support UTF-8
LOGGER.error("Exception in TesterResultSet while getBytes, Impossible. JVMs are required to" +
" support UTF-8", e);
return null;
}
}

2
zeppelin-distribution/src/bin_license/LICENSE
View file

@ -91,6 +91,8 @@ The following components are provided under Apache License.
(Apache 2.0) Lucene Suggest (org.apache.lucene:lucene-suggest:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-suggest)
(Apache 2.0) Elasticsearch: Core (org.elasticsearch:elasticsearch:2.1.0 - http://nexus.sonatype.org/oss-repository-hosting.html/parent/elasticsearch)
(Apache 2.0) Joda convert (org.joda:joda-convert:1.2 - http://joda-convert.sourceforge.net)
(Apache 2.0) Shiro Core (org.apache.shiro:shiro-core:1.2.3 - https://shiro.apache.org)
(Apache 2.0) Shiro Web (org.apache.shiro:shiro-web:1.2.3 - https://shiro.apache.org)
(Apache 2.0) SnakeYAML (org.yaml:snakeyaml:1.15 - http://www.snakeyaml.org)

2
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
View file

@ -131,7 +131,7 @@ public abstract class Interpreter {
static Logger logger = LoggerFactory.getLogger(Interpreter.class);
public static Logger logger = LoggerFactory.getLogger(Interpreter.class);
private InterpreterGroup interpreterGroup;
private URL [] classloaderUrls;
protected Properties property;

8
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterGroup.java
View file

@ -33,6 +33,8 @@ import org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess;
public class InterpreterGroup extends LinkedList<Interpreter>{
String id;
Logger LOGGER = Logger.getLogger(InterpreterGroup.class);
AngularObjectRegistry angularObjectRegistry;
RemoteInterpreterProcess remoteInterpreterProcess; // attached remote interpreter process
@ -100,8 +102,7 @@ public class InterpreterGroup extends LinkedList<Interpreter>{
try {
t.join();
} catch (InterruptedException e) {
Logger logger = Logger.getLogger(InterpreterGroup.class);
logger.error("Can't close interpreter", e);
LOGGER.error("Can't close interpreter", e);
}
}
}
@ -124,8 +125,7 @@ public class InterpreterGroup extends LinkedList<Interpreter>{
try {
t.join();
} catch (InterruptedException e) {
Logger logger = Logger.getLogger(InterpreterGroup.class);
logger.error("Can't close interpreter", e);
LOGGER.error("Can't close interpreter", e);
}
}

1
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteAngularObjectRegistry.java
View file

@ -101,7 +101,6 @@ public class RemoteAngularObjectRegistry extends AngularObjectRegistry {
* this method should be used instead of remove()
* @param name
* @param noteId
* @param emit
* @return
*/
public AngularObject removeAndNotifyRemoteProcess(String name, String noteId) {

1
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterEventPoller.java
View file

@ -124,6 +124,7 @@ public class RemoteInterpreterEventPoller extends Thread {
wait(1000);
}
} catch (InterruptedException ignored) {
logger.info("Error in RemoteInterpreterEventPoller while waitQuietly : ", ignored);
}
}

8
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterProcess.java
View file

@ -121,6 +121,8 @@ public class RemoteInterpreterProcess implements ExecuteResultHandler {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
logger.error("Exception in RemoteInterpreterProcess while synchronized reference " +
"Thread.sleep", e);
}
}
}
@ -177,6 +179,8 @@ public class RemoteInterpreterProcess implements ExecuteResultHandler {
client.shutdown();
} catch (Exception e) {
// safely ignore exception while client.shutdown() may terminates remote process
logger.info("Exception in RemoteInterpreterProcess while synchronized dereference, can " +
"safely ignore exception while client.shutdown() may terminates remote process", e);
} finally {
if (client != null) {
// no longer used
@ -195,6 +199,8 @@ public class RemoteInterpreterProcess implements ExecuteResultHandler {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
logger.error("Exception in RemoteInterpreterProcess while synchronized dereference " +
"Thread.sleep", e);
}
} else {
break;
@ -266,6 +272,8 @@ public class RemoteInterpreterProcess implements ExecuteResultHandler {
client = getClient();
} catch (NullPointerException e) {
// remote process not started
logger.info("NullPointerException in RemoteInterpreterProcess while " +
"updateRemoteAngularObject getClient, remote process not started", e);
return;
} catch (Exception e) {
logger.error("Can't update angular object", e);

9
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterServer.java
View file

@ -114,6 +114,7 @@ public class RemoteInterpreterServer
try {
Thread.sleep(300);
} catch (InterruptedException e) {
logger.info("Exception in RemoteInterpreterServer while shutdown, Thread.sleep", e);
}
}
@ -170,7 +171,7 @@ public class RemoteInterpreterServer
} catch (ClassNotFoundException | NoSuchMethodException | SecurityException
| InstantiationException | IllegalAccessException
| IllegalArgumentException | InvocationTargetException e) {
e.printStackTrace();
logger.error(e.toString(), e);
throw new TException(e);
}
}
@ -225,6 +226,7 @@ public class RemoteInterpreterServer
try {
jobListener.wait(1000);
} catch (InterruptedException e) {
logger.info("Exception in RemoteInterpreterServer while interpret, jobListener.wait", e);
}
}
}
@ -455,6 +457,7 @@ public class RemoteInterpreterServer
try {
eventQueue.wait(1000);
} catch (InterruptedException e) {
logger.info("Exception in RemoteInterpreterServer while getEvent, eventQueue.wait", e);
}
}
@ -468,7 +471,6 @@ public class RemoteInterpreterServer
/**
* called when object is updated in client (web) side.
* @param className
* @param name
* @param noteId noteId where the update issues
* @param object
@ -499,6 +501,7 @@ public class RemoteInterpreterServer
return;
} catch (Exception e) {
// no luck
logger.info("Exception in RemoteInterpreterServer while angularObjectUpdate, no luck", e);
}
}
@ -510,6 +513,7 @@ public class RemoteInterpreterServer
}.getType());
} catch (Exception e) {
// no lock
logger.info("Exception in RemoteInterpreterServer while angularObjectUpdate, no lock", e);
}
}
@ -544,6 +548,7 @@ public class RemoteInterpreterServer
}.getType());
} catch (Exception e) {
// nolock
logger.info("Exception in RemoteInterpreterServer while angularObjectAdd, nolock", e);
}
// try string object type at last

5
zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterUtils.java
View file

@ -17,6 +17,9 @@
package org.apache.zeppelin.interpreter.remote;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
@ -26,6 +29,7 @@ import java.net.Socket;
*
*/
public class RemoteInterpreterUtils {
static Logger LOGGER = LoggerFactory.getLogger(RemoteInterpreterUtils.class);
public static int findRandomAvailablePortOnAllLocalInterfaces() throws IOException {
int port;
try (ServerSocket socket = new ServerSocket(0);) {
@ -43,6 +47,7 @@ public class RemoteInterpreterUtils {
discover.close();
return true;
} catch (IOException e) {
LOGGER.info("Exception in RemoteInterpreterUtils while checkIfRemoteEndpointAccessible", e);
return false;
}
}

5
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/FIFOScheduler.java
View file

@ -24,6 +24,8 @@ import java.util.List;
import java.util.concurrent.ExecutorService;
import org.apache.zeppelin.scheduler.Job.Status;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* FIFOScheduler runs submitted job sequentially
@ -36,6 +38,8 @@ public class FIFOScheduler implements Scheduler {
Job runningJob = null;
private String name;
static Logger LOGGER = LoggerFactory.getLogger(FIFOScheduler.class);
public FIFOScheduler(String name, ExecutorService executor, SchedulerListener listener) {
this.name = name;
this.executor = executor;
@ -107,6 +111,7 @@ public class FIFOScheduler implements Scheduler {
try {
queue.wait(500);
} catch (InterruptedException e) {
LOGGER.error("Exception in FIFOScheduler while run queue.wait", e);
}
continue;
}

10
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java
View file

@ -78,6 +78,8 @@ public abstract class Job {
Date dateFinished;
Status status;
static Logger LOGGER = LoggerFactory.getLogger(Job.class);
transient boolean aborted = false;
String errorMessage;
@ -172,14 +174,14 @@ public abstract class Job {
dateFinished = new Date();
progressUpdator.terminate();
} catch (NullPointerException e) {
logger().error("Job failed", e);
LOGGER.error("Job failed", e);
progressUpdator.terminate();
this.exception = e;
result = e.getMessage();
errorMessage = getStack(e);
dateFinished = new Date();
} catch (Throwable e) {
logger().error("Job failed", e);
LOGGER.error("Job failed", e);
progressUpdator.terminate();
this.exception = e;
result = e.getMessage();
@ -248,10 +250,6 @@ public abstract class Job {
return dateFinished;
}
private Logger logger() {
return LoggerFactory.getLogger(Job.class);
}
protected void setResult(Object result) {
this.result = result;
}

1
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/JobProgressPoller.java
View file

@ -57,6 +57,7 @@ public class JobProgressPoller extends Thread {
try {
Thread.sleep(intervalMs);
} catch (InterruptedException e) {
logger.error("Exception in JobProgressPoller while run Thread.sleep", e);
}
}
}

5
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/ParallelScheduler.java
View file

@ -24,6 +24,8 @@ import java.util.List;
import java.util.concurrent.ExecutorService;
import org.apache.zeppelin.scheduler.Job.Status;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Parallel scheduler runs submitted job concurrently.
@ -37,6 +39,8 @@ public class ParallelScheduler implements Scheduler {
private String name;
private int maxConcurrency;
static Logger LOGGER = LoggerFactory.getLogger(ParallelScheduler.class);
public ParallelScheduler(String name, ExecutorService executor, SchedulerListener listener,
int maxConcurrency) {
this.name = name;
@ -107,6 +111,7 @@ public class ParallelScheduler implements Scheduler {
try {
queue.wait(500);
} catch (InterruptedException e) {
LOGGER.error("Exception in MockInterpreterAngular while interpret queue.wait", e);
}
continue;
}

16
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/RemoteScheduler.java
View file

@ -17,12 +17,6 @@
package org.apache.zeppelin.scheduler;
import java.util.Collection;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import org.apache.thrift.TException;
import org.apache.zeppelin.interpreter.InterpreterResult;
import org.apache.zeppelin.interpreter.InterpreterResult.Code;
@ -32,6 +26,12 @@ import org.apache.zeppelin.scheduler.Job.Status;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.Collection;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.concurrent.ExecutorService;
/**
* RemoteScheduler runs in ZeppelinServer and proxies Scheduler running on RemoteInterpreter
*/
@ -67,6 +67,7 @@ public class RemoteScheduler implements Scheduler {
try {
queue.wait(500);
} catch (InterruptedException e) {
logger.error("Exception in RemoteScheduler while run queue.wait", e);
}
continue;
}
@ -86,6 +87,8 @@ public class RemoteScheduler implements Scheduler {
try {
queue.wait(500);
} catch (InterruptedException e) {
logger.error("Exception in RemoteScheduler while jobRunner.isJobSubmittedInRemote " +
"queue.wait", e);
}
}
}
@ -194,6 +197,7 @@ public class RemoteScheduler implements Scheduler {
try {
this.wait(interval);
} catch (InterruptedException e) {
logger.error("Exception in RemoteScheduler while run this.wait", e);
}
}

4
zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/SchedulerFactory.java
View file

@ -32,7 +32,7 @@ import org.slf4j.LoggerFactory;
* TODO(moon) : add description.
*/
public class SchedulerFactory implements SchedulerListener {
private final Logger logger = LoggerFactory.getLogger(SchedulerFactory.class);
private static final Logger logger = LoggerFactory.getLogger(SchedulerFactory.class);
ExecutorService executor;
Map<String, Scheduler> schedulers = new LinkedHashMap<String, Scheduler>();
@ -46,7 +46,7 @@ public class SchedulerFactory implements SchedulerListener {
try {
singleton = new SchedulerFactory();
} catch (Exception e) {
e.printStackTrace();
logger.error(e.toString(), e);
}
}
}

1
zeppelin-interpreter/src/test/java/org/apache/zeppelin/interpreter/remote/mock/MockInterpreterAngular.java
View file

@ -90,6 +90,7 @@ public class MockInterpreterAngular extends Interpreter {
try {
Thread.sleep(500); // wait for watcher executed
} catch (InterruptedException e) {
logger.error("Exception in MockInterpreterAngular while interpret Thread.sleep", e);
}
String msg = registry.getAll(context.getNoteId()).size() + " " + Integer.toString(numWatch.get());

5
zeppelin-interpreter/src/test/java/org/apache/zeppelin/scheduler/SleepingJob.java
View file

@ -22,6 +22,8 @@ import java.util.Map;
import org.apache.zeppelin.scheduler.Job;
import org.apache.zeppelin.scheduler.JobListener;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class SleepingJob extends Job{
@ -30,6 +32,8 @@ public class SleepingJob extends Job{
private long start;
private int count;
static Logger LOGGER = LoggerFactory.getLogger(SleepingJob.class);
public SleepingJob(String jobName, JobListener listener, int time){
super(jobName, listener);
@ -44,6 +48,7 @@ public class SleepingJob extends Job{
try {
Thread.sleep(10);
} catch (InterruptedException e) {
LOGGER.error("Exception in MockInterpreterAngular while interpret Thread.sleep", e);
}
if(System.currentTimeMillis() - start>time) break;
}

10
zeppelin-server/pom.xml
View file

@ -269,6 +269,16 @@
<version>1.9.0</version>
<scope>test</scope>
</dependency>
<!-- Apache Shiro -->
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-core</artifactId>
</dependency>
<dependency>
<groupId>org.apache.shiro</groupId>
<artifactId>shiro-web</artifactId>
</dependency>
</dependencies>
<build>

3
zeppelin-server/src/main/java/org/apache/zeppelin/rest/InterpreterRestApi.java
View file

@ -110,9 +110,11 @@ public class InterpreterRestApi {
interpreterFactory.setPropertyAndRestart(settingId,
new InterpreterOption(true), p.getProperties());
} catch (InterpreterException e) {
logger.error("Exception in InterpreterRestApi while updateSetting ", e);
return new JsonResponse(
Status.NOT_FOUND, e.getMessage(), ExceptionUtils.getStackTrace(e)).build();
} catch (IOException e) {
logger.error("Exception in InterpreterRestApi while updateSetting ", e);
return new JsonResponse(
Status.INTERNAL_SERVER_ERROR, e.getMessage(), ExceptionUtils.getStackTrace(e)).build();
}
@ -144,6 +146,7 @@ public class InterpreterRestApi {
try {
interpreterFactory.restart(settingId);
} catch (InterpreterException e) {
logger.error("Exception in InterpreterRestApi while restartSetting ", e);
return new JsonResponse(
Status.NOT_FOUND, e.getMessage(), ExceptionUtils.getStackTrace(e)).build();
}

1
zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java
View file

@ -338,6 +338,7 @@ public class NotebookRestApi {
notebookServer.broadcastNote(note);
return new JsonResponse(Status.OK, "").build();
} catch (IndexOutOfBoundsException e) {
LOG.error("Exception in NotebookRestApi while moveParagraph ", e);
return new JsonResponse(Status.BAD_REQUEST, "paragraph's new index is out of bound").build();
}
}

74
zeppelin-server/src/main/java/org/apache/zeppelin/rest/SecurityRestApi.java Normal file
View file

@ -0,0 +1,74 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.server.JsonResponse;
import org.apache.zeppelin.ticket.TicketContainer;
import org.apache.zeppelin.utils.SecurityUtils;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.Response;
import java.util.HashMap;
import java.util.Map;
/**
* Zeppelin security rest api endpoint.
*
*/
@Path("/security")
@Produces("application/json")
public class SecurityRestApi {
/**
* Required by Swagger.
*/
public SecurityRestApi() {
super();
}
/**
* Get ticket
* Returns username & ticket
* for anonymous access, username is always anonymous.
* After getting this ticket, access through websockets become safe
*
* @return 200 response
*/
@GET
@Path("ticket")
public Response ticket() {
ZeppelinConfiguration conf = ZeppelinConfiguration.create();
String principal = SecurityUtils.getPrincipal();
JsonResponse response;
// ticket set to anonymous for anonymous user. Simplify testing.
String ticket;
if ("anonymous".equals(principal))
ticket = "anonymous";
else
ticket = TicketContainer.instance.getTicket(principal);
Map<String, String> data = new HashMap<>();
data.put("principal", principal);
data.put("ticket", ticket);
response = new JsonResponse(Response.Status.OK, "", data);
return response.build();
}
}

8
zeppelin-server/src/main/java/org/apache/zeppelin/server/CorsFilter.java
View file

@ -19,12 +19,12 @@ package org.apache.zeppelin.server;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import org.apache.zeppelin.utils.SecurityUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.text.DateFormat;
import java.util.Arrays;
import java.util.Date;
import java.util.Locale;
@ -43,6 +43,8 @@ import javax.servlet.http.HttpServletResponse;
*/
public class CorsFilter implements Filter {
private static final Logger LOGGER = LoggerFactory.getLogger(CorsFilter.class);
@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain filterChain)
throws IOException, ServletException {
@ -54,7 +56,7 @@ public class CorsFilter implements Filter {
origin = sourceHost;
}
} catch (URISyntaxException e) {
e.printStackTrace();
LOGGER.error("Exception in WebDriverManager while getWebDriver ", e);
}
if (((HttpServletRequest) request).getMethod().equals("OPTIONS")) {

11
zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java
View file

@ -36,6 +36,7 @@ import org.apache.zeppelin.notebook.repo.NotebookRepo;
import org.apache.zeppelin.notebook.repo.NotebookRepoSync;
import org.apache.zeppelin.rest.InterpreterRestApi;
import org.apache.zeppelin.rest.NotebookRestApi;
import org.apache.zeppelin.rest.SecurityRestApi;
import org.apache.zeppelin.rest.ZeppelinRestApi;
import org.apache.zeppelin.scheduler.SchedulerFactory;
import org.apache.zeppelin.search.SearchService;
@ -135,6 +136,7 @@ public class ZeppelinServer extends Application {
try {
System.in.read();
} catch (IOException e) {
LOG.error("Exception in ZeppelinServer while main ", e);
}
System.exit(0);
}
@ -226,6 +228,12 @@ public class ZeppelinServer extends Application {
cxfContext.addFilter(new FilterHolder(CorsFilter.class), "/*",
EnumSet.allOf(DispatcherType.class));
cxfContext.addFilter(org.apache.shiro.web.servlet.ShiroFilter.class, "/*",
EnumSet.allOf(DispatcherType.class));
cxfContext.addEventListener(new org.apache.shiro.web.env.EnvironmentLoaderListener());
return cxfContext;
}
@ -273,6 +281,9 @@ public class ZeppelinServer extends Application {
InterpreterRestApi interpreterApi = new InterpreterRestApi(replFactory);
singletons.add(interpreterApi);
SecurityRestApi securityApi = new SecurityRestApi();
singletons.add(securityApi);
return singletons;
}
}

2
zeppelin-server/src/main/java/org/apache/zeppelin/socket/Message.java
View file

@ -103,6 +103,8 @@ public class Message {
public OP op;
public Map<String, Object> data = new HashMap<String, Object>();
public String ticket = "anonymous";
public String principal = "anonymous";
public Message(OP op) {
this.op = op;

20
zeppelin-server/src/main/java/org/apache/zeppelin/socket/NotebookServer.java
View file

@ -41,6 +41,7 @@ import org.apache.zeppelin.scheduler.Job.Status;
import org.apache.zeppelin.scheduler.JobListener;
import org.apache.zeppelin.server.ZeppelinServer;
import org.apache.zeppelin.socket.Message.OP;
import org.apache.zeppelin.ticket.TicketContainer;
import org.apache.zeppelin.utils.SecurityUtils;
import org.eclipse.jetty.websocket.WebSocket;
import org.eclipse.jetty.websocket.WebSocketServlet;
@ -71,9 +72,9 @@ public class NotebookServer extends WebSocketServlet implements
try {
return SecurityUtils.isValidOrigin(origin, ZeppelinConfiguration.create());
} catch (UnknownHostException e) {
e.printStackTrace();
LOG.error(e.toString(), e);
} catch (URISyntaxException e) {
e.printStackTrace();
LOG.error(e.toString(), e);
}
return false;
}
@ -96,6 +97,19 @@ public class NotebookServer extends WebSocketServlet implements
try {
Message messagereceived = deserializeMessage(msg);
LOG.debug("RECEIVE << " + messagereceived.op);
LOG.debug("RECEIVE PRINCIPAL << " + messagereceived.principal);
LOG.debug("RECEIVE TICKET << " + messagereceived.ticket);
String ticket = TicketContainer.instance.getTicket(messagereceived.principal);
if (ticket != null && !ticket.equals(messagereceived.ticket))
throw new Exception("Invalid ticket " + messagereceived.ticket + " != " + ticket);
ZeppelinConfiguration conf = ZeppelinConfiguration.create();
boolean allowAnonymous = conf.
getBoolean(ZeppelinConfiguration.ConfVars.ZEPPELIN_ANONYMOUS_ALLOWED);
if (!allowAnonymous && messagereceived.principal.equals("anonymous")) {
throw new Exception("Anonymous access not allowed ");
}
/** Lets be elegant here */
switch (messagereceived.op) {
case LIST_NOTES:
@ -772,7 +786,7 @@ public class NotebookServer extends WebSocketServlet implements
try {
note.persist();
} catch (IOException e) {
e.printStackTrace();
LOG.error(e.toString(), e);
}
}
notebookServer.broadcastNote(note);

82
zeppelin-server/src/main/java/org/apache/zeppelin/ticket/TicketContainer.java Normal file
View file

@ -0,0 +1,82 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.ticket;
import java.util.Calendar;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
/**
* Very simple ticket container
* No cleanup is done, since the same user accross different devices share the same ticket
* The Map size is at most the number of different user names having access to a Zeppelin instance
*/
public class TicketContainer {
private static class Entry {
public final String ticket;
// lastAccessTime still unused
public final long lastAccessTime;
Entry(String ticket) {
this.ticket = ticket;
this.lastAccessTime = Calendar.getInstance().getTimeInMillis();
}
}
private Map<String, Entry> sessions = new ConcurrentHashMap<>();
public static final TicketContainer instance = new TicketContainer();
/**
* For test use
* @param principal
* @param ticket
* @return true if ticket assigned to principal.
*/
public boolean isValid(String principal, String ticket) {
if ("anonymous".equals(principal) && "anonymous".equals(ticket))
return true;
Entry entry = sessions.get(principal);
return entry != null && entry.ticket.equals(ticket);
}
/**
* get or create ticket for Websocket authentication assigned to authenticated shiro user
* For unathenticated user (anonymous), always return ticket value "anonymous"
* @param principal
* @return
*/
public synchronized String getTicket(String principal) {
Entry entry = sessions.get(principal);
String ticket;
if (entry == null) {
if (principal.equals("anonymous"))
ticket = "anonymous";
else
ticket = UUID.randomUUID().toString();
} else {
ticket = entry.ticket;
}
entry = new Entry(ticket);
sessions.put(principal, entry);
return ticket;
}
}

17
zeppelin-server/src/main/java/org/apache/zeppelin/utils/SecurityUtils.java
View file

@ -16,6 +16,7 @@
*/
package org.apache.zeppelin.utils;
import org.apache.shiro.subject.Subject;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
import java.net.InetAddress;
@ -44,4 +45,20 @@ public class SecurityUtils {
"localhost".equals(sourceUriHost) ||
conf.getAllowedOrigins().contains(sourceHost);
}
/**
* Return the authenticated user if any otherwise returns "anonymous"
* @return shiro principal
*/
public static String getPrincipal() {
Subject subject = org.apache.shiro.SecurityUtils.getSubject();
String principal;
if (subject.isAuthenticated()) {
principal = subject.getPrincipal().toString();
}
else {
principal = "anonymous";
}
return principal;
}
}

31
zeppelin-server/src/main/resources/shiro.ini Normal file
View file

@ -0,0 +1,31 @@
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
[users]
# List of users with their password allowed to access Zeppelin.
# To use a different strategy (LDAP / Database / ...) check the shiro doc at http://shiro.apache.org/configuration.html#Configuration-INISections
admin = password
[urls]
# anon means the access is anonymous.
# authcBasic means Basic Auth Security
# To enfore security, comment the line below and uncomment the next one
/** = anon
#/** = authcBasic

7
zeppelin-server/src/test/java/com/webautomation/ScreenCaptureHtmlUnitDriver.java
View file

@ -50,6 +50,8 @@ import com.gargoylesoftware.htmlunit.WebRequest;
import com.gargoylesoftware.htmlunit.WebWindow;
import com.gargoylesoftware.htmlunit.html.HtmlElement;
import com.gargoylesoftware.htmlunit.html.HtmlPage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* from https://code.google.com/p/selenium/issues/detail?id=1361
@ -63,6 +65,8 @@ public class ScreenCaptureHtmlUnitDriver extends HtmlUnitDriver implements Takes
// http://stackoverflow.com/questions/4652777/java-regex-to-get-the-urls-from-css
private final static Pattern cssUrlPattern = Pattern.compile("background(-image)?[\\s]*:[^url]*url[\\s]*\\([\\s]*([^\\)]*)[\\s]*\\)[\\s]*");// ?<url>
static Logger LOGGER = LoggerFactory.getLogger(ScreenCaptureHtmlUnitDriver.class);
public ScreenCaptureHtmlUnitDriver() {
super();
}
@ -88,6 +92,7 @@ public class ScreenCaptureHtmlUnitDriver extends HtmlUnitDriver implements Takes
try {
archive = downloadCssAndImages(getWebClient(), (HtmlPage) getCurrentWindow().getEnclosedPage());
} catch (Exception e) {
LOGGER.error("Exception in ScreenCaptureHtmlUnitDriver while getScreenshotAs ", e);
}
if(target.equals(OutputType.BASE64)){
return target.convertFromBase64Png(new Base64Encoder().encode(archive));
@ -116,6 +121,7 @@ public class ScreenCaptureHtmlUnitDriver extends HtmlUnitDriver implements Takes
window = webClient.getWebWindowByName(page.getUrl().toString()+"_screenshot");
webClient.getPage(window, new WebRequest(page.getUrl()));
} catch (Exception e) {
LOGGER.error("Exception in ScreenCaptureHtmlUnitDriver while downloadCssAndImages ", e);
window = webClient.openWindow(page.getUrl(), page.getUrl().toString()+"_screenshot");
}
@ -148,6 +154,7 @@ public class ScreenCaptureHtmlUnitDriver extends HtmlUnitDriver implements Takes
.replace("resources/", "./").getBytes());
}
} catch (Exception e) {
LOGGER.error("Exception in ScreenCaptureHtmlUnitDriver while resultList.iterator ", e);
}
}
String pagesrc = replaceRemoteUrlsWithLocal(page.getWebResponse().getContentAsString(), urlMapping);

1
zeppelin-server/src/test/java/org/apache/zeppelin/ProcessData.java
View file

@ -233,6 +233,7 @@ public class ProcessData {
String exceptionAsString = sw.toString();
LOG.error(exceptionAsString);
} catch (Exception ignore) {
LOG.info("Exception in ProcessData while buildOutputAndErrorStreamData ", ignore);
}
break;
}

7
zeppelin-server/src/test/java/org/apache/zeppelin/WebDriverManager.java
View file

@ -85,6 +85,7 @@ public class WebDriverManager {
driver = new FirefoxDriver(ffox, profile);
} catch (Exception e) {
LOG.error("Exception in WebDriverManager while FireFox Driver ", e);
}
}
@ -92,6 +93,7 @@ public class WebDriverManager {
try {
driver = new ChromeDriver();
} catch (Exception e) {
LOG.error("Exception in WebDriverManager while ChromeDriver ", e);
}
}
@ -99,6 +101,7 @@ public class WebDriverManager {
try {
driver = new SafariDriver();
} catch (Exception e) {
LOG.error("Exception in WebDriverManager while SafariDriver ", e);
}
}
@ -126,6 +129,7 @@ public class WebDriverManager {
loaded = true;
break;
} catch (TimeoutException e) {
LOG.info("Exception in WebDriverManager while WebDriverWait ", e);
driver.navigate().to(url);
}
}
@ -164,7 +168,6 @@ public class WebDriverManager {
} catch (IOException e) {
LOG.error("Download of firebug version: " + firefoxVersion + ", falied in path " + tempPath);
LOG.error(e.toString());
}
LOG.info("Download of firebug version: " + firefoxVersion + ", successful");
}
@ -178,7 +181,7 @@ public class WebDriverManager {
String versionString = (String) CommandExecutor.executeCommandLocalHost(firefoxVersionCmd, false, ProcessData.Types_Of_Data.OUTPUT);
return Integer.valueOf(versionString.replaceAll("Mozilla Firefox", "").trim().substring(0, 2));
} catch (Exception e) {
e.printStackTrace();
LOG.error("Exception in WebDriverManager while getWebDriver ", e);
return -1;
}
}

3
zeppelin-server/src/test/java/org/apache/zeppelin/ZeppelinIT.java
View file

@ -101,6 +101,7 @@ public class ZeppelinIT {
WebElement element = pollingWait(locator, MAX_BROWSER_TIMEOUT_SEC);
return txt.equals(element.getText());
} catch (TimeoutException e) {
LOG.error("Exception in ZeppelinIT while waitForText ", e);
return false;
}
}
@ -255,6 +256,7 @@ public class ZeppelinIT {
System.out.println("testCreateNotebook Test executed");
} catch (ElementNotVisibleException e) {
LOG.error("Exception in ZeppelinIT while testAngularDisplay ", e);
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
}
@ -279,6 +281,7 @@ public class ZeppelinIT {
try {
Thread.sleep(500); // wait for notebook list updated
} catch (InterruptedException e) {
LOG.error("Exception in ZeppelinIT while createNewNote Thread.sleep", e);
}
}
}

2
zeppelin-server/src/test/java/org/apache/zeppelin/ZeppelinITUtils.java
View file

@ -33,7 +33,7 @@ public class ZeppelinITUtils {
try {
Thread.sleep(millis);
} catch (InterruptedException e) {
e.printStackTrace();
LOG.error("Exception in WebDriverManager while getWebDriver ", e);
}
if (logOutput) {
LOG.info("Finished.");

14
zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java
View file

@ -55,7 +55,7 @@ public abstract class AbstractTestRestApi {
static final String restApiUrl = "/api";
static final String url = getUrlToTest();
protected static final boolean wasRunning = checkIfServerIsRuning();
protected static final boolean wasRunning = checkIfServerIsRunning();
static boolean pySpark = false;
private String getUrl(String path) {
@ -86,7 +86,7 @@ public abstract class AbstractTestRestApi {
try {
ZeppelinServer.main(new String[] {""});
} catch (Exception e) {
e.printStackTrace();
LOG.error("Exception in WebDriverManager while getWebDriver ", e);
throw new RuntimeException(e);
}
}
@ -101,7 +101,7 @@ public abstract class AbstractTestRestApi {
boolean started = false;
while (System.currentTimeMillis() - s < 1000 * 60 * 3) { // 3 minutes
Thread.sleep(2000);
started = checkIfServerIsRuning();
started = checkIfServerIsRunning();
if (started == true) {
break;
}
@ -156,7 +156,7 @@ public abstract class AbstractTestRestApi {
try {
return InetAddress.getLocalHost().getHostName();
} catch (UnknownHostException e) {
e.printStackTrace();
LOG.error("Exception in WebDriverManager while getWebDriver ", e);
return "localhost";
}
}
@ -218,7 +218,7 @@ public abstract class AbstractTestRestApi {
boolean started = true;
while (System.currentTimeMillis() - s < 1000 * 60 * 3) { // 3 minutes
Thread.sleep(2000);
started = checkIfServerIsRuning();
started = checkIfServerIsRunning();
if (started == false) {
break;
}
@ -231,13 +231,14 @@ public abstract class AbstractTestRestApi {
}
}
protected static boolean checkIfServerIsRuning() {
protected static boolean checkIfServerIsRunning() {
GetMethod request = null;
boolean isRunning = true;
try {
request = httpGet("/");
isRunning = request.getStatusCode() == 200;
} catch (IOException e) {
LOG.error("Exception in AbstractTestRestApi while checkIfServerIsRunning ", e);
isRunning = false;
} finally {
if (request != null) {
@ -343,6 +344,7 @@ public abstract class AbstractTestRestApi {
try {
new JsonParser().parse(body);
} catch (JsonParseException e) {
LOG.error("Exception in AbstractTestRestApi while matchesSafely ", e);
isValid = false;
}
return isValid;

58
zeppelin-server/src/test/java/org/apache/zeppelin/rest/SecurityRestApiTest.java Normal file
View file

@ -0,0 +1,58 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.rest;
import com.google.gson.Gson;
import com.google.gson.reflect.TypeToken;
import org.apache.commons.httpclient.methods.GetMethod;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import java.io.IOException;
import java.util.Map;
import static org.junit.Assert.*;
public class SecurityRestApiTest extends AbstractTestRestApi {
Gson gson = new Gson();
@BeforeClass
public static void init() throws Exception {
AbstractTestRestApi.startUp();
}
@AfterClass
public static void destroy() throws Exception {
AbstractTestRestApi.shutDown();
}
@Test
public void testTicket() throws IOException {
GetMethod get = httpGet("/security/ticket");
get.addRequestHeader("Origin", "http://localhost");
Map<String, Object> resp = gson.fromJson(get.getResponseBodyAsString(),
new TypeToken<Map<String, Object>>(){}.getType());
Map<String, String> body = (Map<String, String>) resp.get("body");
assertEquals("anonymous", body.get("principal"));
assertEquals("anonymous", body.get("ticket"));
get.releaseConnection();
}
}

2
zeppelin-server/src/test/java/org/apache/zeppelin/rest/ZeppelinSparkClusterTest.java
View file

@ -60,7 +60,7 @@ public class ZeppelinSparkClusterTest extends AbstractTestRestApi {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
LOG.error("Exception in WebDriverManager while getWebDriver ", e);
}
}
}

16
zeppelin-server/src/test/java/org/apache/zeppelin/socket/NotebookServerTest.java
View file

@ -19,10 +19,7 @@
*/
package org.apache.zeppelin.socket;
import static org.junit.Assert.*;
import java.io.IOException;
import com.google.gson.Gson;
import org.apache.zeppelin.interpreter.InterpreterGroup;
import org.apache.zeppelin.interpreter.InterpreterSetting;
import org.apache.zeppelin.notebook.Note;
@ -34,14 +31,13 @@ import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import com.google.gson.Gson;
import java.net.UnknownHostException;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.List;
import javax.servlet.http.HttpServletRequest;
import static org.junit.Assert.*;
import static org.mockito.Mockito.*;
@ -149,6 +145,8 @@ public class NotebookServerTest extends AbstractTestRestApi {
note = notebookServer.importNote(null, notebook, messageReceived);
} catch (NullPointerException e) {
//broadcastNoteList(); failed nothing to worry.
LOG.error("Exception in NotebookServerTest while testImportNotebook, failed nothing to " +
"worry ", e);
}
assertNotEquals(null, notebook.getNote(note.getId()));

62
zeppelin-server/src/test/java/org/apache/zeppelin/ticket/TicketContainerTest.java Normal file
View file

@ -0,0 +1,62 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.zeppelin.ticket;
import org.junit.Before;
import org.junit.Test;
import java.net.UnknownHostException;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
public class TicketContainerTest {
private TicketContainer container;
@Before
public void setUp() throws Exception {
container = TicketContainer.instance;
}
@Test
public void isValidAnonymous() throws UnknownHostException {
boolean ok = container.isValid("anonymous", "anonymous");
assertTrue(ok);
}
@Test
public void isValidExistingPrincipal() throws UnknownHostException {
String ticket = container.getTicket("someuser1");
boolean ok = container.isValid("someuser1", ticket);
assertTrue(ok);
}
@Test
public void isValidNonExistingPrincipal() throws UnknownHostException {
boolean ok = container.isValid("unknownuser", "someticket");
assertFalse(ok);
}
@Test
public void isValidunkownTicket() throws UnknownHostException {
String ticket = container.getTicket("someuser2");
boolean ok = container.isValid("someuser2", ticket+"makeitinvalid");
assertFalse(ok);
}
}

149
zeppelin-web/src/app/app.js
View file

@ -15,65 +15,96 @@
* limitations under the License.
*/
'use strict';
(function() {
var zeppelinWebApp = angular.module('zeppelinWebApp', [
'ngAnimate',
'ngCookies',
'ngRoute',
'ngSanitize',
'angular-websocket',
'ui.ace',
'ui.bootstrap',
'ui.sortable',
'ngTouch',
'ngDragDrop',
'angular.filter',
'monospaced.elastic',
'puElasticInput',
'xeditable',
'ngToast',
'focus-if',
'ngResource'
])
.filter('breakFilter', function() {
return function (text) {
if (!!text) {
return text.replace(/\n/g, '<br />');
}
};
})
.config(function ($httpProvider, $routeProvider, ngToastProvider) {
// withCredentials when running locally via grunt
$httpProvider.defaults.withCredentials = true;
angular.module('zeppelinWebApp', [
'ngAnimate',
'ngCookies',
'ngRoute',
'ngSanitize',
'angular-websocket',
'ui.ace',
'ui.bootstrap',
'ui.sortable',
'ngTouch',
'ngDragDrop',
'angular.filter',
'monospaced.elastic',
'puElasticInput',
'xeditable',
'ngToast',
'focus-if',
'ngResource'
])
.filter('breakFilter', function() {
return function (text) {
if (!!text) {
return text.replace(/\n/g, '<br />');
}
};
})
.config(function ($routeProvider, ngToastProvider) {
$routeProvider
.when('/', {
templateUrl: 'app/home/home.html'
})
.when('/notebook/:noteId', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/notebook/:noteId/paragraph?=:paragraphId', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/notebook/:noteId/paragraph/:paragraphId?', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/interpreter', {
templateUrl: 'app/interpreter/interpreter.html',
controller: 'InterpreterCtrl'
})
.when('/search/:searchTerm', {
templateUrl: 'app/search/result-list.html',
controller: 'SearchResultCtrl'
})
.otherwise({
redirectTo: '/'
});
$routeProvider
.when('/', {
templateUrl: 'app/home/home.html'
})
.when('/notebook/:noteId', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/notebook/:noteId/paragraph?=:paragraphId', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/notebook/:noteId/paragraph/:paragraphId?', {
templateUrl: 'app/notebook/notebook.html',
controller: 'NotebookCtrl'
})
.when('/interpreter', {
templateUrl: 'app/interpreter/interpreter.html',
controller: 'InterpreterCtrl'
})
.when('/search/:searchTerm', {
templateUrl: 'app/search/result-list.html',
controller: 'SearchResultCtrl'
})
.otherwise({
redirectTo: '/'
});
ngToastProvider.configure({
dismissButton: true,
dismissOnClick: false,
timeout: 6000
ngToastProvider.configure({
dismissButton: true,
dismissOnClick: false,
timeout: 6000
});
});
function auth() {
var $http = angular.injector(['ng']).get('$http');
var baseUrlSrv = angular.injector(['zeppelinWebApp']).get('baseUrlSrv');
// withCredentials when running locally via grunt
$http.defaults.withCredentials = true;
return $http.get(baseUrlSrv.getRestApiBase()+'/security/ticket').then(function(response) {
zeppelinWebApp.run(function($rootScope) {
$rootScope.ticket = angular.fromJson(response.data).body;
});
}, function(errorResponse) {
// Handle error case
});
}
function bootstrapApplication() {
angular.bootstrap(document, ['zeppelinWebApp']);
}
angular.element(document).ready(function() {
auth().then(bootstrapApplication);
});
});
}());

1
zeppelin-web/src/app/home/home.controller.js
View file

@ -14,7 +14,6 @@
'use strict';
angular.module('zeppelinWebApp').controller('HomeCtrl', function($scope, notebookListDataFactory, websocketMsgSrv, $rootScope, arrayOrderingSrv) {
var vm = this;
vm.notes = notebookListDataFactory;
vm.websocketMsgSrv = websocketMsgSrv;

4
zeppelin-web/src/app/notebook/notebook.html
View file

@ -66,7 +66,7 @@ limitations under the License.
ng-Init="init(currentParagraph)"
ng-class="columnWidthClass(currentParagraph.config.colWidth)"
class="paragraph-col">
<div class="new-paragraph" ng-click="insertNew('above')" ng-hide="viewOnly">
<div class="new-paragraph" ng-click="insertNew('above')" ng-hide="viewOnly || asIframe">
<h4 class="plus-sign">&#43;</h4>
</div>
<div id="{{currentParagraph.id}}_paragraphColumn"
@ -75,7 +75,7 @@ limitations under the License.
'lastEmptyParagraph': !paragraph.text && !paragraph.result}"
ng-hide="currentParagraph.config.tableHide && viewOnly">
</div>
<div class="new-paragraph" ng-click="insertNew('below');" ng-hide="!$last || viewOnly">
<div class="new-paragraph" ng-click="insertNew('below');" ng-hide="!$last || viewOnly || asIframe ">
<h4 class="plus-sign">&#43;</h4>
</div>
</div>

2
zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js
View file

@ -960,7 +960,7 @@ angular.module('zeppelinWebApp')
clearUnknownColsFromGraphOption();
// set graph height
var height = $scope.paragraph.config.graph.height;
angular.element('#p' + $scope.paragraph.id + '_resize').height(height);
angular.element('#p' + $scope.paragraph.id + '_graph').height(height);
if (!type || type === 'table') {
setTable($scope.paragraph.result, refresh);

3
zeppelin-web/src/components/navbar/navbar.controller.js
View file

@ -23,6 +23,7 @@ angular.module('zeppelinWebApp').controller('NavCtrl', function($scope, $rootSco
vm.connected = websocketMsgSrv.isConnected();
vm.websocketMsgSrv = websocketMsgSrv;
vm.arrayOrderingSrv = arrayOrderingSrv;
vm.authenticated = $rootScope.ticket.principal !== 'anonymous';
angular.element('#notebook-list').perfectScrollbar({suppressScrollX: true});
@ -51,6 +52,8 @@ angular.module('zeppelinWebApp').controller('NavCtrl', function($scope, $rootSco
websocketMsgSrv.getNotebookList();
}
vm.authenticated = $rootScope.ticket.principal !== 'anonymous';
function isActive(noteId) {
return ($routeParams.noteId === noteId);
}

4
zeppelin-web/src/components/navbar/navbar.html
View file

@ -73,8 +73,8 @@ limitations under the License.
</li>
<li class="server-status">
<i class="fa fa-circle" ng-class="{'server-connected':navbar.connected, 'server-disconnected':!navbar.connected}"></i>
<span ng-show="navbar.connected">Connected</span>
<span ng-show="!navbar.connected">Disconnected</span>
<span ng-show="navbar.authenticated">{{ticket.principal}} connected</span>
<span ng-show="!navbar.authenticated">Disconnected</span>
</li>
</ul>
</div>

4
zeppelin-web/src/components/websocketEvents/websocketEvents.factory.js
View file

@ -28,7 +28,9 @@ angular.module('zeppelinWebApp').factory('websocketEvents', function($rootScope,
});
websocketCalls.sendNewEvent = function(data) {
console.log('Send >> %o, %o', data.op, data);
data.principal = $rootScope.ticket.principal;
data.ticket = $rootScope.ticket.ticket;
console.log('Send >> %o, %o, %o, %o', data.op, data.principal, data.ticket, data);
websocketCalls.ws.send(JSON.stringify(data));
};

2
zeppelin-web/src/index.html
View file

@ -12,7 +12,7 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html ng-app="zeppelinWebApp" ng-controller="MainCtrl" class="no-js">
<html ng-controller="MainCtrl" class="no-js">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta charset="utf-8">

4
zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java
View file

@ -430,7 +430,8 @@ public class ZeppelinConfiguration extends XMLConfiguration {
ZEPPELIN_CONF_DIR("zeppelin.conf.dir", "conf"),
// Allows a way to specify a ',' separated list of allowed origins for rest and websockets
// i.e. http://localhost:8080
ZEPPELIN_ALLOWED_ORIGINS("zeppelin.server.allowed.origins", "*");
ZEPPELIN_ALLOWED_ORIGINS("zeppelin.server.allowed.origins", "*"),
ZEPPELIN_ANONYMOUS_ALLOWED("zeppelin.anonymous.allowed", true);
private String varName;
@SuppressWarnings("rawtypes")
@ -565,6 +566,7 @@ public class ZeppelinConfiguration extends XMLConfiguration {
try {
checkType(value);
} catch (Exception e) {
LOG.error("Exception in ZeppelinConfiguration while isType", e);
return false;
}
return true;

34
zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterFactory.java
View file

@ -17,29 +17,8 @@
package org.apache.zeppelin.interpreter;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Collection;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.Set;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.apache.commons.lang.ArrayUtils;
import org.apache.commons.lang.NullArgumentException;
import org.apache.zeppelin.conf.ZeppelinConfiguration;
@ -54,8 +33,13 @@ import org.apache.zeppelin.scheduler.Job.Status;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import java.io.*;
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.*;
/**
* Manage interpreters.

8
zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java
View file

@ -62,7 +62,7 @@ import com.google.gson.stream.JsonReader;
* Collection of Notes.
*/
public class Notebook {
Logger logger = LoggerFactory.getLogger(Notebook.class);
static Logger logger = LoggerFactory.getLogger(Notebook.class);
@SuppressWarnings("unused") @Deprecated //TODO(bzz): remove unused
private SchedulerFactory schedulerFactory;
@ -292,7 +292,7 @@ public class Notebook {
try {
note.unpersist();
} catch (IOException e) {
e.printStackTrace();
logger.error(e.toString(), e);
}
}
@ -475,7 +475,7 @@ public class Notebook {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
logger.error(e.toString(), e);
}
}
@ -483,7 +483,7 @@ public class Notebook {
try {
releaseResource = (boolean) note.getConfig().get("releaseresource");
} catch (java.lang.ClassCastException e) {
e.printStackTrace();
logger.error(e.toString(), e);
}
if (releaseResource) {
for (InterpreterSetting setting : note.getNoteReplLoader().getInterpreterSettings()) {

2
zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/NotebookTest.java
View file

@ -130,7 +130,7 @@ public class NotebookTest implements JobListenerFactory{
try {
FileUtils.copyDirectory(srcDir, destDir);
} catch (IOException e) {
e.printStackTrace();
logger.error(e.toString(), e);
}
// doesn't have copied notebook in memory before reloading

2
zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/NotebookRepoSyncTest.java
View file

@ -199,7 +199,7 @@ public class NotebookRepoSyncTest implements JobListenerFactory {
try {
FileUtils.copyDirectory(srcDir, destDir);
} catch (IOException e) {
e.printStackTrace();
LOG.error(e.toString(), e);
}
assertEquals(0, notebookRepoSync.list(0).size());
assertEquals(1, notebookRepoSync.list(1).size());

2
zeppelin-zengine/src/test/java/org/apache/zeppelin/notebook/repo/VFSNotebookRepoTest.java
View file

@ -134,7 +134,7 @@ public class VFSNotebookRepoTest implements JobListenerFactory {
try {
notebookRepo.save(note);
} catch (IOException e) {
e.printStackTrace();
LOG.error(e.toString(), e);
}
}
}

145
zeppelin-zengine/src/test/java/org/apache/zeppelin/util/UtilsForTests.java
View file

@ -17,6 +17,9 @@
package org.apache.zeppelin.util;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
@ -27,13 +30,15 @@ import java.util.Map;
public class UtilsForTests {
public static File createTmpDir() throws Exception {
File tmpDir = new File(System.getProperty("java.io.tmpdir")+"/ZeppelinLTest_"+System.currentTimeMillis());
tmpDir.mkdir();
return tmpDir;
static Logger LOGGER = LoggerFactory.getLogger(UtilsForTests.class);
}
/*
public static File createTmpDir() throws Exception {
File tmpDir = new File(System.getProperty("java.io.tmpdir") + "/ZeppelinLTest_" + System.currentTimeMillis());
tmpDir.mkdir();
return tmpDir;
}
/*
private static final String HADOOP_DIST="http://apache.mirror.cdnetworks.com/hadoop/common/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz";
//private static final String HADOOP_DIST="http://www.us.apache.org/dist/hadoop/common/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz";
@ -48,72 +53,72 @@ public class UtilsForTests {
}
*/
public static void delete(File file){
if(file.isFile()) file.delete();
else if(file.isDirectory()){
File [] files = file.listFiles();
if(files!=null && files.length>0){
for(File f : files){
delete(f);
}
}
file.delete();
}
}
/**
* Utility method to create a file (if does not exist) and populate it the the given content
*
* @param path to file
* @param content of the file
* @throws IOException
*/
public static void createFileWithContent(String path, String content) throws IOException {
File f = new File(path);
if (!f.exists()) {
stringToFile(content, f);
public static void delete(File file) {
if (file.isFile()) file.delete();
else if (file.isDirectory()) {
File[] files = file.listFiles();
if (files != null && files.length > 0) {
for (File f : files) {
delete(f);
}
}
file.delete();
}
}
public static void stringToFile(String string, File file) throws IOException{
FileOutputStream out = new FileOutputStream(file);
out.write(string.getBytes());
out.close();
}
/**
* Utility method to create a file (if does not exist) and populate it the the given content
*
* @param path to file
* @param content of the file
* @throws IOException
*/
public static void createFileWithContent(String path, String content) throws IOException {
File f = new File(path);
if (!f.exists()) {
stringToFile(content, f);
}
}
@SuppressWarnings({ "unchecked", "rawtypes" })
public static void setEnv(String k, String v) {
Map<String, String> newenv = new HashMap<String, String>();
newenv.put(k, v);
try {
Class<?> processEnvironmentClass = Class.forName("java.lang.ProcessEnvironment");
Field theEnvironmentField = processEnvironmentClass.getDeclaredField("theEnvironment");
theEnvironmentField.setAccessible(true);
Map<String, String> env = (Map<String, String>) theEnvironmentField.get(null);
env.putAll(newenv);
Field theCaseInsensitiveEnvironmentField = processEnvironmentClass.getDeclaredField("theCaseInsensitiveEnvironment");
theCaseInsensitiveEnvironmentField.setAccessible(true);
Map<String, String> cienv = (Map<String, String>) theCaseInsensitiveEnvironmentField.get(null);
cienv.putAll(newenv);
} catch (NoSuchFieldException e) {
try {
Class[] classes = Collections.class.getDeclaredClasses();
Map<String, String> env = System.getenv();
for(Class cl : classes) {
if("java.util.Collections$UnmodifiableMap".equals(cl.getName())) {
Field field = cl.getDeclaredField("m");
field.setAccessible(true);
Object obj = field.get(env);
Map<String, String> map = (Map<String, String>) obj;
map.clear();
map.putAll(newenv);
}
}
} catch (Exception e2) {
e2.printStackTrace();
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
public static void stringToFile(String string, File file) throws IOException {
FileOutputStream out = new FileOutputStream(file);
out.write(string.getBytes());
out.close();
}
@SuppressWarnings({"unchecked", "rawtypes"})
public static void setEnv(String k, String v) {
Map<String, String> newenv = new HashMap<String, String>();
newenv.put(k, v);
try {
Class<?> processEnvironmentClass = Class.forName("java.lang.ProcessEnvironment");
Field theEnvironmentField = processEnvironmentClass.getDeclaredField("theEnvironment");
theEnvironmentField.setAccessible(true);
Map<String, String> env = (Map<String, String>) theEnvironmentField.get(null);
env.putAll(newenv);
Field theCaseInsensitiveEnvironmentField = processEnvironmentClass.getDeclaredField("theCaseInsensitiveEnvironment");
theCaseInsensitiveEnvironmentField.setAccessible(true);
Map<String, String> cienv = (Map<String, String>) theCaseInsensitiveEnvironmentField.get(null);
cienv.putAll(newenv);
} catch (NoSuchFieldException e) {
try {
Class[] classes = Collections.class.getDeclaredClasses();
Map<String, String> env = System.getenv();
for (Class cl : classes) {
if ("java.util.Collections$UnmodifiableMap".equals(cl.getName())) {
Field field = cl.getDeclaredField("m");
field.setAccessible(true);
Object obj = field.get(env);
Map<String, String> map = (Map<String, String>) obj;
map.clear();
map.putAll(newenv);
}
}
} catch (Exception e2) {
LOGGER.error(e2.toString(), e2);
}
} catch (Exception e1) {
LOGGER.error(e1.toString(), e1);
}
}
}