mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
Merge remote-tracking branch 'upstream/master' into ZEPPELIN-2403
# Conflicts: # pig/src/main/resources/interpreter-setting.json
This commit is contained in:
isys.mreshetov
commit
726c1f31fe
26 changed files with 315 additions and 367 deletions
7
LICENSE
7
LICENSE
|
|
@ -276,3 +276,10 @@ The following components are provided under the BSD 2-Clause license. See file
|
|||
|
||||
(BSD 2 Clause) portions of SQLLine (http://sqlline.sourceforge.net/) - http://sqlline.sourceforge.net/#license
|
||||
jdbc/src/main/java/org/apache/zeppelin/jdbc/SqlCompleter.java
|
||||
|
||||
========================================================================
|
||||
Jython Software License
|
||||
========================================================================
|
||||
The following components are provided under the Jython Software License. See file headers and project links for details.
|
||||
|
||||
(Jython Software License) jython-standalone - http://www.jython.org/
|
||||
|
|
@ -29,31 +29,39 @@ which in turns enables them to handle very large data sets.
|
|||
- No pig alias in the last statement in `%pig.query` (read the examples below).
|
||||
- The last statement must be in single line in `%pig.query`
|
||||
|
||||
## Supported runtime mode
|
||||
- Local
|
||||
- MapReduce
|
||||
- Tez_Local (Only Tez 0.7 is supported)
|
||||
- Tez (Only Tez 0.7 is supported)
|
||||
|
||||
## How to use
|
||||
|
||||
### How to setup Pig
|
||||
### How to setup Pig execution modes.
|
||||
|
||||
- Local Mode
|
||||
|
||||
Nothing needs to be done for local mode
|
||||
Set `zeppelin.pig.execType` as `local`.
|
||||
|
||||
- MapReduce Mode
|
||||
|
||||
HADOOP\_CONF\_DIR needs to be specified in `ZEPPELIN_HOME/conf/zeppelin-env.sh`.
|
||||
Set `zeppelin.pig.execType` as `mapreduce`. HADOOP\_CONF\_DIR needs to be specified in `ZEPPELIN_HOME/conf/zeppelin-env.sh`.
|
||||
|
||||
- Tez Local Mode
|
||||
|
||||
Nothing needs to be done for tez local mode
|
||||
Only Tez 0.7 is supported. Set `zeppelin.pig.execType` as `tez_local`.
|
||||
|
||||
- Tez Mode
|
||||
|
||||
HADOOP\_CONF\_DIR and TEZ\_CONF\_DIR needs to be specified in `ZEPPELIN_HOME/conf/zeppelin-env.sh`.
|
||||
Only Tez 0.7 is supported. Set `zeppelin.pig.execType` as `tez`. HADOOP\_CONF\_DIR and TEZ\_CONF\_DIR needs to be specified in `ZEPPELIN_HOME/conf/zeppelin-env.sh`.
|
||||
|
||||
- Spark Local Mode
|
||||
|
||||
Only Spark 1.6.x is supported, by default it is Spark 1.6.3. Set `zeppelin.pig.execType` as `spark_local`.
|
||||
|
||||
- Spark Mode
|
||||
|
||||
Only Spark 1.6.x is supported, by default it is Spark 1.6.3. Set `zeppelin.pig.execType` as `spark`. For now, only yarn-client mode is supported. To enable it, you need to set property `SPARK_MASTER` to yarn-client and set `SPARK_JAR` to the spark assembly jar.
|
||||
|
||||
### How to choose custom Spark Version
|
||||
|
||||
By default, Pig Interpreter would use Spark 1.6.3 built with scala 2.10, if you want to use another spark version or scala version,
|
||||
you need to rebuild Zeppelin by specifying the custom Spark version via -Dpig.spark.version=<custom_spark_version> and scala version via -Dpig.scala.version=<scala_version> in the maven build command.
|
||||
|
||||
### How to configure interpreter
|
||||
|
||||
|
|
@ -71,7 +79,7 @@ So you can use that to find app running in YARN RM UI.
|
|||
<tr>
|
||||
<td>zeppelin.pig.execType</td>
|
||||
<td>mapreduce</td>
|
||||
<td>Execution mode for pig runtime. local | mapreduce | tez_local | tez </td>
|
||||
<td>Execution mode for pig runtime. local | mapreduce | tez_local | tez | spark_local | spark </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>zeppelin.pig.includeJobStats</td>
|
||||
|
|
@ -93,6 +101,17 @@ So you can use that to find app running in YARN RM UI.
|
|||
<td>default</td>
|
||||
<td>queue name for mapreduce engine</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SPARK_MASTER</td>
|
||||
<td>local</td>
|
||||
<td>local | yarn-client</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SPARK_JAR</td>
|
||||
<td></td>
|
||||
<td>The spark assembly jar, both jar in local or hdfs is supported. Put it on hdfs could have
|
||||
performance benefit</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### Example
|
||||
|
|
|
|||
|
|
@ -128,7 +128,7 @@ Change the following values in the Shiro.ini file, and uncomment the line:
|
|||
|
||||
### LDAP
|
||||
|
||||
Two options exist for configuring an LDAP Realm. The simpler to use is the LdapGroupRealm. How ever it has limited
|
||||
Two options exist for configuring an LDAP Realm. The simpler to use is the LdapGroupRealm. How ever it has limited
|
||||
flexibility with mapping of ldap groups to users and for authorization for user groups. A sample configuration file for
|
||||
this realm is given below.
|
||||
|
||||
|
|
@ -144,7 +144,7 @@ ldapRealm.contextFactory.authenticationMechanism = simple
|
|||
The other more flexible option is to use the LdapRealm. It allows for mapping of ldapgroups to roles and also allows for
|
||||
role/group based authentication into the zeppelin server. Sample configuration for this realm is given below.
|
||||
```
|
||||
[main]
|
||||
[main]
|
||||
ldapRealm=org.apache.zeppelin.realm.LdapRealm
|
||||
|
||||
ldapRealm.contextFactory.authenticationMechanism=simple
|
||||
|
|
@ -162,18 +162,18 @@ ldapRealm.groupObjectClass=groupofnames
|
|||
ldapRealm.userSearchAttributeName = sAMAccountName
|
||||
ldapRealm.memberAttribute=member
|
||||
# force usernames returned from ldap to lowercase useful for AD
|
||||
ldapRealm.userLowerCase = true
|
||||
ldapRealm.userLowerCase = true
|
||||
# ability set searchScopes subtree (default), one, base
|
||||
ldapRealm.userSearchScope = subtree;
|
||||
ldapRealm.groupSearchScope = subtree;
|
||||
ldapRealm.memberAttributeValueTemplate=cn={0},ou=people,dc=hadoop,dc=apache,dc=org
|
||||
ldapRealm.contextFactory.systemUsername=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org
|
||||
ldapRealm.contextFactory.systemUsername=uid=guest,ou=people,dc=hadoop,dc=apache,dc=org
|
||||
ldapRealm.contextFactory.systemPassword=S{ALIAS=ldcSystemPassword}
|
||||
# enable support for nested groups using the LDAP_MATCHING_RULE_IN_CHAIN operator
|
||||
ldapRealm.groupSearchEnableMatchingRuleInChain = true
|
||||
# optional mapping from physical groups to logical application roles
|
||||
ldapRealm.rolesByGroup = LDN_USERS: user_role, NYK_USERS: user_role, HKG_USERS: user_role, GLOBAL_ADMIN: admin_role
|
||||
# optional list of roles that are allowed to authenticate. Incase not present all groups are allowed to authenticate (login).
|
||||
# optional list of roles that are allowed to authenticate. Incase not present all groups are allowed to authenticate (login).
|
||||
# This changes nothing for url specific permissions that will continue to work as specified in [urls].
|
||||
ldapRealm.allowedRolesForAuthentication = admin_role,user_role
|
||||
ldapRealm.permissionsByRole= user_role = *:ToDoItemsJdo:*:*, *:ToDoItem:*:*; admin_role = *
|
||||
|
|
@ -182,12 +182,12 @@ securityManager.realms = $ldapRealm
|
|||
```
|
||||
|
||||
### PAM
|
||||
[PAM](https://en.wikipedia.org/wiki/Pluggable_authentication_module) authentication support allows the reuse of existing authentication
|
||||
[PAM](https://en.wikipedia.org/wiki/Pluggable_authentication_module) authentication support allows the reuse of existing authentication
|
||||
moduls on the host where Zeppelin is running. On a typical system modules are configured per service for example sshd, passwd, etc. under `/etc/pam.d/`. You can
|
||||
either reuse one of these services or create your own for Zeppelin. Activiting PAM authentication requires two parameters:
|
||||
1. realm: The Shiro realm being used
|
||||
2. service: The service configured under `/etc/pam.d/` to be used. The name here needs to be the same as the file name under `/etc/pam.d/`
|
||||
|
||||
|
||||
```
|
||||
[main]
|
||||
pamRealm=org.apache.zeppelin.realm.PamRealm
|
||||
|
|
@ -232,4 +232,3 @@ If you want to grant this permission to other users, you can change **roles[ ]**
|
|||
## Other authentication methods
|
||||
|
||||
- [HTTP Basic Authentication using NGINX](./authentication_nginx.html)
|
||||
|
||||
|
|
|
|||
47
pig/pom.xml
47
pig/pom.xml
|
|
@ -36,9 +36,11 @@
|
|||
<url>http://zeppelin.apache.org</url>
|
||||
|
||||
<properties>
|
||||
<pig.version>0.16.0</pig.version>
|
||||
<pig.version>0.17.0</pig.version>
|
||||
<hadoop.version>2.6.0</hadoop.version>
|
||||
<tez.version>0.7.0</tez.version>
|
||||
<pig.spark.version>1.6.3</pig.spark.version>
|
||||
<pig.scala.version>2.10</pig.scala.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
|
|
@ -68,10 +70,28 @@
|
|||
<dependency>
|
||||
<groupId>org.apache.pig</groupId>
|
||||
<artifactId>pig</artifactId>
|
||||
<classifier>h2</classifier>
|
||||
<version>${pig.version}</version>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<groupId>javax.servlet</groupId>
|
||||
<artifactId>servlet-api</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>org.mortbay.jetty</groupId>
|
||||
<artifactId>servlet-api</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>org.mortbay.jetty</groupId>
|
||||
<artifactId>servlet-api-2.5</artifactId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.python</groupId>
|
||||
<artifactId>jython-standalone</artifactId>
|
||||
<version>2.7.0</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.hadoop</groupId>
|
||||
<artifactId>hadoop-client</artifactId>
|
||||
|
|
@ -82,6 +102,12 @@
|
|||
<groupId>org.apache.tez</groupId>
|
||||
<artifactId>tez-api</artifactId>
|
||||
<version>${tez.version}</version>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<groupId>javax.servlet</groupId>
|
||||
<artifactId>servlet-api</artifactId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
|
|
@ -94,6 +120,12 @@
|
|||
<groupId>org.apache.tez</groupId>
|
||||
<artifactId>tez-dag</artifactId>
|
||||
<version>${tez.version}</version>
|
||||
<exclusions>
|
||||
<exclusion>
|
||||
<groupId>javax.servlet</groupId>
|
||||
<artifactId>servlet-api</artifactId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
|
|
@ -120,6 +152,17 @@
|
|||
<version>${tez.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.apache.spark</groupId>
|
||||
<artifactId>spark-core_${pig.scala.version}</artifactId>
|
||||
<version>${pig.spark.version}</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.spark</groupId>
|
||||
<artifactId>spark-yarn_${pig.scala.version}</artifactId>
|
||||
<version>${pig.spark.version}</version>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>junit</groupId>
|
||||
<artifactId>junit</artifactId>
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ import org.apache.commons.lang.StringUtils;
|
|||
import org.apache.commons.lang.exception.ExceptionUtils;
|
||||
import org.apache.pig.PigServer;
|
||||
import org.apache.pig.impl.logicalLayer.FrontendException;
|
||||
import org.apache.pig.tools.pigscript.parser.ParseException;
|
||||
import org.apache.pig.tools.pigstats.*;
|
||||
import org.apache.zeppelin.interpreter.InterpreterContext;
|
||||
import org.apache.zeppelin.interpreter.InterpreterResult;
|
||||
|
|
@ -98,6 +99,10 @@ public class PigInterpreter extends BasePigInterpreter {
|
|||
listenerMap.put(contextInterpreter.getParagraphId(), scriptListener);
|
||||
pigServer.registerScript(tmpFile.getAbsolutePath());
|
||||
} catch (IOException e) {
|
||||
// 1. catch FrontendException, FrontendException happens in the query compilation phase.
|
||||
// 2. catch ParseException for syntax error
|
||||
// 3. PigStats, This is execution error
|
||||
// 4. Other errors.
|
||||
if (e instanceof FrontendException) {
|
||||
FrontendException fe = (FrontendException) e;
|
||||
if (!fe.getMessage().contains("Backend error :")) {
|
||||
|
|
@ -107,9 +112,12 @@ public class PigInterpreter extends BasePigInterpreter {
|
|||
return new InterpreterResult(Code.ERROR, ExceptionUtils.getStackTrace(e));
|
||||
}
|
||||
}
|
||||
if (e.getCause() instanceof ParseException) {
|
||||
return new InterpreterResult(Code.ERROR, e.getCause().getMessage());
|
||||
}
|
||||
PigStats stats = PigStats.get();
|
||||
if (stats != null) {
|
||||
String errorMsg = PigUtils.extactJobStats(stats);
|
||||
String errorMsg = stats.getDisplayString();
|
||||
if (errorMsg != null) {
|
||||
LOGGER.error("Fail to run pig script, " + errorMsg);
|
||||
return new InterpreterResult(Code.ERROR, errorMsg);
|
||||
|
|
@ -127,7 +135,7 @@ public class PigInterpreter extends BasePigInterpreter {
|
|||
StringBuilder outputBuilder = new StringBuilder();
|
||||
PigStats stats = PigStats.get();
|
||||
if (stats != null && includeJobStats) {
|
||||
String jobStats = PigUtils.extactJobStats(stats);
|
||||
String jobStats = stats.getDisplayString();
|
||||
if (jobStats != null) {
|
||||
outputBuilder.append(jobStats);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ import org.apache.pig.PigServer;
|
|||
import org.apache.pig.data.Tuple;
|
||||
import org.apache.pig.impl.logicalLayer.FrontendException;
|
||||
import org.apache.pig.impl.logicalLayer.schema.Schema;
|
||||
import org.apache.pig.tools.pigscript.parser.ParseException;
|
||||
import org.apache.pig.tools.pigstats.PigStats;
|
||||
import org.apache.pig.tools.pigstats.ScriptState;
|
||||
import org.apache.zeppelin.interpreter.*;
|
||||
|
|
@ -125,8 +126,9 @@ public class PigQueryInterpreter extends BasePigInterpreter {
|
|||
} catch (IOException e) {
|
||||
// Extract error in the following order
|
||||
// 1. catch FrontendException, FrontendException happens in the query compilation phase.
|
||||
// 2. PigStats, This is execution error
|
||||
// 3. Other errors.
|
||||
// 2. catch ParseException for syntax error
|
||||
// 3. PigStats, This is execution error
|
||||
// 4. Other errors.
|
||||
if (e instanceof FrontendException) {
|
||||
FrontendException fe = (FrontendException) e;
|
||||
if (!fe.getMessage().contains("Backend error :")) {
|
||||
|
|
@ -134,9 +136,12 @@ public class PigQueryInterpreter extends BasePigInterpreter {
|
|||
return new InterpreterResult(Code.ERROR, ExceptionUtils.getStackTrace(e));
|
||||
}
|
||||
}
|
||||
if (e.getCause() instanceof ParseException) {
|
||||
return new InterpreterResult(Code.ERROR, e.getMessage());
|
||||
}
|
||||
PigStats stats = PigStats.get();
|
||||
if (stats != null) {
|
||||
String errorMsg = PigUtils.extactJobStats(stats);
|
||||
String errorMsg = stats.getDisplayString();
|
||||
if (errorMsg != null) {
|
||||
return new InterpreterResult(Code.ERROR, errorMsg);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ import org.apache.commons.io.IOUtils;
|
|||
import org.apache.commons.lang.StringUtils;
|
||||
import org.apache.commons.lang.exception.ExceptionUtils;
|
||||
import org.apache.pig.PigRunner;
|
||||
import org.apache.pig.backend.hadoop.executionengine.spark.plan.SparkOperator;
|
||||
import org.apache.pig.backend.hadoop.executionengine.tez.TezExecType;
|
||||
import org.apache.pig.tools.pigstats.InputStats;
|
||||
import org.apache.pig.tools.pigstats.JobStats;
|
||||
|
|
@ -29,6 +30,9 @@ import org.apache.pig.tools.pigstats.OutputStats;
|
|||
import org.apache.pig.tools.pigstats.PigStats;
|
||||
import org.apache.pig.tools.pigstats.mapreduce.MRJobStats;
|
||||
import org.apache.pig.tools.pigstats.mapreduce.SimplePigStats;
|
||||
import org.apache.pig.tools.pigstats.spark.SparkJobStats;
|
||||
import org.apache.pig.tools.pigstats.spark.SparkPigStats;
|
||||
import org.apache.pig.tools.pigstats.spark.SparkScriptState;
|
||||
import org.apache.pig.tools.pigstats.tez.TezDAGStats;
|
||||
import org.apache.pig.tools.pigstats.tez.TezPigScriptStats;
|
||||
import org.slf4j.Logger;
|
||||
|
|
@ -39,10 +43,7 @@ import java.io.FileWriter;
|
|||
import java.io.IOException;
|
||||
import java.lang.reflect.Field;
|
||||
import java.text.SimpleDateFormat;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Date;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.*;
|
||||
|
||||
/**
|
||||
*
|
||||
|
|
@ -66,236 +67,4 @@ public class PigUtils {
|
|||
return createTempPigScript(StringUtils.join(lines, "\n"));
|
||||
}
|
||||
|
||||
public static String extactJobStats(PigStats stats) {
|
||||
if (stats instanceof SimplePigStats) {
|
||||
return extractFromSimplePigStats((SimplePigStats) stats);
|
||||
} else if (stats instanceof TezPigScriptStats) {
|
||||
return extractFromTezPigStats((TezPigScriptStats) stats);
|
||||
} else {
|
||||
throw new RuntimeException("Unrecognized stats type:" + stats.getClass().getSimpleName());
|
||||
}
|
||||
}
|
||||
|
||||
public static String extractFromSimplePigStats(SimplePigStats stats) {
|
||||
|
||||
try {
|
||||
Field userIdField = PigStats.class.getDeclaredField("userId");
|
||||
userIdField.setAccessible(true);
|
||||
String userId = (String) (userIdField.get(stats));
|
||||
Field startTimeField = PigStats.class.getDeclaredField("startTime");
|
||||
startTimeField.setAccessible(true);
|
||||
long startTime = (Long) (startTimeField.get(stats));
|
||||
Field endTimeField = PigStats.class.getDeclaredField("endTime");
|
||||
endTimeField.setAccessible(true);
|
||||
long endTime = (Long) (endTimeField.get(stats));
|
||||
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.UNKNOWN) {
|
||||
LOGGER.warn("unknown return code, can't display the results");
|
||||
return null;
|
||||
}
|
||||
if (stats.getPigContext() == null) {
|
||||
LOGGER.warn("unknown exec type, don't display the results");
|
||||
return null;
|
||||
}
|
||||
|
||||
SimpleDateFormat sdf = new SimpleDateFormat(DATE_FORMAT);
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append("\nHadoopVersion\tPigVersion\tUserId\tStartedAt\tFinishedAt\tFeatures\n");
|
||||
sb.append(stats.getHadoopVersion()).append("\t").append(stats.getPigVersion()).append("\t")
|
||||
.append(userId).append("\t")
|
||||
.append(sdf.format(new Date(startTime))).append("\t")
|
||||
.append(sdf.format(new Date(endTime))).append("\t")
|
||||
.append(stats.getFeatures()).append("\n");
|
||||
sb.append("\n");
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.SUCCESS) {
|
||||
sb.append("Success!\n");
|
||||
} else if (stats.getReturnCode() == PigRunner.ReturnCode.PARTIAL_FAILURE) {
|
||||
sb.append("Some jobs have failed! Stop running all dependent jobs\n");
|
||||
} else {
|
||||
sb.append("Failed!\n");
|
||||
}
|
||||
sb.append("\n");
|
||||
|
||||
Field jobPlanField = PigStats.class.getDeclaredField("jobPlan");
|
||||
jobPlanField.setAccessible(true);
|
||||
PigStats.JobGraph jobPlan = (PigStats.JobGraph) jobPlanField.get(stats);
|
||||
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.SUCCESS
|
||||
|| stats.getReturnCode() == PigRunner.ReturnCode.PARTIAL_FAILURE) {
|
||||
sb.append("Job Stats (time in seconds):\n");
|
||||
sb.append(MRJobStats.SUCCESS_HEADER).append("\n");
|
||||
List<JobStats> arr = jobPlan.getSuccessfulJobs();
|
||||
for (JobStats js : arr) {
|
||||
sb.append(js.getDisplayString());
|
||||
}
|
||||
sb.append("\n");
|
||||
}
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.FAILURE
|
||||
|| stats.getReturnCode() == PigRunner.ReturnCode.PARTIAL_FAILURE) {
|
||||
sb.append("Failed Jobs:\n");
|
||||
sb.append(MRJobStats.FAILURE_HEADER).append("\n");
|
||||
List<JobStats> arr = jobPlan.getFailedJobs();
|
||||
for (JobStats js : arr) {
|
||||
sb.append(js.getDisplayString());
|
||||
}
|
||||
sb.append("\n");
|
||||
}
|
||||
sb.append("Input(s):\n");
|
||||
for (InputStats is : stats.getInputStats()) {
|
||||
sb.append(is.getDisplayString());
|
||||
}
|
||||
sb.append("\n");
|
||||
sb.append("Output(s):\n");
|
||||
for (OutputStats ds : stats.getOutputStats()) {
|
||||
sb.append(ds.getDisplayString());
|
||||
}
|
||||
|
||||
sb.append("\nCounters:\n");
|
||||
sb.append("Total records written : " + stats.getRecordWritten()).append("\n");
|
||||
sb.append("Total bytes written : " + stats.getBytesWritten()).append("\n");
|
||||
sb.append("Spillable Memory Manager spill count : "
|
||||
+ stats.getSMMSpillCount()).append("\n");
|
||||
sb.append("Total bags proactively spilled: "
|
||||
+ stats.getProactiveSpillCountObjects()).append("\n");
|
||||
sb.append("Total records proactively spilled: "
|
||||
+ stats.getProactiveSpillCountRecords()).append("\n");
|
||||
sb.append("\nJob DAG:\n").append(jobPlan.toString());
|
||||
|
||||
return "Script Statistics: \n" + sb.toString();
|
||||
} catch (Exception e) {
|
||||
LOGGER.error("Can not extract message from SimplePigStats", e);
|
||||
return "Can not extract message from SimpelPigStats," + ExceptionUtils.getStackTrace(e);
|
||||
}
|
||||
}
|
||||
|
||||
private static String extractFromTezPigStats(TezPigScriptStats stats) {
|
||||
|
||||
try {
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.UNKNOWN) {
|
||||
LOGGER.warn("unknown return code, can't display the results");
|
||||
return null;
|
||||
}
|
||||
if (stats.getPigContext() == null) {
|
||||
LOGGER.warn("unknown exec type, don't display the results");
|
||||
return null;
|
||||
}
|
||||
|
||||
Field userIdField = PigStats.class.getDeclaredField("userId");
|
||||
userIdField.setAccessible(true);
|
||||
String userId = (String) (userIdField.get(stats));
|
||||
Field startTimeField = PigStats.class.getDeclaredField("startTime");
|
||||
startTimeField.setAccessible(true);
|
||||
long startTime = (Long) (startTimeField.get(stats));
|
||||
Field endTimeField = PigStats.class.getDeclaredField("endTime");
|
||||
endTimeField.setAccessible(true);
|
||||
long endTime = (Long) (endTimeField.get(stats));
|
||||
|
||||
SimpleDateFormat sdf = new SimpleDateFormat(DATE_FORMAT);
|
||||
StringBuilder sb = new StringBuilder();
|
||||
sb.append("\n");
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "HadoopVersion", stats.getHadoopVersion()));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "PigVersion", stats.getPigVersion()));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "TezVersion", TezExecType.getTezVersion()));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "UserId", userId));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "FileName", stats.getFileName()));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "StartedAt", sdf.format(new Date(startTime))));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "FinishedAt", sdf.format(new Date(endTime))));
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", "Features", stats.getFeatures()));
|
||||
sb.append("\n");
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.SUCCESS) {
|
||||
sb.append("Success!\n");
|
||||
} else if (stats.getReturnCode() == PigRunner.ReturnCode.PARTIAL_FAILURE) {
|
||||
sb.append("Some tasks have failed! Stop running all dependent tasks\n");
|
||||
} else {
|
||||
sb.append("Failed!\n");
|
||||
}
|
||||
sb.append("\n");
|
||||
|
||||
// Print diagnostic info in case of failure
|
||||
if (stats.getReturnCode() == PigRunner.ReturnCode.FAILURE
|
||||
|| stats.getReturnCode() == PigRunner.ReturnCode.PARTIAL_FAILURE) {
|
||||
if (stats.getErrorMessage() != null) {
|
||||
String[] lines = stats.getErrorMessage().split("\n");
|
||||
for (int i = 0; i < lines.length; i++) {
|
||||
String s = lines[i].trim();
|
||||
if (i == 0 || !org.apache.commons.lang.StringUtils.isEmpty(s)) {
|
||||
sb.append(String.format("%1$20s: %2$-100s%n", i == 0 ? "ErrorMessage" : "", s));
|
||||
}
|
||||
}
|
||||
sb.append("\n");
|
||||
}
|
||||
}
|
||||
|
||||
Field tezDAGStatsMapField = TezPigScriptStats.class.getDeclaredField("tezDAGStatsMap");
|
||||
tezDAGStatsMapField.setAccessible(true);
|
||||
Map<String, TezDAGStats> tezDAGStatsMap =
|
||||
(Map<String, TezDAGStats>) tezDAGStatsMapField.get(stats);
|
||||
int count = 0;
|
||||
for (TezDAGStats dagStats : tezDAGStatsMap.values()) {
|
||||
sb.append("\n");
|
||||
sb.append("DAG " + count++ + ":\n");
|
||||
sb.append(dagStats.getDisplayString());
|
||||
sb.append("\n");
|
||||
}
|
||||
|
||||
sb.append("Input(s):\n");
|
||||
for (InputStats is : stats.getInputStats()) {
|
||||
sb.append(is.getDisplayString().trim()).append("\n");
|
||||
}
|
||||
sb.append("\n");
|
||||
sb.append("Output(s):\n");
|
||||
for (OutputStats os : stats.getOutputStats()) {
|
||||
sb.append(os.getDisplayString().trim()).append("\n");
|
||||
}
|
||||
return "Script Statistics:\n" + sb.toString();
|
||||
} catch (Exception e) {
|
||||
LOGGER.error("Can not extract message from SimplePigStats", e);
|
||||
return "Can not extract message from SimpelPigStats," + ExceptionUtils.getStackTrace(e);
|
||||
}
|
||||
}
|
||||
|
||||
public static List<String> extractJobIds(PigStats stat) {
|
||||
if (stat instanceof SimplePigStats) {
|
||||
return extractJobIdsFromSimplePigStats((SimplePigStats) stat);
|
||||
} else if (stat instanceof TezPigScriptStats) {
|
||||
return extractJobIdsFromTezPigStats((TezPigScriptStats) stat);
|
||||
} else {
|
||||
throw new RuntimeException("Unrecognized stats type:" + stat.getClass().getSimpleName());
|
||||
}
|
||||
}
|
||||
|
||||
public static List<String> extractJobIdsFromSimplePigStats(SimplePigStats stat) {
|
||||
List<String> jobIds = new ArrayList<>();
|
||||
try {
|
||||
Field jobPlanField = PigStats.class.getDeclaredField("jobPlan");
|
||||
jobPlanField.setAccessible(true);
|
||||
PigStats.JobGraph jobPlan = (PigStats.JobGraph) jobPlanField.get(stat);
|
||||
List<JobStats> arr = jobPlan.getJobList();
|
||||
for (JobStats js : arr) {
|
||||
jobIds.add(js.getJobId());
|
||||
}
|
||||
return jobIds;
|
||||
} catch (Exception e) {
|
||||
LOGGER.error("Can not extract jobIds from SimpelPigStats", e);
|
||||
throw new RuntimeException("Can not extract jobIds from SimpelPigStats", e);
|
||||
}
|
||||
}
|
||||
|
||||
public static List<String> extractJobIdsFromTezPigStats(TezPigScriptStats stat) {
|
||||
List<String> jobIds = new ArrayList<>();
|
||||
try {
|
||||
Field tezDAGStatsMapField = TezPigScriptStats.class.getDeclaredField("tezDAGStatsMap");
|
||||
tezDAGStatsMapField.setAccessible(true);
|
||||
Map<String, TezDAGStats> tezDAGStatsMap =
|
||||
(Map<String, TezDAGStats>) tezDAGStatsMapField.get(stat);
|
||||
for (TezDAGStats dagStats : tezDAGStatsMap.values()) {
|
||||
LOGGER.debug("Tez JobId:" + dagStats.getJobId());
|
||||
jobIds.add(dagStats.getJobId());
|
||||
}
|
||||
return jobIds;
|
||||
} catch (Exception e) {
|
||||
LOGGER.error("Can not extract jobIds from TezPigScriptStats", e);
|
||||
throw new RuntimeException("Can not extract jobIds from TezPigScriptStats", e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
"envName": null,
|
||||
"propertyName": "zeppelin.pig.execType",
|
||||
"defaultValue": "mapreduce",
|
||||
"description": "local | mapreduce | tez",
|
||||
"description": "local | mapreduce | tez_local | tez | spark_local | spark",
|
||||
"type": "string"
|
||||
},
|
||||
"zeppelin.pig.includeJobStats": {
|
||||
|
|
@ -17,6 +17,20 @@
|
|||
"defaultValue": false,
|
||||
"description": "flag to include job stats in output",
|
||||
"type": "checkbox"
|
||||
},
|
||||
"SPARK_MASTER": {
|
||||
"envName": "SPARK_MASTER",
|
||||
"propertyName": "SPARK_MASTER",
|
||||
"defaultValue": "local",
|
||||
"description": "local | yarn-client",
|
||||
"type": "string"
|
||||
},
|
||||
"SPARK_JAR": {
|
||||
"envName": "SPARK_JAR",
|
||||
"propertyName": "SPARK_JAR",
|
||||
"defaultValue": "",
|
||||
"description": "spark assembly jar uploaded in hdfs",
|
||||
"type": "textarea"
|
||||
}
|
||||
},
|
||||
"editor": {
|
||||
|
|
@ -29,13 +43,6 @@
|
|||
"name": "query",
|
||||
"className": "org.apache.zeppelin.pig.PigQueryInterpreter",
|
||||
"properties": {
|
||||
"zeppelin.pig.execType": {
|
||||
"envName": null,
|
||||
"propertyName": "zeppelin.pig.execType",
|
||||
"defaultValue": "mapreduce",
|
||||
"description": "local | mapreduce | tez",
|
||||
"type": "string"
|
||||
},
|
||||
"zeppelin.pig.maxResult": {
|
||||
"envName": null,
|
||||
"propertyName": "zeppelin.pig.maxResult",
|
||||
|
|
|
|||
|
|
@ -0,0 +1,149 @@
|
|||
/**
|
||||
* Licensed to the Apache Software Foundation (ASF) under one
|
||||
* or more contributor license agreements. See the NOTICE file
|
||||
* distributed with this work for additional information
|
||||
* regarding copyright ownership. The ASF licenses this file
|
||||
* to you under the Apache License, Version 2.0 (the
|
||||
* "License"); you may not use this file except in compliance
|
||||
* with the License. You may obtain a copy of the License at
|
||||
* <p>
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* <p>
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
|
||||
package org.apache.zeppelin.pig;
|
||||
|
||||
import org.apache.commons.io.IOUtils;
|
||||
import org.apache.zeppelin.interpreter.InterpreterContext;
|
||||
import org.apache.zeppelin.interpreter.InterpreterResult;
|
||||
import org.junit.After;
|
||||
import org.junit.Test;
|
||||
|
||||
import java.io.File;
|
||||
import java.io.FileWriter;
|
||||
import java.io.IOException;
|
||||
import java.util.Properties;
|
||||
|
||||
import static org.junit.Assert.assertEquals;
|
||||
import static org.junit.Assert.assertTrue;
|
||||
|
||||
|
||||
public class PigInterpreterSparkTest {
|
||||
private PigInterpreter pigInterpreter;
|
||||
private InterpreterContext context;
|
||||
|
||||
public void setUpSpark(boolean includeJobStats) {
|
||||
Properties properties = new Properties();
|
||||
properties.put("zeppelin.pig.execType", "spark_local");
|
||||
properties.put("zeppelin.pig.includeJobStats", includeJobStats + "");
|
||||
pigInterpreter = new PigInterpreter(properties);
|
||||
pigInterpreter.open();
|
||||
context = new InterpreterContext(null, "paragraph_id", null, null, null, null, null, null, null, null,
|
||||
null, null);
|
||||
|
||||
}
|
||||
@After
|
||||
public void tearDown() {
|
||||
pigInterpreter.close();
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testBasics() throws IOException {
|
||||
setUpSpark(false);
|
||||
|
||||
String content = "1\tandy\n"
|
||||
+ "2\tpeter\n";
|
||||
File tmpFile = File.createTempFile("zeppelin", "test");
|
||||
FileWriter writer = new FileWriter(tmpFile);
|
||||
IOUtils.write(content, writer);
|
||||
writer.close();
|
||||
|
||||
// simple pig script using dump
|
||||
String pigscript = "a = load '" + tmpFile.getAbsolutePath() + "';"
|
||||
+ "dump a;";
|
||||
InterpreterResult result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
|
||||
assertTrue(result.message().get(0).getData().contains("(1,andy)\n(2,peter)"));
|
||||
|
||||
// describe
|
||||
pigscript = "a = load '" + tmpFile.getAbsolutePath() + "' as (id: int, name: bytearray);"
|
||||
+ "describe a;";
|
||||
result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
|
||||
assertTrue(result.message().get(0).getData().contains("a: {id: int,name: bytearray}"));
|
||||
|
||||
// syntax error (compilation error)
|
||||
pigscript = "a = loa '" + tmpFile.getAbsolutePath() + "';"
|
||||
+ "describe a;";
|
||||
result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.ERROR, result.code());
|
||||
assertTrue(result.message().get(0).getData().contains("Syntax error, unexpected symbol at or near 'a'"));
|
||||
|
||||
// syntax error
|
||||
pigscript = "a = load '" + tmpFile.getAbsolutePath() + "';"
|
||||
+ "foreach a generate $0;";
|
||||
result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.ERROR, result.code());
|
||||
assertTrue(result.message().get(0).getData().contains("expecting one of"));
|
||||
}
|
||||
|
||||
@Test
|
||||
public void testIncludeJobStats() throws IOException {
|
||||
setUpSpark(true);
|
||||
|
||||
String content = "1\tandy\n"
|
||||
+ "2\tpeter\n";
|
||||
File tmpFile = File.createTempFile("zeppelin", "test");
|
||||
FileWriter writer = new FileWriter(tmpFile);
|
||||
IOUtils.write(content, writer);
|
||||
writer.close();
|
||||
|
||||
// simple pig script using dump
|
||||
String pigscript = "a = load '" + tmpFile.getAbsolutePath() + "';"
|
||||
+ "dump a;";
|
||||
InterpreterResult result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
|
||||
assertTrue(result.message().get(0).getData().contains("Spark Job"));
|
||||
assertTrue(result.message().get(0).getData().contains("(1,andy)\n(2,peter)"));
|
||||
|
||||
// describe
|
||||
pigscript = "a = load '" + tmpFile.getAbsolutePath() + "' as (id: int, name: bytearray);"
|
||||
+ "describe a;";
|
||||
result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.SUCCESS, result.code());
|
||||
// no job is launched, so no jobStats
|
||||
assertTrue(result.message().get(0).getData().contains("a: {id: int,name: bytearray}"));
|
||||
|
||||
// syntax error (compilation error)
|
||||
pigscript = "a = loa '" + tmpFile.getAbsolutePath() + "';"
|
||||
+ "describe a;";
|
||||
result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.ERROR, result.code());
|
||||
// no job is launched, so no jobStats
|
||||
assertTrue(result.message().get(0).getData().contains("Syntax error, unexpected symbol at or near 'a'"));
|
||||
|
||||
// execution error
|
||||
pigscript = "a = load 'invalid_path';"
|
||||
+ "dump a;";
|
||||
result = pigInterpreter.interpret(pigscript, context);
|
||||
assertEquals(InterpreterResult.Type.TEXT, result.message().get(0).getType());
|
||||
assertEquals(InterpreterResult.Code.ERROR, result.code());
|
||||
assertTrue(result.message().get(0).getData().contains("Failed to read data from"));
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
2
pom.xml
2
pom.xml
|
|
@ -107,7 +107,7 @@
|
|||
<commons.collections.version>3.2.1</commons.collections.version>
|
||||
<commons.logging.version>1.1.1</commons.logging.version>
|
||||
<commons.cli.version>1.3.1</commons.cli.version>
|
||||
<shiro.version>1.2.3</shiro.version>
|
||||
<shiro.version>1.3.2</shiro.version>
|
||||
|
||||
<!-- test library versions -->
|
||||
<junit.version>4.12</junit.version>
|
||||
|
|
|
|||
|
|
@ -95,8 +95,8 @@ The following components are provided under Apache License.
|
|||
(Apache 2.0) Lucene Suggest (org.apache.lucene:lucene-suggest:5.3.1 - http://lucene.apache.org/lucene-parent/lucene-suggest)
|
||||
(Apache 2.0) Elasticsearch: Core (org.elasticsearch:elasticsearch:2.1.0 - http://nexus.sonatype.org/oss-repository-hosting.html/parent/elasticsearch)
|
||||
(Apache 2.0) Joda convert (org.joda:joda-convert:1.8.1 - http://joda-convert.sourceforge.net)
|
||||
(Apache 2.0) Shiro Core (org.apache.shiro:shiro-core:1.2.3 - https://shiro.apache.org)
|
||||
(Apache 2.0) Shiro Web (org.apache.shiro:shiro-web:1.2.3 - https://shiro.apache.org)
|
||||
(Apache 2.0) Shiro Core (org.apache.shiro:shiro-core:1.3.2 - https://shiro.apache.org)
|
||||
(Apache 2.0) Shiro Web (org.apache.shiro:shiro-web:1.3.2 - https://shiro.apache.org)
|
||||
(Apache 2.0) SnakeYAML (org.yaml:snakeyaml:1.15 - http://www.snakeyaml.org)
|
||||
(Apache 2.0) Protocol Buffers (com.google.protobuf:protobuf-java:2.5.0 - https://github.com/google/protobuf/releases)
|
||||
(Apache 2.0) Alluxio Shell (org.alluxio:alluxio-shell:1.0.0 - http://alluxio.org)
|
||||
|
|
|
|||
|
|
@ -478,7 +478,6 @@
|
|||
</dependencies>
|
||||
</dependencyManagement>
|
||||
</profile>
|
||||
|
||||
<profile>
|
||||
<id>using-source-tree</id>
|
||||
<activation>
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@
|
|||
"angular-resource": "1.5.7",
|
||||
"angular-bootstrap": "~2.5.0",
|
||||
"angular-websocket": "~1.0.13",
|
||||
"ace-builds": "1.2.6",
|
||||
"ace-builds": "1.2.7",
|
||||
"angular-ui-ace": "0.1.3",
|
||||
"jquery.scrollTo": "~1.4.13",
|
||||
"nvd3": "~1.8.5",
|
||||
|
|
@ -46,13 +46,14 @@
|
|||
"src-noconflict/mode-python.js",
|
||||
"src-noconflict/mode-sql.js",
|
||||
"src-noconflict/mode-markdown.js",
|
||||
"src-noconflict/mode-pig.js",
|
||||
"src-noconflict/mode-sh.js",
|
||||
"src-noconflict/mode-r.js",
|
||||
"src-noconflict/keybinding-emacs.js",
|
||||
"src-noconflict/ext-language_tools.js",
|
||||
"src-noconflict/theme-chrome.js"
|
||||
],
|
||||
"version": "1.2.6",
|
||||
"version": "1.2.7",
|
||||
"name": "ace-builds"
|
||||
},
|
||||
"highlightjs": {
|
||||
|
|
@ -65,7 +66,7 @@
|
|||
}
|
||||
},
|
||||
"resolutions": {
|
||||
"ace-builds": "1.2.6",
|
||||
"ace-builds": "1.2.7",
|
||||
"angular": ">=1.5.0 <1.6"
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -59,6 +59,7 @@ module.exports = function(config) {
|
|||
'bower_components/ace-builds/src-noconflict/mode-python.js',
|
||||
'bower_components/ace-builds/src-noconflict/mode-sql.js',
|
||||
'bower_components/ace-builds/src-noconflict/mode-markdown.js',
|
||||
'bower_components/ace-builds/src-noconflict/mode-pig.js',
|
||||
'bower_components/ace-builds/src-noconflict/mode-sh.js',
|
||||
'bower_components/ace-builds/src-noconflict/mode-r.js',
|
||||
'bower_components/ace-builds/src-noconflict/keybinding-emacs.js',
|
||||
|
|
|
|||
|
|
@ -146,6 +146,7 @@ let zeppelinWebApp = angular.module('zeppelinWebApp', requiredModules)
|
|||
|
||||
// handel logout on API failure
|
||||
.config(function ($httpProvider, $provide) {
|
||||
$httpProvider.defaults.headers.common['X-Requested-With'] = 'XMLHttpRequest'
|
||||
$provide.factory('httpInterceptor', function ($q, $rootScope) {
|
||||
return {
|
||||
'responseError': function (rejection) {
|
||||
|
|
@ -175,12 +176,24 @@ function auth () {
|
|||
},
|
||||
crossDomain: true
|
||||
})
|
||||
return $http.get(baseUrlSrv.getRestApiBase() + '/security/ticket').then(function (response) {
|
||||
let config = {headers: { 'X-Requested-With': 'XMLHttpRequest' }}
|
||||
return $http.get(baseUrlSrv.getRestApiBase() + '/security/ticket', config).then(function (response) {
|
||||
zeppelinWebApp.run(function ($rootScope) {
|
||||
$rootScope.ticket = angular.fromJson(response.data).body
|
||||
|
||||
$rootScope.ticket.screenUsername = $rootScope.ticket.principal
|
||||
if ($rootScope.ticket.principal.startsWith('#Pac4j')) {
|
||||
let re = ', name=(.*?),'
|
||||
$rootScope.ticket.screenUsername = $rootScope.ticket.principal.match(re)[1]
|
||||
}
|
||||
})
|
||||
}, function (errorResponse) {
|
||||
// Handle error case
|
||||
let redirect = errorResponse.headers('Location')
|
||||
if (errorResponse.status === 401 && redirect !== undefined) {
|
||||
// Handle page redirect
|
||||
window.location.href = redirect
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -63,14 +63,8 @@ limitations under the License.
|
|||
<ul class="noDot">
|
||||
<li class="liVertical" ng-repeat="repo in repositories">
|
||||
<a tabindex="0" class="btn btn-info" role="button"
|
||||
popover-trigger="focus"
|
||||
popover-placement="right"
|
||||
popover-html-unsafe="<label>URL: </label>
|
||||
{{repo.url}}<br>
|
||||
<label>Username: </label>
|
||||
{{repo.authentication.username}}<br>
|
||||
<label>Proxy host: </label>
|
||||
{{repo.proxy.host}}">
|
||||
tooltip-append-to-body="true" tooltip-placement="top-left"
|
||||
uib-tooltip="{{repo.url}}">
|
||||
<span class="fa fa-database"></span>
|
||||
{{repo.id}}
|
||||
<span ng-if="!isDefaultRepository(repo.id)" class="fa fa-close blackOpc"
|
||||
|
|
|
|||
|
|
@ -94,7 +94,7 @@ limitations under the License.
|
|||
</div>
|
||||
|
||||
<span
|
||||
ng-if="(type == 'TABLE' || type == 'NETWORK') && !config.helium.activeApp && graphMode!='table' && !asIframe && !viewOnly"
|
||||
ng-if="(type == 'TABLE' || type == 'NETWORK') && !config.helium.activeApp && !asIframe && !viewOnly"
|
||||
style="margin-left:10px; cursor:pointer; display: inline-block; vertical-align:top; position: relative; line-height:30px;">
|
||||
<a class="btnText" ng-click="toggleGraphSetting()">
|
||||
settings <span ng-class="config.graph.optionOpen ? 'fa fa-caret-up' : 'fa fa-caret-down'"></span>
|
||||
|
|
|
|||
|
|
@ -47,11 +47,10 @@ limitations under the License.
|
|||
style="font-weight: 300; font-size: 13px; margin-left: 1px;"
|
||||
ng-style="prop.name === 'xAxis' ? { 'background-color': '#906ebd' } : prop.name === 'yAxis' ? { 'background-color': '#cd5c5c' } : prop.name === 'group' ? { 'background-color': '#5782bd' } : ''">
|
||||
{{prop.name}}
|
||||
<a tabindex="0" class="fa fa-info-circle" role="button" popover-placement="top"
|
||||
ng-if="prop.tooltip"
|
||||
style="color: white;"
|
||||
popover-trigger="focus"
|
||||
popover-html-unsafe="{{prop.tooltip}}"></a>
|
||||
<a tabindex="0" class="fa fa-info-circle" role="button"
|
||||
ng-if="prop.tooltip" style="color: white;"
|
||||
tooltip-placement="top" tooltip-append-to-body="true"
|
||||
uib-tooltip="{{prop.tooltip}}"></a>
|
||||
</span>
|
||||
<ul data-drop="true"
|
||||
ng-model="config[prop.name]"
|
||||
|
|
|
|||
|
|
@ -34,12 +34,12 @@ export default class ScatterchartVisualization extends Nvd3ChartVisualization {
|
|||
},
|
||||
{
|
||||
name: 'size',
|
||||
tooltip: `<li>Size option is valid only when you drop numeric field here.</li>
|
||||
<li>When data in each axis are discrete,
|
||||
'number of values in corresponding coordinate' will be used as size.</li>
|
||||
<li>Zeppelin consider values as discrete when the values contain string value
|
||||
or the number of distinct values are bigger than 5% of total number of values.</li>
|
||||
<li>Size field button turns to grey when the option you chose is not valid.</li>`
|
||||
tooltip: `This option is only valid for numeric fields.
|
||||
When data in each axis is discrete,
|
||||
'number of values in corresponding coordinate' will be used.
|
||||
Zeppelin considers values as discrete when input values contain a string
|
||||
or the number of distinct values is greater than 5% of the total number of values.
|
||||
This field turns grey when the selected option is invalid.`
|
||||
}
|
||||
]
|
||||
this.columnselector = new ColumnselectorTransformation(config, this.columnselectorProps)
|
||||
|
|
|
|||
|
|
@ -95,6 +95,7 @@ function NavCtrl ($scope, $rootScope, $http, $routeParams, $location,
|
|||
$http.post(logoutURL).error(function () {
|
||||
$rootScope.userName = ''
|
||||
$rootScope.ticket.principal = ''
|
||||
$rootScope.ticket.screenUsername = ''
|
||||
$rootScope.ticket.ticket = ''
|
||||
$rootScope.ticket.roles = ''
|
||||
BootstrapDialog.show({
|
||||
|
|
@ -131,6 +132,7 @@ function NavCtrl ($scope, $rootScope, $http, $routeParams, $location,
|
|||
})
|
||||
|
||||
$scope.$on('loginSuccess', function (event, param) {
|
||||
$rootScope.ticket.screenUsername = $rootScope.ticket.principal
|
||||
listConfigurations()
|
||||
loadNotes()
|
||||
getHomeNote()
|
||||
|
|
|
|||
|
|
@ -90,7 +90,7 @@ limitations under the License.
|
|||
<i ng-if="!navbar.connected" class="fa fa-circle server-disconnected"
|
||||
uib-tooltip="WebSocket Disconnected" tooltip-placement="bottom" style="margin-top: 7px; vertical-align: top"></i>
|
||||
<button ng-if="ticket" class="nav-btn dropdown-toggle" type="button" data-toggle="dropdown" style="margin:11px 5px 0 0; padding-left: 0px;">
|
||||
<span class="username">{{ticket.principal}}</span>
|
||||
<span class="username">{{ticket.screenUsername}}</span>
|
||||
<span class="caret" style="margin-bottom: 8px"></span>
|
||||
</button>
|
||||
<span ng-if="!ticket" style="margin: 5px;"></span>
|
||||
|
|
|
|||
|
|
@ -1,24 +0,0 @@
|
|||
/*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
angular.module('zeppelinWebApp').directive('popoverHtmlUnsafePopup', popoverHtmlUnsafePopup)
|
||||
|
||||
function popoverHtmlUnsafePopup () {
|
||||
return {
|
||||
restrict: 'EA',
|
||||
replace: true,
|
||||
scope: {title: '@', content: '@', placement: '@', animation: '&', isOpen: '&'},
|
||||
templateUrl: 'components/popover-html-unsafe/popover-html-unsafe-popup.html'
|
||||
}
|
||||
}
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
<div class="popover {{placement}}" ng-class="{ in: isOpen(), fade: animation() }">
|
||||
<!--
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
-->
|
||||
<div class="arrow"></div>
|
||||
|
||||
<div class="popover-inner">
|
||||
<h3 class="popover-title" ng-bind="title" ng-show="title"></h3>
|
||||
<div class="popover-content" bind-html-unsafe="content"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
/*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
angular.module('zeppelinWebApp').directive('popoverHtmlUnsafe', popoverHtmlUnsafe)
|
||||
|
||||
function popoverHtmlUnsafe ($uibTooltip) {
|
||||
'ngInject'
|
||||
|
||||
return $uibTooltip('popoverHtmlUnsafe', 'popover', 'click')
|
||||
}
|
||||
|
|
@ -140,6 +140,7 @@ limitations under the License.
|
|||
<script src="bower_components/ace-builds/src-noconflict/mode-python.js"></script>
|
||||
<script src="bower_components/ace-builds/src-noconflict/mode-sql.js"></script>
|
||||
<script src="bower_components/ace-builds/src-noconflict/mode-markdown.js"></script>
|
||||
<script src="bower_components/ace-builds/src-noconflict/mode-pig.js"></script>
|
||||
<script src="bower_components/ace-builds/src-noconflict/mode-sh.js"></script>
|
||||
<script src="bower_components/ace-builds/src-noconflict/mode-r.js"></script>
|
||||
<script src="bower_components/ace-builds/src-noconflict/keybinding-emacs.js"></script>
|
||||
|
|
|
|||
|
|
@ -56,8 +56,6 @@ import './components/interpreter/widget/widget.number.directive.js'
|
|||
import './components/expandCollapse/expandCollapse.directive.js'
|
||||
import './components/noteName-create/notename.controller.js'
|
||||
import './components/noteName-import/notenameImport.controller.js'
|
||||
import './components/popover-html-unsafe/popover-html-unsafe.directive.js'
|
||||
import './components/popover-html-unsafe/popover-html-unsafe-popup.directive.js'
|
||||
import './components/editor/codeEditor.directive.js'
|
||||
import './components/ngenter/ngenter.directive.js'
|
||||
import './components/dropdowninput/dropdowninput.directive.js'
|
||||
|
|
|
|||
Loading…
Reference in a new issue