mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
Apply auto TOC to all of docs under docs/interpreter/
This commit is contained in:
AhyoungRyu
parent
587d4baea0
commit
9c5f76bbb1
17 changed files with 179 additions and 116 deletions
|
|
@ -6,7 +6,11 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Alluxio Interpreter for Apache Zeppelin
|
||||
# Alluxio Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Alluxio](http://alluxio.org/) is a memory-centric distributed storage system enabling reliable data sharing at memory-speed across cluster frameworks.
|
||||
|
||||
## Configuration
|
||||
|
|
|
|||
|
|
@ -6,7 +6,9 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Cassandra CQL Interpreter for Apache Zeppelin
|
||||
# Cassandra CQL Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
|
|
|
|||
|
|
@ -6,7 +6,11 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Elasticsearch Interpreter for Apache Zeppelin
|
||||
# Elasticsearch Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Elasticsearch](https://www.elastic.co/products/elasticsearch) is a highly scalable open-source full-text search and analytics engine. It allows you to store, search, and analyze big volumes of data quickly and in near real time. It is generally used as the underlying engine/technology that powers applications that have complex search features and requirements.
|
||||
|
||||
## Configuration
|
||||
|
|
|
|||
|
|
@ -6,7 +6,11 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Flink interpreter for Apache Zeppelin
|
||||
# Flink interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Apache Flink](https://flink.apache.org) is an open source platform for distributed stream and batch data processing. Flink’s core is a streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams. Flink also builds batch processing on top of the streaming engine, overlaying native iteration support, managed memory, and program optimization.
|
||||
|
||||
## How to start local Flink cluster, to test the interpreter
|
||||
|
|
|
|||
|
|
@ -6,7 +6,11 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Geode/Gemfire OQL Interpreter for Apache Zeppelin
|
||||
# Geode/Gemfire OQL Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
|
|
@ -33,7 +37,7 @@ This interpreter supports the [Geode](http://geode.incubator.apache.org/) [Objec
|
|||
|
||||
This [Video Tutorial](https://www.youtube.com/watch?v=zvzzA9GXu3Q) illustrates some of the features provided by the `Geode Interpreter`.
|
||||
|
||||
### Create Interpreter
|
||||
## Create Interpreter
|
||||
By default Zeppelin creates one `Geode/OQL` instance. You can remove it or create more instances.
|
||||
|
||||
Multiple Geode instances can be created, each configured to the same or different backend Geode cluster. But over time a `Notebook` can have only one Geode interpreter instance `bound`. That means you _cannot_ connect to different Geode clusters in the same `Notebook`. This is a known Zeppelin limitation.
|
||||
|
|
@ -42,10 +46,10 @@ To create new Geode instance open the `Interpreter` section and click the `+Crea
|
|||
|
||||
> Note: The `Name` of the instance is used only to distinguish the instances while binding them to the `Notebook`. The `Name` is irrelevant inside the `Notebook`. In the `Notebook` you must use `%geode.oql` tag.
|
||||
|
||||
### Bind to Notebook
|
||||
## Bind to Notebook
|
||||
In the `Notebook` click on the `settings` icon in the top right corner. The select/deselect the interpreters to be bound with the `Notebook`.
|
||||
|
||||
### Configuration
|
||||
## Configuration
|
||||
You can modify the configuration of the Geode from the `Interpreter` section. The Geode interpreter expresses the following properties:
|
||||
|
||||
<table class="table-configuration">
|
||||
|
|
@ -71,12 +75,12 @@ You can modify the configuration of the Geode from the `Interpreter` section. T
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
### How to use
|
||||
## How to use
|
||||
> *Tip 1: Use (CTRL + .) for OQL auto-completion.*
|
||||
|
||||
> *Tip 2: Always start the paragraphs with the full `%geode.oql` prefix tag! The short notation: `%geode` would still be able run the OQL queries but the syntax highlighting and the auto-completions will be disabled.*
|
||||
|
||||
#### Create / Destroy Regions
|
||||
### Create / Destroy Regions
|
||||
The OQL specification does not support [Geode Regions](https://cwiki.apache.org/confluence/display/GEODE/Index#Index-MainConceptsandComponents) mutation operations. To `create`/`destroy` regions one should use the [GFSH](http://geode-docs.cfapps.io/docs/tools_modules/gfsh/chapter_overview.html) shell tool instead. In the following it is assumed that the GFSH is colocated with Zeppelin server.
|
||||
|
||||
```bash
|
||||
|
|
@ -97,7 +101,7 @@ EOF
|
|||
|
||||
Above snippet re-creates two regions: `regionEmployee` and `regionCompany`. Note that you have to explicitly specify the locator host and port. The values should match those you have used in the Geode Interpreter configuration. Comprehensive list of [GFSH Commands by Functional Area](http://geode-docs.cfapps.io/docs/tools_modules/gfsh/gfsh_quick_reference.html).
|
||||
|
||||
#### Basic OQL
|
||||
### Basic OQL
|
||||
```sql
|
||||
%geode.oql
|
||||
SELECT count(*) FROM /regionEmployee
|
||||
|
|
@ -136,7 +140,7 @@ SELECT e.key, e.value FROM /regionEmployee.entrySet e
|
|||
|
||||
> Note: You can have multiple queries in the same paragraph but only the result from the first is displayed. [[1](https://issues.apache.org/jira/browse/ZEPPELIN-178)], [[2](https://issues.apache.org/jira/browse/ZEPPELIN-212)].
|
||||
|
||||
#### GFSH Commands From The Shell
|
||||
### GFSH Commands From The Shell
|
||||
Use the Shell Interpreter (`%sh`) to run OQL commands form the command line:
|
||||
|
||||
```bash
|
||||
|
|
@ -145,7 +149,7 @@ source /etc/geode/conf/geode-env.sh
|
|||
gfsh -e "connect" -e "list members"
|
||||
```
|
||||
|
||||
#### Apply Zeppelin Dynamic Forms
|
||||
### Apply Zeppelin Dynamic Forms
|
||||
You can leverage [Zeppelin Dynamic Form](../manual/dynamicform.html) inside your OQL queries. You can use both the `text input` and `select form` parameterization features
|
||||
|
||||
```sql
|
||||
|
|
@ -153,7 +157,10 @@ You can leverage [Zeppelin Dynamic Form](../manual/dynamicform.html) inside your
|
|||
SELECT * FROM /regionEmployee e WHERE e.employeeId > ${Id}
|
||||
```
|
||||
|
||||
#### Geode REST API
|
||||
### Auto-completion
|
||||
The Geode Interpreter provides a basic auto-completion functionality. On `(Ctrl+.)` it list the most relevant suggestions in a pop-up window.
|
||||
|
||||
## Geode REST API
|
||||
To list the defined regions you can use the [Geode REST API](http://geode-docs.cfapps.io/docs/geode_rest/chapter_overview.html):
|
||||
|
||||
```
|
||||
|
|
@ -182,6 +189,3 @@ http://<geode server hostname>phd1.localdomain:8484/gemfire-api/v1/
|
|||
http-service-port=8484
|
||||
start-dev-rest-api=true
|
||||
```
|
||||
|
||||
### Auto-completion
|
||||
The Geode Interpreter provides a basic auto-completion functionality. On `(Ctrl+.)` it list the most relevant suggestions in a pop-up window.
|
||||
|
|
|
|||
|
|
@ -6,16 +6,22 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## HBase Shell Interpreter for Apache Zeppelin
|
||||
# HBase Shell Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[HBase Shell](http://hbase.apache.org/book.html#shell) is a JRuby IRB client for Apache HBase. This interpreter provides all capabilities of Apache HBase shell within Apache Zeppelin. The interpreter assumes that Apache HBase client software has been installed and it can connect to the Apache HBase cluster from the machine on where Apache Zeppelin is installed.
|
||||
To get start with HBase, please see [HBase Quickstart](https://hbase.apache.org/book.html#quickstart)
|
||||
To get start with HBase, please see [HBase Quickstart](https://hbase.apache.org/book.html#quickstart).
|
||||
|
||||
## HBase release supported
|
||||
By default, Zeppelin is built against HBase 1.0.x releases. To work with HBase 1.1.x releases, use the following build command:
|
||||
|
||||
```bash
|
||||
# HBase 1.1.4
|
||||
mvn clean package -DskipTests -Phadoop-2.6 -Dhadoop.version=2.6.0 -P build-distr -Dhbase.hbase.version=1.1.4 -Dhbase.hadoop.version=2.6.0
|
||||
```
|
||||
|
||||
To work with HBase 1.2.0+, use the following build command:
|
||||
|
||||
```bash
|
||||
|
|
@ -94,4 +100,4 @@ And then to put data into that table
|
|||
put 'test', 'row1', 'cf:a', 'value1'
|
||||
```
|
||||
|
||||
For more information on all commands available, refer to [HBase shell commands](https://learnhbase.wordpress.com/2013/03/02/hbase-shell-commands/)
|
||||
For more information on all commands available, refer to [HBase shell commands](https://learnhbase.wordpress.com/2013/03/02/hbase-shell-commands/).
|
||||
|
|
|
|||
|
|
@ -6,8 +6,11 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## HDFS File System Interpreter for Apache Zeppelin
|
||||
# HDFS File System Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Hadoop File System](http://hadoop.apache.org/) is a distributed, fault tolerant file system part of the hadoop project and is often used as storage for distributed processing engines like [Hadoop MapReduce](http://hadoop.apache.org/) and [Apache Spark](http://spark.apache.org/) or underlying file systems like [Alluxio](http://www.alluxio.org/).
|
||||
|
||||
## Configuration
|
||||
|
|
@ -44,13 +47,17 @@ It supports the basic shell file commands applied to HDFS, it currently only sup
|
|||
|
||||
> **Tip :** Use ( Ctrl + . ) for autocompletion.
|
||||
|
||||
### Create Interpreter
|
||||
## Create Interpreter
|
||||
|
||||
In a notebook, to enable the **HDFS** interpreter, click the **Gear** icon and select **HDFS**.
|
||||
|
||||
|
||||
#### WebHDFS REST API
|
||||
## WebHDFS REST API
|
||||
You can confirm that you're able to access the WebHDFS API by running a curl command against the WebHDFS end point provided to the interpreter.
|
||||
|
||||
Here is an example:
|
||||
|
||||
```bash
|
||||
$> curl "http://localhost:50070/webhdfs/v1/?op=LISTSTATUS"
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,9 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Hive Interpreter for Apache Zeppelin
|
||||
The [Apache Hive](https://hive.apache.org/) ™ data warehouse software facilitates querying and managing large datasets residing in distributed storage. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. At the same time this language also allows traditional map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or inefficient to express this logic in HiveQL.
|
||||
# Hive Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Important Notice
|
||||
Hive Interpreter will be deprecated and merged into JDBC Interpreter. You can use Hive Interpreter by using JDBC Interpreter with same functionality. See the example below of settings and dependencies.
|
||||
|
|
@ -52,7 +53,6 @@ Hive Interpreter will be deprecated and merged into JDBC Interpreter. You can us
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
----
|
||||
|
||||
### Configuration
|
||||
<table class="table-configuration">
|
||||
|
|
@ -115,6 +115,10 @@ Hive Interpreter will be deprecated and merged into JDBC Interpreter. You can us
|
|||
|
||||
This interpreter provides multiple configuration with `${prefix}`. User can set a multiple connection properties by this prefix. It can be used like `%hive(${prefix})`.
|
||||
|
||||
## Overview
|
||||
|
||||
The [Apache Hive](https://hive.apache.org/) ™ data warehouse software facilitates querying and managing large datasets residing in distributed storage. Hive provides a mechanism to project structure onto this data and query the data using a SQL-like language called HiveQL. At the same time this language also allows traditional map/reduce programmers to plug in their custom mappers and reducers when it is inconvenient or inefficient to express this logic in HiveQL.
|
||||
|
||||
## How to use
|
||||
Basically, you can use
|
||||
|
||||
|
|
|
|||
|
|
@ -6,16 +6,18 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Ignite Interpreter for Apache Zeppelin
|
||||
# Ignite Interpreter for Apache Zeppelin
|
||||
|
||||
### Overview
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Apache Ignite](https://ignite.apache.org/) In-Memory Data Fabric is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies.
|
||||
|
||||
|
||||
|
||||
You can use Zeppelin to retrieve distributed data from cache using Ignite SQL interpreter. Moreover, Ignite interpreter allows you to execute any Scala code in cases when SQL doesn't fit to your requirements. For example, you can populate data into your caches or execute distributed computations.
|
||||
|
||||
### Installing and Running Ignite example
|
||||
## Installing and Running Ignite example
|
||||
In order to use Ignite interpreters, you may install Apache Ignite in some simple steps:
|
||||
|
||||
1. Download Ignite [source release](https://ignite.apache.org/download.html#sources) or [binary release](https://ignite.apache.org/download.html#binaries) whatever you want. But you must download Ignite as the same version of Zeppelin's. If it is not, you can't use scala code on Zeppelin. You can find ignite version in Zeppelin at the pom.xml which is placed under `path/to/your-Zeppelin/ignite/pom.xml` ( Of course, in Zeppelin source release ). Please check `ignite.version` .<br>Currently, Zeppelin provides ignite only in Zeppelin source release. So, if you download Zeppelin binary release( `zeppelin-0.5.0-incubating-bin-spark-xxx-hadoop-xx` ), you can not use ignite interpreter on Zeppelin. We are planning to include ignite in a future binary release.
|
||||
|
|
@ -31,7 +33,7 @@ In order to use Ignite interpreters, you may install Apache Ignite in some simpl
|
|||
$ nohup java -jar </path/to/your Jar file name>
|
||||
```
|
||||
|
||||
### Configuring Ignite Interpreter
|
||||
## Configuring Ignite Interpreter
|
||||
At the "Interpreters" menu, you may edit Ignite interpreter or create new one. Zeppelin provides these properties for Ignite.
|
||||
|
||||
<table class="table-configuration">
|
||||
|
|
@ -69,14 +71,14 @@ At the "Interpreters" menu, you may edit Ignite interpreter or create new one. Z
|
|||
|
||||
|
||||
|
||||
### Interpreter Binding for Zeppelin Notebook
|
||||
## How to use
|
||||
After configuring Ignite interpreter, create your own notebook. Then you can bind interpreters like below image.
|
||||
|
||||
|
||||
|
||||
For more interpreter binding information see [here](http://zeppelin.apache.org/docs/manual/interpreters.html).
|
||||
|
||||
### How to use Ignite SQL interpreter
|
||||
### Ignite SQL interpreter
|
||||
In order to execute SQL query, use ` %ignite.ignitesql ` prefix. <br>
|
||||
Supposing you are running `org.apache.ignite.examples.streaming.wordcount.StreamWords`, then you can use "words" cache( Of course you have to specify this cache name to the Ignite interpreter setting section `ignite.jdbc.url` of Zeppelin ).
|
||||
For example, you can select top 10 words in the words cache using the following query
|
||||
|
|
|
|||
|
|
@ -7,7 +7,11 @@ group: manual
|
|||
{% include JB/setup %}
|
||||
|
||||
|
||||
## Generic JDBC Interpreter for Apache Zeppelin
|
||||
# Generic JDBC Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
|
||||
This interpreter lets you create a JDBC connection to any data source, by now it has been tested with:
|
||||
|
||||
|
|
@ -16,16 +20,14 @@ This interpreter lets you create a JDBC connection to any data source, by now it
|
|||
* MariaDB
|
||||
* Redshift
|
||||
* Apache Hive
|
||||
* Apache Drill
|
||||
* Details on using [Drill JDBC Driver](https://drill.apache.org/docs/using-the-jdbc-driver)
|
||||
* Apache Phoenix
|
||||
* Apache Tajo
|
||||
* Apache Drill (Details on using [Drill JDBC Driver](https://drill.apache.org/docs/using-the-jdbc-driverde* Apache Tajo
|
||||
|
||||
If someone else used another database please report how it works to improve functionality.
|
||||
|
||||
### Create Interpreter
|
||||
## Create Interpreter
|
||||
|
||||
When create a interpreter by default use PostgreSQL with the next properties:
|
||||
When you create a interpreter by default use PostgreSQL with the next properties:
|
||||
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
|
|
@ -56,7 +58,7 @@ When create a interpreter by default use PostgreSQL with the next properties:
|
|||
|
||||
It is not necessary to add driver jar to the classpath for PostgreSQL as it is included in Zeppelin.
|
||||
|
||||
#### Simple connection
|
||||
### Simple connection
|
||||
|
||||
Prior to creating the interpreter it is necessary to add maven coordinate or path of the JDBC driver to the Zeppelin classpath. To do this you must edit dependencies artifact(ex. `mysql:mysql-connector-java:5.1.38`) in interpreter menu as shown:
|
||||
|
||||
|
|
@ -95,7 +97,7 @@ To create the interpreter you need to specify connection parameters as shown in
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
#### Multiple connections
|
||||
### Multiple connections
|
||||
|
||||
JDBC interpreter also allows connections to multiple data sources. It is necessary to set a prefix for each connection to reference it in the paragraph in the form of `%jdbc(prefix)`. Before you create the interpreter it is necessary to add each driver's maven coordinates or JDBC driver's jar file path to the Zeppelin classpath. To do this you must edit the dependencies of JDBC interpreter in interpreter menu as following:
|
||||
|
||||
|
|
@ -151,10 +153,10 @@ You can add all the jars you need to make multiple connections into the same JDB
|
|||
</table>
|
||||
|
||||
|
||||
### Bind to Notebook
|
||||
## Bind to Notebook
|
||||
In the `Notebook` click on the `settings` icon at the top-right corner. Use select/deselect to specify the interpreters to be used in the `Notebook`.
|
||||
|
||||
### More Properties
|
||||
## More Properties
|
||||
You can modify the interpreter configuration in the `Interpreter` section. The most common properties are as follows, but you can specify other properties that need to be connected.
|
||||
|
||||
<table class="table-configuration">
|
||||
|
|
@ -197,9 +199,11 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
### Examples
|
||||
#### Hive
|
||||
##### Properties
|
||||
## Examples
|
||||
|
||||
### Hive
|
||||
|
||||
#### Properties
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
|
|
@ -222,7 +226,8 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
<td>hive_password</td>
|
||||
</tr>
|
||||
</table>
|
||||
##### Dependencies
|
||||
|
||||
#### Dependencies
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Artifact</th>
|
||||
|
|
@ -237,8 +242,9 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
<td></td>
|
||||
</tr>
|
||||
</table>
|
||||
#### Phoenix
|
||||
##### Properties
|
||||
|
||||
### Phoenix
|
||||
#### Properties
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
|
|
@ -261,7 +267,7 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
<td>phoenix_password</td>
|
||||
</tr>
|
||||
</table>
|
||||
##### Dependencies
|
||||
#### Dependencies
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Artifact</th>
|
||||
|
|
@ -272,8 +278,9 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
<td></td>
|
||||
</tr>
|
||||
</table>
|
||||
#### Tajo
|
||||
##### Properties
|
||||
|
||||
### Tajo
|
||||
#### Properties
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
|
|
@ -288,7 +295,8 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
<td>jdbc:tajo://localhost:26002/default</td>
|
||||
</tr>
|
||||
</table>
|
||||
##### Dependencies
|
||||
|
||||
#### Dependencies
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Artifact</th>
|
||||
|
|
@ -300,9 +308,9 @@ To develop this functionality use this [method](http://docs.oracle.com/javase/7/
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
### How to use
|
||||
## How to use
|
||||
|
||||
#### Reference in paragraph
|
||||
### Reference in paragraph
|
||||
|
||||
Start the paragraphs with the `%jdbc`, this will use the `default` prefix for connection. If you want to use other connection you should specify the prefix of it as follows `%jdbc(prefix)`:
|
||||
|
||||
|
|
@ -311,6 +319,7 @@ Start the paragraphs with the `%jdbc`, this will use the `default` prefix for co
|
|||
SELECT * FROM db_name;
|
||||
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```sql
|
||||
|
|
@ -319,7 +328,7 @@ SELECT * FROM db_name;
|
|||
|
||||
```
|
||||
|
||||
#### Apply Zeppelin Dynamic Forms
|
||||
### Apply Zeppelin Dynamic Forms
|
||||
|
||||
You can leverage [Zeppelin Dynamic Form](../manual/dynamicform.html) inside your queries. You can use both the `text input` and `select form` parametrization features
|
||||
|
||||
|
|
@ -330,5 +339,5 @@ FROM demo.performers
|
|||
WHERE name='{{performer=Sheryl Crow|Doof|Fanfarlo|Los Paranoia}}'
|
||||
```
|
||||
|
||||
### Bugs & Contacts
|
||||
## Bugs & Reporting
|
||||
If you find a bug for this interpreter, please create a [JIRA]( https://issues.apache.org/jira/browse/ZEPPELIN-382?jql=project%20%3D%20ZEPPELIN) ticket.
|
||||
|
|
|
|||
|
|
@ -6,14 +6,16 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Lens Interpreter for Apache Zeppelin
|
||||
# Lens Interpreter for Apache Zeppelin
|
||||
|
||||
### Overview
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Apache Lens](https://lens.apache.org/) provides an Unified Analytics interface. Lens aims to cut the Data Analytics silos by providing a single view of data across multiple tiered data stores and optimal execution environment for the analytical query. It seamlessly integrates Hadoop with traditional data warehouses to appear like one.
|
||||
|
||||
|
||||
|
||||
### Installing and Running Lens
|
||||
## Installing and Running Lens
|
||||
In order to use Lens interpreters, you may install Apache Lens in some simple steps:
|
||||
|
||||
1. Download Lens for latest version from [the ASF](http://www.apache.org/dyn/closer.lua/lens/2.3-beta). Or the older release can be found [in the Archives](http://archive.apache.org/dist/lens/).
|
||||
|
|
@ -24,7 +26,7 @@ In order to use Lens interpreters, you may install Apache Lens in some simple st
|
|||
./bin/lens-ctl start (or stop)
|
||||
```
|
||||
|
||||
### Configuring Lens Interpreter
|
||||
## Configuring Lens Interpreter
|
||||
At the "Interpreters" menu, you can edit Lens interpreter or create new one. Zeppelin provides these properties for Lens.
|
||||
|
||||
<table class="table-configuration">
|
||||
|
|
@ -163,7 +165,7 @@ query execute cube select customer_city_name, product_details.description, produ
|
|||
|
||||
These are just examples that provided in advance by Lens. If you want to explore whole tutorials of Lens, see the [tutorial video](https://cwiki.apache.org/confluence/display/LENS/2015/07/13/20+Minute+video+demo+of+Apache+Lens+through+examples).
|
||||
|
||||
### Lens UI Service
|
||||
## Lens UI Service
|
||||
Lens also provides web UI service. Once the server starts up, you can open the service on http://serverhost:19999/index.html and browse. You may also check the structure that you made and use query easily here.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,8 +6,12 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Livy Interpreter for Apache Zeppelin
|
||||
Livy is an open source REST interface for interacting with Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in YARN.
|
||||
# Livy Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Livy](http://livy.io/) is an open source REST interface for interacting with Spark from anywhere. It supports executing snippets of code or programs in a Spark context that runs locally or in YARN.
|
||||
|
||||
* Interactive Scala, Python and R shells
|
||||
* Batch submissions in Scala, Java, Python
|
||||
|
|
@ -16,13 +20,12 @@ Livy is an open source REST interface for interacting with Spark from anywhere.
|
|||
* Does not require any code change to your programs
|
||||
|
||||
### Requirements
|
||||
|
||||
Additional requirements for the Livy interpreter are:
|
||||
|
||||
* Spark 1.3 or above.
|
||||
* Livy server.
|
||||
|
||||
### Configuration
|
||||
## Configuration
|
||||
We added some common configurations for spark, and you can set any configuration you want.
|
||||
This link contains all spark configurations: http://spark.apache.org/docs/latest/configuration.html#available-properties.
|
||||
And instead of starting property with `spark.` it should be replaced with `livy.spark.`.
|
||||
|
|
@ -101,8 +104,6 @@ Example: `spark.master` to `livy.spark.master`
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
|
||||
## How to use
|
||||
Basically, you can use
|
||||
|
||||
|
|
@ -136,7 +137,7 @@ hello("livy")
|
|||
When Zeppelin server is running with authentication enabled, then this interpreter utilizes Livy’s user impersonation feature i.e. sends extra parameter for creating and running a session ("proxyUser": "${loggedInUser}"). This is particularly useful when multi users are sharing a Notebook server.
|
||||
|
||||
|
||||
### Apply Zeppelin Dynamic Forms
|
||||
## Apply Zeppelin Dynamic Forms
|
||||
You can leverage [Zeppelin Dynamic Form]({{BASE_PATH}}/manual/dynamicform.html). You can use both the `text input` and `select form` parameterization features.
|
||||
|
||||
```
|
||||
|
|
@ -159,7 +160,7 @@ The session would have timed out, you may need to restart the interpreter.
|
|||
|
||||
> Blacklisted configuration values in session config: spark.master
|
||||
|
||||
edit `conf/spark-blacklist.conf` file in livy server and comment out `#spark.master` line.
|
||||
Edit `conf/spark-blacklist.conf` file in livy server and comment out `#spark.master` line.
|
||||
|
||||
if you choose to work on livy in `apps/spark/java` directory in https://github.com/cloudera/hue ,
|
||||
copy `spark-user-configurable-options.template` to `spark-user-configurable-options.conf` file in livy server and comment out `#spark.master`
|
||||
If you choose to work on livy in `apps/spark/java` directory in [https://github.com/cloudera/hue](https://github.com/cloudera/hue),
|
||||
copy `spark-user-configurable-options.template` to `spark-user-configurable-options.conf` file in livy server and comment out `#spark.master`.
|
||||
|
|
|
|||
|
|
@ -6,9 +6,11 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Markdown Interpreter for Apache Zeppelin
|
||||
# Markdown Interpreter for Apache Zeppelin
|
||||
|
||||
### Overview
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Markdown](http://daringfireball.net/projects/markdown/) is a plain text formatting syntax designed so that it can be converted to HTML.
|
||||
Zeppelin uses markdown4j. For more examples and extension support, please checkout [here](https://code.google.com/p/markdown4j/).
|
||||
In Zeppelin notebook, you can use ` %md ` in the beginning of a paragraph to invoke the Markdown interpreter and generate static html from Markdown plain text.
|
||||
|
|
@ -17,7 +19,7 @@ In Zeppelin, Markdown interpreter is enabled by default.
|
|||
|
||||
<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/markdown-interpreter-setting.png" width="60%" />
|
||||
|
||||
### Example
|
||||
## Example
|
||||
The following example demonstrates the basic usage of Markdown in a Zeppelin notebook.
|
||||
|
||||
<img src="{{BASE_PATH}}/assets/themes/zeppelin/img/docs-img/markdown-example.png" width="70%" />
|
||||
|
|
|
|||
|
|
@ -6,7 +6,12 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
# PostgreSQL, HAWQ Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Important Notice
|
||||
|
||||
Postgresql Interpreter will be deprecated and merged into JDBC Interpreter. You can use Postgresql by using JDBC Interpreter with same functionality. See the example below of settings and dependencies.
|
||||
|
||||
### Properties
|
||||
|
|
@ -44,10 +49,19 @@ Postgresql Interpreter will be deprecated and merged into JDBC Interpreter. You
|
|||
<td></td>
|
||||
</tr>
|
||||
</table>
|
||||
---
|
||||
|
||||
----
|
||||
## Overview
|
||||
|
||||
## PostgreSQL, HAWQ Interpreter for Apache Zeppelin
|
||||
[<img align="right" src="http://img.youtube.com/vi/wqXXQhJ5Uk8/0.jpg" alt="zeppelin-view" hspace="10" width="250"></img>](https://www.youtube.com/watch?v=wqXXQhJ5Uk8)
|
||||
|
||||
This interpreter seamlessly supports the following SQL data processing engines:
|
||||
|
||||
* [PostgreSQL](http://www.postgresql.org/) - OSS, Object-relational database management system (ORDBMS)
|
||||
* [Apache HAWQ](http://pivotal.io/big-data/pivotal-hawq) - Powerful [Open Source](https://wiki.apache.org/incubator/HAWQProposal) SQL-On-Hadoop engine.
|
||||
* [Greenplum](http://pivotal.io/big-data/pivotal-greenplum-database) - MPP database built on open source PostgreSQL.
|
||||
|
||||
This [Video Tutorial](https://www.youtube.com/watch?v=wqXXQhJ5Uk8) illustrates some of the features provided by the `Postgresql Interpreter`.
|
||||
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
|
|
@ -62,17 +76,7 @@ Postgresql Interpreter will be deprecated and merged into JDBC Interpreter. You
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
[<img align="right" src="http://img.youtube.com/vi/wqXXQhJ5Uk8/0.jpg" alt="zeppelin-view" hspace="10" width="250"></img>](https://www.youtube.com/watch?v=wqXXQhJ5Uk8)
|
||||
|
||||
This interpreter seamlessly supports the following SQL data processing engines:
|
||||
|
||||
* [PostgreSQL](http://www.postgresql.org/) - OSS, Object-relational database management system (ORDBMS)
|
||||
* [Apache HAWQ](http://pivotal.io/big-data/pivotal-hawq) - Powerful [Open Source](https://wiki.apache.org/incubator/HAWQProposal) SQL-On-Hadoop engine.
|
||||
* [Greenplum](http://pivotal.io/big-data/pivotal-greenplum-database) - MPP database built on open source PostgreSQL.
|
||||
|
||||
This [Video Tutorial](https://www.youtube.com/watch?v=wqXXQhJ5Uk8) illustrates some of the features provided by the `Postgresql Interpreter`.
|
||||
|
||||
### Create Interpreter
|
||||
## Create Interpreter
|
||||
By default Zeppelin creates one `PSQL` instance. You can remove it or create new instances.
|
||||
|
||||
Multiple PSQL instances can be created, each configured to the same or different backend databases. But over time a `Notebook` can have only one PSQL interpreter instance `bound`. That means you _cannot_ connect to different databases in the same `Notebook`. This is a known Zeppelin limitation.
|
||||
|
|
@ -81,10 +85,10 @@ To create new PSQL instance open the `Interpreter` section and click the `+Creat
|
|||
|
||||
> Note: The `Name` of the instance is used only to distinct the instances while binding them to the `Notebook`. The `Name` is irrelevant inside the `Notebook`. In the `Notebook` you must use `%psql.sql` tag.
|
||||
|
||||
### Bind to Notebook
|
||||
## Bind to Notebook
|
||||
In the `Notebook` click on the `settings` icon in the top right corner. The select/deselect the interpreters to be bound with the `Notebook`.
|
||||
|
||||
### Configuration
|
||||
## Configuration
|
||||
You can modify the configuration of the PSQL from the `Interpreter` section. The PSQL interpreter expenses the following properties:
|
||||
|
||||
<table class="table-configuration">
|
||||
|
|
@ -120,12 +124,12 @@ You can modify the configuration of the PSQL from the `Interpreter` section. Th
|
|||
</tr>
|
||||
</table>
|
||||
|
||||
### How to use
|
||||
## How to use
|
||||
```
|
||||
Tip: Use (CTRL + .) for SQL auto-completion.
|
||||
```
|
||||
|
||||
#### DDL and SQL commands
|
||||
### DDL and SQL commands
|
||||
Start the paragraphs with the full `%psql.sql` prefix tag! The short notation: `%psql` would still be able run the queries but the syntax highlighting and the auto-completions will be disabled.
|
||||
|
||||
You can use the standard CREATE / DROP / INSERT commands to create or modify the data model:
|
||||
|
|
@ -154,7 +158,7 @@ select count(*) from mytable;
|
|||
select * from mytable;
|
||||
```
|
||||
|
||||
#### PSQL command line tools
|
||||
### PSQL command line tools
|
||||
Use the Shell Interpreter (`%sh`) to access the command line [PSQL](http://www.postgresql.org/docs/9.4/static/app-psql.html) interactively:
|
||||
|
||||
```bash
|
||||
|
|
@ -179,7 +183,7 @@ This will produce output like this:
|
|||
retail_demo | gpadmin
|
||||
```
|
||||
|
||||
#### Apply Zeppelin Dynamic Forms
|
||||
### Apply Zeppelin Dynamic Forms
|
||||
You can leverage [Zeppelin Dynamic Form](../manual/dynamicform.html) inside your queries. You can use both the `text input` and `select form` parametrization features
|
||||
|
||||
```sql
|
||||
|
|
@ -191,7 +195,7 @@ ORDER BY count ${order=DESC,DESC|ASC}
|
|||
LIMIT ${limit=10};
|
||||
```
|
||||
|
||||
#### Example HAWQ PXF/HDFS Tables
|
||||
### Example HAWQ PXF/HDFS Tables
|
||||
Create HAWQ external table that read data from tab-separated-value data in HDFS.
|
||||
|
||||
```sql
|
||||
|
|
@ -209,5 +213,5 @@ And retrieve content
|
|||
select * from retail_demo.payment_methods_pxf
|
||||
```
|
||||
|
||||
### Auto-completion
|
||||
## Auto-completion
|
||||
The PSQL Interpreter provides a basic auto-completion functionality. On `(Ctrl+.)` it list the most relevant suggestions in a pop-up window. In addition to the SQL keyword the interpreter provides suggestions for the Schema, Table, Column names as well.
|
||||
|
|
|
|||
|
|
@ -6,7 +6,9 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Python 2 & 3 Interpreter for Apache Zeppelin
|
||||
# Python 2 & 3 Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Configuration
|
||||
<table class="table-configuration">
|
||||
|
|
@ -63,8 +65,6 @@ print (z.select("f1",[("o1","1"),("o2","2")],"2"))
|
|||
print("".join(z.checkbox("f3", [("o1","1"), ("o2","2")],["1"])))
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Zeppelin features not fully supported by the Python Interpreter
|
||||
|
||||
* Interrupt a paragraph execution (`cancel()` method) is currently only supported in Linux and MacOs. If interpreter runs in another operating system (for instance MS Windows) , interrupt a paragraph will close the whole interpreter. A JIRA ticket ([ZEPPELIN-893](https://issues.apache.org/jira/browse/ZEPPELIN-893)) is opened to implement this feature in a next release of the interpreter.
|
||||
|
|
@ -105,7 +105,6 @@ rates = pd.read_csv("bank.csv", sep=";")
|
|||
z.show(rates)
|
||||
```
|
||||
|
||||
|
||||
## Technical description
|
||||
|
||||
For in-depth technical details on current implementation plese reffer [python/README.md](https://github.com/apache/zeppelin/blob/master/python/README.md)
|
||||
For in-depth technical details on current implementation plese reffer [python/README.md](https://github.com/apache/zeppelin/blob/master/python/README.md).
|
||||
|
|
|
|||
|
|
@ -6,17 +6,20 @@ group: manual
|
|||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
## Scalding Interpreter for Apache Zeppelin
|
||||
# Scalding Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
[Scalding](https://github.com/twitter/scalding) is an open source Scala library for writing MapReduce jobs.
|
||||
|
||||
### Building the Scalding Interpreter
|
||||
## Building the Scalding Interpreter
|
||||
You have to first build the Scalding interpreter by enable the **scalding** profile as follows:
|
||||
|
||||
```
|
||||
mvn clean package -Pscalding -DskipTests
|
||||
```
|
||||
|
||||
### Enabling the Scalding Interpreter
|
||||
## Enabling the Scalding Interpreter
|
||||
In a notebook, to enable the **Scalding** interpreter, click on the **Gear** icon,select **Scalding**, and hit **Save**.
|
||||
|
||||
<center>
|
||||
|
|
@ -27,7 +30,7 @@ In a notebook, to enable the **Scalding** interpreter, click on the **Gear** ico
|
|||
|
||||
</center>
|
||||
|
||||
### Configuring the Interpreter
|
||||
## Configuring the Interpreter
|
||||
|
||||
Scalding interpreter runs in two modes:
|
||||
|
||||
|
|
@ -65,9 +68,9 @@ For reducer estimation, you need to add something like:
|
|||
If you want to control the maximum number of open interpreters, you have to select "scoped" interpreter for note
|
||||
option and set max.open.instances argument.
|
||||
|
||||
### Testing the Interpreter
|
||||
## Testing the Interpreter
|
||||
|
||||
#### Local mode
|
||||
### Local mode
|
||||
|
||||
In example, by using the [Alice in Wonderland](https://gist.github.com/johnynek/a47699caa62f4f38a3e2) tutorial,
|
||||
we will count words (of course!), and plot a graph of the top 10 words in the book.
|
||||
|
|
@ -111,7 +114,7 @@ If you click on the icon for the pie chart, you should be able to see a chart li
|
|||
|
||||
|
||||
|
||||
#### HDFS mode
|
||||
### HDFS mode
|
||||
|
||||
**Test mode**
|
||||
|
||||
|
|
@ -146,7 +149,7 @@ a.toList
|
|||
|
||||
This command should create a map reduce job.
|
||||
|
||||
### Future Work
|
||||
## Future Work
|
||||
* Better user feedback (hadoop url, progress updates)
|
||||
* Ability to cancel jobs
|
||||
* Ability to dynamically load jars without restarting the interpreter
|
||||
|
|
|
|||
|
|
@ -7,8 +7,14 @@ group: manual
|
|||
{% include JB/setup %}
|
||||
|
||||
|
||||
## Spark Interpreter for Apache Zeppelin
|
||||
[Apache Spark](http://spark.apache.org) is supported in Zeppelin with
|
||||
# Spark Interpreter for Apache Zeppelin
|
||||
|
||||
<div id="toc"></div>
|
||||
|
||||
## Overview
|
||||
[Apache Spark](http://spark.apache.org) is a fast and general-purpose cluster computing system.
|
||||
It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs
|
||||
Apache Spark is supported in Zeppelin with
|
||||
Spark Interpreter group, which consists of five interpreters.
|
||||
|
||||
<table class="table-configuration">
|
||||
|
|
@ -200,13 +206,13 @@ Here are few examples:
|
|||
|
||||
* SPARK\_SUBMIT\_OPTIONS in conf/zeppelin-env.sh
|
||||
|
||||
export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.2.0 --jars /path/mylib1.jar,/path/mylib2.jar --files /path/mylib1.py,/path/mylib2.zip,/path/mylib3.egg"
|
||||
export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.2.0 --jars /path/mylib1.jar,/path/mylib2.jar --files /path/mylib1.py,/path/mylib2.zip,/path/mylib3.egg"
|
||||
|
||||
* SPARK_HOME/conf/spark-defaults.conf
|
||||
|
||||
spark.jars /path/mylib1.jar,/path/mylib2.jar
|
||||
spark.jars.packages com.databricks:spark-csv_2.10:1.2.0
|
||||
spark.files /path/mylib1.py,/path/mylib2.egg,/path/mylib3.zip
|
||||
spark.jars /path/mylib1.jar,/path/mylib2.jar
|
||||
spark.jars.packages com.databricks:spark-csv_2.10:1.2.0
|
||||
spark.files /path/mylib1.py,/path/mylib2.egg,/path/mylib3.zip
|
||||
|
||||
### 3. Dynamic Dependency Loading via %dep interpreter
|
||||
> Note: `%dep` interpreter is deprecated since v0.6.0.
|
||||
|
|
@ -344,7 +350,7 @@ select * from ${table=defaultTableName} where text like '%${search}%'
|
|||
To learn more about dynamic form, checkout [Dynamic Form](../manual/dynamicform.html).
|
||||
|
||||
|
||||
### Interpreter setting option.
|
||||
## Interpreter setting option
|
||||
|
||||
Interpreter setting can choose one of 'shared', 'scoped', 'isolated' option. Spark interpreter creates separate scala compiler per each notebook but share a single SparkContext in 'scoped' mode (experimental). It creates separate SparkContext per each notebook in 'isolated' mode.
|
||||
|
||||
|
|
@ -354,7 +360,7 @@ Logical setup with Zeppelin, Kerberos Key Distribution Center (KDC), and Spark o
|
|||
|
||||
<img src="../assets/themes/zeppelin/img/docs-img/kdc_zeppelin.png">
|
||||
|
||||
####Configuration Setup
|
||||
### Configuration Setup
|
||||
|
||||
1. On the server that Zeppelin is installed, install Kerberos client modules and configuration, krb5.conf.
|
||||
This is to make the server communicate with KDC.
|
||||
|
|
|
|||
Loading…
Reference in a new issue