Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

merge with Ayoung's

Browse source
This commit is contained in:
astroshim 2016-08-11 20:04:22 +09:00
commit 4c8d72def0
2 changed files with 12 additions and 13 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

BIN
docs/assets/themes/zeppelin/img/docs-img/zeppelin_yarn_conf.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 73 KiB

After

Width:  |  Height:  |  Size: 212 KiB

25
docs/install/spark_cluster_mode.md
View file

@ -1,7 +1,7 @@
---
layout: page
title: "Apache Zeppelin on Spark cluster mode"
description: ""
description: "This document will guide you how you can build and configure the environment on 3 types of Spark cluster manager with Apache Zeppelin using docker scripts."
group: install
---
<!--
@ -56,12 +56,12 @@ spark_standalone bash;
```
### 3. Configure Spark interpreter in Zeppelin
Set Spark master as `spark://localhost:7077` in Zeppelin **Interpreters** setting page.
Set Spark master as `spark://<hostname>:7077` in Zeppelin **Interpreters** setting page.
<img src="../assets/themes/zeppelin/img/docs-img/standalone_conf.png" />
### 4. Run Zeppelin with Spark interpreter
After running single paragraph with Spark interpreter in Zeppelin, browse `https://localhost:8080` and check whether Spark cluster is running well or not.
After running single paragraph with Spark interpreter in Zeppelin, browse `https://<hostname>:8080` and check whether Spark cluster is running well or not.
<img src="../assets/themes/zeppelin/img/docs-img/spark_ui.png" />
@ -115,29 +115,28 @@ docker run -it \
You can simply verify the processes of Spark and YARN is running well in Docker with below command.
```
ps -ef
ps -ef | grep spark
```
and also You can see the hdfs web ui on `http://localhost:50070/` and yarn ui on `http://localhost:8088/cluster` and spark ui on `http://localhost:8080/`.
You can also check each application web UI for HDFS on `http://<hostname>:50070/`, YARN on `http://<hostname>:8088/cluster` and Spark on `http://<hostname>:8080/`.
### 4. Configure Spark interpreter in Zeppelin
Set following configurations to the `zeppelin-env.sh`.
Set following configurations to `conf/zeppelin-env.sh`.
```
export MASTER=yarn-client
export HADOOP_CONF_DIR=[PATH OF HADOOP CONF]
export SPARK_HOME=[PATH OF SPARK HOME]
export HADOOP_CONF_DIR=[your_hadoop_conf_path]
export SPARK_HOME=[your_spark_home_path]
```
Hadoop configuration path(HADOOP_CONF_DIR) is the `scripts/docker/spark-cluster-managers/spark_yarn_cluster/hdfs_conf`.
Please make sure the spark master as `yarn-client` in Zeppelin **Interpreters** setting page.
`HADOOP_CONF_DIR`(Hadoop configuration path) is defined in `/scripts/docker/spark-cluster-managers/spark_yarn_cluster/hdfs_conf`.
Don't forget to set Spark `master` as `yarn-client` in Zeppelin **Interpreters** setting page like below.
<img src="../assets/themes/zeppelin/img/docs-img/zeppelin_yarn_conf.png" />
### 5. Run Zeppelin with Spark interpreter
After running single paragraph with Spark interpreter in Zeppelin, browse `http://localhost:8088/cluster/apps` and check zeppelin application running well or not.
After running a single paragraph with Spark interpreter in Zeppelin, browse `http://<hostname>:8088/cluster/apps` and check Zeppelin application is running well or not.
<img src="../assets/themes/zeppelin/img/docs-img/yarn_applications.png" />