Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

fixed felixcheung pointed out.

Browse source
This commit is contained in:
astroshim 2016-08-11 19:59:43 +09:00
parent 633c930af7
commit 8c62cf138b
6 changed files with 11 additions and 14 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/_includes/themes/zeppelin/_navigation.html
View file

@ -105,7 +105,7 @@
<li class="title"><span><b>Advanced</b><span></li>
<li><a href="{{BASE_PATH}}/install/virtual_machine.html">Zeppelin on Vagrant VM</a></li>
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#spark-standalone-mode">Zeppelin on Spark Cluster Mode (Standalone)</a></li>
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#spark-standalone-mode">Zeppelin on Spark Cluster Mode (Yarn)</a></li>
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#spark-standalone-mode">Zeppelin on Spark Cluster Mode (YARN)</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Contibute</b><span></li>
<li><a href="{{BASE_PATH}}/development/writingzeppelininterpreter.html">Writing Zeppelin Interpreter</a></li>

2
docs/index.md
View file

@ -170,7 +170,7 @@ Join to our [Mailing list](https://zeppelin.apache.org/community.html) and repor
* Advanced
* [Apache Zeppelin on Vagrant VM](./install/virtual_machine.html)
* [Zeppelin on Spark Cluster Mode (Standalone via Docker)](./install/spark_cluster_mode.html#spark-standalone-mode)
* [Zeppelin on Spark Cluster Mode (Yarn via Docker)](./install/spark_cluster_mode.html#spark-yarn-mode)
* [Zeppelin on Spark Cluster Mode (YARN via Docker)](./install/spark_cluster_mode.html#spark-yarn-mode)
* Contribute
* [Writing Zeppelin Interpreter](./development/writingzeppelininterpreter.html)
* [Writing Zeppelin Application (Experimental)](./development/writingzeppelinapplication.html)

8
docs/install/spark_cluster_mode.md
View file

@ -72,8 +72,8 @@ ps -ef | grep spark
```
## Spark on Yarn mode
You can simply set up [Spark on Yarn](http://spark.apache.org/docs/latest/running-on-yarn.html) docker environment with below steps.
## Spark on YARN mode
You can simply set up [Spark on YARN](http://spark.apache.org/docs/latest/running-on-yarn.html) docker environment with below steps.
> **Note :** Since Apache Zeppelin and Spark use same `8080` port for their web UI, you might need to change `zeppelin.server.port` in `conf/zeppelin-site.xml`.
@ -111,9 +111,9 @@ docker run -it \
spark_yarn bash;
```
### 3. Verify running Spark on Yarn.
### 3. Verify running Spark on YARN.
You can simply verify the processes of Spark and Yarn is running well in Docker with below command.
You can simply verify the processes of Spark and YARN is running well in Docker with below command.
```

1
scripts/docker/spark-cluster-managers/spark_standalone/Dockerfile
View file

@ -13,7 +13,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM centos:centos6
MAINTAINER hsshim@nflabs.com
ENV SPARK_PROFILE 1.6
ENV SPARK_VERSION 1.6.2

8
scripts/docker/spark-cluster-managers/spark_yarn_cluster/Dockerfile
View file

@ -13,10 +13,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
FROM centos:centos6
MAINTAINER hsshim@nflabs.com
ENV SPARK_PROFILE 1.6
ENV SPARK_VERSION 1.6.1
ENV SPARK_PROFILE 2.0
ENV SPARK_VERSION 2.0.0
ENV HADOOP_PROFILE 2.3
ENV HADOOP_VERSION 2.3.0
@ -73,7 +72,6 @@ RUN rm /usr/local/hadoop/lib/native/*
RUN curl -Ls http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64.tar|tar -x -C /usr/local/hadoop/lib/native/
# install spark
#RUN curl -s https://www.apache.org/dist/spark/spark-1.6.1/spark-1.6.1-bin-hadoop2.3.tgz | tar -xz -C /usr/local/
RUN curl -s http://archive.apache.org/dist/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE.tgz | tar -xz -C /usr/local/
RUN cd /usr/local && ln -s spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE spark
ENV SPARK_HOME /usr/local/spark
@ -106,4 +104,4 @@ EXPOSE 8030 8031 8032 8033 8040 8042 8088
#spark
EXPOSE 8080 7077 8888 8081
ENTRYPOINT ["/etc/entrypoint.sh"]
ENTRYPOINT ["/etc/entrypoint.sh"]

4
scripts/docker/spark-cluster-managers/spark_yarn_cluster/entrypoint.sh
View file

@ -32,7 +32,7 @@ service sshd start
$HADOOP_PREFIX/sbin/start-dfs.sh
$HADOOP_PREFIX/sbin/start-yarn.sh
$HADOOP_PREFIX/bin/hdfs dfsadmin -safemode leave && $HADOOP_PREFIX/bin/hdfs dfs -put $SPARK_HOME-1.6.1-bin-hadoop2.3/lib /spark
$HADOOP_PREFIX/bin/hdfs dfsadmin -safemode leave && $HADOOP_PREFIX/bin/hdfs dfs -put $SPARK_HOME-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE/lib /spark
# start spark
export SPARK_MASTER_OPTS="-Dspark.driver.port=7001 -Dspark.fileserver.port=7002
@ -57,4 +57,4 @@ then
/usr/sbin/sshd -D -d
else
/bin/bash -c "$*"
fi
fi