mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
Adding docs for the Scalding interpreter
This commit is contained in:
parent
8004b3945e
commit
d4cf3085cd
6 changed files with 73 additions and 0 deletions
|
|
@ -45,6 +45,7 @@
|
|||
<li><a href="{{BASE_PATH}}/interpreter/lens.html">Lens</a></li>
|
||||
<li><a href="{{BASE_PATH}}/interpreter/markdown.html">Markdown</a></li>
|
||||
<li><a href="{{BASE_PATH}}/interpreter/postgresql.html">Postgresql, hawq</a></li>
|
||||
<li><a href="{{BASE_PATH}}/interpreter/scalding.html">Scalding</a></li>
|
||||
<li><a href="{{BASE_PATH}}/pleasecontribute.html">Shell</a></li>
|
||||
<li><a href="{{BASE_PATH}}/interpreter/spark.html">Spark</a></li>
|
||||
<li><a href="{{BASE_PATH}}/pleasecontribute.html">Tajo</a></li>
|
||||
|
|
|
|||
Binary file not shown.
|
After Width: | Height: | Size: 13 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
BIN
docs/assets/themes/zeppelin/img/docs-img/scalding-pie.png
Normal file
BIN
docs/assets/themes/zeppelin/img/docs-img/scalding-pie.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 96 KiB |
|
|
@ -41,6 +41,7 @@ limitations under the License.
|
|||
* [lens](./interpreter/lens.html)
|
||||
* [md](./interpreter/markdown.html)
|
||||
* [postgresql, hawq](./interpreter/postgresql.html)
|
||||
* [scalding](./interpreter/scalding.html)
|
||||
* [sh](./pleasecontribute.html)
|
||||
* [spark](./interpreter/spark.html)
|
||||
* [tajo](./pleasecontribute.html)
|
||||
|
|
|
|||
71
docs/interpreter/scalding.md
Normal file
71
docs/interpreter/scalding.md
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
layout: page
|
||||
title: "Scalding Interpreter"
|
||||
description: ""
|
||||
group: manual
|
||||
---
|
||||
{% include JB/setup %}
|
||||
|
||||
|
||||
## Scalding Interpreter for Apache Zeppelin
|
||||
[Scalding](https://github.com/twitter/scalding) is an open source Scala library for writing MapReduce jobs.
|
||||
|
||||
### Enabling the Scalding Interpreter
|
||||
|
||||
In a notebook, to enable the **Scalding** interpreter, click on the **Gear** icon,select **Scalding**, and hit **Save**.
|
||||
|
||||
<center>
|
||||

|
||||
|
||||

|
||||
</center>
|
||||
|
||||
### Configuring the Interpreter
|
||||
Zeppelin comes with a pre-configured Scalding interpreter in local mode, so you do not need to install anything.
|
||||
|
||||
### Testing the Interpreter
|
||||
|
||||
In example, by using the [Alice in Wonderland](https://gist.github.com/johnynek/a47699caa62f4f38a3e2) tutorial, we will count words (of course!), and plot a graph of the top 10 words in the book.
|
||||
|
||||
```
|
||||
%scalding
|
||||
|
||||
import scala.io.Source
|
||||
|
||||
// Get the Alice in Wonderland book from gutenberg.org:
|
||||
val alice = Source.fromURL("http://www.gutenberg.org/files/11/11.txt").getLines
|
||||
val aliceLineNum = alice.zipWithIndex.toList
|
||||
val alicePipe = TypedPipe.from(aliceLineNum)
|
||||
|
||||
// Now get a list of words for the book:
|
||||
val aliceWords = alicePipe.flatMap { case (text, _) => text.split("\\s+").toList }
|
||||
|
||||
// Now lets add a count for each word:
|
||||
val aliceWithCount = aliceWords.filterNot(_.equals("")).map { word => (word, 1L) }
|
||||
|
||||
// let's sum them for each word:
|
||||
val wordCount = aliceWithCount.group.sum
|
||||
|
||||
print ("Here are the top 10 words\n")
|
||||
val top10 = wordCount
|
||||
.groupAll
|
||||
.sortBy { case (word, count) => -count }
|
||||
.take(10)
|
||||
top10.dump
|
||||
|
||||
```
|
||||
```
|
||||
%scalding
|
||||
|
||||
val table = "words\t count\n" + top10.toIterator.map{case (k, (word, count)) => s"$word\t$count"}.mkString("\n")
|
||||
print("%table " + table)
|
||||
|
||||
```
|
||||
|
||||
If you click on the icon for the pie chart, you should be able to see a chart like this:
|
||||

|
||||
|
||||
### Current Status & Future Work
|
||||
The current implementation of the Scalding interpreter does not support canceling jobs, or fine-grained progress updates.
|
||||
|
||||
The pre-configured Scalding interpreter only supports Scalding in local mode. Hadoop mode for Scalding is currently unsupported, and will be future work (contributions welcome!).
|
||||
Loading…
Reference in a new issue