Elgato_dark/zeppelin
1
0
Fork 0
mirror of https://github.com/apache/zeppelin synced 2026-05-24 09:38:26 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
zeppelin/scio/README.md
Rafal Wojdyla f127237fb1 Closes [ZEPPELIN-1505] Add Scio interpreter 
### What is this PR for?

Closes #ZEPPELIN-1505. Adds Scio interpreter. Scio is a Scala DSL on top of Dataflow/Beam.
### What type of PR is it?

Improvement - ZEPPELIN-1505
### Todos
- [x] - test integration with zeppelin context (zeppelin context is too tightly coupled withs spark)
- [x] - what to do about code completion?
- [x] - add more tests?
- [x] - add helpers to display data
- [x] - add doc in `docs/interpreter/scio`
### What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-1505
### How should this be tested?

```
mvn -pl scio,zeppelin-display,zeppelin-interpreter -Dtest='org.apache.zeppelin.scio.*' -DfailIfNoTests=false test
```
### Screenshots

![GitHub Logo](http://i.imgur.com/w6Cp5kp.png)
### Questions:
- Does the licenses files need update? ~~no~~ yes
- Is there breaking changes for older versions? no
- Does this needs documentation? yes (included in the PR)

Author: Rafal Wojdyla <rav@spotify.com>

Closes #1471 from ravwojdyla/scio and squashes the following commits:

d6fbc4e [Rafal Wojdyla] Add runner doc links
7e6fdec [Rafal Wojdyla] Fix indentation
d8de7c8 [Rafal Wojdyla] Remove optional flink deps from Beam
508705f [Rafal Wojdyla] Remove duplicates
49cf0eb [Rafal Wojdyla] Add scio to beam group
cd79fc8 [Rafal Wojdyla] Add .bigquery cache to gitignore
b961791 [Rafal Wojdyla] Check the message content
1e30f76 [Rafal Wojdyla] Simplify SCollection implicits
3c519f1 [Rafal Wojdyla] Fix doc style
e9579d8 [Rafal Wojdyla] Clarify Context sharing + add docs about display helpers
327273e [Rafal Wojdyla] Add Zeppelin custom ContextAndArgs
0920fdd [Rafal Wojdyla] Remove obsolete deps
8f25f71 [Rafal Wojdyla] Upgrade scio to 0.2.4
3275185 [Rafal Wojdyla] Add license
bd4df5e [Rafal Wojdyla] Fix documentation style
dcbb197 [Rafal Wojdyla] Add documentation link
e635674 [Rafal Wojdyla] Add tests for DisplayHelpers
c0f8ccf [Rafal Wojdyla] Fix style and number of records for take
9dcc8ce [Rafal Wojdyla] Add display helpers for Tap[T] and Future[Tap[T]]
4014c81 [Rafal Wojdyla] Parse params generic params
0305a3c [Rafal Wojdyla] Style fix
a92494b [Rafal Wojdyla] Style + use `split` to support both scala 2.{10,11}
b884b72 [Rafal Wojdyla] Fix tests - add exception and check messages
99a7daa [Rafal Wojdyla] Progress should be 0
7b54e49 [Rafal Wojdyla] No need to override delegation for completion
12f0096 [Rafal Wojdyla] Clean up tests
93233a8 [Rafal Wojdyla] Rename argz param, fix style
0a3b49a [Rafal Wojdyla] Add Scio doc
61850d7 [Rafal Wojdyla] Add TableRow display helper + style
be252f8 [Rafal Wojdyla] Add avro display
89a2811 [Rafal Wojdyla] Add limit of display
9a21aa0 [Rafal Wojdyla] Add display helpers
6ff4e95 [Rafal Wojdyla] Inject argz to the Scio interpreter
570cfaa [Rafal Wojdyla] Add Scio interpreter tests
38abaf9 [Rafal Wojdyla] Add Scio interpreter
7b596ea [Rafal Wojdyla] Generalize SparkOutputStream
2016-11-14 11:00:15 -08:00

356 B
Raw Blame History

Scio interpreter for Apache Zeppelin

Raison d'être:

Provide Scio Interpreter for Zeppelin.

Build

mvn -pl zeppelin-interpreter,zeppelin-display,scio -DskipTests package

Test

mvn -pl scio,zeppelin-display,zeppelin-interpreter -Dtest='org.apache.zeppelin.scio.*' -DfailIfNoTests=false test