Commit graph

18 commits

Author SHA1 Message Date
Mina Lee
60089f0f58 [ZEPPELIN-1566] Make paragraph editable with double click
### What is this PR for?
This PR enables edit on double click for markdown/angular paragraph.  Users can change `editOnDblClick` field to be `false` by editting `interpreter/md/interpreter-setting.json` or  `conf/interpreter.json`. In the same context, users can set other type paragraphs to be editable on double click by setting `editOnDblClick` to be true.

This PR also fixes bug that syntax highlight doesn't work on pasted code.

### What type of PR is it?
Feature

### Todos
* [x] Create test
* [x] Update docs

### What is the Jira issue?
[ZEPPELIN-1566](https://issues.apache.org/jira/browse/ZEPPELIN-1566)

### How should this be tested?
1. Create new markdown interpreter
2. Create notebook and run markdown paragraph
3. Double click markdown paragraph to edit

### Screenshots (if appropriate)

**Edit on double click and hide editor on paragraph run**
![oct-20-2016 12-28-54](https://cloud.githubusercontent.com/assets/8503346/19545401/ca2a69a8-96c0-11e6-9e70-ca930fd7cc8e.gif)

**Syntax highlight on paste**

Before
![oct-20-2016 12-24-54](https://cloud.githubusercontent.com/assets/8503346/19545333/46a2f2a8-96c0-11e6-910a-2a216da5603b.gif)

After
![oct-20-2016 12-25-06](https://cloud.githubusercontent.com/assets/8503346/19545338/4d3bb852-96c0-11e6-8dc4-ff839234876a.gif)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes

Author: Mina Lee <minalee@apache.org>

Closes #1540 from minahlee/ZEPPELIN-1566 and squashes the following commits:

89ea628 [Mina Lee] Remove redundant code
4ffc446 [Mina Lee] Enable focus first paragraph on note reload
c566706 [Mina Lee] Check null condition of editor setting and return default setting
38d5e35 [Mina Lee] Update document about how to make paragraph editable on double click
b42039e [Mina Lee] Add integration test for editOnDblClick
0a26207 [Mina Lee] Add editOnDblClick field in interpreter-setting.json
93abe6a [Mina Lee] Make paragraph editable on doubleclick if editOnDblClick set true in interpreter-setting.json
359dc0b [Mina Lee] Split getAndSetEditorSetting method into getEditorSetting and setEditorLanguage method
2016-10-29 10:30:22 +09:00
Mina Lee
a3ca800311 [ZEPPELIN-1026] set syntax highlight based on default bound interpreter
### What is this PR for?
This is complete work of #1148. Comments and tasks on #1148 has been handled in this PR.
- Add syntax language information in `interpreter-setting.json`
- When user type `%replName` in paragraph, back-end check if the interpreter name with `replName` exists, and return language information to front-end if it does
- If user doesn't specify `%replName`, default interpreter's language will be used
- Using alias name for paragraph syntax highlight

### What type of PR is it?
[Bug Fix | Improvement]

### What is the Jira issue?
[ZEPPELIN-1026](https://issues.apache.org/jira/browse/ZEPPELIN-1026)

### How should this be tested?
1. Create new note and make markdown interpreter to be default.
2. See if markdown syntax is applied.

### Screenshots (if appropriate)
#### Case 1. When the default interpreter set to python interpreter.
**Before**
Has `scala` as syntax highlight language when %python is not set.
<img width="665" alt="screen shot 2016-07-07 at 10 46 20 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655312/af67a302-4494-11e6-949e-793ad0515d7a.png">

**After**
Has `python` as syntax highlight language even when %python is not set.
<img width="666" alt="screen shot 2016-07-07 at 10 44 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/16655248/769d8ba4-4494-11e6-9b3c-dc5e026e9c53.png">

#### Case 2. When use alias name as repl name.
**Before**
<img width="742" alt="screen shot 2016-09-08 at 4 22 39 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353471/620c5ede-75e2-11e6-9d01-0726bc900dc0.png">

**After**
<img width="741" alt="screen shot 2016-09-08 at 4 34 57 pm" src="https://cloud.githubusercontent.com/assets/8503346/18353487/6cdaa406-75e2-11e6-831a-08e0fa3a85d8.png">

### Further possible improvements
There are still several cases that Zeppelin doesn't handle syntax highlight well. These can be handled with another jira ticket/PR.
1. When default bound interpreter changes, syntax highlight is not changed accordingly
2. When copy/paste code, syntax highlight won't be applied properly since Zeppelin only checks changes when cursor is in first line.

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes(for creating new interpreter)

Author: Mina Lee <minalee@apache.org>

Closes #1415 from minahlee/ZEPPELIN-1026 and squashes the following commits:

c66fb0e [Mina Lee] Move getEditorSetting to InterpreterFactory class
2d56222 [Mina Lee] Add description about default syntax highlight in doc
08ccad9 [Mina Lee] Fix test
0874522 [Mina Lee] Change condition for triggering 'getAndSetEditorSetting' to reduce front-end <-> back-end communication
9e4f2e9 [Mina Lee] Change the way to read interpreter language from interpreter-setting.json after #1145
75543b3 [Mina Lee] Add test
565d9d0 [Mina Lee] [DOC] Setting syntax highlight when writing new interpreter
20132ca [Mina Lee] Get paragraph editor mode from backend
52f4207 [Mina Lee] Align comments for readability
26cbbb8 [Mina Lee] Add editor field
2016-09-23 16:47:26 +09:00
Damien CORNEAU
56bbff3d3d [ZEPPELIN-1414] Remove CONTRIBUTING.md files
### What is this PR for?
Following the PR #1356, we now have the contribution documentation to on Apache Zeppelin Website.

We can now remove the contributing files from the repository, and replace the links to that documentation.

### What type of PR is it?
Documentation

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1414

### How should this be tested?
Just check if there is other CONTRIBUTING.md files and if there is other links to replace

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Damien CORNEAU <corneadoug@gmail.com>

Closes #1406 from corneadoug/ZEPPELIN-1414 and squashes the following commits:

4347043 [Damien CORNEAU] change a how to contribute link to the webiste in the docs
faf3e11 [Damien CORNEAU] Make links to contribution material point to the website
cb4b5f7 [Damien CORNEAU] Remove contributing files from RAT
7810f57 [Damien CORNEAU] Remove contributing files
2016-09-09 08:04:10 +09:00
AhyoungRyu
85d4df4f0c [ZEPPELIN-1219] Add searching feature to Zeppelin docs site
### What is this PR for?
As more and more document pages are added, it's really hard to find specific pages. So I added searching feature to Zeppelin documentation site([jekyll](https://jekyllrb.com/) based site) using [lunr.js](http://lunrjs.com/).

 - **How does it work?**

  I created [`search_data.json`](6e02423f54/docs/search_data.json) which is used for docs info template. `lunr.js` combines all of the text from all of the docs in `docs/` into `_site/search_data.json`. It looks like below.
![screen shot 2016-08-03 at 4 49 59 am](https://cloud.githubusercontent.com/assets/10060731/17342828/f2908be8-5935-11e6-8eee-b189677c0531.png)
All the info are comes from [Jekyll YAML front matter](https://jekyllrb.com/docs/frontmatter/) variables. (i.e. title, group, description.. that's why I rewrote all docs' title and description.)
[search.js](6e02423f54/docs/assets/themes/zeppelin/js/search.js) will do this job using this data!

### What type of PR is it?
Improvement & Feature

### Todos
* [x] - Keep consistency for all docs pages' `Title`
* [x] - Add some overview sentences to all docs pages' `Description` section (this will be used as the result preview)
* [x] - Add apache license header to all docs page (some pages are missing the license header currently)
* [x] - Add LICENSE for `lunr.min.js`

### What is the Jira issue?
[ZEPPELIN-1219](https://issues.apache.org/jira/browse/ZEPPELIN-1219)

### How should this be tested?
1. Apply this patch and build `ZEPPELIN_HOME/docs` dir -> please see [docs/README.md#build-documentation](https://github.com/apache/zeppelin/tree/master/docs#build-documentation)
2. Click `search` icon in navbar and go to `search.html` page
3. Type anything you want to search in the search bar (i.e. type `python`, `spark`, `dynamic` ... )

### Screenshots (if appropriate)
![screen shot 2016-08-03 at 4 42 28 pm](https://cloud.githubusercontent.com/assets/10060731/17357851/d092e2ca-5999-11e6-9917-a3d4113e6e43.png)

![search](https://cloud.githubusercontent.com/assets/10060731/17357828/b2486cd6-5999-11e6-873b-121fac033b03.gif)

### Questions:
* Does the licenses files need update? Yes, for `lunr.min.js`
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: AhyoungRyu <fbdkdud93@hanmail.net>

Closes #1266 from AhyoungRyu/ZEPPELIN-1219 and squashes the following commits:

7ec8854 [AhyoungRyu] Modify 'no result' sentence
91b71a7 [AhyoungRyu] Remove Apache license header since JSON doesn't allow comment
34afd5d [AhyoungRyu] Add Apache license header to search_data.json
6784282 [AhyoungRyu] Minor search page UI update
0389d28 [AhyoungRyu] Make index.md not to be searched
9f1ba42 [AhyoungRyu] Disable enterkey press & change icon
bd4956a [AhyoungRyu] Add docs.js & search.js to exclude list in pom.xml
624b051 [AhyoungRyu] Add Apache license header to search.js
1381152 [AhyoungRyu] Fix search result skipping issue
6e775f5 [AhyoungRyu] Make pleasecontribute.md not to be searched
ee11136 [AhyoungRyu] Fix some typos
fa01299 [AhyoungRyu] Refine 'description' in some docs as @bzz suggested
da0cff9 [AhyoungRyu] Exclude lunr.min.js
36ba7f1 [AhyoungRyu] Add lunr.min.js license info
f6a05a6 [AhyoungRyu] Apply css style for the search results
68eb997 [AhyoungRyu] Attach 'Apache Zeppelin ZEPPELIN_VERSION Documentation: ' to title
d908c37 [AhyoungRyu] Add searching page
a951fa6 [AhyoungRyu] Add search icon to navbar
0688a79 [AhyoungRyu] Keep consistency all docs' front matter for the right search result
040f532 [AhyoungRyu] Add template for storing docs info based on jekyll front matter
0705bd6 [AhyoungRyu] Add js files: lunr.min.js & search.js
2016-08-10 12:39:22 +09:00
AhyoungRyu
16b320ff92 [DOC][ZEPPELIN-1209] Remove a useless sentence about default interpreter in docs
### What is this PR for?
As new interpreter registration mechanism which was started in [ZEPPELIN-804](https://issues.apache.org/jira/browse/ZEPPELIN-804), we can't set default interpreter anymore using `zeppelin-site.xml` as described in [https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html#apache-zeppelin-configuration](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html#apache-zeppelin-configuration) (see `zeppelin.interpreters` property description in the configuration table). So we need to remove the related contents in Zeppelin docs site.

Below pages will be updated:
 - [https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html#apache-zeppelin-configuration](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html#apache-zeppelin-configuration)
 - [https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/development/writingzeppelininterpreter.html#060-and-later](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/development/writingzeppelininterpreter.html#060-and-later)

### What type of PR is it?
Documentation

### What is the Jira issue?
[ZEPPELIN-1209](https://issues.apache.org/jira/browse/ZEPPELIN-1209)

### How should this be tested?
No need to test. Just removed two sentences about the default interpreter setting.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: AhyoungRyu <fbdkdud93@hanmail.net>

Closes #1207 from AhyoungRyu/ZEPPELIN-1209 and squashes the following commits:

61108f4 [AhyoungRyu] Add a sentence about the deprecation
9f44f22 [AhyoungRyu] Remove useless sentence about default interpreter in docs
2016-08-03 17:51:38 +09:00
Zak Hassan
4e6c7990cc [ZEPPELIN-1202] Documentation typo under writing interpreter docs
### What is this PR for?
To correct typo found in documentation

### What type of PR is it?
 Documentation

### Todos
* [x] - Replace sentence with incorrect english docs with correction.

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-1202

### How should this be tested?
Build docs and see correct docs on `https://zeppelin.apache.org/docs/latest/development/writingzeppelininterpreter.html`
### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no

Author: Zak Hassan <zak.hassan1010@gmail.com>

Closes #1198 from zmhassan/ZEPPELIN-1202 and squashes the following commits:

5b3d708 [Zak Hassan] Making correction to Json snippet inside docs.
f3e2150 [Zak Hassan] Corrected typo of interpreter-setting.json file should not be interpareter-setting.json
8d6e068 [Zak Hassan] Documentation typo creation.
2016-07-19 16:00:26 +09:00
AhyoungRyu
5975125f18 [ZEPPELIN-1018] Apply auto "Table of Contents" generator to Zeppelin docs website
### What is this PR for?
I added auto TOC(Table of Contents) generator for Zeppelin documentation website. TOC can help people looking through whole contents at a glance and finding what they want quickly.

I just added `<div id="toc"></div>`  to the each documentation header. [`toc`](https://github.com/apache/zeppelin/compare/master...AhyoungRyu:ZEPPELIN-1018?expand=1#diff-85af09fb498a5667ea455391533f945dR3)  recognize `<h2>` & `<h3>` as a title in the docs  and it automatically generate TOC. So I set a rule for this work. (I'll write this rule on `docs/CONTRIBUTING.md` or [docs/howtocontributewebsite](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/howtocontributewebsite.html)).

```
# Level-1 Heading  <- Use only for the main title of the page
## Level-2 Heading <- Start with this one
### Level-3 heading <- Only use this one for child of Level-2

toc only recognize Level-2 & Level-3
```

Please see the below attached screenshot image.

### What type of PR is it?
Improvement & Documentation

### Todos
* [x] - Add TOC generator
* [x] - Apply TOC(`<div id="toc"></div>`) to every documentation and reorganize each headers(apply the above rule)
* [x] - Fix some broken code block in several docs
* [x] - Apply TOC to `r.md` (Currently R docs has some duplicated info since [this one](d5e87fb8ba) and [this one](7d6cc7e991) )
* [x] - Apply TOC to `install.md` after #1010 merged
* [x] - Apply TOC to `interpreterinstallation.md` after #1042 merged

### What is the Jira issue?
[ZEPPELIN-1018](https://issues.apache.org/jira/browse/ZEPPELIN-1018)

### How should this be tested?
1. Apply this patch and build `docs/` with [this guide](https://github.com/apache/zeppelin/tree/master/docs#build-documentation)
2.  Visit some docs page. Then you can see TOC in the header of page.

### Screenshots (if appropriate)
 - Automatically generated TOC in Spark interpreter docs page
<img width="831" alt="screen shot 2016-06-16 at 9 37 18 pm" src="https://cloud.githubusercontent.com/assets/10060731/16140902/945b9c7a-340a-11e6-91f3-b6174738bed0.png">

### Questions:
* Does the licenses files need update?
No. Actually I used [jekyll-table-of-contents#copyright](https://github.com/ghiculescu/jekyll-table-of-contents#copyright). But I don't need to add a license for this :)
* Is there breaking changes for older versions? No
* Does this needs documentation? Maybe

Author: AhyoungRyu <fbdkdud93@hanmail.net>

Closes #1031 from AhyoungRyu/ZEPPELIN-1018 and squashes the following commits:

e66397b [AhyoungRyu] Apply TOC to interpreterinstallation.md
009579b [AhyoungRyu] Add more info to 'What is the next?' in install.md
04cf501 [AhyoungRyu] Revert 'where to start' section
b7cbe5f [AhyoungRyu] Fix typo
cf0911c [AhyoungRyu] Rename license file
388f35a [AhyoungRyu] Add jekyll-table-of-contents license info
6394c70 [AhyoungRyu] Fix image path in python.md
d00e4b1 [AhyoungRyu] Move interpreter/screenshot/ -> asset/../img/docs-img/
3ffb383 [AhyoungRyu] Remove duplicated info in r.md & apply toc
a03ca99 [AhyoungRyu] Exclude toc.js from pom.xml
3fae7df [AhyoungRyu] Apply auto generated toc to install.md
d114a9d [AhyoungRyu] Address @felixcheung feedback
6a788fe [AhyoungRyu] Resize TOC tab indent
6760c00 [AhyoungRyu] Apply auto TOC to all of docs under docs/storage/
fbde57f [AhyoungRyu] Apply auto TOC to all of docs under docs/quickstart/
db76eb6 [AhyoungRyu] Apply auto TOC to all of docs under docs/install/
f35db47 [AhyoungRyu] Apply auto TOC to all of docs under docs/displaysystem/
b05365f [AhyoungRyu] Apply auto TOC to all of docs under docs/rest-api/
163691c [AhyoungRyu] Apply auto TOC to all of docs under docs/manual/
bef398e [AhyoungRyu] Apply auto TOC to all of docs under docs/development/
9c5f76b [AhyoungRyu] Apply auto TOC to all of docs under docs/interpreter/
587d4ba [AhyoungRyu] Apply auto TOC to all of docs under docs/security/
1f10b97 [AhyoungRyu] Change toc configuration
78dca9e [AhyoungRyu] Add toc.js for auto generating TOC
2016-06-25 22:57:44 -07:00
Mina Lee
81adf09f21 [ZEPPELIN-980] Move git repository from incubator-zeppelin to zeppelin
### What is this PR for?
git repo infra have moved from incubator-zeppelin to zeppelin

### What type of PR is it?
Hot Fix

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Mina Lee <minalee@apache.org>

Closes #983 from minahlee/ZEPPELIN-980 and squashes the following commits:

7542c31 [Mina Lee] remove travis url after travis migration is done
3793b38 [Mina Lee] Move git repository from incubator-zeppelin to zeppelin
2016-06-10 09:02:41 -07:00
Jongyoul Lee
43baa0af49 ZEPPELIN-925 Merge HiveInterpreter into JDBCInterpreter
### What is this PR for?
This removes hive module and adds example setting for using Hive in a JdbcInterpreter by using loading dynamic dependencies. It reduces Zeppelin's binary size.

There's no codes' modification except removing hive directory and remove the module from pom.xml

### What type of PR is it?
[Feature]

### Todos
* [x] - Remove hive module
* [x] - Add an example for using Hive in JDBC

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-925

### How should this be tested?
Set the interpreter properties and test it

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Jongyoul Lee <jongyoul@gmail.com>

Closes #943 from jongyoul/ZEPPELIN-925 and squashes the following commits:

eb50c78 [Jongyoul Lee] Fixed typo
45ec667 [Jongyoul Lee] Updated style of jdbc.md
f6b94d6 [Jongyoul Lee] Update hive.md
6aa9c7b [Jongyoul Lee] Revert "Remove hive.md from docs/interpreter"
f6573e3 [Jongyoul Lee] Deleted hive module from pom.xml
f720ed5 [Jongyoul Lee] Updated Documentation
9c58640 [Jongyoul Lee] Remove hive.md from docs/interpreter
cdf1cfc [Jongyoul Lee] Remove hive module and its directory
a38dc28 [Jongyoul Lee] Update example for hive in jdbc.md
2016-06-09 22:46:10 +09:00
AhyoungRyu
58cb48a24a [HOTFIX] ZEPPELIN-959: Fix odd code block in writingzeppelininterpreter.md
### What is this PR for?
Currently some code blocks in [docs/development/writingzeppelininterpreter.html](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/development/writingzeppelininterpreter.html) & [docs/storage/storage.html#S3](https://zeppelin.apache.org/docs/0.6.0-SNAPSHOT/storage/storage.html) are broken.

But it seems the broken code block  in `docs/storage/storage.html` is already handled in #962. So I just updated in `docs/development/writingzeppelininterpreter.html`

### What type of PR is it?
Hot Fix

### Todos

### What is the Jira issue?
[ZEPPELIN-959](https://issues.apache.org/jira/browse/ZEPPELIN-959)

### How should this be tested?
See the attached images :)

### Screenshots (if appropriate)
- Before
![docs-before2](https://cloud.githubusercontent.com/assets/10060731/15812570/6d70ba16-2b6b-11e6-8486-86052e8ceeed.gif)

- After
<img width="1077" alt="screen shot 2016-06-05 at 7 38 50 pm" src="https://cloud.githubusercontent.com/assets/10060731/15812563/4b36e696-2b6b-11e6-8f14-f4572331eec6.png">

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: AhyoungRyu <fbdkdud93@hanmail.net>

Closes #969 from AhyoungRyu/ZEPPELIN-959 and squashes the following commits:

8f8cc70 [AhyoungRyu] ZEPPELIN-959: Fix odd code block in writingzeppelininterpreter.md
879b8f9 [AhyoungRyu] Revert "ZEPPELIN-959: Fix odd code blocks in some documentations"
967b2fa [AhyoungRyu] ZEPPELIN-959: Fix odd code blocks in some documentations
2016-06-06 11:25:27 -07:00
AhyoungRyu
7c50cacc84 [HOTFIX] ZEPPELIN-931: fix interpreter listing bug
### What is this PR for?
Currently available interpreter list is not shown in `Creating New Interpreter` section. It seems this bug was generated after #835 was merged. So I temporally deactivated [3 SerializedName code lines](6d7f1bc434).

### What type of PR is it?
Bug Fix

### Todos
* [x] - Fix interpreter listing bug when creating new interpreter

### What is the Jira issue?
[ZEPPELIN-931](https://issues.apache.org/jira/browse/ZEPPELIN-931)

### How should this be tested?
1. Build latest master branch and browse Zeppelin home
2. Create new interpreter -> You can not see the available interpreter list in this step like below attached screenshot
3. Apply this patch
4. Build again and browse  -> You can see the available interpreter list as normal

### Screenshots (if appropriate)
 - **Before**
<img width="1273" alt="screen shot 2016-06-01 at 12 36 42 pm" src="https://cloud.githubusercontent.com/assets/10060731/15723066/9082435e-27f5-11e6-9783-df44638dbbec.png">

 - **After**
<img width="1273" alt="screen shot 2016-06-01 at 12 33 06 pm" src="https://cloud.githubusercontent.com/assets/10060731/15723067/92bcc8ce-27f5-11e6-82f5-6c0db7b4342c.png">

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: AhyoungRyu <fbdkdud93@hanmail.net>
Author: Jongyoul Lee <jongyoul@gmail.com>
Author: Ah young <fbdkdud93@nflabs.com>

Closes #945 from AhyoungRyu/ZEPPELIN-931 and squashes the following commits:

711eb54 [Ah young] Merge pull request #2 from jongyoul/ZEPPELIN-931
6121f9b [Jongyoul Lee] - Fixed documentation
6e7dac9 [Ah young] Merge pull request #1 from jongyoul/ZEPPELIN-931
fed1b40 [Jongyoul Lee] - Fixed fieldName in interpreter-setting.json
6d7f1bc [AhyoungRyu] ZEPPELIN-931: fix interpreter listing bug
2016-06-02 14:48:13 +09:00
Jongyoul Lee
f8e1f6c4fa ZEPPELIN-804 Refactoring registration mechanism on Interpreters
### What is this PR for?
This PR enable Zeppelin server register Interpreters without any dependencies of their own. For instance, we should build `spark` with `spark-dependencies` even we use our own Spark cluster because current initialisation mechanism needs to all of its dependencies.

### What type of PR is it?
[Improvement]

### Todos
* [x] - Add RegisteredInterpreter from interpreter-setting.json in a jar or interpreter/{interpreter}/interpreter-setting.json
* [x] - Adjust it to Spark*Interpreter

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-804

### How should this be tested?
1. Prepare your own spark cluster - e.g. standalone, Yarn, Mesos -
1. rm -rf interpreter
1. mvn clean package -DskipTests -pl 'zeppelin-display,zeppelin-interpreter,zeppelin-server,zeppelin-web,zeppelin-zengine,angular,jdbc,spark'
1. bin/zeppelin-daemon.sh start
1. Check error in log
1. apply patch
1. mvn clean package -DskipTests -pl 'zeppelin-display,zeppelin-interpreter,zeppelin-server,zeppelin-web,zeppelin-zengine,angular,jdbc,spark'
1. bin/zeppelin-daemon.sh start
1. run some paragraph with simple command like `sc.version`

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

### Description

#### This PR introduce three initialisation mechanism including current one.
* {interpreter_dir}/{interpreter_group}/interpreter-setting.json
* interpreter-settings.json in your interpreter jar
* Current static initialization

#### Initialization step
1. Get {interpreter_dir} from Configuration
1. Get the list of {interpreter_dir}/[{interpreter_group1},{interpreter_group2}...]
1. Find {interpreter_dir}/{interpreter_group1}/interpreter-setting.json
1. Find interpreter-setting.json in the resources of {interpreter_dir}/{interpreter_group1}/**/*.jar
1. Adopt static init
1. Repeat them from the second step with {interpreter_group2}

Author: Jongyoul Lee <jongyoul@gmail.com>

Closes #835 from jongyoul/ZEPPELIN-804 and squashes the following commits:

823321e [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Added documentation
25bc501 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Ignored while registering a new interpreter with existing interpreter key
312dd77 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Reverted log4j setting
81ab361 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Changed logger setting only for test. This will be reverted after test
e8f990f [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed some unicode characters in interpreter-setting.json
3ad41bb [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Checked if path exists or not
1b3cd0c [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Checked if path exists or not
844dccb [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed logic to check for supporting legacy mechanism
c5b7d54 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Extracted new initialization logic into another methods
5d63d91 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Removed static initialisation of Spark*Interpreter
0d720c0 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Removed a interpreter which fails to initialize
d343636 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Excluded SparkRInterpreter from interpreter-setting.json
519f057 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Excluded interpreter-setting.json from rat check
00f55a8 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Excluded interpreter-setting.json from rat check
1fa2e52 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed test environments
8a90fe4 [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed test environments
d54f98e [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Changed Spark*Interpreter to use interpreter-setting.json
48ac41d [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Fixed the style
ca7b96c [Jongyoul Lee] ZEPPELIN-804 Refactoring registration mechanism on Interpreters - Added a new initialization mechanism to use interpreter-setting.json - Adjusted new mechanism to SparkInterpreter for verification
2016-05-31 11:50:08 +09:00
Cheng-Yu Hsu
a313e492c4 Fix typos in docs
### What is this PR for?

1. Fix some typos in docs.
2. Remove trailing white spaces for each line.
3. Remove leading white spaces if a line contains no content.
4. Add trailing new line for each file.

### What type of PR is it?
Improvement | Documentation

### Todos
None

### What is the Jira issue?
N/A

### How should this be tested?
Build the doc site and check.

### Screenshots (if appropriate)
N/A

### Questions:
* Does the licenses files need update? *no*
* Is there breaking changes for older versions? *no*
* Does this needs documentation? *no*

Author: Cheng-Yu Hsu <m@cyhsu.me>

Closes #852 from cyhsutw/fix-typos-in-docs and squashes the following commits:

6f5f46b [Cheng-Yu Hsu] fix typos in docs
2016-04-24 18:32:44 +09:00
Lee moon soo
b88f52e3cf [ZEPPELIN-513] Dedicated interpreter session per notebook
### What is this PR for?

Currently, all notebooks that binded to the same interpreter setting shares the same interpreter instance.
For example, let's say there're `spark-production` interpreter setting and all notebooks are using it.
Then all notebooks share a single instance of SparkInterpreter, and that result all the variables are being shared across the notebook.

But sometimes isolated name space for variables per notebook is preferred. In this case we can create per notebook interpreter instance.

This PR provides generalized facility to provides per notebook interpreter instance, not only for SparkInterpreter but for any other existing interpreter without modification.

**Design**

It provides toggle switch that user turn on / turn off this mode per interpreter setting.
When per notebook interpreter is turned on, it still creates a single remote interpreter process per interpreter setting. But inside of the process, each notebook will have it's own interpreter instance and the instance is terminated automatically on removal of notebook or unbinding note from interpreter setting.

**SparkInterpreter**

Because of the design keeps single remote interpreter process and creates multiple interpreter instance inside of it, SparkInterpreter become a problem. While multiple SparkContext can not be created on a single process. So this PR try to, create per notebook scala compiler instance, but share single SparkContext instance. In this way, each notebook can run job concurrently by leveraging fairscheduler of SparkContext without running multiple Spark application (SparkContext).

And this approach is inspired by https://github.com/piyush-mukati/incubator-zeppelin/tree/parallel_scheduler_support_spark, this PR trying to generalize it for all other interpreters.

### What type of PR is it?
Feature

### Todos
* [x] - Per note interpreter session
* [x] - Scala compiler instance per note and share a single SparkContext (SparkInterpreter)
* [x] - Handle Spark class server correctly
* [x] - Handle SparkInterpreter Progressbar correctly

### Is there a relevant Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-513

### How should this be tested?

Check "Per note session" of your Spark interpreter setting.
And create two different notebook and check scala variables are not shared.

### Screenshots (if appropriate)

Checkbox to enable
![image](https://cloud.githubusercontent.com/assets/1540981/13270280/e6db3686-da41-11e5-9408-8eaa9c2a7350.png)

How it works for SparkInterpreter
![per_note_session](https://cloud.githubusercontent.com/assets/1540981/13135855/3c01f26c-d5c9-11e5-864b-30fc4e9f61d1.gif)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? yes

Author: Lee moon soo <moon@apache.org>

This patch had conflicts when merged, resolved by
Committer: Lee moon soo <moon@apache.org>

Closes #703 from Leemoonsoo/notebook_interpreter_session and squashes the following commits:

222066c [Lee moon soo] Use nanoTime
8287eb9 [Lee moon soo] Fix style
45af2df [Lee moon soo] Prevent infinitely waiting loop
438b212 [Lee moon soo] Fix typo on the comment
b091b7e [Lee moon soo] Move numReferenceOfSparkContext.incrementAndGet() to the end of open() method
4d2533f [Lee moon soo] Per note session -> Separate Interpreter for each note
7b073f6 [Lee moon soo] Merge branch 'master' into notebook_interpreter_session
feabb5f [Lee moon soo] Handle NPE when closing interpreters in group spark.
5047217 [Lee moon soo] Restore angularObject correctly
1f03ebb [Lee moon soo] Merge branch 'master' into notebook_interpreter_session
7730cf3 [Lee moon soo] Add document for per note session mode
dedf93f [Lee moon soo] Nullcheck
72b2235 [Lee moon soo] Protect SparkContext creation
f2299d6 [Lee moon soo] Handle scheduler termination correctly when unbind, bind interpreter
269c27d [Lee moon soo] Fix unittest
4299b87 [Lee moon soo] Share a SparkContext across ScalaCompilers
cc33c25 [Lee moon soo] Add more unittests
6dd981a [Lee moon soo] update interpreter module test
1d1638c [Lee moon soo] Fix test
58e5f91 [Lee moon soo] Update spark interpreter
366b651 [Lee moon soo] Don't destroy interpreter on note memove or change bindings when interpreter.option.perNoteSession is false
fc9cb3f [Lee moon soo] interpreter session aware interpreter factory
ed1ab0d [Lee moon soo] zeppelin-interpreter note session support
5787984 [Lee moon soo] Add option for per note session interpreter
2016-02-28 08:18:00 -08:00
Felix Cheung
c484bf74ef [ZEPPELIN-632][DOC] Document steps to contribute a new interpreter
### What is this PR for?
Instead of going through a list of items one by one in PR review, we should have a checklist of TODOs for a potential interpreter contributor to review and follow

### What type of PR is it?
Documentation

### Todos
* [x] - Update doc

### Is there a relevant Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-632

### How should this be tested?
N/A

### Screenshots (if appropriate)
N/A

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? This is.

Author: Felix Cheung <felixcheung_m@hotmail.com>

Closes #675 from felixcheung/createinterpreterdoc and squashes the following commits:

0feb8fe [Felix Cheung] add link
2a4aa4b [Felix Cheung] add link to pom src
dc47a58 [Felix Cheung] small wording update
78d9127 [Felix Cheung] doc update
2016-02-06 10:57:12 -08:00
Ryu Ah young
8c848f0de9 Fix some typos, grammars and Increase readability documentation
### What is this PR for?
It would be better if Zeppelin provides looking good documentation.

### What type of PR is it?
Improvement

### Todos
* [x] - Fix typos, grammars and Increase readability writingzeppelininterpreter.md
* [x] - Also spark.md
* [x] - tutorial.md
* [x] - cassandra.md
* [x] - elasticsearch.md
* [x] - flink.md
* [x] - hive.md
* [x] - lens.md
* [x] - markdown.md
* [x] - rest-interpreter.md

### Is there a relevant Jira issue?
No : )

### How should this be tested?
These documentations are in [here](http://zeppelin.incubator.apache.org/docs/0.6.0-incubating-SNAPSHOT/).

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No

Author: Ryu Ah young <fbdkdud93@hanmail.net>

Closes #578 from AhyoungRyu/Fix-Typo and squashes the following commits:

925f30e [Ryu Ah young] Fix the list of 'Configure your Interpreter'
9b768e9 [Ryu Ah young] Change .md -> .html in interpreter.md
785dcbb [Ryu Ah young] Fix odd space in spark.md
a5d26d7 [Ryu Ah young] Fix version 1.5.1 -> 1.5.2 in spark.md
ef77027 [Ryu Ah young] Fix Upper case -> Lower case in cassandra.md
0918f79 [Ryu Ah young] Fix grammar error and Increase readability interpreters.md
f3d0173 [Ryu Ah young] Add numbering for dividing the table list and Fix grammar error rest-interpreter.md
fe4dfff [Ryu Ah young] Fix image size unit px -> % and Increase readability markdown.md
11b32db [Ryu Ah young] Remove useless 'to' in lens.md
3eec4cf [Ryu Ah young] Remove numbering flink.md
aac2e01 [Ryu Ah young] Add short description about Apache hive and Increase readability hive.md
d08092a [Ryu Ah young] Add highlight for scala code and Increase readability flink.md
8690e02 [Ryu Ah young] Increase readability and remove useless <hr> tag elasticsearch.md
9c008cf [Ryu Ah young] Increase readability and Remove useless <br>, <hr> tag cassandra.md
3041b95 [Ryu Ah young] Change rebuild -> rebuilding in spark.md
99b1e9e [Ryu Ah young] Remove unnecessary space in writingzeppelininterpreter.md
c63e59f [Ryu Ah young] Increase readability and delete useless <br> tag tutorial.md
e867df3 [Ryu Ah young] Increase readability and delete useless <br>, <hr> tag spark.md
f72b1cb [Ryu Ah young] Add .sh next to .bin/zeppein-daemon
466f82c [Ryu Ah young] Fix some typos and grammars in writingzeppelininterpreter.md
2016-01-17 08:20:36 -08:00
Lee moon soo
e4ff4c0353 ZEPPELIN-448 Update documentation for Spark configuration and BASE_PATH change
https://issues.apache.org/jira/browse/ZEPPELIN-448

Addresses update documentation for Spark configuration.

And BASE_PATH change after #430, #431 is deployed.
(currently deployed website has this change applied)

Author: Lee moon soo <moon@apache.org>

Closes #448 from Leemoonsoo/update_install and squashes the following commits:

54e2edf [Lee moon soo] add newline
cbd95d6 [Lee moon soo] Add description for included version of spark when SPARK_HOME is not set
00099ad [Lee moon soo] Add description for bumping up version
cf99212 [Lee moon soo] Fix typo
ab6d267 [Lee moon soo] Update BASE_PATH
73a737c [Lee moon soo] Remove instruction for 0.5.0 as zeppelin has documentation per version
4369f3e [Lee moon soo] Fix link
dcdaa74 [Lee moon soo] Add more configurations possible
8c9361d [Lee moon soo] Change font style
de74e6b [Lee moon soo] Update install and configuring spark
2015-11-21 15:25:14 +09:00
Lee moon soo
c2cbafd1d8 ZEPPELIN-412 Documentation based on Zeppelin version
https://issues.apache.org/jira/browse/ZEPPELIN-412

To provide documentation based on Zeppelin version, like Spark, Flink project does, it need to separate documentations from website.

* docs will be kept in Zeppelin main source tree and being built and published under 'docs' menu on website with specific version number.
* website will be kept in gh-pages branch and provides menu for multiple version of docs.

This PR removes unnecessary pages, which is provided by website. (for example download page)

This is the screenshot after applying this PR

![image](https://cloud.githubusercontent.com/assets/1540981/11163334/53a14c7a-8b0e-11e5-80cb-961bb8a15faa.png)

![image](https://cloud.githubusercontent.com/assets/1540981/11163335/5acc9f22-8b0e-11e5-8329-273bee738cc9.png)

Author: Lee moon soo <moon@apache.org>

Closes #430 from Leemoonsoo/ZEPPELIN-412 and squashes the following commits:

35da7f2 [Lee moon soo] Remove docs dir
5e4ce12 [Lee moon soo] Update readme
0635cbb [Lee moon soo] Remove unnecessary pages
e21cdd2 [Lee moon soo] Style font size
b5fe812 [Lee moon soo] Change title to overview
469b850 [Lee moon soo] Get remove unnecessary menu
2015-11-18 09:08:54 +09:00
Renamed from docs/docs/development/writingzeppelininterpreter.md (Browse further)