6 commits

Author	SHA1	Message	Date
Jeff Zhang	4c8f20ae33	ZEPPELIN-3051. Support Interpreter Process Recovery ... ### What is this PR for? This PR is for the purpose of recover running interpreter process when zeppelin server is restarted. This would be useful when restarting zeppelin without interrupt current running interpreter processes, should be useful when admin do maintenance or upgrading. Interface `RecoveryStorage` is used for storing the information of running interpreter process. Currently it only has one implementation `FileSystemRecoveryStorage`, other implementation could be done later (such as zookeeper based). `InterpreterLauncher` is the component where to recover the running interpreter process. Test: * RecoveryTest.java * FileSystemRecoveryStorageTest.java Design Doc: https://docs.google.com/document/d/1Plm3Hd40aGdNaXmjdsoY4ek3f-gTijTMGMkNjAZN39Y/edit?usp=sharing ### What type of PR is it? [Feature] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3051 ### How should this be tested? Unit test & Integration Test is added. Also manually verified. ### Screenshots (if appropriate) ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2668 from zjffdu/ZEPPELIN-3051 and squashes the following commits: a4c9b9c [Jeff Zhang] address comments 575b7b9 [Jeff Zhang] fix the pid of interpreter process id 02b118f [Jeff Zhang] address comments da7cbb9 [Jeff Zhang] ZEPPELIN-3051. Support Interpreter Process Recovery	2017-12-14 16:20:26 +08:00
Jeff Zhang	13f8e6cc65	ZEPPELIN-3085 Introduce generic ConfInterpreter for more fine-grained control of interpreter setting ... ### What is this PR for? Zeppelin's interpreter setting is shared by all the users and notes, if you want to have different setting you have to create new interpreter, e.g. you can create `spark_jar1` for running spark with dependency jar1 and `spark_jar2` for running spark with dependency jar2. This approach works, but not so convenient. `ConfInterpreter` can provide more fine-grained control on interpreter setting and more flexibility. `ConfInterpreter` is a generic interpreter that could be used by any interpreters. The input format should be property file format. In the first paragraph, we use ConfInterpreter to make custom configuration of spark interpreter (set app name, yarn-client mode & add spark-csv dependencies). Then you can run the second paragraph which use spark-csv.  It can be used to make custom setting for any interpreter. `ConfInterpreter` would run in the zeppelin server side, it would update the interpreter properties before you launch the interpreter process, so it needs to run before interpreter process launched. And when interpreter process is launched is determined by interpreter mode setting. So users needs to understand the interpreter mode setting of zeppelin and be aware when interpreter process is launched. E.g. If we set spark interpreter setting as isolated per note. Under this setting, each note will launch one interpreter process. In this scenario, user need to put `ConfInterpreter` as the first paragraph as the above example. Otherwise the customized setting can not be applied (The paragraph using ConfInterpreter will report ERROR). ### What type of PR is it? [Feature | Documentation] ### Todos * [ ] - Task ### What is the Jira issue? * https://issues.apache.org/jira/browse/ZEPPELIN-3085 ### How should this be tested? * Unit test, System test is added, also manually verified it. ### Screenshots (if appropriate)  ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2692 from zjffdu/ZEPPELIN-3085 and squashes the following commits: 87ce20f [Jeff Zhang] ZEPPELIN-3085. Introduce generic ConfInterpreter for more fine-grained control of interpreter setting	2017-12-14 13:57:52 +08:00
Jeff Zhang	d925e052ad	[Doc] Doc for LifecycleManager ... ### What is this PR for? Documentation for LifecycleManager ### What type of PR is it? [ Documentation ] ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: Jeff Zhang <zjffdu@apache.org> Closes #2644 from zjffdu/Doc_Lifecycle and squashes the following commits: 5aa0c16 [Jeff Zhang] [Doc] Doc for LifecycleManager	2017-11-06 18:26:53 +08:00
1ambda	86bc9332e0	[ZEPPELIN-2582][DOCS] docs for interpreter binding modes ... ### What is this PR for? Updated `interpreter_binding_mode.md` since users are sometimes confused what this mode means and there is already opened JIRA issue. This documentation will be helpful to Zeppelin users. **disclaimer: content was copied from [here](https://medium.com/leemoonsoo/apache-zeppelin-interpreter-mode-explained-bae0525d0555) with author's consent.** ### What type of PR is it? [Documentation] ### Todos DONE ### What is the Jira issue? [ZEPPELIN-2582](https://issues.apache.org/jira/browse/ZEPPELIN-2582) ### How should this be tested? 1. setup rvm 2.1.0+ and install required bundles for `docs/` 2. bundle exec jekyll serve --watch 3. http://localhost:4000/usage/interpreter/interpreter_binding_mode.html ### Screenshots (if appropriate)       ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - This PR is about docs. Author: 1ambda <1amb4a@gmail.com> Closes #2437 from 1ambda/ZEPPELIN-2582/docs-for-interpreter-modes and squashes the following commits: e0bcf26c [1ambda] fix: Docs a60fdbf2 [1ambda] fix: Correct sentences cdeb7e8a [1ambda] docs: Update imgs d60c48ea [1ambda] fix: clarify description c2317d92 [1ambda] fix: Use basepath 53411350 [1ambda] docs: Remove multiple spark context boxes in scoped mode image 07c4b02a [1ambda] fix: Provide more specific example for per user mode e385ecf5 [1ambda] docs: Use new images fc69bc60 [1ambda] docs: Add UI images 56f1a68b [1ambda] docs: Add description for per-user 963f9ce7 [1ambda] docs: Add column to explain sharing objects 33b0736e [1ambda] fix: typo 316c5634 [1ambda] docs: Fix incorrect info for scoped mode, isolated mode 729b56a2 [1ambda] fix: Remove code perspective af442263 [1ambda] docs: Use lowercase for word note 9ae9552a [1ambda] docs: emphasize per note mode in sopced, iolsated 66d02ba7 [1ambda] docs: Enhance description for ioslated mode 83eb2e5f [1ambda] docs: enhance desc for scoped mode 1eafb647 [1ambda] docs: Add descroptoin for per user, per note 43e8b023 [1ambda] docs: Add summary table of @cacti77 b43a4daf [1ambda] docs: Update interpreter binding mode page 56836205 [1ambda] feat: Add images	2017-07-07 16:23:49 +09:00
1ambda	1c23f21388	[ZEPPELIN-2707][DOCS][HOTFIX] fix: broken image URLs in 0.8.0-SNAPSHOT doc ... ### What is this PR for? fix: broken image URLs in 0.8.0-SNAPSHOT doc using the path `/asset` (the absolute path) for image URLs is actually invalid. That's because each version has its own image directory. So they should use the relative path. `{{BASE_PATH}}` ``` ➜ asf-zeppelin tree site | grep asset ├── assets # root asset, we shouldn't use it in versioned doc. │ │ ├── assets │ │ ├── assets │ │ ├── assets │ │ ├── assets │ │ ├── assets │ │ ├── assets │ │ ├── assets │ │ ├── assets │ │ ├── assets │ ├── assets ``` ### What type of PR is it? [Bug Fix] ### Todos DONE ### What is the Jira issue? [ZEPPELIN-2707](https://issues.apache.org/jira/browse/ZEPPELIN-2707) ### How should this be tested? 1. cd `docs/` 2. build: `bundle exec jekyll build --safe` 3. check whether links in `_site` include `/docs/0.8.0-SNAPSHOT` as prefix or not ### Screenshots (if appropriate) #### Current http://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/usage/interpreter/overview.html  #### After  ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1amb4a@gmail.com> Closes #2450 from 1ambda/ZEPPELIN-2707/should-use-its-own-asset-directory and squashes the following commits: fb70214a [1ambda] fix: Use its own asset dir	2017-06-30 10:33:01 +09:00
1ambda	4b6d3e5574	[ZEPPELIN-2596] Improving documentation page ... ### What is this PR for? Improving documentation page. Please check *TODO* and *Screenshots* sections for detail. The motivation is described in [the JIRA ticket](https://issues.apache.org/jira/browse/ZEPPELIN-2583) and discussion is ongoing on the mailing list. ### What type of PR is it? [Improvement | Documentation] ### Todos * [x] - improved the navbar style * [x] - improved the main page * [x] - re-organized content structure * [x] - added tutorial pages: `spark_with_zeppelin.md`, `python_with_zeppelin.md`, `sql_with_zeppelin.md` for overview * [x] - added `multi_user_support.md` page to provide overview * [x] - added the empty `interpreter_binding_mode` page. This will be handed in the different issue: [ZEPPELIN-2582](https://issues.apache.org/jira/browse/ZEPPELIN-2582) * [x] - added the empty `trouble_shooting` page. This can be filled in the following PRs. * [x] - added the empty `useful_developer_tools` page. This can be filled in the following PRs. ### What is the Jira issue? [ZEPPELIN-2596](https://issues.apache.org/jira/browse/ZEPPELIN-2596) ### How should this be tested? 1. checkout 2. `cd docs` 3. `bundle install` (make sure that you have ruby 2.1.0+ and bundle) 4. `bundle exec jekyll serve --watch` 5. open `localhost:4000` ### Screenshots (if appropriate) #### better navbar: before  #### better navbar: after  #### improved main page: before  #### improved main page: after  #### organized content structure: before  #### organized content structure: after  ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - related with docs Author: 1ambda <1amb4a@gmail.com> Closes #2371 from 1ambda/updating-version-doc and squashes the following commits: eb02fa967 [1ambda] fix: navbar focus color applies after folding 026379ed6 [1ambda] fix: Remove docs/.listen_test a7dd4737b [1ambda] fix: sora's comment 1.2 18c5058f7 [1ambda] fix: resolve description in python_with_zeppelin.md d3ad67c73 [1ambda] fix: sora's comment 4 d133dbbcc [1ambda] fix: resolve sora's comment 3 513c6ff2c [1ambda] fix: resolve sora's comment 1.1 4c2946928 [1ambda] fix: resovle sora's comment 2 1c3946ac6 [1ambda] fix: sora's comment 1 4d6e4267f [1ambda] fix: Resolve sola's comment 3 d0524cafe [1ambda] fix: Set less shadow for nav 5f1f998ba [1ambda] docs: Add useful_develop_tools.md 9dfd62c74 [1ambda] fix: Typo in installation.md 30f7d7e06 [1ambda] fix: Typo in helium ctrl d6877e792 [1ambda] docs: Add python_with_zeppelin.md 7027e96c0 [1ambda] docs: Improve python conda, docker doc style e55b50a9d [1ambda] fix: Invalid URLs 75ddeeaff [1ambda] docs: replace URIs in interpreter 5b43993a4 [1ambda] docs: Add sql_with_zeppelin 053794e84 [1ambda] docs: Add spark_with_zeppelin.md d4d88b9c7 [1ambda] docs: Improve proxy doc b46cdd126 [1ambda] docs: Add empty interpreter_binding_mode.md 06fcb239e [1ambda] docs: Add empty personalized_mode.md 4991cf0a7 [1ambda] docs: Update upgrading.md 53142b7a0 [1ambda] fix: Simplify install.md 8a5c1e721 [1ambda] docs: Add multi_user_support.md 34095775e [1ambda] fix: Increase font size to 15px a03b04b33 [1ambda] fix: Remove sample text from trouble_shooting.md 199842590 [1ambda] fix: Remove docker doc link 66a2a7d26 [1ambda] docs: Improve impersonation page 0a6e3fc1d [1ambda] docs: Improve install doc ccd999ed5 [1ambda] docs: Improve helium doc f8d742d08 [1ambda] fix: an invalid link in navbar b7aa5f884 [1ambda] fix: URLs in development 61a175d94 [1ambda] docs: Update install.md 4c56de5c4 [1ambda] fix: URLs in setup 0b1d63513 [1ambda] fix: URLs in quickstart 28970a4fe [1ambda] feat: Add docs/usage 735946bca [1ambda] feat: rename /quickstart b351cf237 [1ambda] fix: Add missing links b70770b4f [1ambda] feat: Change URLs in nav, index 94e80aef6 [1ambda] fix: doens't display navbar version in small 6e0cab110 [1ambda] feat: Update doc section names b9ce256ff [1ambda] feat: Hide version in navbar when md f8bab52be [1ambda] fix: Better image display in index.md eeb37d5b5 [1ambda] fix: Add RL padding for mobile browser ceb60b5ee [1ambda] feat: Style collapsed nav for mobile browser 4ebafb4b6 [1ambda] commit	2017-06-23 17:44:13 +09:00