mirror of
https://github.com/apache/zeppelin
synced 2026-05-24 09:38:26 +00:00
Adding external libraries to livy
This commit is contained in:
parent
03a1e80f8f
commit
98090abc07
2 changed files with 44 additions and 1 deletions
|
|
@ -30,7 +30,7 @@ We added some common configurations for spark, and you can set any configuration
|
|||
This link contains all spark configurations: http://spark.apache.org/docs/latest/configuration.html#available-properties.
|
||||
And instead of starting property with `spark.` it should be replaced with `livy.spark.`.
|
||||
Example: `spark.master` to `livy.spark.master`
|
||||
|
||||
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Property</th>
|
||||
|
|
@ -102,8 +102,41 @@ Example: `spark.master` to `livy.spark.master`
|
|||
<td></td>
|
||||
<td>Upper bound for the number of executors.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>livy.spark.executor.extraClassPath</td>
|
||||
<td></td>
|
||||
<td>Adding extra libraries to the executors</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>livy.spark.driver.extraClassPath</td>
|
||||
<td></td>
|
||||
<td>Adding extra libraries to the driver</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Adding External libraries
|
||||
You should put the libraries manually on the livy server and then set the paths of the libraries to those properties `livy.spark.executor.extraClassPath` and `livy.spark.driver.extraClassPath`
|
||||
|
||||
Example
|
||||
|
||||
<table class="table-configuration">
|
||||
<tr>
|
||||
<th>Property</th>
|
||||
<th>Default</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>livy.spark.executor.extraClassPath</td>
|
||||
<td>path/to/jar1.jar:path/to/jar2.jar</td>
|
||||
<td>Adding extra libraries to the executors</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>livy.spark.driver.extraClassPath</td>
|
||||
<td>path/to/jar1.jar:path/to/jar2.jar</td>
|
||||
<td>Adding extra libraries to the driver</td>
|
||||
<tr>
|
||||
</table>
|
||||
|
||||
## How to use
|
||||
Basically, you can use
|
||||
|
||||
|
|
|
|||
|
|
@ -81,6 +81,16 @@
|
|||
"propertyName": "zeppelin.livy.keytab",
|
||||
"defaultValue": "",
|
||||
"description": "Kerberos keytab to authenticate livy"
|
||||
},
|
||||
"livy.spark.executor.extraClassPath": {
|
||||
"propertyName": "livy.spark.executor.extraClassPath",
|
||||
"defaultValue": "",
|
||||
"description": "Adding extra libraries to the executors"
|
||||
},
|
||||
"livy.spark.driver.extraClassPath": {
|
||||
"propertyName": "livy.spark.driver.extraClassPath",
|
||||
"defaultValue": "",
|
||||
"description": "Adding extra libraries to the driver"
|
||||
}
|
||||
}
|
||||
},
|
||||
|
|
|
|||
Loading…
Reference in a new issue