### What is this PR for? `AdvancedTransformation` has more detailed options while providing existing features of `PivotTransformation` and `ColumnselectorTransformation` which Zeppelin already has  Here are some features which advanced-transformation can provide. 1. **(screenshot)** multiple sub charts 2. **(screenshot)** parameter widgets: `input`, `checkbox`, `option`, `textarea` 3. **(screenshot)** expand/fold axis and parameter panels 4. **(screenshot)** clear axis and parameter panels 5. **(screenshot)** remove duplicated columns in an axis 6. **(screenshot)** limit column count in an axis 7. configurable char axes: `valueType`, `axisType`, `description`, ... 8. configurable chart parameters 9. lazy transformation 10. parsing parameters automatically based on their type: `int`, `float`, `string`, `JSON` 11. multiple transformation methods 12. re-initialize whole configuration based on spec hash. 13. **(screenshot)** shared axis #### API Details: Spec `AdvancedTransformation` requires `spec` which includes axis and parameter details for charts. - Let's create 2 sub-charts called `simple-line` and `step-line`. - Each sub chart can have different `axis` and `parameter` depending on their requirements. ```js constructor(targetEl, config) { super(targetEl, config) const spec = { charts: { 'simple-line': { sharedAxis: true, /** set if you want to share axes between sub charts, default is `false` */ axis: { 'xAxis': { dimension: 'multiple', axisType: 'key', }, 'yAxis': { dimension: 'multiple', axisType: 'aggregator'}, 'category': { dimension: 'multiple', axisType: 'group', }, }, parameter: { 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, 'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', }, 'dashLength': { valueType: 'int', defaultValue: 0, description: 'the length of dash', }, }, }, 'step-line': { axis: { 'xAxis': { dimension: 'single', axisType: 'unique', }, 'yAxis': { dimension: 'multiple', axisType: 'value', }, }, parameter: { 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, 'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', }, 'noStepRisers': { valueType: 'boolean', defaultValue: false, description: 'no risers in step line', widget: 'checkbox', }, }, }, } this.transformation = new AdvancedTransformation(config, spec) } ``` #### API Details: Axis Spec | Field Name | Available Values (type) | Description | | --- | --- | --- | |`dimension` | `single` | Axis can contains only 1 column | |`dimension` | `multiple` | Axis can contains multiple columns | |`axisType` | `key` | Column(s) in this axis will be used as `key` like in `PivotTransformation`. These columns will be served in `column.key` | |`axisType` | `aggregator` | Column(s) in this axis will be used as `value` like in `PivotTransformation`. These columns will be served in `column.aggregator` | |`axisType` | `group` | Column(s) in this axis will be used as `group` like in `PivotTransformation`. These columns will be served in `column.group` | |`axisType` | (string) | Any string value can be used here. These columns will be served in `column.custom` | |`maxAxisCount` | (int) | The maximum column count that this axis can contains. (unlimited if `undefined`) | |`valueType` | (string) | Describe the value type just for annotation | Here is an example. ```js axis: { 'xAxis': { dimension: 'multiple', axisType: 'key', }, 'yAxis': { dimension: 'multiple', axisType: 'aggregator'}, 'category': { dimension: 'multiple', axisType: 'group', maxAxisCount: 2, valueType: 'string', }, }, ``` #### API Details: Parameter Spec | Field Name | Available Values (type) | Description | | --- | --- | --- | |`valueType` | `string` | Parameter which has string value | |`valueType` | `int` | Parameter which has int value | |`valueType` | `float` | Parameter which has float value | |`valueType` | `boolean` | Parameter which has boolean value used with `checkbox` widget usually | |`valueType` | `JSON` | Parameter which has JSON value used with `textarea` widget usually. `defaultValue` should be `""` (empty string). This ||`defaultValue` | (any) | Default value of this parameter. `JSON` type should have `""` (empty string) | |`description` | (string) | Description of this parameter. This value will be parsed as HTML for pretty output | |`widget` | `input` | Use [input](https://developer.mozilla.org/en/docs/Web/HTML/Element/input) widget. This is the default widget (if `widget` is undefined)| |`widget` | `checkbox` | Use [checkbox](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input/checkbox) widget. | |`widget` | `textarea` | Use [textarea](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/textarea) widget. | |`widget` | `option` | Use [select + option](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/select) widget. This parameter should have `optionValues` field as well. | |`optionValues` | (Array<string>) | Available option values used with the `option` widget | Here is an example. ```js parameter: { // string type, input widget 'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', }, // boolean type, checkbox widget 'inverted': { widget: 'checkbox', valueType: 'boolean', defaultValue: false, description: 'invert x and y axes', }, // string type, option widget with `optionValues` 'graphType': { widget: 'option', valueType: 'string', defaultValue: 'line', description: 'graph type', optionValues: [ 'line', 'smoothedLine', 'step', ], }, // HTML in `description` 'dateFormat': { valueType: 'string', defaultValue: '', description: 'format of date (<a href="https://docs.amcharts.com/3/javascriptcharts/AmGraph#dateFormat">doc</a>) (e.g YYYY-MM-DD)', }, // JSON type, textarea widget 'yAxisGuides': { widget: 'textarea', valueType: 'JSON', defaultValue: '', description: 'guides of yAxis ', }, ``` #### API Details: Transformer Spec `AdvancedTransformation` supports 3 transformation methods. The return value will depend on the transformation method type. ```js const spec = { charts: { 'simple': { /** default value of `transform.method` is the flatten cube. */ axis: { ... }, parameter: { ... } }, 'cube-group': { transform: { method: 'cube', }, axis: { ... }, parameter: { ... }, } 'no-group': { transform: { method: 'raw', }, axis: { ... }, parameter: { ... }, } ``` | Field Name | Available Values (type) | Description | | --- | --- | --- | |`method` | `object` | designed for [amcharts: serial](https://www.amcharts.com/demos/date-based-data/) | |`method` | `array` | designed for [highcharts: column](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-basic/) | |`method` | `drill-down` | designed for [highcharts: drill-down](http://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/highcharts/demo/column-drilldown/) | |`method` | `raw` | will return the original `tableData.rows` | Whatever you specified as `transform.method`, the `transformer` value will be always function for lazy computation. ```js // advanced-transformation.util#getTransformer if (transformSpec.method === 'raw') { transformer = () => { return rows; } } else if (transformSpec.method === 'array') { transformer = () => { ... return { ... } } } ``` #### Feature Details: Automatic parameter parsing Advanced transformation will parse parameter values automatically based on their type: `int`, `float`, `string`, `JSON` - See also `advanced-transformation-util.js#parseParameter` #### Feature Details: re-initialize the whole configuration based on spec hash ```js // advanced-transformation-util#initializeConfig const currentVersion = JSON.stringify(spec) if (!config.spec || !config.spec.version || config.spec.version !== currentVersion) { spec.version = currentVersion // reset config... } ``` #### Feature Details: Shared Axes If you set `sharedAxis` to `true` in chart specification, then these charts will share their axes. (default is `false`) ```js const spec = { charts: { 'column': { transform: { method: 'array', }, sharedAxis: true, axis: { ... }, parameter: { ... }, }, 'stacked': { transform: { method: 'array', }, sharedAxis: true, axis: { ... } parameter: { ... }, }, ```  #### API Details: Usage in Visualization#render() Let's assume that we want to create 2 sub-charts called `basic` and `no-group`. - https://github.com/1ambda/zeppelin-ultimate-line-chart (an practical example) ```js drawBasicChart(parameter, column, transformer) { const { ... } = transformer() } drawNoGroupChart(parameter, column, transformer) { const { ... } = transformer() } render(data) { const { chart, parameter, column, transformer, } = data if (chart === 'basic') { this.drawBasicChart(parameter, column, transformer) } else if (chart === 'no-group') { this.drawNoGroupChart(parameter, column, transformer) } } ``` ### What type of PR is it? [Feature] ### Todos NONE ### What is the Jira issue? [ZEPPELIN-2217](https://issues.apache.org/jira/browse/ZEPPELIN-2217) ### How should this be tested? 1. Clone https://github.com/1ambda/zeppelin-ultimate-line-chart 2. Create a symbolic link `ultimate-line-chart.json` into `$ZEPPELIN_HOME/helium` 3. Modify the `artifact` value to proper absolute path considering your local machine. 4. Install the above visualization in `localhost:9000/#helium` 5. Test it ### Screenshots (if appropriate) #### 1. *(screenshot)* multiple sub charts  #### 2. *(screenshot)* parameter widgets: `input`, `checkbox`, `option`, `textarea`  #### 3. *(screenshot)* expand/fold axis and parameter panels  #### 4. *(screenshot)* clear axis and parameter panels  #### 5. *(screenshot)* remove duplicated columns in an axis  #### 6. *(screenshot)* limit column count in an axis  ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - NO Author: 1ambda <1amb4a@gmail.com> Closes #2098 from 1ambda/ZEPPELIN-2217/advanced-transformation and squashes the following commits:6cde7c9[1ambda] fix reset params when spec changec75a3f2[1ambda] fix: Reset persisted axis6a2130a[1ambda] fix: clear config only when axis changed5464e84[1ambda] fix: Optimize array 2 key method9beb1e7[1ambda] fix: Type error2408225[1ambda] test: Add test for array 2keybf56761[1ambda] feat: Add array:2-key transform method7c6768f[1ambda] feat: Use axisSpec.desc as tooltipf98d4c9[1ambda] fix: Remove invalid key prop5cf2ece[1ambda] feat: Add minAxisCount4887800[1ambda] fix: Remove local module yarn caches3e29572[1ambda] refactor: copyModule funcc91a033[1ambda] fix: Set yarn cache dir in helium-bundles04b5140[1ambda] fix: Import a-tr0a876cf[1ambda] docs: Update index.md380b1af[1ambda] docs: Fix typo and add desc for existing trs908214b[1ambda] docs: Move experimental tagsa009627[1ambda] feat: Allow dup aggr axis3b44e92[1ambda] fix: Remove unuse constab6c22e[1ambda] test: Add test for drill-down method756107a[1ambda] test: Add array transformation methodd819c73[1ambda] test: Add object methodbf00fba[1ambda] test: Add MockTableData39fe5ae[1ambda] test: Add test for getColumnsFromAxis4c393b4[1ambda] fix: Add polyfill for es6 funcs in teste92c787[1ambda] test: Add test for rmDup, aplMaxAxisCount843f45d[1ambda] test: Add test for getCurrent* funcsae5277c[1ambda] test: Add test for initializeConfigc14a9dc7[1ambda] test: Add tests for widget, paramsc510af1[1ambda] docs: Add doc for Transformation52db37b[1ambda] feat: Show panel menus only when opened17ad4a4[1ambda] feat: Support chartChanged, parameterChangedc0d33d3[1ambda] fix: Sort selectors in drilldown methodcfd6fef[1ambda] feat: sharedAxis9af80ce[1ambda] style: Indent79b5654[1ambda] fix: return the same info in transform7bee464[1ambda] fix: Keynamesee8788e[1ambda] feat: Support drill-down666025a[1ambda] fix: DON'T reset current chartae1891f[1ambda] add array:key transform4167a2e[1ambda] fix: Sort keyNames912b5b7[1ambda] fix: Persist initialized configf1f6b0c[1ambda] feat: Support ARRAY transform.method812f9a2[1ambda] fix: Set proper aggr value when 0 group20f9437[1ambda] fix: getCube func25d51a9[1ambda] DON'T display aggr.name when aggrColumns.length == 1f37e13d[1ambda] fix: Add 'object' transform.methodda2370c[1ambda] fix: Add resetAxis, Param funcs2370682[1ambda] fix: average is not caculated correctlydd08e38[1ambda] fix: Set param panel height to 400881695a[1ambda] feat: clear chart, param separately4d0d62b[1ambda] fix: DON'T clean panel config92676d1[1ambda] fix: limit parameter panel height to 370cc29060[1ambda] feat: parse param description as HTML9a2d227[1ambda] fix: Stop event propagation in widgetsfcc625c[1ambda] feat: Automatic param parsingb4d774c[1ambda] fix: Dont close param panel when enter088705b[1ambda] refactor: Remove util and add Widget funcsbf88b4f[1ambda] feat: textare widget and update hook4e73012[1ambda] feat: widget checkbox11b7eaa[1ambda] feat: option widget5d3efc9[1ambda] fix: Change panel headerb1d9d31[1ambda] feat: Save and close with enter key53f508c[1ambda] feat: custom axisSpec0dbc431[1ambda] feat: Support transformer94d837a[1ambda] feat: Automatic spec versioning74b8b4e[1ambda] fix: Duplicated radio btn id, name5b88f08[1ambda] fix: Modify margin of subchart radio btns019892c[1ambda] feat: Support transform: flatten0484e1e[1ambda] feat: Support maxAxisCount in axisSpec936901b[1ambda] feat: Support undefined valueType in axisSpec7a454ff[1ambda] feat: Cube Transformationf0ed02f[1ambda] feat: Support same axis types49985c6[1ambda] refactor: Refine axis, param specd89e223[1ambda] feat: advanced-transformation-api75569ce[1ambda] feat: Support multiple charts in UIe1fcc2e[1ambda] feat: Support multiple charts97be629[1ambda] fix: Add singleDimensionAggregatorChanged676bd7e[1ambda] refactor: Refine transform API9fb398e[1ambda] feat: Add clearConfiga8a4fb1[1ambda] refactor: Add getAxisInSingleDimension func9768ecf[1ambda] feat: Add groupBase axis option91ae54d[1ambda] fix: Overflow issue in single aggr10c80fc[1ambda] feat: AdvancedTransformation
10 KiB
| layout | title | description | group |
|---|---|---|---|
| page | Transformations for Zeppelin Visualization | Description for Transformations | development |
{% include JB/setup %}
Transformations for Zeppelin Visualization
Overview
Transformations
- renders setting which allows users to set columns and
- transforms table rows according to the configured columns.
Zeppelin provides 4 types of transformations.
1. PassthroughTransformation
PassthroughTransformation is the simple transformation which does not convert original tabledata at all.
See passthrough.js
2. ColumnselectorTransformation
ColumnselectorTransformation is uses when you need N axes but do not need aggregation.
3. PivotTransformation
PivotTransformation provides group by and aggregation. Every chart using PivotTransformation has 3 axes. Keys, Groups and Values.
See pivot.js
4. AdvancedTransformation
AdvancedTransformation has more detailed options while providing existing features of PivotTransformation and ColumnselectorTransformation
- multiple sub charts
- configurable chart axes
- parameter widgets:
input,checkbox,option,textarea - parsing parameters automatically based on their types
- expand / fold axis and parameter panels
- multiple transformation methods while supporting lazy converting
- re-initialize the whole configuration based on spec hash.
Spec
AdvancedTransformation requires spec which includes axis and parameter details for charts.
Let's create 2 sub-charts called line and no-group. Each sub chart can have different axis and parameter depending on their requirements.
class AwesomeVisualization extends Visualization {
constructor(targetEl, config) {
super(targetEl, config)
const spec = {
charts: {
'line': {
transform: { method: 'object', },
sharedAxis: false, /** set if you want to share axes between sub charts, default is `false` */
axis: {
'xAxis': { dimension: 'multiple', axisType: 'key', description: 'serial', },
'yAxis': { dimension: 'multiple', axisType: 'aggregator', description: 'serial', },
'category': { dimension: 'multiple', axisType: 'group', description: 'categorical', },
},
parameter: {
'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', },
'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', },
'lineWidth': { valueType: 'int', defaultValue: 0, description: 'width of line', },
},
},
'no-group': {
transform: { method: 'object', },
sharedAxis: false,
axis: {
'xAxis': { dimension: 'single', axisType: 'key', },
'yAxis': { dimension: 'multiple', axisType: 'value', },
},
parameter: {
'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', },
'yAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of yAxis', },
},
},
}
this.transformation = new AdvancedTransformation(config, spec)
}
...
// `render` will be called whenever `axis` or `parameter` is changed
render(data) {
const { chart, parameter, column, transformer, } = data
if (chart === 'line') {
const transformed = transformer()
// draw line chart
} else if (chart === 'no-group') {
const transformed = transformer()
// draw no-group chart
}
}
}
Spec: axis
| Field Name | Available Values (type) | Description |
|---|---|---|
dimension |
single |
Axis can contains only 1 column |
dimension |
multiple |
Axis can contains multiple columns |
axisType |
key |
Column(s) in this axis will be used as key like in PivotTransformation. These columns will be served in column.key |
axisType |
aggregator |
Column(s) in this axis will be used as value like in PivotTransformation. These columns will be served in column.aggregator |
axisType |
group |
Column(s) in this axis will be used as group like in PivotTransformation. These columns will be served in column.group |
axisType |
(string) | Any string value can be used here. These columns will be served in column.custom |
maxAxisCount (optional) |
(int) | The max number of columns that this axis can contain. (unlimited if undefined) |
minAxisCount (optional) |
(int) | The min number of columns that this axis should contain to draw chart. (1 in case of single dimension) |
description (optional) |
(string) | Description for the axis. |
Here is an example.
axis: {
'xAxis': { dimension: 'multiple', axisType: 'key', },
'yAxis': { dimension: 'multiple', axisType: 'aggregator'},
'category': { dimension: 'multiple', axisType: 'group', maxAxisCount: 2, valueType: 'string', },
},
Spec: sharedAxis
If you set sharedAxis: false for sub charts, then their axes are persisted in global space (shared). It's useful for when you creating multiple sub charts sharing their axes but have different parameters. For example,
basic-column,stacked-column,percent-columnpieanddonut
Here is an example.
const spec = {
charts: {
'column': {
transform: { method: 'array', },
sharedAxis: true,
axis: { ... },
parameter: { ... },
},
'stacked': {
transform: { method: 'array', },
sharedAxis: true,
axis: { ... }
parameter: { ... },
},
Spec: parameter
| Field Name | Available Values (type) | Description |
|---|---|---|
valueType |
string |
Parameter which has string value |
valueType |
int |
Parameter which has int value |
valueType |
float |
Parameter which has float value |
valueType |
boolean |
Parameter which has boolean value used with checkbox widget usually |
valueType |
JSON |
Parameter which has JSON value used with textarea widget usually. defaultValue should be "" (empty string). This |
description |
(string) | Description of this parameter. This value will be parsed as HTML for pretty output |
widget |
input |
Use input widget. This is the default widget (if widget is undefined) |
widget |
checkbox |
Use checkbox widget. |
widget |
textarea |
Use textarea widget. |
widget |
option |
Use select + option widget. This parameter should have optionValues field as well. |
optionValues |
(Array) | Available option values used with the option widget |
Here is an example.
parameter: {
// string type, input widget
'xAxisUnit': { valueType: 'string', defaultValue: '', description: 'unit of xAxis', },
// boolean type, checkbox widget
'inverted': { widget: 'checkbox', valueType: 'boolean', defaultValue: false, description: 'invert x and y axes', },
// string type, option widget with `optionValues`
'graphType': { widget: 'option', valueType: 'string', defaultValue: 'line', description: 'graph type', optionValues: [ 'line', 'smoothedLine', 'step', ], },
// HTML in `description`
'dateFormat': { valueType: 'string', defaultValue: '', description: 'format of date (<a href="https://docs.amcharts.com/3/javascriptcharts/AmGraph#dateFormat">doc</a>) (e.g YYYY-MM-DD)', },
// JSON type, textarea widget
'yAxisGuides': { widget: 'textarea', valueType: 'JSON', defaultValue: '', description: 'guides of yAxis ', },
Spec: transform
| Field Name | Available Values (type) | Description |
|---|---|---|
method |
object |
designed for rows requiring object manipulation |
method |
array |
designed for rows requiring array manipulation |
method |
array:2-key |
designed for xyz charts (e.g bubble chart) |
method |
drill-down |
designed for drill-down charts |
method |
raw |
will return the original tableData.rows |
Whatever you specified as transform.method, the transformer value will be always function for lazy computation.
// advanced-transformation.util#getTransformer
if (transformSpec.method === 'raw') {
transformer = () => { return rows; }
} else if (transformSpec.method === 'array') {
transformer = () => {
...
return { ... }
}
}
Here is actual usage.
class AwesomeVisualization extends Visualization {
constructor(...) { /** setup your spec */ }
...
// `render` will be called whenever `axis` or `parameter` are changed
render(data) {
const { chart, parameter, column, transformer, } = data
if (chart === 'line') {
const transformed = transformer()
// draw line chart
} else if (chart === 'no-group') {
const transformed = transformer()
// draw no-group chart
}
}
...
}