diff options
| author | 窦凤虎 <[email protected]> | 2024-09-10 12:11:14 +0000 |
|---|---|---|
| committer | 窦凤虎 <[email protected]> | 2024-09-10 12:11:14 +0000 |
| commit | 7d4b6d052d9584e38a1c0deec18408f74b07655e (patch) | |
| tree | 90eb508ca3cd69e9a48531237c9e713e704a8a1c /docs/processor/split-processor.md | |
| parent | e9f8f7a435d805fa6d0ec8b06715f0f68be326ff (diff) | |
| parent | 4bb87b62cd7d3dd12bd19e643aaffda53e35e57a (diff) | |
Merge branch 'fix/bootstrap' into 'develop'v1.6.0-2-SNAPSHOT
Fix/bootstrap
See merge request galaxy/platform/groot-stream!105
Diffstat (limited to 'docs/processor/split-processor.md')
| -rw-r--r-- | docs/processor/split-processor.md | 49 |
1 files changed, 49 insertions, 0 deletions
diff --git a/docs/processor/split-processor.md b/docs/processor/split-processor.md new file mode 100644 index 0000000..e1a1163 --- /dev/null +++ b/docs/processor/split-processor.md @@ -0,0 +1,49 @@ +# Split Processor + +> Split the output of a data processing pipeline into multiple streams based on certain conditions. + +## Description + +Using the flink side Outputs send data from a stream to multiple downstream consumers. This is useful when you want to separate or filter certain elements of a stream without disrupting the main processing flow. For example, side outputs can be used for error handling, conditional routing, or extracting specific subsets of the data. + +## Options + +| name | type | required | default value | +|-------------------|--------|----------|--------------------------------------------------------------------------------------------| +| type | String | Yes | The type of the processor, now only support ` com.geedgenetworks.core.split.SplitOperator` | +| rules | Array | Yes | Array of Object. Defining rules for labeling Side Output Tag | +| [rule.]tag | String | Yes | The tag name of the side output | +| [rule.]expression | String | Yes | The expression to evaluate the event. | + +## Usage Example + +This example uses a split processor to split the data into two streams based on the value of the `decoded_as` field. + +```yaml +splits: + decoded_as_split: + type: split + rules: + - tag: http_tag + expression: event.decoded_as == 'HTTP' + - tag: dns_tag + expression: event.decoded_as == 'DNS' + + +topology: + - name: inline_source + downstream: [decoded_as_split] + - name: decoded_as_split + tags: [http_tag, dns_tag] + downstream: [ projection_processor, aggregate_processor] + - name: projection_processor + downstream: [ print_sink ] + - name: aggregate_processor + downstream: [ print_sink ] + - name: print_sink + downstream: [] +``` + + + + |
