diff options
| author | doufenghu <[email protected]> | 2024-11-01 20:40:46 +0800 |
|---|---|---|
| committer | doufenghu <[email protected]> | 2024-11-01 20:40:46 +0800 |
| commit | 5818ed2ac9ca31a35a55f330160a9cf7f63bf6f3 (patch) | |
| tree | 0d2f00c6d6c1791de8c5588572e0e7fb538803f2 /docs/processor | |
| parent | e25eabde3ccb3f0d52346cb11cac757763c41be8 (diff) | |
[Improve][docs] Add a description of the new features for version 1.7.1-SNAPSHOT.
Diffstat (limited to 'docs/processor')
| -rw-r--r-- | docs/processor/udaf.md | 38 | ||||
| -rw-r--r-- | docs/processor/udf.md | 52 |
2 files changed, 89 insertions, 1 deletions
diff --git a/docs/processor/udaf.md b/docs/processor/udaf.md index 66d6ad5..f305201 100644 --- a/docs/processor/udaf.md +++ b/docs/processor/udaf.md @@ -9,7 +9,9 @@ - [First Value](#First-Value) - [Last Value](#Last-Value) - [Long Count](#Long-Count) +- [Max](#Max) - [MEAN](#Mean) +- [Min](#Min) - [Number SUM](#Number-SUM) - [HLLD](#HLLD) - [Approx Count Distinct HLLD](#Approx-Count-Distinct-HLLD) @@ -116,6 +118,23 @@ Example output_fields: [sessions] ``` +### Max + +MAX is used to get the maximum value of the field in the group of events. + +```MAX(filter, lookup_fields, output_fields)``` +- filter: optional +- lookup_fields: required. Now only support one field. +- output_fields: optional. If not set, the output field name is `lookup_field_name`. + +Example + +```yaml +- function: MAX + lookup_fields: [receive_time] + output_fields: [receive_time] +``` + ### Mean MEAN is used to calculate the mean value of the field in the group of events. The lookup field value must be a number. @@ -135,6 +154,25 @@ Example output_fields: [received_bytes_mean] ``` + +### Min + +MIN is used to get the minimum value of the field in the group of events. + +```MIN(filter, lookup_fields, output_fields)``` +- filter: optional +- lookup_fields: required. Now only support one field. +- output_fields: optional. If not set, the output field name is `lookup_field_name`. + +Example + +```yaml +- function: MIN + lookup_fields: [receive_time] + output_fields: [receive_time] +``` + + ### Number SUM NUMBER_SUM is used to sum the value of the field in the group of events. The lookup field value must be a number. diff --git a/docs/processor/udf.md b/docs/processor/udf.md index e480275..7f5c656 100644 --- a/docs/processor/udf.md +++ b/docs/processor/udf.md @@ -10,11 +10,13 @@ - [Current Unix Timestamp](#current-unix-timestamp) - [Domain](#domain) - [Drop](#drop) +- [Encrypt](#encrypt) - [Eval](#eval) - [Flatten](#flatten) - [From Unix Timestamp](#from-unix-timestamp) - [Generate String Array](#generate-string-array) - [GeoIP Lookup](#geoip-lookup) +- [HMAC](#hmac) - [JSON Extract](#json-extract) - [Path Combine](#path-combine) - [Rename](#rename) @@ -174,6 +176,30 @@ Example: filter: event.server_ip == '4.4.4.4' ``` +### Encrypt + +Encrypt function is used to encrypt the field value by the specified algorithm. + +Note: This feature allows you to use a third-party RESTful API to retrieve encrypted fields. By using these fields as criteria, you can determine whether the current field is encrypted. You must also set the projection.encrypt.schema.registry.uri as a job property. +For example, setting `projection.encrypt.schema.registry.uri=127.0.0.1:9999/v1/schema/session_record?option=encrypt_fields` will return the encrypted fields in an array format. + +```ENCRYPT(filter, lookup_fields, output_fields[, parameters])``` +- filter: optional +- lookup_fields: required +- output_fields: required +- parameters: required + - identifier: `<String>` required. The identifier of the encryption algorithm. Supports `aes-128-gcm96`, `aes-256-gcm96`, and `sm4-gcm96`. + +Example: +Encrypt the phone number by the AES-128-GCM96 algorithm. Here phone_number will replace the original value with the encrypted value. +```yaml +- function: ENCRYPT + lookup_fields: [phone_number] + output_fields: [phone_number] + parameters: + identifier: aes-128-gcm96 +``` + ### Eval Eval function is used to adds or removes fields from events by evaluating an value expression. @@ -383,6 +409,29 @@ Example: CITY: server_administrative_area ``` +### HMAC + +HMAC function is used to generate the hash-based message authentication code (HMAC) by the specified algorithm. + +```HMAC(filter, lookup_fields, output_fields[, parameters])``` +- filter: optional +- lookup_fields: required +- output_fields: required +- parameters: required + - secret_key: `<String>` required. The secret key used to generate the HMAC. + - output_format: `<String>` required. Enum: `HEX`, `BASE64`. Default is `BASE64`. + +Example: + +```yaml + - function: HMAC + lookup_fields: [phone_number] + output_fields: [phone_number_hmac] + parameters: + secret_key: abcdefg + output_format: BASE64 +``` + ### JSON Extract JSON extract function is used to extract the value from json string. @@ -604,4 +653,5 @@ Example: output_fields: [log_uuid] ``` -Result: such as 2ed6657d-e927-568b-95e1-2665a8aea6a2.
\ No newline at end of file +Result: such as 2ed6657d-e927-568b-95e1-2665a8aea6a2. + |
