summaryrefslogtreecommitdiff
path: root/docs/processor
diff options
context:
space:
mode:
authordoufenghu <[email protected]>2024-11-01 20:40:46 +0800
committerdoufenghu <[email protected]>2024-11-01 20:40:46 +0800
commit5818ed2ac9ca31a35a55f330160a9cf7f63bf6f3 (patch)
tree0d2f00c6d6c1791de8c5588572e0e7fb538803f2 /docs/processor
parente25eabde3ccb3f0d52346cb11cac757763c41be8 (diff)
[Improve][docs] Add a description of the new features for version 1.7.1-SNAPSHOT.
Diffstat (limited to 'docs/processor')
-rw-r--r--docs/processor/udaf.md38
-rw-r--r--docs/processor/udf.md52
2 files changed, 89 insertions, 1 deletions
diff --git a/docs/processor/udaf.md b/docs/processor/udaf.md
index 66d6ad5..f305201 100644
--- a/docs/processor/udaf.md
+++ b/docs/processor/udaf.md
@@ -9,7 +9,9 @@
- [First Value](#First-Value)
- [Last Value](#Last-Value)
- [Long Count](#Long-Count)
+- [Max](#Max)
- [MEAN](#Mean)
+- [Min](#Min)
- [Number SUM](#Number-SUM)
- [HLLD](#HLLD)
- [Approx Count Distinct HLLD](#Approx-Count-Distinct-HLLD)
@@ -116,6 +118,23 @@ Example
output_fields: [sessions]
```
+### Max
+
+MAX is used to get the maximum value of the field in the group of events.
+
+```MAX(filter, lookup_fields, output_fields)```
+- filter: optional
+- lookup_fields: required. Now only support one field.
+- output_fields: optional. If not set, the output field name is `lookup_field_name`.
+
+Example
+
+```yaml
+- function: MAX
+ lookup_fields: [receive_time]
+ output_fields: [receive_time]
+```
+
### Mean
MEAN is used to calculate the mean value of the field in the group of events. The lookup field value must be a number.
@@ -135,6 +154,25 @@ Example
output_fields: [received_bytes_mean]
```
+
+### Min
+
+MIN is used to get the minimum value of the field in the group of events.
+
+```MIN(filter, lookup_fields, output_fields)```
+- filter: optional
+- lookup_fields: required. Now only support one field.
+- output_fields: optional. If not set, the output field name is `lookup_field_name`.
+
+Example
+
+```yaml
+- function: MIN
+ lookup_fields: [receive_time]
+ output_fields: [receive_time]
+```
+
+
### Number SUM
NUMBER_SUM is used to sum the value of the field in the group of events. The lookup field value must be a number.
diff --git a/docs/processor/udf.md b/docs/processor/udf.md
index e480275..7f5c656 100644
--- a/docs/processor/udf.md
+++ b/docs/processor/udf.md
@@ -10,11 +10,13 @@
- [Current Unix Timestamp](#current-unix-timestamp)
- [Domain](#domain)
- [Drop](#drop)
+- [Encrypt](#encrypt)
- [Eval](#eval)
- [Flatten](#flatten)
- [From Unix Timestamp](#from-unix-timestamp)
- [Generate String Array](#generate-string-array)
- [GeoIP Lookup](#geoip-lookup)
+- [HMAC](#hmac)
- [JSON Extract](#json-extract)
- [Path Combine](#path-combine)
- [Rename](#rename)
@@ -174,6 +176,30 @@ Example:
filter: event.server_ip == '4.4.4.4'
```
+### Encrypt
+
+Encrypt function is used to encrypt the field value by the specified algorithm.
+
+Note: This feature allows you to use a third-party RESTful API to retrieve encrypted fields. By using these fields as criteria, you can determine whether the current field is encrypted. You must also set the projection.encrypt.schema.registry.uri as a job property.
+For example, setting `projection.encrypt.schema.registry.uri=127.0.0.1:9999/v1/schema/session_record?option=encrypt_fields` will return the encrypted fields in an array format.
+
+```ENCRYPT(filter, lookup_fields, output_fields[, parameters])```
+- filter: optional
+- lookup_fields: required
+- output_fields: required
+- parameters: required
+ - identifier: `<String>` required. The identifier of the encryption algorithm. Supports `aes-128-gcm96`, `aes-256-gcm96`, and `sm4-gcm96`.
+
+Example:
+Encrypt the phone number by the AES-128-GCM96 algorithm. Here phone_number will replace the original value with the encrypted value.
+```yaml
+- function: ENCRYPT
+ lookup_fields: [phone_number]
+ output_fields: [phone_number]
+ parameters:
+ identifier: aes-128-gcm96
+```
+
### Eval
Eval function is used to adds or removes fields from events by evaluating an value expression.
@@ -383,6 +409,29 @@ Example:
CITY: server_administrative_area
```
+### HMAC
+
+HMAC function is used to generate the hash-based message authentication code (HMAC) by the specified algorithm.
+
+```HMAC(filter, lookup_fields, output_fields[, parameters])```
+- filter: optional
+- lookup_fields: required
+- output_fields: required
+- parameters: required
+ - secret_key: `<String>` required. The secret key used to generate the HMAC.
+ - output_format: `<String>` required. Enum: `HEX`, `BASE64`. Default is `BASE64`.
+
+Example:
+
+```yaml
+ - function: HMAC
+ lookup_fields: [phone_number]
+ output_fields: [phone_number_hmac]
+ parameters:
+ secret_key: abcdefg
+ output_format: BASE64
+```
+
### JSON Extract
JSON extract function is used to extract the value from json string.
@@ -604,4 +653,5 @@ Example:
output_fields: [log_uuid]
```
-Result: such as 2ed6657d-e927-568b-95e1-2665a8aea6a2. \ No newline at end of file
+Result: such as 2ed6657d-e927-568b-95e1-2665a8aea6a2.
+