diff options
| author | root <[email protected]> | 2024-08-22 10:26:59 +0000 |
|---|---|---|
| committer | root <[email protected]> | 2024-08-22 10:26:59 +0000 |
| commit | 54a70f19d9f5aa50142aed1779132639c34cc5dd (patch) | |
| tree | cf5fd594f5822c0f49195fbfa97c65f0b1df9839 /docs | |
| parent | 72cf89723d7d751470204c95e633f7b518e5952c (diff) | |
rename terminology "group" to "object"
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/api_reference.md | 102 | ||||
| -rw-r--r-- | docs/configuration_management.md | 4 | ||||
| -rw-r--r-- | docs/getting_started.md | 34 | ||||
| -rw-r--r-- | docs/group_hierarchy.md | 158 | ||||
| -rw-r--r-- | docs/history.md | 6 | ||||
| -rw-r--r-- | docs/imgs/NOT-clause-objects.png (renamed from docs/imgs/NOT-clause-groups.png) | bin | 43309 -> 43309 bytes | |||
| -rw-r--r-- | docs/imgs/object-hierarchy1.png (renamed from docs/imgs/group-hierarchy1.png) | bin | 63866 -> 63866 bytes | |||
| -rw-r--r-- | docs/imgs/object-hierarchy2.png (renamed from docs/imgs/group-hierarchy2.png) | bin | 62829 -> 62829 bytes | |||
| -rw-r--r-- | docs/imgs/object-hierarchy3.png (renamed from docs/imgs/group-hierarchy3.png) | bin | 61114 -> 61114 bytes | |||
| -rw-r--r-- | docs/logical_combinations.md | 92 | ||||
| -rw-r--r-- | docs/maat_table.md | 90 | ||||
| -rw-r--r-- | docs/monitor_tools.md | 2 | ||||
| -rw-r--r-- | docs/object_hierarchy.md | 158 | ||||
| -rw-r--r-- | docs/overview.md | 14 | ||||
| -rw-r--r-- | docs/terminology.md | 30 |
15 files changed, 345 insertions, 345 deletions
diff --git a/docs/api_reference.md b/docs/api_reference.md index 48c1b29..b5eaab1 100644 --- a/docs/api_reference.md +++ b/docs/api_reference.md @@ -16,7 +16,7 @@ After that, call maat_new to create a maat instance. Next, for the two main usag * Call the maat_scan_xx interface to perform the scan task and return the hit rule_id. - * If you are interested in the hit item, group, or path information, you can call the maat_state_get_xx interface as needed. + * If you are interested in the hit item, object, or path information, you can call the maat_state_get_xx interface as needed. * Callback Scenario @@ -55,7 +55,7 @@ After that, call maat_new to create a maat instance. Next, for the two main usag * [maat_options_set_json_file_decrypt_key](#maat_options_set_json_file_decrypt_key) * [maat_options_set_expr_engine](#maat_options_set_expr_engine) * [maat_options_set_hit_path_enabled](#maat_options_set_hit_path_enabled) -* [maat_options_set_hit_group_enabled](#maat_options_set_hit_group_enabled) +* [maat_options_set_hit_object_enabled](#maat_options_set_hit_object_enabled) ## maat instance @@ -72,12 +72,12 @@ After that, call maat_new to create a maat instance. Next, for the two main usag * [maat_state_get_hit_paths](#maat_state_get_hit_paths) * [maat_state_get_rule_table_ids](#maat_state_get_rule_table_ids) * [maat_state_get_scan_count](#maat_state_get_scan_count) -* [maat_state_get_direct_hit_groups](#maat_state_get_direct_hit_groups) -* [maat_state_get_direct_hit_group_cnt](#maat_state_get_direct_hit_group_cnt) -* [maat_state_get_indirect_hit_groups](#maat_state_get_indirect_hit_groups) -* [maat_state_get_indirect_hit_group_cnt](#maat_state_get_indirect_hit_group_cnt) -* [maat_state_get_last_hit_groups](#maat_state_get_last_hit_groups) -* [maat_state_get_last_hit_group_cnt](#maat_state_get_last_hit_group_cnt) +* [maat_state_get_direct_hit_objects](#maat_state_get_direct_hit_objects) +* [maat_state_get_direct_hit_object_cnt](#maat_state_get_direct_hit_object_cnt) +* [maat_state_get_indirect_hit_objects](#maat_state_get_indirect_hit_objects) +* [maat_state_get_indirect_hit_object_cnt](#maat_state_get_indirect_hit_object_cnt) +* [maat_state_get_last_hit_objects](#maat_state_get_last_hit_objects) +* [maat_state_get_last_hit_object_cnt](#maat_state_get_last_hit_object_cnt) ## maat scan @@ -86,7 +86,7 @@ After that, call maat_new to create a maat instance. Next, for the two main usag * [maat_scan_ipv4](#maat_scan_ipv4) * [maat_scan_ipv6](#maat_scan_ipv6) * [maat_scan_string](#maat_scan_string) -* [maat_scan_group](#maat_scan_group) +* [maat_scan_object](#maat_scan_object) * [maat_scan_not_logic](#maat_scan_not_logic) ## maat stream @@ -500,13 +500,13 @@ By default, maat does not store the hit path, and you can call this interface if **Returns**: success(0), failure(-1) -### maat_options_set_hit_group_enabled +### maat_options_set_hit_object_enabled ```c -int maat_options_set_hit_group_enabled(struct maat_options *opts); +int maat_options_set_hit_object_enabled(struct maat_options *opts); ``` -By default, maat does not store the hit groups, and you can call this interface if you need to get this information. +By default, maat does not store the hit objects, and you can call this interface if you need to get this information. **Parameters**: * opts - The pointer of maat_options structure. @@ -548,7 +548,7 @@ Destroy a maat instance. struct maat_state *maat_state_new(struct maat *instance, int thread_id); ``` -Before performing the actual scanning task, you need to call this interface to create a per-thread maat_state structure, which will be used as an input parameter when calling the maat_scan_xx interface to store the intermediate scanning state. This is especially useful in composite scanning scenarios to accurately provide all hit results from multiple scans. Additionally, hit paths, group_id of hits, and other information generated by the scanning are temporarily stored in it, and can be retrieved through the maat_state_get_xx interface. +Before performing the actual scanning task, you need to call this interface to create a per-thread maat_state structure, which will be used as an input parameter when calling the maat_scan_xx interface to store the intermediate scanning state. This is especially useful in composite scanning scenarios to accurately provide all hit results from multiple scans. Additionally, hit paths, object_id of hits, and other information generated by the scanning are temporarily stored in it, and can be retrieved through the maat_state_get_xx interface. **Parameters**: * instance - Maat instance created by maat_new(). @@ -609,7 +609,7 @@ int maat_state_set_scan_rule_table(struct maat_state *state, int rule_table_id); ``` -Maat supports loading multiple rule tables and building their respective runtime. Within the table schema, there is a rule table that has a `default_rule_table` field, indicating that maat will use the runtime of this table to calculate which rule_id will be hit by the hit group_id. If the caller wants to use another rule table runtime for the above calculation, this interface can be invoked to set it up. +Maat supports loading multiple rule tables and building their respective runtime. Within the table schema, there is a rule table that has a `default_rule_table` field, indicating that maat will use the runtime of this table to calculate which rule_id will be hit by the hit object_id. If the caller wants to use another rule table runtime for the above calculation, this interface can be invoked to set it up. **Parameters**: * state - Pointer to the maat state structure. @@ -670,106 +670,106 @@ Gets the number of scans since maat state was created. **Returns**: The number of scans. -### maat_state_get_direct_hit_groups +### maat_state_get_direct_hit_objects ```c -int maat_state_get_direct_hit_groups(struct maat_state *state, - struct maat_hit_group *group_array, +int maat_state_get_direct_hit_objects(struct maat_state *state, + struct maat_hit_object *object_array, size_t array_size); ``` -Get the direct hit_groups, including the table_id from the scan input, the item_id and its group_id in the corresponding item table. +Get the direct hit_objects, including the table_id from the scan input, the item_id and its object_id in the corresponding item table. -`Note`: hit_groups may be duplicated. +`Note`: hit_objects may be duplicated. **Parameters**: * state - Pointer to the maat state structure. -* group_array - Array to store the maat_hit_groups. +* object_array - Array to store the maat_hit_objects. * array_size - Size of the array. -**Returns**: The number of direct hit_groups on success, and -1 is returned on failure. +**Returns**: The number of direct hit_objects on success, and -1 is returned on failure. -### maat_state_get_direct_hit_group_cnt +### maat_state_get_direct_hit_object_cnt ```c -size_t maat_state_get_direct_hit_group_cnt(struct maat_state *state); +size_t maat_state_get_direct_hit_object_cnt(struct maat_state *state); ``` -Get the number of direct hit_groups. +Get the number of direct hit_objects. **Parameters**: * state - Pointer to the maat state structure. -**Returns**: The number of direct hit_groups. +**Returns**: The number of direct hit_objects. -### maat_state_get_indirect_hit_groups +### maat_state_get_indirect_hit_objects ```c -int maat_state_get_indirect_hit_groups(struct maat_state *state, - struct maat_hit_group *group_array, +int maat_state_get_indirect_hit_objects(struct maat_state *state, + struct maat_hit_object *object_array, size_t array_size); ``` -Gets the indirect hit_groups, including the table_id from the scan input, the id of the super groups referencing the hit items. +Gets the indirect hit_objects, including the table_id from the scan input, the id of the super objects referencing the hit items. -`Note`: hit_groups may be duplicated. +`Note`: hit_objects may be duplicated. **Parameters**: * state - Pointer to the maat state structure. -* group_array - Array to store the maat_hit_groups. +* object_array - Array to store the maat_hit_objects. * array_size - Size of the array. -**Returns**: The number of indirect hit_groups on success, and -1 is returned on failure. +**Returns**: The number of indirect hit_objects on success, and -1 is returned on failure. -### maat_state_get_indirect_hit_group_cnt +### maat_state_get_indirect_hit_object_cnt ```c -size_t maat_state_get_indirect_hit_group_cnt(struct maat_state *state); +size_t maat_state_get_indirect_hit_object_cnt(struct maat_state *state); ``` -Get the number of indirect hit_groups. +Get the number of indirect hit_objects. **Parameters**: * state - Pointer to the maat state structure. -**Returns**: The number of indirect hit_groups. +**Returns**: The number of indirect hit_objects. -### maat_state_get_last_hit_groups +### maat_state_get_last_hit_objects ```c -int maat_state_get_last_hit_groups(struct maat_state *state, - struct maat_hit_group *group_array, +int maat_state_get_last_hit_objects(struct maat_state *state, + struct maat_hit_object *object_array, size_t array_size); ``` -Get all hit_groups in the latest scan, including the table_id from the scan input, the item_id and its group_id in the corresponding item table, the id of the super groups referencing the hit items. +Get all hit_objects in the latest scan, including the table_id from the scan input, the item_id and its object_id in the corresponding item table, the id of the super objects referencing the hit items. -`Note`: hit_groups is not duplicated. +`Note`: hit_objects is not duplicated. **Parameters**: * state - Pointer to the maat state structure. -* group_array - Array to store the maat_hit_groups. +* object_array - Array to store the maat_hit_objects. * array_size - Size of the array. -**Returns**: The number of all hit_groups in the latest scan on success, and -1 is returned on failure. +**Returns**: The number of all hit_objects in the latest scan on success, and -1 is returned on failure. -### maat_state_get_last_hit_group_cnt +### maat_state_get_last_hit_object_cnt ```c -size_t maat_state_get_last_hit_group_cnt(struct maat_state *state); +size_t maat_state_get_last_hit_object_cnt(struct maat_state *state); ``` -Get the number of all hit_groups in the latest scan. +Get the number of all hit_objects in the latest scan. **Parameters**: * state - Pointer to the maat state structure. -**Returns**: The number of all hit_groups in the latest scan. +**Returns**: The number of all hit_objects in the latest scan. ### maat_scan_flag @@ -841,16 +841,16 @@ int maat_scan_string(struct maat *instance, int table_id, Same as above, the difference is that the data to be scanned is a string. -### maat_scan_group +### maat_scan_object ```c -int maat_scan_group(struct maat *instance, int table_id, - struct maat_hit_group *groups, size_t n_group, +int maat_scan_object(struct maat *instance, int table_id, + struct maat_hit_object *objects, size_t n_object, long long *results, size_t n_result, size_t *n_hit_result, struct maat_state *state); ``` -Different from the above interface, the input to this interface is the hit item_id and group_id. Therefore, there is no need to execute the underlying scanning engine. It only needs to calculate the hit rule_id based on the hit group_id. +Different from the above interface, the input to this interface is the hit item_id and object_id. Therefore, there is no need to execute the underlying scanning engine. It only needs to calculate the hit rule_id based on the hit object_id. ### maat_scan_not_logic diff --git a/docs/configuration_management.md b/docs/configuration_management.md index 513adb7..efa2f6e 100644 --- a/docs/configuration_management.md +++ b/docs/configuration_management.md @@ -9,7 +9,7 @@ Maat supports three configuration loading modes: ## 1.<a name='Redis mode'></a> Redis mode -As shown in the diagram below, the data source is stored in the relational database `MariaDB`. An adapter writes this data into the `leader redis`, which is then distributed through redis's `leader-follower synchronization` mechanism. The storage structure design in redis for configurations does not need to consider the logical relationships between rule, group, and item. Maat will construct these logical relationships internally after loading the configurations from redis. +As shown in the diagram below, the data source is stored in the relational database `MariaDB`. An adapter writes this data into the `leader redis`, which is then distributed through redis's `leader-follower synchronization` mechanism. The storage structure design in redis for configurations does not need to consider the logical relationships between rule, object, and item. Maat will construct these logical relationships internally after loading the configurations from redis. <img src="./imgs/sync-with-redis.png" width="600" height="300" > @@ -26,7 +26,7 @@ As shown in the diagram below, the data source is stored in the relational datab | MAAT_LABEL_INDEX | Label index | Sorted Set, where the element is the configuration table name, rule_id, and the score is the label_id | | | EFFECTIVE_RULE:TableName,ID OBSOLETE_RULE:TableName,ID | Main configuration | string | Active configurations, loaded one by one by MAAT. | | SEQUENCE_REGION | Region ID generation sequence | INTEGER | Used by producers to generate unique region_id(item_id) | -| SEQUENCE_GROUP | Group ID generation sequence | INTEGER | Used by producers to generate unique group_id | +| SEQUENCE_OBJECT | Object ID generation sequence | INTEGER | Used by producers to generate unique object_id | | EXPIRE_OP_LOCK | Distributed lock | string "locked" | Ensures that at most one writer performs eviction. | `Note`: The following api supports writing one line of configuration to redis. diff --git a/docs/getting_started.md b/docs/getting_started.md index f87b01e..cc92a95 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -31,7 +31,7 @@ A complete use case consists of three parts: ### Case1 -In the scanning scenario, it is necessary to configure the schema of multiple tables, including the item table, group2rule table, and rule table. If there is also group nesting involved, the schema of the group2group table needs to be configured. +In the scanning scenario, it is necessary to configure the schema of multiple tables, including the item table, object2rule table, and rule table. If there is also object nesting involved, the schema of the object2object table needs to be configured. **(1) table schema** @@ -53,12 +53,12 @@ Table schema is stored in a json file(such as table_info.conf), which is loaded }, { "table_id":1, - "table_name":"GROUP2RULE", - "table_type":"group2rule", - "associated_rule_table_id":0, /* associate rule table_id, group2rule table shares the same runtime with the corresponding rule table, so it needs to be mapped to the corresponding rule table. */ + "table_name":"OBJECT2RULE", + "table_type":"object2rule", + "associated_rule_table_id":0, /* associate rule table_id, object2rule table shares the same runtime with the corresponding rule table, so it needs to be mapped to the corresponding rule table. */ "valid_column":3, "custom": { - "group_id":1, + "object_id":1, "rule_id":2, "negate_option":4, "attribute_name":5, @@ -67,13 +67,13 @@ Table schema is stored in a json file(such as table_info.conf), which is loaded }, { "table_id":2, - "table_name":"GROUP2GROUP", - "table_type":"group2group", + "table_name":"OBJECT2OBJECT", + "table_type":"object2object", "valid_column":4, "custom": { - "group_id":1, - "included_sub_group_ids":2, - "excluded_sub_group_ids":3 + "object_id":1, + "included_sub_object_ids":2, + "excluded_sub_object_ids":3 } }, { @@ -83,7 +83,7 @@ Table schema is stored in a json file(such as table_info.conf), which is loaded "valid_column":7, "custom": { "item_id":1, - "group_id":2, + "object_id":2, "keywords":3, "expr_type":4, "match_method":5, @@ -99,8 +99,8 @@ Configurations are stored in a json file(such as maat_json.json), which is loade ```json { "rule_table": "RULE", - "group2rule_table": "GROUP2RULE", - "group2group_table": "GROUP2GROUP", + "object2rule_table": "OBJECT2RULE", + "object2object_table": "OBJECT2OBJECT", "rules": [ { "rule_id": 123, @@ -110,9 +110,9 @@ Configurations are stored in a json file(such as maat_json.json), which is loade "do_log": 1, "user_region": "anything", "is_valid": "yes", - "groups": [ + "objects": [ { - "group_name": "Untitled", + "object_name": "Untitled", "regions": [ { "table_name": "HTTP_URL", @@ -216,8 +216,8 @@ In the callback scenario, only the schema of the corresponding table needs to be ```json { "rule_table": "RULE", - "group2rule_table": "GROUP2RULE", - "group2group_table": "GROUP2GROUP", + "object2rule_table": "OBJECT2RULE", + "object2object_table": "OBJECT2OBJECT", "plugin_table": { "table_name": "TEST_IP_PLUGIN_WITH_EXDATA", "table_content": [ diff --git a/docs/group_hierarchy.md b/docs/group_hierarchy.md deleted file mode 100644 index 32fa852..0000000 --- a/docs/group_hierarchy.md +++ /dev/null @@ -1,158 +0,0 @@ -# Group(Object) hierarchies - -A group can reference other groups, and can also be referenced by other groups. For example, group_A references group_B, group_A is the superior group of group_B, and group_B is the subordinate group of group_A. There are two reference relationships between groups: `include` and `exclude`. - -## Include - -Include is equivalent to the inclusion semantics in set theory. For example, when group_A is included by group_B, if a traffic attribute satisfies group_A, group_B is satisfied. - -## Exclude - -A group defines a subset of an group type, such as network addresses or port numbers. The definition is made using items, which can be used to add to or exclude from the group definition. Groups can also have subordinate groups whose definitions are included in the superior group. - -There are rules of precedence to take into account when defining groups: - -- Excluding has precedence over including in the same group. - -- Items in a superior group have precedence over items in a subordinate group. - -- Items in a superior group are not taken into account in a subordinate group, if the subordinate group is used directly in a rule. - -- Peer groups (different subordinate groups of the same superior group) do not affect each other. - -In short, to determine the set defined by a group, perform the following calculation: - -1. For each subordinate group (remember sibling groups do not affect each other): - - Add included items. - - - Subtract excluded items. - -2. Add included items in the group itself, overriding any excludes in the subordinate groups. - -3. Subtract excluded items in the group itself. - -The following figure shows a group with an included set and an excluded subset. - -<img src="./imgs/exclude.png" width="350" height="200" > - -Now, consider adding a subordinate group. The subordinate group also has an included set and an excluded subset. If the superior group is used, the result is shown in the following figure “A superior and subordinate group”. - -<img src="./imgs/hierarchy-and-exclude.png" width="350" height="230" > - -As can be seen, the excluded item in the subordinate group is overwritten since it is in the included set of the superior group. Also, the excluded item from the superior group excludes part of the included item in the subordinate. - -If only the subordinate group is used in a rule condition, the superior group items are disregarded, leaving the set shown in the next figure, “The subordinate group”. - -<img src="./imgs/sub-exclude.png" width="350" height="230" > - -Restrictions: - -- A group can only include or exclude groups of the same type. - -- A group should include at least one subordinate group. (Exclude only is not allowed) - -- Traffic attribute using stream scan cannot allow use object(group) with exclude, i.e., keywords object on HTTP Response Body, Email attachment. - -Now, let's see a graph of hierarchy example, where the dotted line means exclude. The matched subordinate groups and activated superiors are listed in the following table. - -Now, let's see a graph of hierarchy example, where the dotted line means exclude. If the matched subordinate groups: **{g11, g13}**, then activated superiors: **{g2, g7}**. - -<img src="./imgs/group-hierarchy1.png" width="750" height="350" > - -The analysis process is as follows: - -**matched groups {g11, g13}** - -**Level 1**: g11 => incl{g6, g7} excl{null}, g13 => incl{g3} excl{g4}; new matched groups {g6, g7, g3}, `all matched groups` {g11, g13, g6, g7, g3} - -**new matched groups {g6, g7, g3}** - -**Level 2**: g6 => incl{g1} excl{g2, g3}, g7 => incl{g2, g4} excl{g6}; new matched groups {g1, g2, g4}, `all matched groups` {g11, g13, g7, g1, g2, g4} - -**new matched groups {g1, g2, g4}** - -**Level 3**: end - -Check the validity of `all matched groups` {g11, g13, g7, g1, g2, g4}: - -1. g11, g13 are the originally matched nodes, so keep {g11, g13}. - -2. g7 is derived from the match of g11, implying that g10 is not matched (if g10 is matched, then g7 is not matched, as the exclude priority is higher), so keep g7. - -3. g1 is derived from the match of g6, but since g6 is excluded due to both g11 and g7, and g6 excludes g7 with higher priority, g1 is no longer matched. Therefore, remove g1 from the set. - -3. g1 is derived from the match of g6, which in turn is derived from the match of g11. However, since g7 is also matched and g6 excludes g7 with higher priority, g7's match causes g6 to be unmatched, and consequently, g1 is no longer matched. Therefore, g1 is removed from the set. - -4. g2 is derived from the match of g7, with the implied condition that g6 is not matched, so keep g2. - -5. Finally, `all matched groups` {g11, g13, g7, g2}, `matched super groups` {g2, g7}. - -Remove the dotted line from g7 to g6, as shown in the diagram below. If the matched subordinate groups: **{g11, g13}**, then activated groups: **{g1, g6, g7}**. - -<img src="./imgs/group-hierarchy2.png" width="750" height="350" > - -The analysis process is as follows: - -**matched groups {g11, g13}** - -**Level 1**: g11 => incl{g6, g7} excl{null}, g13 => incl{g3} excl{g4}; new matched groups {g6, g7, g3}, `all matched groups` {g11, g13, g6, g7, g3} - -**new matched groups {g6, g7, g3}** - -**Level 2**: g6 => incl{g1} excl{g2, g3}, g7 => incl{g2, g4} excl{null}; new matched groups {g1, g2, g4}, `all matched groups` {g11, g13, g6, g1, g7, g2, g4} - -**new matched groups {g1, g2, g4}** - -**Level 3**: end - -Check the validity of `all matched groups` {g11, g13, g6, g1, g7, g2, g4}: - -1. g11, g13 are the originally matched nodes, so keep {g11, g13}. - -2. g6 is derived from the match of g11, implying the g10 is not matched(if g10 is matched, then g7 is not matched, as the exclude priority is higher), so keep g6. - -3. g1 is derived from the match of g6, which in turn is derived from the match of g11, implying the g5 is not matched, so keep g1. - -4. g7 is derived from the match of g11, implying the g10 is not matched, so keep g7. - -5. g2 is derived from the match of g7. However, since g6 is also matched and g2 excludes g6 with higher priority, g6's match causes g2 to be unmatched. Therefore, g2 is removed from the set. - -6. g4 is derived from the match of g7. However, since g13 is also matched and g4 excludes g13 with higher priority, g13's match causes g4 to be unmatched. Therefore, g4 is removed from the set. - -7. Finally, `all matched groups` {g11, g13, g1, g6, g7}, `matched super groups` {g1, g6, g7}. - -Again remove the dotted line from g13 to g4, as shown in the diagram below. If the matched subordinate groups: **{g11, g13}**, then activated superiors: **{g1, g4, g6, g7}**. - -<img src="./imgs/group-hierarchy3.png" width="750" height="350" > - -The analysis process is as follows: - -**matched groups {g11, g13}** - -**Level 1**: g11 => incl{g6, g7} excl{null}, g13 => incl{g3} excl{null}; new matched groups {g6, g7, g3}, `all matched groups` {g11, g13, g6, g7, g3} - -**new matched groups {g6, g7, g3}** - -**Level 2**: g6 => incl{g1} excl{g2, g3}, g7 => incl{g2, g4} excl{null}; new matched groups {g1, g2, g4}, `all matched groups` {g11, g13, g6, g1, g7, g2, g4} - -**new matched groups {g1, g2, g4}** - -**Level 3**: end - -Check the validity of `all matched groups` {g11, g13, g6, g1, g7, g2, g4}: - -1. g11, g13 are the originally matched nodes, so keep {g11, g13}. - -2. g6 is derived from the match of g11, implying the g10 is not matched(if g10 is matched, then g7 is not matched, as the exclude priority is higher), so keep g6. - -3. g1 is derived from the match of g6, which in turn is derived from the match of g11, implying the g5 is not matched, so keep g1. - -4. g7 is derived from the match of g11, implying the g10 is not matched, so keep g7. - -5. g2 is derived from the match of g7. However, since g6 is also matched and g2 excludes g6 with higher priority, g6's match causes g2 to be unmatched. Therefore, g2 is removed from the set. - -6. g4 is derived from the match of g13, so keep g4. - -7. Finally, `all matched groups` {g11, g13, g6, g1, g7, g4}, `matched super groups` {g1, g4, g6, g7}. - -The above example is strictly consistent with the case `GroupExcludeTest.level_3_function` in the [test/group_nesting](../test/group_nesting/) test. Please run it to see the actual execution results.
\ No newline at end of file diff --git a/docs/history.md b/docs/history.md index 429a429..e9c99d0 100644 --- a/docs/history.md +++ b/docs/history.md @@ -8,15 +8,15 @@ | v3.1.20| 2021-4-28 | Add explanation for loading gzip-compressed JSON files | Zheng Chao | | v3.1.5 | 2021-3-12 | Add matching examples for callback class FQDN table | Zheng Chao | | v3.1.4 | 2020-11-04| When content foreign key is "null," it represents an empty file | Zheng Chao | -| v3.1.1 | 2020-9-27 | Add explanation for creating attributes on multiple different types of physical tables | Zheng Chao | +| v3.1.1 | 2020-9-27 | Add explanation for creating virtual tables on multiple different types of physical tables | Zheng Chao | | v3.1.0 | 2020-9-18 | Add FQDN callback table | Zheng Chao | | v3.0.4 | 2020-8-17 | Add extended numerical domain configuration interval_plus | Zheng Chao | -| v3.0 | 2020-6-29 | Maat 3.0, support conditions, original grouping table split into grouping relationship table and grouping compilation table | Zheng Chao | +| v3.0 | 2020-6-29 | Maat 3.0, support clauses, original grouping table split into grouping relationship table and grouping compilation table | Zheng Chao | | v2.8 | 2020-3-13 | Support composite tables | Zheng Chao | | v2.8 | 2020-2-11 | Support Hit Path | Zheng Chao | | v2.8 | 2020-2-4 | Support sorting strategies by Evaluation Order | Zheng Chao | | v2.8 | 2020-1-22 | Maat JSON file encryption support, see 17.3 | Zheng Chao | -| v2.8 | 2019-7-28 | 1. No longer compatible with no grouping tables 2. Support attributes | Zheng Chao | +| v2.8 | 2019-7-28 | 1. No longer compatible with no grouping tables 2. Support virtual tables | Zheng Chao | | v2.7.2 | 2019-7-10 | Expand IP configuration table to support CIDR format | Zheng Chao | | v2.7.1 | 2019-5-23 | Rename rich IP class configuration table to extended IP configuration table | Zheng Chao | | v2.7.0 | 2019-5-12 | 1. Add explanation for subgroup relationships 2. Add explanation for rich IP class configuration | Zheng Chao | diff --git a/docs/imgs/NOT-clause-groups.png b/docs/imgs/NOT-clause-objects.png Binary files differindex bc42519..bc42519 100644 --- a/docs/imgs/NOT-clause-groups.png +++ b/docs/imgs/NOT-clause-objects.png diff --git a/docs/imgs/group-hierarchy1.png b/docs/imgs/object-hierarchy1.png Binary files differindex 8bf11ee..8bf11ee 100644 --- a/docs/imgs/group-hierarchy1.png +++ b/docs/imgs/object-hierarchy1.png diff --git a/docs/imgs/group-hierarchy2.png b/docs/imgs/object-hierarchy2.png Binary files differindex 751605d..751605d 100644 --- a/docs/imgs/group-hierarchy2.png +++ b/docs/imgs/object-hierarchy2.png diff --git a/docs/imgs/group-hierarchy3.png b/docs/imgs/object-hierarchy3.png Binary files differindex 0a879b3..0a879b3 100644 --- a/docs/imgs/group-hierarchy3.png +++ b/docs/imgs/object-hierarchy3.png diff --git a/docs/logical_combinations.md b/docs/logical_combinations.md index ffe1041..3c09331 100644 --- a/docs/logical_combinations.md +++ b/docs/logical_combinations.md @@ -1,26 +1,26 @@ # Logical combinations -When you understand the [configuration relationship](./overview.md#12-configuration-relationship) and [group hierarchy](./group_hierarchy.md), you will find multiple layers of logical relationships. Are all these logical relationships necessary? If yes, what are the use cases targeted by each? This document serves as a cheat sheet for you to understand the logical combinations of maat rules. +When you understand the [configuration relationship](./overview.md#12-configuration-relationship) and [object hierarchy](./object_hierarchy.md), you will find multiple layers of logical relationships. Are all these logical relationships necessary? If yes, what are the use cases targeted by each? This document serves as a cheat sheet for you to understand the logical combinations of maat rules. Before describing specific rules, we need to define the syntax of logical combinations. A rule is written using infix notation, which consists of operands, operators, and parentheses. The operands are rule IDs, and operators are "!(NOT)", "&(AND)", "|(OR)". Before showing how to configure the specific rules, we need some raw materials (items). The following is the configuration information for the different types of item tables: **keywords(table_id=1)** -| item_id | group_id | keywords | +| item_id | object_id | keywords | | ------- | -------- | ------------- | | 101 | 201 | www.baidu.com | | 102 | 202 | baidu.com | **ip address(table_id=2)** -| item_id | group_id | ip range | +| item_id | object_id | ip range | | ------- | -------- | --------------------------- | | 110 | 210 | 192.168.1.1 ~ 192.168.1.255 | | 111 | 211 | 192.168.1.11 ~ 192.168.1.20 | | 112 | 212 | 192.168.1.21 ~ 192.168.1.30 | **port(table_id=3)** -| item_id | group_id | interval range | +| item_id | object_id | interval range | | ------- | -------- | -------------- | | 120 | 220 | 100 ~ 200 | | 121 | 221 | 80 ~ 80 | @@ -29,7 +29,7 @@ Before showing how to configure the specific rules, we need some raw materials ( * [AND logic](#and-logic) * [OR logic](#or-logic) * [NOT logic](#not-logic) -* [Group exclude](#group-exclude) +* [Object exclude](#object-exclude) ## AND logic @@ -37,62 +37,62 @@ Before showing how to configure the specific rules, we need some raw materials ( * case1: Deny hosts with source IP addresses in the range of 192.168.1.11 to 192.168.1.20 from accessing the website www.baidu.com. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * www.baidu.com => group_id(201) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * www.baidu.com => object_id(201) ```bash rule(rule) = condition1 & condition2 - = {attribute1, group1, condition_index1} & {attribute2, group2, condition_index2} + = {attribute1, object1, condition_index1} & {attribute2, object2, condition_index2} = {2, 211, 1} & {1, 201, 2} ``` * case2: To block the traffic whose source IP address is 192.168.1.11 to 192.168.1.20 and source port 80. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * 端口80 ~ 80 => group_id(221) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * 端口80 ~ 80 => object_id(221) ```bash rule(rule) = condition1 & condition2 - = {attribute1, group1, condition_index1} & {attribute2, group2, condition_index2} + = {attribute1, object1, condition_index1} & {attribute2, object2, condition_index2} = {2, 211, 1} & {3, 221, 2} ``` The JSON configuration for the logical `AND` can be referenced at [unit_test Json configuration](../test/maat_json.json) with rule_id=152. -group_name: "152_mail_addr" and group_name: "interval_group_refered" are two conditions of this rule, with a logical `AND` relationship between them. +object_name: "152_mail_addr" and object_name: "interval_object_refered" are two conditions of this rule, with a logical `AND` relationship between them. ## OR logic -`Note`: Multiple groups under the same condition have a logical 'OR' relationship. +`Note`: Multiple objects under the same condition have a logical 'OR' relationship. * case1: Deny hosts with source IP addresses in the range of 192.168.1.11 to 192.168.1.30 from accessing the website www.baidu.com. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * 192.168.1.21 ~ 192.168.1.30 => group_id(212) - * www.baidu.com => group_id(201) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * 192.168.1.21 ~ 192.168.1.30 => object_id(212) + * www.baidu.com => object_id(201) ```bash rule(rule) = condition1 & condition2 - = {attribute1, (group1 | group2), condition_index1} & {attribute2, group3, condition_index2} + = {attribute1, (object1 | object2), condition_index1} & {attribute2, object3, condition_index2} = {2, (211 | 212), 1} & {1, 201, 2} ``` * case2: To block the traffic whose source ip address is 192.168.1.11 to 192.168.1.20 and source port 80 or 443. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * port 80 ~ 80 => group_id(221) - * port 443 ~ 443 => group_id(222) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * port 80 ~ 80 => object_id(221) + * port 443 ~ 443 => object_id(222) ```bash rule(rule) = condition1 & condition2 - = {attribute1, group1, condition_index1} & {attribute2, (group2 | group3), condition_index2} + = {attribute1, object1, condition_index1} & {attribute2, (object2 | object3), condition_index2} = {2, 211, 1} & {3, (221 | 222), 2} ``` The JSON configuration for the logical `OR` can be referenced at [unit_test Json configuration](../test/maat_json.json) with rule_id=152. -group_name: "152_mail_addr" contains two regions(items) with a logical `OR` relationship between them. +object_name: "152_mail_addr" contains two regions(items) with a logical `OR` relationship between them. ## NOT logic @@ -101,76 +101,76 @@ group_name: "152_mail_addr" contains two regions(items) with a logical `OR` rela * case1: Hosts with source ip addresses ranging from 192.168.1.11 to 192.168.1.20 are allowed to access websites other than www.baidu.com. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * www.baidu.com => group_id(201) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * www.baidu.com => object_id(201) ```bash rule(rule) = condition1 & !condition2 - = {attribute1, group1, condition_index1} & !{attribute2, group2, condition_index2} + = {attribute1, object1, condition_index1} & !{attribute2, object2, condition_index2} = {2, 211, 1} & !{1, 201, 2} ``` * case2: To block the traffic whose source ip address is in 192.168.1.11 to 192.168.1.20 and the source port is not 80 or 443. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * port 80 ~ 80 => group_id(221) - * port 443 ~ 443 => group_id(222) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * port 80 ~ 80 => object_id(221) + * port 443 ~ 443 => object_id(222) ```bash rule(rule) = condition1 & !condition2 - = {attribute1, group1, condition_index1} & !{attribute2, (group2 | group3), condition_index2} + = {attribute1, object1, condition_index1} & !{attribute2, (object2 | object3), condition_index2} = {2, 211, 1} & !{3, (221 | 222), 2} ``` The JSON configuration for the logical `OR` can be referenced at [unit_test Json configuration](../test/maat_json.json) with rule_id=145. -The group_name: "123_IP_group" is a negate condition of this rule. +The object_name: "123_IP_object" is a negate condition of this rule. -## Group exclude +## Object exclude -`Note`: The `exclude` relationship only applies between groups of the same type, such as super_group1 = include {group1}, exclude {group2}. +`Note`: The `exclude` relationship only applies between objects of the same type, such as super_object1 = include {object1}, exclude {object2}. -Constraint: The super group cannot consist only of exclude groups; it must contain at least one include group. +Constraint: The super object cannot consist only of exclude objects; it must contain at least one include object. * case1: Deny hosts with source IP addresses in the range of 192.168.1.1 to 192.168.1.255 but not in the range of 192.168.1.11 to 192.168.1.20 from accessing the website www.baidu.com. - * 192.168.1.1 ~ 192.168.1.255 => group_id(210) - * 192.168.1.11 ~ 192.168.20 => group_id(211) - * www.baidu.com => group_id(201) + * 192.168.1.1 ~ 192.168.1.255 => object_id(210) + * 192.168.1.11 ~ 192.168.20 => object_id(211) + * www.baidu.com => object_id(201) -In this case, you need to configure super_group1 first. +In this case, you need to configure super_object1 first. ```bash -super_group1 = group1 exclude group2 +super_object1 = object1 exclude object2 = 210 exclude 211 ``` And then configure the rule. ```bash rule(rule) = condition1 & condition2 - = {attribute1, super_group1, condition_index1} & {attribute2, group2, condition_index2} + = {attribute1, super_object1, condition_index1} & {attribute2, object2, condition_index2} = {2, (210 exclude 211), 1} & {1, 201, 2} ``` * case2: Deny hosts with source IP addresses in the range of 192.168.1.11 to 192.168.1.20 from accessing subdomains of baidu.com except for www.baidu.com. - * 192.168.1.11 ~ 192.168.1.20 => group_id(211) - * www.baidu.com => group_id(201) - * baidu.com => group_id(202) + * 192.168.1.11 ~ 192.168.1.20 => object_id(211) + * www.baidu.com => object_id(201) + * baidu.com => object_id(202) -In this case, you need to configure super_group2 first. +In this case, you need to configure super_object2 first. ```bash -super_group2 = group2 exclude group3 +super_object2 = object2 exclude object3 = 202 exclude 201 ``` And then configure the rule. ```bash rule(rule) = condition1 & condition2 - = {attribute1, group1, condition_index1} & {attribute2, super_group2, condition_index2} + = {attribute1, object1, condition_index1} & {attribute2, super_object2, condition_index2} = {2, 211, 1} & {1, (202 exclude 201), 2} ``` The JSON configuration for the logical `OR` can be referenced at [unit_test Json configuration](../test/maat_json.json) with rule_id=200. -The group_name: "ExcludeLogicGroup200" is a super group that includes an include group "ExcludeLogicGroup200_1" and an exclude group "ExcludeLogicGroup200_2".
\ No newline at end of file +The object_name: "ExcludeLogicObject200" is a super object that includes an include object "ExcludeLogicObject200_1" and an exclude object "ExcludeLogicObject200_2".
\ No newline at end of file diff --git a/docs/maat_table.md b/docs/maat_table.md index f1bd1a4..aade183 100644 --- a/docs/maat_table.md +++ b/docs/maat_table.md @@ -12,8 +12,8 @@ Maat tables are divided into two categories: physical tables that actually exist The types of physical tables are as follows: - [item table](#11-item-table) - [rule table](#12-rule-table) -- [group2rule table](#13-group2rule-table) -- [group2group table](#14-group2group-table) +- [object2rule table](#13-object2rule-table) +- [object2object table](#14-object2object-table) - [plugin table](#15-plugin-table) - [ip_plugin table](#16-ip_plugin-table) - [fqdn_plugin table](#17-fqdn_plugin-table) @@ -39,11 +39,11 @@ Each item table must has the following columns: - item_id: In a maat instance, the item id is globally unique, meaning that the item id of different tables must not be duplicate. -- group_id: Indicate the group to which the item belongs, an item belongs to only one group. +- object_id: Indicate the object to which the item belongs, an item belongs to only one object. - is_valid: In incremental updates, 1(valid means add) 0(invalid means del) -The range of item_id(group_id, rule_id) is 0~2^63,which is 8 bytes. +The range of item_id(object_id, rule_id) is 0~2^63,which is 8 bytes. #### 1.1.1 <a name='exprtable'></a> expr item table @@ -52,7 +52,7 @@ Describe matching rules for strings. | **FieldName** | **type** | **constraint** | | ---------------- | -------------- | ------- | | **item_id** | LONG LONG | primary key | -| **group_id** | LONG LONG | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | LONG LONG | leaf object id, can be referenced by object2object & object2rule table | | **keywords** | VARCHAR2(1024) | field to match during scanning | | **expr_type** | INT | 0(keywords), 1(AND expr), 2(regular expr), 3(substring with offset) | **match_method** | INT | only useful when expr_type is 0. 0(sub), 1(suffix), 2(prefix), 3(exactly) | @@ -68,7 +68,7 @@ The table schema is stored in table_info.conf. "valid_column":7, //7th column(is_valid field) "custom": { "item_id":1, //1st column(item_id field) - "group_id":2, //2nd column(group_id field) + "object_id":2, //2nd column(object_id field) "keywords":3, //3rd column(keywords field) "expr_type":4, //4th column(expr_type field) "match_method":5,//5th column(match_method field) @@ -86,7 +86,7 @@ The table schema is stored in table_info.conf. "valid_column":7, "custom": { "item_id":1, - "group_id":2, + "object_id":2, "keywords":3, "expr_type":4, "match_method":5, @@ -158,7 +158,7 @@ Describe extended matching rules for strings by adding the district column. | **FieldName** | **type** | **constraint** | | ---------------- | -------------- | ------- | | **item_id** | LONG LONG | primary key | -| **group_id** | LONG LONG | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | LONG LONG | leaf object id, can be referenced by object2object & object2rule table | | **district** | VARCHAR2(1024) | describe the effective position of the keywords | | **keywords** | VARCHAR2(1024) | field to match during scanning | | **expr_type** | INT | 0(keywords), 1(AND expr), 2(regular expr), 3(substring with offset) @@ -183,7 +183,7 @@ Describe matching rules for IP address. Both the address and port are represente | **FieldName** | **type** | **constraint** | | -------------- | ------------ | -------------- | | **item_id** | LONG LONG | primary key | -| **group_id** | LONG LONG | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | LONG LONG | leaf object id, can be referenced by object2object & object2rule table | | **addr_type** | INT | Ipv4 = 4 Ipv6 = 6 | | **addr_format**| VARCHAR2(40) | ip addr format, single/range/CIDR/mask | | **ip1** | VARCHAR2(40) | start ip | @@ -197,7 +197,7 @@ Determine whether an integer is within a certain numerical range. | **FieldName** | **type** | **constraint** | | ---------------- | -------- | -------------- | | **item_id** | INT | primary key | -| **group_id** | INT | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | INT | leaf object id, can be referenced by object2object & object2rule table | | **low_boundary** | INT | lower bound of the numerical range(including lb), 0 ~ (2^32 - 1)| | **up_boundary** | INT | upper bound of the numerical range(including ub), 0 ~ (2^32 - 1)| | **is_valid** | INT | 0(invalid), 1(valid) | @@ -209,7 +209,7 @@ Describe extended matching rules for integer by adding the district column. | **FieldName** | **type** | **constraint** | | ---------------- | -------- | -------------- | | **item_id** | INT | primary key | -| **group_id** | INT | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | INT | leaf object id, can be referenced by object2object & object2rule table | | **district** | VARCHAR2(1024)| describe the effective position of the keywords | | **low_boundary** | INT | lower bound of the numerical range(including lb), 0 ~ (2^32 - 1)| | **up_boundary** | INT | upper bound of the numerical range(including ub), 0 ~ (2^32 - 1)| @@ -220,7 +220,7 @@ Describe extended matching rules for integer by adding the district column. | **FieldName** | **type** | **constraint** | | ------------- | -------- | -------------- | | **item_id** | INT | primary key | -| **group_id** | INT | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | INT | leaf object id, can be referenced by object2object & object2rule table | | **flag** | INT | flag, 0 ~ (2^32 - 1)| | **flag_mask** | INT | flag_mask, 0 ~ (2^32 - 1)| | **is_valid** | INT | 0(invalid), 1(valid) | @@ -230,7 +230,7 @@ Describe extended matching rules for integer by adding the district column. | **FieldName** | **type** | **constraint** | | ------------- | -------- | -------------- | | **item_id** | INT | primary key | -| **group_id** | INT | leaf group id, can be referenced by group2group & group2rule table | +| **object_id** | INT | leaf object id, can be referenced by object2object & object2rule table | | **district** | INT | describe the effective position of the flag | | **flag** | INT | flag, 0 ~ (2^32 - 1)| | **flag_mask** | INT | flag_mask, 0 ~ (2^32 - 1)| @@ -247,30 +247,30 @@ Describe the specific policy, one maat instance can has multiple rule tables wit | **is_valid** | INT | 0(invalid),1(valid) | | **condition_num** | INT | no more than 8 conditions | -### 1.3 <a name='Group2RuleTable'></a> group2rule table +### 1.3 <a name='Object2RuleTable'></a> object2rule table -Describe the relationship between group and rule. +Describe the relationship between object and rule. | **FieldName** | **type** | **constraint** | | ----------------- | ------------- | -------------- | -| **group_ids** | VARCHAR(256) | group ids are separated by commas(g1,g2,g3) | +| **object_ids** | VARCHAR(256) | object ids are separated by commas(g1,g2,g3) | | **rule_id** | LONG LONG | rule id | | **is_valid** | INT | 0(invalid), 1(valid) | | **negate_option** | INT | logical 'NOT', identify a negate condition, 0(no) 1(yes) | | **attribute** | VARCHAR2(256) | attribute name, NOT NULL | -| **Nth_condition** | INT | the condition seq in (conjunctive normal form)CNF, from 0 to 7. groups with the same condition ID are logical 'OR' | +| **Nth_condition** | INT | the condition seq in (conjunctive normal form)CNF, from 0 to 7. objects with the same condition ID are logical 'OR' | -NOTE: If group_id is invalid in xx_item table, it must be marked as invalid in this table. +NOTE: If object_id is invalid in xx_item table, it must be marked as invalid in this table. -### 1.4 <a name='Group2GroupTable'></a> group2group table +### 1.4 <a name='Object2ObjectTable'></a> object2object table -Describe the relationship between groups. +Describe the relationship between objects. | **FieldName** | **type** | **constraint** | | ---------------------- | ------------ | ---------------| -| **group_id** | LONG LONG | reference from xx_item table's group_id | -| **incl_sub_group_ids** | VARCHAR(256) | included sub group ids are separated by commas(g1,g2,g3)| -| **excl_sub_group_ids** | VARCHAR(256) | excluded sub group ids are separated by commas(g4,g5)| +| **object_id** | LONG LONG | reference from xx_item table's object_id | +| **incl_sub_object_ids** | VARCHAR(256) | included sub object ids are separated by commas(g1,g2,g3)| +| **excl_sub_object_ids** | VARCHAR(256) | excluded sub object ids are separated by commas(g4,g5)| | **is_valid** | Bool | (invalid), 1(valid) | @@ -359,7 +359,7 @@ For example: HTTP_REGION is the conjunction of HTTP_URL and HTTP_HOST. "valid_column":7, "custom": { "item_id":1, - "group_id":2, + "object_id":2, "keywords":3, "expr_type":4, "match_method":5, @@ -396,13 +396,13 @@ Before notifying the callback table, Maat fetches the foreign keys to local file By matching the tags accepted by Maat with the configuration tags, selective configuration loading is achieved. Configuration tags are a collection of tag arrays, denoted as "tag_sets", while Maat accepts tags are tag arrays denoted as "tags". -Configuration tags are tags stored on compilation configurations or group configurations, identifying where the configuration is effective in which Maat instances. It consists of multiple tag sets, where multiple tags within a set are ANDed, and multiple values of a tag are ORed. +Configuration tags are tags stored on compilation configurations or object configurations, identifying where the configuration is effective in which Maat instances. It consists of multiple tag sets, where multiple tags within a set are ANDed, and multiple values of a tag are ORed. ## 2. Table runtime -Maat loads the configuration of different types of tables into memory to form the corresponding runtime for each table. We can see all table types from the table schema, and the runtime for the item table is similar, as it is an abstraction of the scanning engine. When we provide the data to be scanned and call the corresponding scanning interface, we can return whether the item is hit or not, and if it is hit, we can return the corresponding item’s group_id. +Maat loads the configuration of different types of tables into memory to form the corresponding runtime for each table. We can see all table types from the table schema, and the runtime for the item table is similar, as it is an abstraction of the scanning engine. When we provide the data to be scanned and call the corresponding scanning interface, we can return whether the item is hit or not, and if it is hit, we can return the corresponding item’s object_id. -From the [configuration relationship](./overview.md#12-configuration-relationship) diagram, we can see how the hit group is referenced by other groups or rules. If a hit group is referenced by other groups or rules, there will be one or more hit paths that follow the `item_id -> group_id` {-> super group_id} `-> rule_id`. This requires two special runtimes: group2group_runtime and rule_runtime. +From the [configuration relationship](./overview.md#12-configuration-relationship) diagram, we can see how the hit object is referenced by other objects or rules. If a hit object is referenced by other objects or rules, there will be one or more hit paths that follow the `item_id -> object_id` {-> super object_id} `-> rule_id`. This requires two special runtimes: object2object_runtime and rule_runtime. Based on this, we can divide the runtime into the following three categories: @@ -412,8 +412,8 @@ Based on this, we can divide the runtime into the following three categories: * flag_runtime * interval_runtime -2. group & rule table runtime - * group2group_runtime +2. object & rule table runtime + * object2object_runtime * rule_runtime 3. xx_plugin table runtime @@ -431,15 +431,15 @@ Among the four types of runtimes mentioned above, `expr_runtime` is relatively u **Note**: Due to the inability to unify the native rulescan usage with hyperscan, a partial refactoring has been done on rulescan. The refactored rulescan follows the same interface and usage as hyperscan, making it compatible with the design of the expr_matcher abstraction layer. -### 2.2 group & rule table runtime +### 2.2 object & rule table runtime -#### 2.2.1 group2group runtime +#### 2.2.1 object2object runtime -The `group2group_runtime` is a runtime that is built based on the reference relationships between groups, which are stored in the [group2group table](#14-group2group-table). From the [group hierarchy](./group_hierarchy.md), we can understand that if a hit occurs in a leaf group that is referenced by other groups, there may be certain super groups that are also hit. This is exactly the functionality provided by this runtime. +The `object2object_runtime` is a runtime that is built based on the reference relationships between objects, which are stored in the [object2object table](#14-object2object-table). From the [object hierarchy](./object_hierarchy.md), we can understand that if a hit occurs in a leaf object that is referenced by other objects, there may be certain super objects that are also hit. This is exactly the functionality provided by this runtime. #### 2.2.2 rule runtime -In addition to the rule table, there is also the group2rule table in the table schema. However, from a runtime perspective, the configurations of these two tables together constitute rule_runtime. This means that there is no standalone group2rule_runtime. Rule_runtime is the most complex among all runtime types because it serves multiple functions. +In addition to the rule table, there is also the object2rule table in the table schema. However, from a runtime perspective, the configurations of these two tables together constitute rule_runtime. This means that there is no standalone object2rule_runtime. Rule_runtime is the most complex among all runtime types because it serves multiple functions. **Note:** This will involve the terminology of [condition](./terminology.md#condition). @@ -449,7 +449,7 @@ In addition to the rule table, there is also the group2rule table in the table s * rule2 = condition1 & condition2 = {attribute1, g2} & {attribute2, g3} - Given the matched attribute_id and group_id, all matching rule_ids can be provided. For example, if scanning attribute1 matches g2 and attribute2 matches g3, rule_runtime will return the matched rule_id 2. + Given the matched attribute_id and object_id, all matching rule_ids can be provided. For example, if scanning attribute1 matches g2 and attribute2 matches g3, rule_runtime will return the matched rule_id 2. 2. For expressions with negate-conditions, returning the matched rule_id: @@ -459,9 +459,9 @@ In addition to the rule table, there is also the group2rule table in the table s If scanning attribute1 matches g1 and attribute2 matches g3, rule_runtime will return the matched rule_id 4. -3. If a rule_id is matched, the full hit path can be obtained: **item_id -> group_id ->** {super_group_id} -> condition{**attribute_id, negate_option, condition_index} -> rule_id**. If the matched group is not referenced by a rule, a half hit path can be obtained: **item_id -> group_id** -> {super_group_id}. +3. If a rule_id is matched, the full hit path can be obtained: **item_id -> object_id ->** {super_object_id} -> condition{**attribute_id, negate_option, condition_index} -> rule_id**. If the matched object is not referenced by a rule, a half hit path can be obtained: **item_id -> object_id** -> {super_object_id}. -4. Getting the matched group_ids and the count of hit groups. +4. Getting the matched object_ids and the count of hit objects. The internal structure of rule_runtime is as follows, including the control plane for configuration loading and the data plane for external calls. @@ -469,33 +469,33 @@ The internal structure of rule_runtime is as follows, including the control plan * **Control plane** -Rule runtime loads the rule table and group2rule table configurations into memory, assigning a unique condition_id to all conditions of each rule. The following three parts are constructed based on the condition_id: +Rule runtime loads the rule table and object2rule table configurations into memory, assigning a unique condition_id to all conditions of each rule. The following three parts are constructed based on the condition_id: 1. All condition_ids under the same rule are used to construct AND expressions, and all rule AND expressions are used to build a bool_matcher. -2. For negate_option=0 (conditions), a `condition_id hash` is built, key:{group_id, attribute_id, negate_option}, value:condition_id. +2. For negate_option=0 (conditions), a `condition_id hash` is built, key:{object_id, attribute_id, negate_option}, value:condition_id. -3. For negate_option=1 (negate-conditions), a `NOT_condition_id hash` is built, key:{group_id, attribute_id, negate_option}, value:condition_id. +3. For negate_option=1 (negate-conditions), a `NOT_condition_id hash` is built, key:{object_id, attribute_id, negate_option}, value:condition_id. * **Data Plane** On the data plane, services are provided externally through the maat API, primarily with the following three types of interfaces: -1. **maat_scan_xx**: This interface dynamically generates the hit {item_id, group_id}. +1. **maat_scan_xx**: This interface dynamically generates the hit {item_id, object_id}. -* The hit item_id and group_id form a half-hit path. +* The hit item_id and object_id form a half-hit path. -* The group_id that is hit and the scanned `attribute_id` form the key {group_id, attribute_id, 0}. This key is used to find the `hit condition_ids` in the condition_id hash. +* The object_id that is hit and the scanned `attribute_id` form the key {object_id, attribute_id, 0}. This key is used to find the `hit condition_ids` in the condition_id hash. -* Use the key {group_id, attribute_id, 1} to search for NOT_condition_ids in the NOT_condition_id hash and cache them as `exclude condition_ids`. These condition_ids need to be removed from all condition_ids that are eventually hit. This is because the scan hit {group_id, attribute_id, 0} => condition_id, leading to the deduction that {group_id, attribute_id, 1} => NOT_condition_id does not hit. +* Use the key {object_id, attribute_id, 1} to search for NOT_condition_ids in the NOT_condition_id hash and cache them as `exclude condition_ids`. These condition_ids need to be removed from all condition_ids that are eventually hit. This is because the scan hit {object_id, attribute_id, 0} => condition_id, leading to the deduction that {object_id, attribute_id, 1} => NOT_condition_id does not hit. -* Identify the group_ids in attribute_id table that appear in the NOT_condition and add them to the `NOT_condition_group` set. Ensure that this set does not contain any group_id that was hit during scanning. If any such group_id is present, remove it from the set to form the final `NOT_condition_group` for the attribute_id table. +* Identify the object_ids in attribute_id table that appear in the NOT_condition and add them to the `NOT_condition_object` set. Ensure that this set does not contain any object_id that was hit during scanning. If any such object_id is present, remove it from the set to form the final `NOT_condition_object` for the attribute_id table. * Use the hit condition_ids to determine if there are any hit rule_ids. If there are, populate the half-hit path which will become full-hit path. 2. **maat_scan_not_logic**: This interface is used to activate negate-condition logic. -* Traverse the `NOT_condition_group` of `attribute_id`. For each `group_id`, form a key `{group_id, attribute_id, 1}` to obtain the `NOT_condition_id`. If it is in the `exclude condition_ids` set, ignore it; otherwise, add it to the `all hit condition_ids` set as a hit `NOT_condition_id`, and record the half-hit path of the negate-condition. +* Traverse the `NOT_condition_object` of `attribute_id`. For each `object_id`, form a key `{object_id, attribute_id, 1}` to obtain the `NOT_condition_id`. If it is in the `exclude condition_ids` set, ignore it; otherwise, add it to the `all hit condition_ids` set as a hit `NOT_condition_id`, and record the half-hit path of the negate-condition. * Use the `all hit condition_ids` to calculate if there are any newly hit rule_ids. If there are, populate the half-hit path of the negate-condition which will become full-hit path. diff --git a/docs/monitor_tools.md b/docs/monitor_tools.md index 894f4ed..1b0fd6b 100644 --- a/docs/monitor_tools.md +++ b/docs/monitor_tools.md @@ -24,7 +24,7 @@ The statistical information in the above figure is divided into two parts, part1 - NOT_condition_num: total number of negate-conditions -- excl_grp_num: total number of exclude groups +- excl_grp_num: total number of exclude objects - garbage_queue_len: number of elements in the garbage queue diff --git a/docs/object_hierarchy.md b/docs/object_hierarchy.md new file mode 100644 index 0000000..8254373 --- /dev/null +++ b/docs/object_hierarchy.md @@ -0,0 +1,158 @@ +# Object(Object) hierarchies + +A object can reference other objects, and can also be referenced by other objects. For example, object_A references object_B, object_A is the superior object of object_B, and object_B is the subordinate object of object_A. There are two reference relationships between objects: `include` and `exclude`. + +## Include + +Include is equivalent to the inclusion semantics in set theory. For example, when object_A is included by object_B, if a traffic attribute satisfies object_A, object_B is satisfied. + +## Exclude + +A object defines a subset of an object type, such as network addresses or port numbers. The definition is made using items, which can be used to add to or exclude from the object definition. Objects can also have subordinate objects whose definitions are included in the superior object. + +There are rules of precedence to take into account when defining objects: + +- Excluding has precedence over including in the same object. + +- Items in a superior object have precedence over items in a subordinate object. + +- Items in a superior object are not taken into account in a subordinate object, if the subordinate object is used directly in a rule. + +- Peer objects (different subordinate objects of the same superior object) do not affect each other. + +In short, to determine the set defined by a object, perform the following calculation: + +1. For each subordinate object (remember sibling objects do not affect each other): + - Add included items. + + - Subtract excluded items. + +2. Add included items in the object itself, overriding any excludes in the subordinate objects. + +3. Subtract excluded items in the object itself. + +The following figure shows a object with an included set and an excluded subset. + +<img src="./imgs/exclude.png" width="350" height="200" > + +Now, consider adding a subordinate object. The subordinate object also has an included set and an excluded subset. If the superior object is used, the result is shown in the following figure “A superior and subordinate object”. + +<img src="./imgs/hierarchy-and-exclude.png" width="350" height="230" > + +As can be seen, the excluded item in the subordinate object is overwritten since it is in the included set of the superior object. Also, the excluded item from the superior object excludes part of the included item in the subordinate. + +If only the subordinate object is used in a rule condition, the superior object items are disregarded, leaving the set shown in the next figure, “The subordinate object”. + +<img src="./imgs/sub-exclude.png" width="350" height="230" > + +Restrictions: + +- A object can only include or exclude objects of the same type. + +- A object should include at least one subordinate object. (Exclude only is not allowed) + +- Traffic attribute using stream scan cannot allow use object(object) with exclude, i.e., keywords object on HTTP Response Body, Email attachment. + +Now, let's see a graph of hierarchy example, where the dotted line means exclude. The matched subordinate objects and activated superiors are listed in the following table. + +Now, let's see a graph of hierarchy example, where the dotted line means exclude. If the matched subordinate objects: **{g11, g13}**, then activated superiors: **{g2, g7}**. + +<img src="./imgs/object-hierarchy1.png" width="750" height="350" > + +The analysis process is as follows: + +**matched objects {g11, g13}** + +**Level 1**: g11 => incl{g6, g7} excl{null}, g13 => incl{g3} excl{g4}; new matched objects {g6, g7, g3}, `all matched objects` {g11, g13, g6, g7, g3} + +**new matched objects {g6, g7, g3}** + +**Level 2**: g6 => incl{g1} excl{g2, g3}, g7 => incl{g2, g4} excl{g6}; new matched objects {g1, g2, g4}, `all matched objects` {g11, g13, g7, g1, g2, g4} + +**new matched objects {g1, g2, g4}** + +**Level 3**: end + +Check the validity of `all matched objects` {g11, g13, g7, g1, g2, g4}: + +1. g11, g13 are the originally matched nodes, so keep {g11, g13}. + +2. g7 is derived from the match of g11, implying that g10 is not matched (if g10 is matched, then g7 is not matched, as the exclude priority is higher), so keep g7. + +3. g1 is derived from the match of g6, but since g6 is excluded due to both g11 and g7, and g6 excludes g7 with higher priority, g1 is no longer matched. Therefore, remove g1 from the set. + +3. g1 is derived from the match of g6, which in turn is derived from the match of g11. However, since g7 is also matched and g6 excludes g7 with higher priority, g7's match causes g6 to be unmatched, and consequently, g1 is no longer matched. Therefore, g1 is removed from the set. + +4. g2 is derived from the match of g7, with the implied condition that g6 is not matched, so keep g2. + +5. Finally, `all matched objects` {g11, g13, g7, g2}, `matched super objects` {g2, g7}. + +Remove the dotted line from g7 to g6, as shown in the diagram below. If the matched subordinate objects: **{g11, g13}**, then activated objects: **{g1, g6, g7}**. + +<img src="./imgs/object-hierarchy2.png" width="750" height="350" > + +The analysis process is as follows: + +**matched objects {g11, g13}** + +**Level 1**: g11 => incl{g6, g7} excl{null}, g13 => incl{g3} excl{g4}; new matched objects {g6, g7, g3}, `all matched objects` {g11, g13, g6, g7, g3} + +**new matched objects {g6, g7, g3}** + +**Level 2**: g6 => incl{g1} excl{g2, g3}, g7 => incl{g2, g4} excl{null}; new matched objects {g1, g2, g4}, `all matched objects` {g11, g13, g6, g1, g7, g2, g4} + +**new matched objects {g1, g2, g4}** + +**Level 3**: end + +Check the validity of `all matched objects` {g11, g13, g6, g1, g7, g2, g4}: + +1. g11, g13 are the originally matched nodes, so keep {g11, g13}. + +2. g6 is derived from the match of g11, implying the g10 is not matched(if g10 is matched, then g7 is not matched, as the exclude priority is higher), so keep g6. + +3. g1 is derived from the match of g6, which in turn is derived from the match of g11, implying the g5 is not matched, so keep g1. + +4. g7 is derived from the match of g11, implying the g10 is not matched, so keep g7. + +5. g2 is derived from the match of g7. However, since g6 is also matched and g2 excludes g6 with higher priority, g6's match causes g2 to be unmatched. Therefore, g2 is removed from the set. + +6. g4 is derived from the match of g7. However, since g13 is also matched and g4 excludes g13 with higher priority, g13's match causes g4 to be unmatched. Therefore, g4 is removed from the set. + +7. Finally, `all matched objects` {g11, g13, g1, g6, g7}, `matched super objects` {g1, g6, g7}. + +Again remove the dotted line from g13 to g4, as shown in the diagram below. If the matched subordinate objects: **{g11, g13}**, then activated superiors: **{g1, g4, g6, g7}**. + +<img src="./imgs/object-hierarchy3.png" width="750" height="350" > + +The analysis process is as follows: + +**matched objects {g11, g13}** + +**Level 1**: g11 => incl{g6, g7} excl{null}, g13 => incl{g3} excl{null}; new matched objects {g6, g7, g3}, `all matched objects` {g11, g13, g6, g7, g3} + +**new matched objects {g6, g7, g3}** + +**Level 2**: g6 => incl{g1} excl{g2, g3}, g7 => incl{g2, g4} excl{null}; new matched objects {g1, g2, g4}, `all matched objects` {g11, g13, g6, g1, g7, g2, g4} + +**new matched objects {g1, g2, g4}** + +**Level 3**: end + +Check the validity of `all matched objects` {g11, g13, g6, g1, g7, g2, g4}: + +1. g11, g13 are the originally matched nodes, so keep {g11, g13}. + +2. g6 is derived from the match of g11, implying the g10 is not matched(if g10 is matched, then g7 is not matched, as the exclude priority is higher), so keep g6. + +3. g1 is derived from the match of g6, which in turn is derived from the match of g11, implying the g5 is not matched, so keep g1. + +4. g7 is derived from the match of g11, implying the g10 is not matched, so keep g7. + +5. g2 is derived from the match of g7. However, since g6 is also matched and g2 excludes g6 with higher priority, g6's match causes g2 to be unmatched. Therefore, g2 is removed from the set. + +6. g4 is derived from the match of g13, so keep g4. + +7. Finally, `all matched objects` {g11, g13, g6, g1, g7, g4}, `matched super objects` {g1, g4, g6, g7}. + +The above example is strictly consistent with the case `ObjectExcludeTest.level_3_function` in the [test/object_nesting](../test/object_nesting/) test. Please run it to see the actual execution results.
\ No newline at end of file diff --git a/docs/overview.md b/docs/overview.md index bb3f795..82b44ba 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -7,7 +7,7 @@ Before proceeding, please make sure you are familiar with the [terminology](./te As mentioned in the readme, maat has two typical usage patterns: **Pattern 1** -* Update rules in the item table, group2rule table, and rule table +* Update rules in the item table, object2rule table, and rule table * Call the maat scanning api to determine if the actual traffic hits the effective rules * If a rule is hit, maat can provide detailed information about the hit rule @@ -20,15 +20,15 @@ As mentioned in the readme, maat has two typical usage patterns: Different types of configurations are stored in different tables. For all configuration types, please refer to the [table schema](./maat_table.md#1-table-schema). -The physical tables are mainly divided into three categories: the item table, group rule relationship table (rule table, group2rule table, group2group table), and xx_plugin table. The first two types of tables are used for maat traffic scanning, while the xx_plugin table is used as a callback table, which can obtain the detailed configuration information for a specific key. +The physical tables are mainly divided into three categories: the item table, object rule relationship table (rule table, object2rule table, object2object table), and xx_plugin table. The first two types of tables are used for maat traffic scanning, while the xx_plugin table is used as a callback table, which can obtain the detailed configuration information for a specific key. ### 1.2 Configuration relationship -As shown in the diagram below, maat organizes and abstracts configurations using terms such as item, group, literal, condition, rule, etc., allowing users to flexibly configure various policies. The term "literal" is an internal concept in maat and is not visible to external users. +As shown in the diagram below, maat organizes and abstracts configurations using terms such as item, object, literal, condition, rule, etc., allowing users to flexibly configure various policies. The term "literal" is an internal concept in maat and is not visible to external users. -In addition, groups support nesting. For more detailed information, please refer to [group hierarchy](./group_hierarchy.md). +In addition, objects support nesting. For more detailed information, please refer to [object hierarchy](./object_hierarchy.md). -If we define literal_id = {attribute_id, group_id}, then a literal is composed of one or more literal_ids. The multiple literal_ids that form the same condition have a logical “OR” relationship. The multiple conditions that form the same rule have a logical “AND” relationship, and there can be a maximum of 8 conditions within the same rule. In addition, the condition itself supports logical "NOT". +If we define literal_id = {attribute_id, object_id}, then a literal is composed of one or more literal_ids. The multiple literal_ids that form the same condition have a logical “OR” relationship. The multiple conditions that form the same rule have a logical “AND” relationship, and there can be a maximum of 8 conditions within the same rule. In addition, the condition itself supports logical "NOT". <img src="./imgs/rule_diagram.png" width="800" height="450" > @@ -56,7 +56,7 @@ The diagram illustrates the overall architecture of maat, including the control * **Data Plane** - When calling the maat scanning interface, it subsequently calls the table runtime of the corresponding table, then proceeds to the scanning engine. Upon the scanning engine returning a hit group, it further searches for the matching `rule_id` through group2group runtime, group2rule runtime, and rule runtime, which is then returned to the caller. In addition, if the caller is interested in the hit path, they can also retrieve it through the interfaces provided by maat. + When calling the maat scanning interface, it subsequently calls the table runtime of the corresponding table, then proceeds to the scanning engine. Upon the scanning engine returning a hit object, it further searches for the matching `rule_id` through object2object runtime, object2rule runtime, and rule runtime, which is then returned to the caller. In addition, if the caller is interested in the hit path, they can also retrieve it through the interfaces provided by maat. The scanning mentioned above all uses the `effective runtime`. If there are configuration changes, it will trigger the construction of `updating runtime`. Once this construction is completed, it will become effective runtime, and the original effective runtime will be put into the garbage collection queue waiting to be recycled. @@ -102,7 +102,7 @@ maat_framework_perf_gtest: The performance test set is mainly used to test the t * benchmark: Maat performance benchmark test, testing the scanning time of different scanning interfaces under different scale rule sets. -* group_nesting: Functionality and performance test set for group nesting. +* object_nesting: Functionality and performance test set for object nesting. * ipport_plugin: Functionality and performance test set for the ipport_plugin table. diff --git a/docs/terminology.md b/docs/terminology.md index 723b24a..4956b93 100644 --- a/docs/terminology.md +++ b/docs/terminology.md @@ -1,7 +1,7 @@ # Terminology * [Item](#item) -* [Group(Object)](#groupobject) +* [Object(Object)](#objectobject) * [Rule(Policy)](#rulepolicy) * [Condition(Condition)](#condition) * [Literal](#literal) @@ -32,27 +32,27 @@ As a filter for network attributes, the smallest unit of a rule There are multiple types of items stored in corresponding tables such as string, IP and numerical range, more details can be found in [Item table](./maat_table.md#11-item-table). -## Group(Object) +## Object(Object) -A group defines a set that can contain different types of items and can also reference other groups. The definition is made using items, which can be used to add to or exclude from the group definition. Groups can also have subordinate groups whose definitions are included in or excluded from the superior group. +A object defines a set that can contain different types of items and can also reference other objects. The definition is made using items, which can be used to add to or exclude from the object definition. Objects can also have subordinate objects whose definitions are included in or excluded from the superior object. -- An item only belongs to one group, but one group can has multiple items. The multiple items under the same group are logical 'OR' relationships. e.g.(g1 = item1 | item2). +- An item only belongs to one object, but one object can has multiple items. The multiple items under the same object are logical 'OR' relationships. e.g.(g1 = item1 | item2). -- A group can be included or excluded by other groups. For example, if group1 and group2 is included by group3, then the group3 is called the superior(super) group, and group1(group2) is called the subordinate(sub) group. There's a logical 'OR' relationship between the included sub groups under the same group, e.g.(g3 = incl-g1 | incl-g2). There's a logical 'AND' relationship between included group and excluded group under the same group, e.g.(g4 = incl-g1 & excl-g2). +- A object can be included or excluded by other objects. For example, if object1 and object2 is included by object3, then the object3 is called the superior(super) object, and object1(object2) is called the subordinate(sub) object. There's a logical 'OR' relationship between the included sub objects under the same object, e.g.(g3 = incl-g1 | incl-g2). There's a logical 'AND' relationship between included object and excluded object under the same object, e.g.(g4 = incl-g1 & excl-g2). -- Group supports multi-level nesting, see [group hierarchy](./overview.md#groupobject-nesting-and-hierarchies) +- Object supports multi-level nesting, see [object hierarchy](./overview.md#objectobject-nesting-and-hierarchies) -- A Group can be referenced by different rules. +- A Object can be referenced by different rules. -The relationship between group and group is stored in the [group2group table](./maat_table.md#14-group2group-table). +The relationship between object and object is stored in the [object2object table](./maat_table.md#14-object2object-table). ## Rule(Policy) -A conjunctive normal form(CNF) consisting of multiple groups and attributes. +A conjunctive normal form(CNF) consisting of multiple objects and attributes. `Note`: A rule can contain up to 8 conditions and multiple conditions in the same rule can be logical 'AND' and logical 'NOT' relationships. -The relationship between group and rule is stored in the [group2rule table](./maat_table.md#13-group2rule-table). +The relationship between object and rule is stored in the [object2rule table](./maat_table.md#13-object2rule-table). <img src="./imgs/CNF.jpg" alt="exclude" style="zoom:80%" /> @@ -64,11 +64,11 @@ Conditions are divided into two categories based on whether they contain the log ## Literal -A Literal consists of `attribute_id(attribute id)` and `group_id`. During the rules loading process, a unique condition_id will be generated based on the combination of attribute_id and group_id in the same condition. +A Literal consists of `attribute_id(attribute id)` and `object_id`. During the rules loading process, a unique condition_id will be generated based on the combination of attribute_id and object_id in the same condition. ## Physical table -Different rules are stored in different tables in the actual database, including [item table](./maat_table.md#11-item-table), [rule table](./maat_table.md#12-rule-table), [group2rule table](./maat_table.md#13-group2rule-table), [group2group table](./maat_table.md#14-group2group-table), and [xx_plugin table](./maat_table.md#15-plugin-table), and so on. +Different rules are stored in different tables in the actual database, including [item table](./maat_table.md#11-item-table), [rule table](./maat_table.md#12-rule-table), [object2rule table](./maat_table.md#13-object2rule-table), [object2object table](./maat_table.md#14-object2object-table), and [xx_plugin table](./maat_table.md#15-plugin-table), and so on. ## Attribute @@ -86,7 +86,7 @@ Defines the type of table and the configuration format, determining the specific ## Table runtime -The runtime generated by loading the configuration in the table into memory. Different tables have different runtimes. The group2rule table is merged with the corresponding rule table to generate a rule runtime, meaning there is no separate group2rule runtime. +The runtime generated by loading the configuration in the table into memory. Different tables have different runtimes. The object2rule table is merged with the corresponding rule table to generate a rule runtime, meaning there is no separate object2rule runtime. Different scanning api use runtimes of different tables. For example, the HTTP_URL table is of type expr, and its corresponding scanning interface is maat_scan_string. Therefore, when calling this scanning interface, the API internally uses the runtime of the HTTP_URL table to perform the actual scanning task. @@ -112,11 +112,11 @@ Maat supports not only block-based scanning but also stream-based scanning. For ## Half/Full hit -From the diagram of [configuration relationship](./overview.md#12-configuration-relationship), it can be seen that if the group that is hit is not referenced by rule, or even if referenced by rule, but after logical operations no rule is hit, then this hit is called a half hit; if a rule is hit, then this hit is called a full hit. +From the diagram of [configuration relationship](./overview.md#12-configuration-relationship), it can be seen that if the object that is hit is not referenced by rule, or even if referenced by rule, but after logical operations no rule is hit, then this hit is called a half hit; if a rule is hit, then this hit is called a full hit. ## Hit path -From the relationship of item, group, and rule mentioned above, if a scan hits a certain rule, there must be a logical path composed of item_id -> group_id -> rule_id. Maat names this path the hit path. If a group has nested references, the hit path should be item_id -> sub_group_id -> group_id -> rule_id. +From the relationship of item, object, and rule mentioned above, if a scan hits a certain rule, there must be a logical path composed of item_id -> object_id -> rule_id. Maat names this path the hit path. If a object has nested references, the hit path should be item_id -> sub_object_id -> object_id -> rule_id. ## Redis |
