Extend query/check scope decentrally

[arlimus]

Kudos to @JosiahOne for reporting and suggesting this idea!

Problem

Variants are centrally managed and cannot be easily extended by other users.

Example: Let's say we write a "Cloud security policy". One of the checks says "Don't expose SSH on containers". We could implement this for AWS, Azure and GCP via variants and release a policy. That policy would not answer the check "Don't expose SSH on containers" for all 3 managed clouds. However, it wouldn't easily be extensible. Someone outside of Mondoo could not add another cloud coverage to that check without asking us to change our (centrally managed) policy.

Current variants are shaped like this:

queries:
- uid: cloud-ssh-check
  variants:
  - uid: aws-ssh-check
  - uid: azure-ssh-check
  - uid: gcp-ssh-check

# implementations of referenced checks
- uid: aws-ssh-check
  mql: ...
  ...

Discussion

Our goal is to allow users to extend checks in a decentralized way.

Option A:

queries:
- uid: cloud-ssh-check
  # metadata ...

We leave the variant container empty, i.e. it has no specific queries that it implements. It also doesn't require any filters.

Then we implement queries that attach to the container, which can come from the same or a different policy or bundle:

queries:
- uid: aws-ssh-check
  extends: cloud-ssh-check
  filters: ...
  mql: ...

This can also report into multiple variant queries:

queries:
- uid: aws-ssh-check
  extends:
  - uid: cloud-ssh-check
  - uid: my-custom-ssh
  ...

• specify UIDs or MRNs. If the referenced query is in the same bundle (or namespace) you can use UIDs, otherwise you need MRNs
• the implementing policies must be activated for the run, in order to report into the aggregation layer
  • example1: user activates a generic cloud-policy, without activating any specific e.g. aws-policy. We now run cloud-ssh-check as the variant container, but we don't have aws-ssh-check in the execution. In this case we get an unknown status for cloud-ssh-check
  • example2: user activates the specific aws-policy, but not the generic policy. In this case all referenced checks (e.g. including aws-ssh-check) are executed and collected. However no reporting into cloud-ssh-check is done because we didn't activate cloud-policy
  • example3: user activates both the generic cloud-policy and specific aws-policy. Now that we have both cloud-ssh-check and aws-ssh-check in the run, the latter reports into the former and we get results for both
• long-term replace variants with this approach, i.e. deprecate the variants: ... approach listed above

Clarification

Reporting: If a base and variant query both are present in an execution we

• a: Only report the variant query in the CLI, if that policy has been selected (eg: cloud-ssh-check)
• b: Report both the variant and base query in the CLI (eg: both cloud-ssh-check and aws-ssh-check)

Scoring: If both the base and variant policy are selected we

• a: Only score the check once and not twice
• b: Score the check for both the base and variant

Extensions: Additional mappings:

• a: Cannot be specified. Only authors of the base query are able to specify policy mappings (eg: aws-ssh-check can map to other policies, but not the other way around)
• b: Can be specified implicitly. I.e. authors of both the base query (eg: aws-ssh-check) as well as authors who manage overlays of this query (eg in an overlay policy) can extend existing mappings. However: mappings cannot be removed, they can only be extended.
• c: Can be managed explicitly. This allow authors of other policies to extend existing queries or modify existing mappings. This is much the same way that mappings work in framework. For this, a query map can be modified via the action field.

Illustration of example c:

- uid: aws-ssh-check
  extends:
  - uid: cloud-ssh-check
    action: deactivate
  - uid: my-custom-ssh
    # implicitly uses action: activatae

Solution

...

[scottford-io]

@arlimus I assume that docs and remediation steps would work the same in that you can have individual steps with each child variant?

queries:
- uid: aws-ssh-check
  extends: cloud-ssh-check
  filters: ...
  mql: ...
  docs:
    remediation:
      - id: console
        desc: |
          # Markdown goes here
           
          1. Step 1
          2. Step 2
          3. etc

[JosiahOne]

Thanks for the writeup @arlimus! I do think Option A, as-is, would be very useful.

However, I'll note that this doesn't entirely cover my desired workflow (or if it does, it's not clear to me how):

Today my biggest annoyance with writing policies is that I have to write multiple queries corresponding to resources that feel the same. For example, if I want to "prevent SSH on port 22" I have to write MQL rules for:

1. AWS security groups as returned by the API
2. AWS security groups as defined in .tf files
3. AWS security groups as defined in .tf state
4. Azure security groups (or whatever they're called these days)

Today these are written as variants within one file. With Option A, I could shard these into separate files. That's great for maintenance but still frustrating from an authorship perspective.

What I'd really like is the ability to write queries against multiple resource types at once because they all adhere to the same schema.

Example Sketch

An example of how this might look in cnspec (maybe this is "Option B") is:

schemas:
- uid: network-rules
  schema: |
{
    "$schema": "https://json-schema.org/draft/2019-09/schema",
    "$id": "http://example.com/example.json",
    "title": "Cloud Provider Network Restriction Rule Schema",
    "type": "array",
    "default": [],
    "items": {
        "title": "A Schema",
        "type": "object",
        "default": {},
        "required": [
            "ipPermissions"
        ],
        "properties": {
            "ipPermissions": {
                "title": "The ipPermissions Schema",
                "type": "array",
                "default": [],
                "items": {
                    "title": "A Schema",
                    "type": "object",
                    "default": {},
                    "required": [
                        "ipRanges",
                        "fromPort",
                        "toPort"
                    ],
                    "properties": {
                        "ipRanges": {
                            "title": "The ipRanges Schema",
                            "type": "array",
                            "default": [],
                            "items": {
                                "title": "A Schema",
                                "type": "string",
                                "default": ""
                            }
                        },
                        "fromPort": {
                            "title": "The fromPort Schema",
                            "type": "integer",
                            "default": 0
                        },
                        "toPort": {
                            "title": "The toPort Schema",
                            "type": "integer",
                            "default": 0
                        }
                    }
                }
            }
        }
    }
}
  output: networkRules

queries:
- uid: generic-no-server-admin-ingress-check
  schema: network-rules
  uses: [ aws-api-network-rules, tf-plan-aws-network-rules, tf-state-aws-network-rules, azure-network-rules ] 
  filters: ...
  mql: |
    networkRules.where(ipPermissions.any(
    ipRanges.contains('0.0.0.0/0'))).all(
      ipPermissions.none(fromPort >= 22 || toPort <= 22 && toPort != 0 )
    )
    networkRules.where(ipPermissions.any(
    ipRanges.contains('::/0'))).all(
      ipPermissions.none(fromPort >= 22 || toPort <= 22 && toPort != 0 )
    )
    networkRules.where(ipPermissions.any(
    ipRanges.contains('0.0.0.0/0'))).all(
      ipPermissions.none(fromPort >= 3389 || toPort <= 3389 && toPort != 0 )
    )
    networkRules.where(ipPermissions.any(
    ipRanges.contains('::/0'))).all(
      ipPermissions.none(fromPort >= 3389 || toPort <= 3389 && toPort != 0 )
    )

- uid: aws-api-network-rules
  implements: network-rules
  mql: aws.ec2.securityGroups

- uid: tf-plan-aws-network-rules
  implements: network-rules
  mql: ...

...

The important bits are:

1. Add a schemas section to define reusable "resource types" that can be queried against without caring about which provider actually generated the data.
2. Allow specific rules to specify the schema they are written against.
3. Provide a new uses field that indicates which resources from different providers will be queried and normalized so minimal queries can be written.
4. The mql rule will be able to reference the results of uses queries
5. "Normalizers" can be written by using a implements keyword. These rules will pull resources from a specific provider and return data that matches the schema indicated by implements.

Mondoo could provide a whole slew of "normalizers" or schemas for similar types of data. Rule authors from specific companies would then only have to worry about writing against those schemas, not against each provider's output. This would, IMO, significantlly lessen the cognitive overhead of Mondoo.