Typical use question: Are analyze and verify-policy usually targeting the same component?

[jgsuess]

Macaron seems to have two execution phases: analyze and verify-policy.

To model the configuration with a little more interesting detail, I would like to model these two calls. Now for me the question is: Would a typical user always analyze and verify the same component, or would it be common to analyze a component and then verify only on specific dependencies of that component.

I know that analyze may bring in the data for dependencies, so a verify call might turn up something useful. But I am not sure what a developers typical workflow would entail. I am talking about someone who is just an average user, not an expert in the configuration, as that is what I am building my case for.

To motivate what I am about to do: The calls below are using parameters for remote path, branch and commit. My goal is to extract them to the configuration model and then generate the shell call dynamically. And this prompts the question whether I need one or two sets of data for analyze and verify separately.

run_macaron_analysis:
  extends: .run_macaron
  stage: execute_macaron_analysis
  script:
    - >-
      ${MACARON} analyze -rp
      https://github.com/micronaut-projects/micronaut-core -b 4.0.x -d
      82d115b4901d10226552ac67b0a10978cd5bc603

run_macaron_policy:
  extends: .run_macaron
  stage: execute_macaron_policy_check
  script:
    - >-
      ${MACARON} verify-policy -d ${OUTPUT_DIRECTORY}/macaron.db
      --file ${CI_PROJECT_DIR}/reference/oci-micronaut.dl
      || true
    - >-
      ${MACARON} verify-policy -d ${OUTPUT_DIRECTORY}/macaron.db
      --file ${CI_PROJECT_DIR}/target/Basic_mapstruct_mapstruct.dl
      || true

[behnazh-w]

Thanks @jgsuess for the discussion.

I know that analyze may bring in the data for dependencies, so a verify call might turn up something useful. But I am not sure what a developers typical workflow would entail. I am talking about someone who is just an average user, not an expert in the configuration, as that is what I am building my case for.

Right now we have two types of users in mind:

1. Average users who are not familiar with declarative policies
2. Advanced users who are willing to write their own policies

For the former, we are working on template policies that only require changing the main target (e.g., repository path of the top level artifact) in the Datalog policy to be used by a project. Right now the user needs to change the target manually, however we can make it even easier for the user and add a command to Macaron to automatically generate a policy that already contains the right target, which I think is similar to your goal.

For the latter, we want to give a lot of flexibility to the user to write/adjust their policies, using the template policies to get started.

Note that for both cases we still want to separate the analyze and verify-policy commands because verify-policy uses the cached results in the database and can enforce temporal policies to find regressions, and in general we can enforce policies with minimal overhead. But as explained, we can make Datalog policies almost transparent to average users. So, all they would need to do is to run analyze, generate-policy (which doesn't exist yet), and then verify-policy.

[jgsuess]

Thanks. That is very useful! I am indeed preparing a setup for a local (and hopefully later remote) IDE that is able to generate configurations to run macaron via docker locally and remote as part of a build. I hope that such an IDE setup will make macaron accessible to high-level users. I currently have an element called 'enforcement' and I will probably stay with running and querying the outcomes in one go to ensure this is simple.

Later I would like to add a view that can show and query the DB content by transforming it to a metamodel that can be displayed nicely. At that time, changing the model to allow to specify database locations and querying their state seems the right thing. I thin without a means to easily inspect the DB writing queries would be a blind thing. Different if storage was one of the other souffle options (CSV, compressed CSV), but given it is DB, I guess it would be a bit hard.

The templating that I am reusing includes various modes of authoring support for the templates, including full generation, protected regions or just patches. This will allow very flexible integration of generated and handwritten items that most template systems do not offer. Full generation writes over or appends to a file, protected regions reparses and respects specific regions marked as changed and patch looks for diff matches in the file and replaces them.