Cross component security group design

[aaronwalker]

Description

Right now each Cfhighlander component is have to solve the problem of how it manages security groups and has to solve the problem of wiring them together with other components. This tends to create a tight coupling between components or as a consumer of a component you have to provide excessive additional configuration.

Proposal

To create a mechanism either as a component that can be inlined or in the Cfhighlander core to support automatic wiring of security groups without creating component dependencies

[Guslington]

My preference would be solve this with inlining using security group component(s). I think it would give the user more freedom to deal with security groups without having them feel like they have to contribute to the core cfhighlander project to make a change.

[aaronwalker]

My preference would be solve this with inlining using security group component(s). I think it would give the user more freedom to deal with security groups without having them feel like they have to contribute to the core cfhighlander project to make a change.

@Guslington I kinda agree but it does create boiler plate code in that most components would want to include security group handling. I was thinking to see if we could maybe have the concept of core components which automatically get inlined unless you disable/override them

[toshke]

@aaronwalker @Guslington
I like mix of the concepts - have a list of standard components, such as security groups, route53 records (perhaps ecs service?), and placing config file sg.config.yaml or dns.config.yaml
alongside with the component would automatically enable inlining this component.

Another option would be to have rendering of this groups using helper cfndsl functions (look at standard extensions idea in theonestack/hl-component-vpc#11 in theonestack/hl-component-bastion#5) automatically if certain configuration key is set - and add this to default cfndsl template code, e.g. in ERB template

# component config yaml
securityGroups:
  bastion:
    export: true
  ops:
    import: true

dns:
  bastion: bastion.$EnvironmentName.$DnsDomain

# cfndsl.template.erb

# will render required parameters for sg-to-sg configuration
render_sgs(securityGroups) if defined? securityGroups

# will render required HostedZoneId parameter as well
render_dns(dns) if defined? dns

I like the path where we can add "traits" to components, though not sure on how to avoid tweaking core code in order to add / remove a trait.

[aaronwalker]

@toshke do you think we could add some kind of plugin registration hook to allow injection of cfndsl or even the native cfn.

So instead it call out to any registered plugin allow it to inject resources during rendering.

something like

# cfndsl.template.erb

plugins.each do |plugin|
  plugin.render(context)
end if defined? plugins

You then declare what plugins you require in the top-level (parent component) and provide the corresponding config

[toshke]

@aaronwalker I like the idea, much more flexible than hardcoding functionality into the template.
plugins could hook into current phases of template compilation

1. Component configuration generation
2. Component cfndsl rendering
3. Component resulting template rendering, be it YAML or JSON. Though for predictable results this should be done before inlining or find replace Ref with other function types such as FnImportValue when extending a template cfhighlander#59

With such plugin system, some of the issues raised could be implemented, such as removing resources in extended templates.

Also, current "standard extensions" idea may fit into the plugin system, e.g. allowing for syntax below, compared to current DependsOn

CfhighlanderTemplate do
    Plugins ['stdext']
end

[Guslington]

@toshke @aaronwalker LGTM

@toshke i like the simple syntax of Plugins ['array']