[RFC] VPC

[t-richard]

The goal of this discussion is to get feedback on a potential "VPC" construct.

The current implementation is setting the VPC config on all functions in the stack.

The idea would be to build this as a foundation for other constructs that need Lambda functions to be in a VPC such as

• RDS
• ElastiCache (Redis/memcached)
• ElasticSearch
• Amazon MQ (RabbitMQ/ActiveMQ)
• DocumentDB (mongoDB)
• DynamoDB DAX
• and much more

I think a good start would be to be on par with the main features provided by serverless-vpc-plugin.

Quick start

constructs:
  vpc:
    type: vpc

How it works

The basis of this is the awesome L2 VPC Construct provided by the CDK.

The defaults of the construct are generally what you want:

• a VPC
• an Internet Gateway
• as many public subnets as they are AZs in the region
• as many private subnets as they are AZs in the region
• a NAT Gateway in each public Subnet to allow outbound traffic to the internet from the instances/functions
• route tables properly configured to direct outbound traffic coming from the private subnets to the NAT gateways

But this setup has a significant cost mainly because of the NAT gateways, so that's why it's important to be able to configure it for less critical apps or dev/staging environments.

Configuration reference

Configuring NAT Gateways

We have two options here :

• natGateways which determines the number of gateways to provision. If not set or set to a high number, this will provision as many NAT as there are AZs in the region
• natGatewayType can be either gateway which will provision the fully managed NAT Gateway from AWS; or instance which will provision a small t3.nano NAT Instance

Used together, they can help people save hundreds of dollars on dev/test environments with this kind of config

constructs:
  vpc:
    type: vpc
    natGateways: 1
    natGatewayType: instance

There are of course more options to consider : maximum number of AZs to use, maybe bastion hosts or use Session Manager to do it and provide a Lift command to connect, and many more !

Challenges

I see several challenges to this.

Should we allow to specifiy which functions have to be in the generated VPC ? Or apply it to all functions ? If we choose the later, this construct can only be used once in a given app.

How to get the maximum number of AZs right ? The CDK is supposedly able to guess this from the region and account ID, but my tests were not conclusive about this. This causes problems because us-west-1 has 3 AZs but you only get access to 2 of them by default (you can check that by running aws ec2 describe-availability-zones --region=us-west-1 with credentials/profiles configured).
If Lift generated subnets in the 3 AZs, the deployment would fail. This can be documented but maybe there's a better approach.

How to make the other (future) constructs "auto-discover" the VPC and use its configuration to generate required resources like Subnet groups.

Draft Pull Request

See a draft implementation here #65

Would love to get your feedback about this

[mnapoli]

Thanks for opening this, continuing our Slack discussion here.

As you mention, we have 2 scenarios:

• in dev, we probably want 1 AZ, and use the cheapest NAT instance
• in prod, we want multiple AZ + managed NAT Gateway

I'll be opening a separate discussion to see how we address this, as we'll get the same challenge with other constructs (e.g. RDS database).

In the meantime, I suggest we focus on the production use case for now.

For the record we discussed VPC config separately with Marco and came to the conclusion that for production scenarios, it would make sense to go by default with:

• 2 AZ (fine for most production use cases), with an setting to increase that number
• 1 (managed) NAT Gateway per AZ

I don't think we want to provide an option to use NAT instances for the production version (at least at first).

As for any extra feature (bastion, etc.) those are great ideas, but I would start without them at first TBH.

How to get the maximum number of AZs right ? The CDK is supposedly able to guess this from the region and account ID, but my tests were not conclusive about this. This causes problems because us-west-1 has 3 AZs but you only get access to 2 of them by default (you can check that by running aws ec2 describe-availability-zones --region=us-west-1 with credentials/profiles configured).
If Lift generated subnets in the 3 AZs, the deployment would fail. This can be documented but maybe there's a better approach.

Oh that's too bad. I think it is tracked by this bug report: aws/aws-cdk#6015

I'm fine with documenting that at first. Maybe if there are only a few problematic regions we could add an error message/warning in the code itself?

[mnapoli]

Opening a separate thread for "should it be a construct?"

We initially planned the VPC to be "hidden": it would be created/enabled as soon as you would import a construct that requires it. E.g. if you use a database (RDS) construct.

And to be clear: it would automatically apply to all functions.

Not having it as a classic construct solves the issue of "you can't have 2 VPC constructs".

However, we still need to put it in the YAML file somewhere. One possibility could be:

service: myapp
provider:
  lift:
    vpc:
      az: 3 # optional setting

Any other idea? (also ping @fredericbarthelet)

[t-richard]

In my opinion, not making it a construct would be very limitative.

Here is my try at "it should be a construct"

Being a construct would keep the syntax consistent which is easier for the user.

It would allow to use references (variables), and allow users to use the VPC construct alone to do things not provided by Lift.

Lets say i want to create a DocumentDB cluster which is not supported by Lift, I could just use custom CF and use variables exposed by Lift to wire up the Cluster to the VPC created by Lift.

It would also allow to support more advanced use cases in the future : only given functions to put in a VPC, having several VPCs in the same stack, add commands (to enable a bastion for example ?)

Having it created behind the scene is great because the user does not have to care or even to know about the VPC. But I think it's important to make the VPC explicit and available because we won't be able to support all VPC-dependent services so this empowers users. The experience would be great for supported services but very poor to extend.

And on the other end of the spectrum, users might want to handle VPC by themselves (eg. when putting several apps in the same VPC) and might want to be able to use a RDS construct and being able to provide the VPC. This might be easier if the VPC is explicit.

As a start, we would just have to make sure there is proper documentation and error messages if the VPC is not created or if two of them are present in the stack (in the beginning, but this could change later).

Either way, there are challenges, pros and cons. But this seems like the simplest and comprehensive solution.

[mnapoli]

Thanks for those details. I'm leaning in "let's make it a construct" because it would allow us to ship this ASAP, without taking too many risks with the architecture (not introducing the concept of "shared resources") and see the limitations afterwards.

We can easily change course, for example:

• ship a VPC construct
• we realize the UX is not ideal
• we iterate and ship a new (better) "VPC" concept (e.g. the one we mentioned, integrated in the "provider")
• we deprecate the old VPC construct

We don't loose much with that strategy, we don't paint ourselves into a corner. I like that.

@fredericbarthelet are you ok if we first try with a VPC construct (like any other construct) and see how it goes?