[RFC] Static Website

[fredericbarthelet]

The goal of this discussion is to get feedback on the "Static Website" component.

If you are new to Lift, a quick intro: it's a Serverless plugin that can be installed via npm and enabled in any serverless.yml file.

Here is what we are planning so far.

Use case

Deploying static websites and single-page applications, for example React, VueJS or Angular apps.

⚠️ Websites rendered server-side, i.e. with a backend language, will be covered by another component.

Quick start

# serverless.yml

# [...]

static-websites:

  landing:
    path: 'landing/dist'

The example above will create an S3 bucket to host the static website, and set up the CloudFront CDN.

Note: this component takes about 5 minutes to deploy the first time.

In the first version, the website will be deployed manually to S3 using the AWS CLI:

aws s3 sync landing/dist s3://bucket-name/ --delete

In the future, deployment will happen automatically during serverless deploy, including CDN cache invalidation.

How it works

The component will deploy:

• an S3 bucket with OAI
• CloudFront CDN in front of S3 to provide HTTPS and caching at edge

Cache invalidation

Quick intro on how caching with CloudFront+S3 works:

• By default, files are cached in CloudFront, not in browsers. Browsers send "If-Modified" requests.
  • Cache busting works by simply invalidating CloudFront.
• With Cache-Control: max-age=3600 in S3 (on each object): CloudFront + browsers will cache for 1 hour.
  • Cache busting in CloudFront is possible with an invalidation. In the browser, the only solution is versioning.
• With Cache-Control: max-age=60 and Cache-Control: s-maxage=3600 in S3 (on each object): browsers will cache for 1 minute (max-age) and CloudFront for 1 hour (s-maxage).
  • Cache busting in CloudFront is possible with an invalidation. In the browser, the only solution is versioning.

Here are the use cases we plan on supporting:

1. I want a simple solution (no versioning) with edge caching.

   static-websites:
     landing:
       path: 'public/landing'

   Lift deploys without Cache-Control. CloudFront will cache, but not browsers. On deployment, Lift creates a CloudFront invalidation that clears the CloudFront cache.

   After the deployment, every new page load should get the latest version.

   If a SPA was loaded in a browser before the deployment, every new asset loaded will be the new version.

   Note: CloudFront invalidations are free up to 30 invalidations per day. Then it's 0.005$ per invalidation.

2. I want the most aggressive caching.

   static-websites:
     landing:
       path: 'public/landing'
       browserCaching: true # could also be a duration, e.g. 3600

   In that scenario, website assets must contain a version number or hash in their filename (for example in React).

   Lift deploys with Cache-Control=3600 on all files except HTML. CloudFront will cache everything, browsers will cache everything except HTML files.

   Why not HTML? So that browsers always reload the latest HTML version that points to the latest filenames.

   On deployment, Lift uploads the new files and creates a CloudFront invalidation which clears the CloudFront cache.

   New page loads will receive the latest HTML files, pointing to the latest assets versions.

   If a SPA was loaded in a browser before the deployment, it will keep trying to load old asset versions. This is why older versions are kept for 1 day in the S3 bucket. Lift clears old asset versions after 1 day.

Configuration reference

Custom domain

static-websites:
  landing:
    domain: mywebsite.com
    # or (CloudFront accepts multiple domains)
    domain:
      - mywebsite.com
      - app.mywebsite.com
    # ARN of an ACM certificate for the domain, registered in us-east-1
    certificate: arn:aws:acm:us-east-1:123456615250:certificate/0a28e63d-d3a9-4578-9f8b-14347bfe8123

The configuration above will activate the custom domain mywebsite.com on CloudFront, using the provided HTTPS certificate.

The certificate must be registered in ACM in the us-east-1 region before deploying.

After deploying, a CloudFront domain will be generated. Create a CNAME DNS entry for your domain that points to that xxx.cloudfront.net domain.

Browser caching

static-websites:
  landing:
    path: 'public/landing'
    browserCaching: true # could also be a duration, e.g. 3600

See the previous section about "Cache invalidation" for more details.

Security

Ideally, we want some security headers to be present in responses.

How can we implement that? Lambda@Edge? → heavy solution

---

Feedback is welcome, just post a reply to this discussion 🙏

[M1ke]

including CDN cache invalidation

I wonder could it also handle wiping the bucket content. E.g. for create-react-app each build generates custom JS files with a hash in the name relating to the build content. This means cache invalidation isn't needed, but also means that your bucket over time (especially if you're doing rapid development, CI/CD etc) fills up with lots of versions of very similar JS files, all but one of which are essentially old code. This could even be a security risk, e.g. you accidentally publish a key and then quickly upload a new build, but your file with the key is still there.

For good reason S3 doesn't make it easy to empty a bucket but for some application configs it'd be completely legitimate to have this as part of the framework.

[M1ke]

Just on that last point:

with an option to disable the pruning of course

With option 2 you can have an option to disable, as the process should be safe given it's aware of only specific files to delete.

With option 1 the deletion definitely needs to be opt-in as people can be using the bucket for anything else - it's bad practice but a general delete command can be very dangerous!

[mnapoli]

@M1ke this is a good point (sorry for not getting back sooner on that). The bucket for the static website is really a bucket containing code artifacts.

It isn't meant to store content at all, that's why it's 100% public and there are no backups enabled.

And to be clear: on every deployment made via serverless deploy, any extra file will be deleted. Just like when deploying to Netlify for example.

And yeah, we should 100% protect against mistakes. Besides documentation, do you see any way we can prevent accidental misuse? (i.e. no data uploaded in there)

[dhrrgn]

And yeah, we should 100% protect against mistakes. Besides documentation, do you see any way we can prevent accidental misuse? (i.e. no data uploaded in there)

Since the website bucket is created and controlled via the plugin, I think the chance of misuse should be minimal. You'd have to go out of your way and try to do it.

That being said, the only thing I can think of is adding a check pre-deploy to make sure there are no non-deployed artifacts. This would slow down deploys though. It'd have to look something like:

• Add metadata to each deployed object via --metadata on the s3 sync call, like --metadata "DeploymentArtifact=1

Pre-deploy:

1. List all objects (recursively in case there are >1000)
2. For each object, make a GetObjectMetadata request and if any objects are missing the DeploymentArtifact user metadata, abort the deploy with an error.

That sucks though, and adds a bunch of overhead for an extreme edge case (IMO).

If someone really needed to use the same bucket, they could add a policy to prevent deletion on their objects...which would probably break this component though since emptyBucket would error (not sure how s3 sync handles that...)

[mnapoli]

@dhrrgn that is a very good idea actually!

I am working on the deployment and I need to list all objects anyway (to upload only changed files). So that could be doable without overhead. I'll keep that in mind and probably try it later in the next weeks.

[dhrrgn]

@mnapoli Ah, very nice. It is annoying that you can't request user metadata in the list request. But, the headObject requests should be quick. Could make it faster by sending 10-20 async requests at a time and using Promise.all and rejecting if an unknown object.

An alternative is similar to what I suggested previously: During deploy, add the keys for the uploaded objects to the CF output, or to a (non-public) manifest file. Then it's pretty trivial to detect changed/new/unknown files in the bucket. 🤷‍♂️

[M1ke]

Also whilst you're mentioning Lambda@Edge, worth noting that you can also do that for static site "neat" URLs, e.g. you have a directory structure like

public/
  - about/
  - - index.html
  - contact/
  - - index.html
  - index.html

So you want domain.com/ to get the bottom index.html file, domain.com/about to get the one in that directory etc. You can't do that natively but Lambda@Edge on Cloudfront can rewrite it for you. I've documented that here and also published a Terraform module to do that simply for those using TF to manage infrastructure.

[mnapoli]

domain.com/about/ should work, do you mean that domain.com/about won't? (i.e. the difference is the trailing /?)

[M1ke]

I know that when I initially build my static site in 2018 I found that the structure I was used to using from static site generators like Jekyll which would turn about.md into about/index.html got a lot of not found errors. So I wrote that module, tested it, and it's been running ever since. Maybe it's not needed for every case, would make sense for Cloudfront to do that natively, but I'd need to check!

[M1ke]

@mnapoli seems we don't need Lambda@Edge for those rewrites any more!

https://aws.amazon.com/about-aws/whats-new/2021/05/cloudfront-functions/

[mnapoli]

@M1ke Yep I was waiting exactly for that feature ^^

[andrea-cristaudo]

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/example-function-add-index.html

[wysow]

And just a thought but this feature should also work for static assets of a dynamic website I think...

[mnapoli]

Server-side websites (or backend websites) will be addressed in a separate component as this is a different use case that leads to a different solution. It will also let us provide options specific to server-side websites.

[andrea-cristaudo]

One common use case for static websites is redirecting www to non-www addresses or viceversa. With a web server is the easiest thing in the world but with aws-static-website-stack is a very complex task.
It should be very easy to implement in application level (something like what wordpress does) but it'd be very nice to get it in infrastructure level.

[mnapoli]

That's a very good point, TBH I never thought about how to solve that without Lambda@Edge (which is a PITA, if we can avoid it that's really great).

With S3 alone IIRC there's a technique with having 2 separate buckets (one for www, for for the root domain), but I'm not sure if it works with CloudFront. I'll have to research that.

[andrea-cristaudo]

Found out how to do without lambda@edge, it requires (for https support) one additional s3 bucket, one additional cloudfront instance and the dns record pointing to the cf instance.
I'll try to create a PR this week.

[mnapoli]

@andrea-cristaudo thanks. FYI don't bother with the PR yet, the codebase is very early and unstable.

Also I might have an idea on how to implement that in a simpler way!

[M1ke]

Yeah S3 is the way to go for www. redirects, there's a Terraform example here

[mnapoli]

CloudFront Functions are out, this might help here too.

[guillaumem-theodo]

About Security topic, adding default Security headers is a must have on this component. A set of default security headers could be a nice feature. Implementation using Lambda@Edge.

Something like:

static-websites:
  landing:
    path: 'public/landing'
    securityHeaders: yes

or (adding additional headers)

static-websites:
  landing:
    path: 'public/landing'
    securityHeaders:
      default: yes
      'X-XSS-Protection': '1; mode=block'
      ...

[mnapoli]

CloudFront Functions are out and could help with that. It seems there will even be CloudFormation support with inline code, and CDK support as well: https://github.com/aws/aws-cdk/pull/14511/files#diff-3b0cd2595ecdf6f4a567197fcc6399233a11280b54a69b3730458f021fe6c933R616-R621

I'm waiting for CloudFormation support before trying out further (at the moment it's not documented yet).

[mnapoli]

CloudFormation is finally documented and supported: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudfront-function.html

I'll try that next week hopefully.

[andrea-cristaudo]

Example of adding security headers with CloudFront Functions: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/example-function-add-security-headers.html

[mnapoli]

Thank you @andrea-cristaudo!

[mnapoli]

Finally managed to implement security headers using CloudFront Functions: #21

Any review is welcome :)

[andrea-cristaudo]

Maybe would be useful to enable brotli compression as well?
It's not enough to set "compress: true" for cache behaviour but it's required to use a CachePolicy with a ParametersInCacheKeyAndForwardedToOrigin instance with EnableAcceptEncodingBrotli set to true. Here the link to the documentation: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-cloudfront-cachepolicy-parametersincachekeyandforwardedtoorigin.html

[mnapoli]

🤯 nice! Thanks a lot, will try to apply that next week!

[mnapoli]

I've opened #22 to apply that recommendation, thank you!

Do you know any downside of switching to using "CachePolicy"? I haven't found any, but just in case.

[Pigius]

Out of curiosity, do you plan to support NextJS to the static website construct as well?

[Pigius]

@t-richard I played around with NextJS and static-website construct for a while and managed to do deployment after exporting to static HTML. However, not every project with nextJS can be exported this way (Not all functionalities are compatible with next export.).

However the deployment works, I even made a simple Proof of Concept: https://github.com/Pigius/nextjs-app-service. Thanks a lot for the help!

[mnapoli]

@Pigius very interesting, thanks for sharing!

I'd be interested to know what's missing to properly support NextJS (you mention missing features). Or maybe as @t-richard mentions, we would need backend capabilities (Lambda or Lambda@Edge) to fully support NextJS?

[Pigius]

You're welcome. I did not mention any missing features at all, but I think what already @t-richard mentioned will make a job (SSR construct).

[t-richard]

@Pigius nice to know it worked !

These frameworks (Next, Nuxt, etc) are kinda limited when using only static generation. They are truly powerful when you enable server-side rendering (and that's their main purpose IMO).

In the case of SSR, then again, the Server-side website construct could be the place for that. I've only done SSR on Lambda with classic Lambdas, not Lambda@Edge and I think it is too restrictive for SSR to work properly.

[Pigius]

I see your point. Thanks!

[andrea-cristaudo]

Would be useful to let protect a static website with some kind of auth system. It would be great to provide staging versions of a website not public available.

Maybe with Cloudfront function could be achieved something like basic authentication with username/password credentials?

[mnapoli]

There seems to be interest in this (#78 mentions that too), so I've turned your comment into a new issue: #79.

Please add a 👍 on #79: we use these to prioritize new features.