Skip to content

Commit

Permalink
cloned sdk site
Browse files Browse the repository at this point in the history
  • Loading branch information
werzl committed Sep 9, 2020
1 parent e4c44eb commit fa31dbc
Show file tree
Hide file tree
Showing 82 changed files with 12,739 additions and 0 deletions.
28 changes: 28 additions & 0 deletions .github/workflows/CD.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: CD

on:
push:
branches: master

jobs:

build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- uses: actions/setup-node@v1

- name: Installation ⏳
run: yarn

- name: Build 🧱
run: yarn build

- name: Deploy 🚀
uses: JamesIves/github-pages-deploy-action@releases/v3
with:
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }}
BRANCH: gh-pages
FOLDER: build
21 changes: 21 additions & 0 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: CI

on:
pull_request:
branches: master

jobs:

build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2

- uses: actions/setup-node@v1

- name: Installation ⏳
run: yarn

- name: Build 🧱
run: yarn build
21 changes: 21 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Dependencies
/node_modules

# Production
/build

# Generated files
.docusaurus
.cache-loader

# Misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local

debug.log*
npm-debug.log*
yarn-debug.log*
yarn-error.log*
202 changes: 202 additions & 0 deletions docs/doc1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
---
id: doc1
title: Style Guide
sidebar_label: Style Guide
---

You can write content using [GitHub-flavored Markdown syntax](https://github.github.com/gfm/).

## Markdown Syntax

To serve as an example page when styling markdown based Docusaurus sites.

## Headers

# H1 - Create the best documentation

## H2 - Create the best documentation

### H3 - Create the best documentation

#### H4 - Create the best documentation

##### H5 - Create the best documentation

###### H6 - Create the best documentation

---

## Emphasis

Emphasis, aka italics, with _asterisks_ or _underscores_.

Strong emphasis, aka bold, with **asterisks** or **underscores**.

Combined emphasis with **asterisks and _underscores_**.

Strikethrough uses two tildes. ~~Scratch this.~~

---

## Lists

1. First ordered list item
1. Another item ⋅⋅\* Unordered sub-list.
1. Actual numbers don't matter, just that it's a number ⋅⋅1. Ordered sub-list
1. And another item.

⋅⋅⋅You can have properly indented paragraphs within list items. Notice the blank line above, and the leading spaces (at least one, but we'll use three here to also align the raw Markdown).

⋅⋅⋅To have a line break without a paragraph, you will need to use two trailing spaces.⋅⋅ ⋅⋅⋅Note that this line is separate, but within the same paragraph.⋅⋅ ⋅⋅⋅(This is contrary to the typical GFM line break behaviour, where trailing spaces are not required.)

- Unordered list can use asterisks

* Or minuses

- Or pluses

---

## Links

[I'm an inline-style link](https://www.google.com)

[I'm an inline-style link with title](https://www.google.com "Google's Homepage")

[I'm a reference-style link][arbitrary case-insensitive reference text]

[I'm a relative reference to a repository file](../blob/master/LICENSE)

[You can use numbers for reference-style link definitions][1]

Or leave it empty and use the [link text itself].

URLs and URLs in angle brackets will automatically get turned into links. http://www.example.com or <http://www.example.com> and sometimes example.com (but not on Github, for example).

Some text to show that the reference links can follow later.

[arbitrary case-insensitive reference text]: https://www.mozilla.org
[1]: http://slashdot.org
[link text itself]: http://www.reddit.com

---

## Images

Here's our logo (hover to see the title text):

Inline-style: ![alt text](https://github.com/adam-p/markdown-here/raw/master/src/common/images/icon48.png 'Logo Title Text 1')

Reference-style: ![alt text][logo]

[logo]: https://github.com/adam-p/markdown-here/raw/master/src/common/images/icon48.png 'Logo Title Text 2'

---

## Code

```javascript
var s = 'JavaScript syntax highlighting';
alert(s);
```

```python
s = "Python syntax highlighting"
print(s)
```

```
No language indicated, so no syntax highlighting.
But let's throw in a <b>tag</b>.
```

```js {2}
function highlightMe() {
console.log('This line can be highlighted!');
}
```

---

## Tables

Colons can be used to align columns.

| Tables | Are | Cool |
| ------------- | :-----------: | -----: |
| col 3 is | right-aligned | \$1600 |
| col 2 is | centered | \$12 |
| zebra stripes | are neat | \$1 |

There must be at least 3 dashes separating each header cell. The outer pipes (|) are optional, and you don't need to make the raw Markdown line up prettily. You can also use inline Markdown.

| Markdown | Less | Pretty |
| -------- | --------- | ---------- |
| _Still_ | `renders` | **nicely** |
| 1 | 2 | 3 |

---

## Blockquotes

> Blockquotes are very handy in email to emulate reply text. This line is part of the same quote.
Quote break.

> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can _put_ **Markdown** into a blockquote.
---

## Inline HTML

<dl>
<dt>Definition list</dt>
<dd>Is something people use sometimes.</dd>

<dt>Markdown in HTML</dt>
<dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd>
</dl>

---

## Line Breaks

Here's a line for us to start with.

This line is separated from the one above by two newlines, so it will be a _separate paragraph_.

This line is also a separate paragraph, but... This line is only separated by a single newline, so it's a separate line in the _same paragraph_.

---

## Admonitions

:::note

This is a note

:::

:::tip

This is a tip

:::

:::important

This is important

:::

:::caution

This is a caution

:::

:::warning

This is a warning

:::
5 changes: 5 additions & 0 deletions docs/documentation-and-guides.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: Documentation and Guides
---

The pages here describe the Glasswall Product APIs and how to use them.
7 changes: 7 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Getting Started
---

# File Drop

The perfect introduction is to try out the Glasswall technology with our [File Drop web page](https://file-drop.co.uk/ "Glasswall Filedrop Page").
61 changes: 61 additions & 0 deletions docs/how-tos/ci-cd-pipeline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: Set-up of CI/CD for GitHub Repository

sidebar_label: CI/CD Setup
---

This guide will walk through the steps to set-up and create a CI/CD pipeline for your GitHub Repository

![](/img/CI-CD.png)
## Steps
### Branch Policy
1. On the GitHub repo, create a new branch from 'master' called 'develop'
2. Go to Settings > Branches, select 'develop' from the drop down list to make it the default branch, click update
3. Under 'Branch protection rules', click 'Add rule'
4. Add 'master' for 'Branch name pattern', check 'Restrict who can push to matching branches' and save.
5. Add new branch protection rule for 'develop', with the following settings:
- Require pull request reviews before merging
- Require status checks to pass before merging
- Require branches to be up to date before merging
### Repo Security
Next we need to add in the 'Glasswall-Github' user as an Admin of the repo so that we can perform branch merges from the actions, and use submodules.
1. On the GitHub repo, go to Settings > Manage access > Invite teams or people
2. Search for 'Glasswall-Github' and select the Admin role.
3. Go to to Settings > Secrets > Add a new secret
4. Name the secret 'TOKEN_GITHUB' and use the Personal Access Token which can be given to you by IT Support, as the value.
### Gated Build
Next step is to set-up a Gated build which will build our code and run tests on pull requests
1. On the GitHub repo, go to Actions > New workflow > set up a workflow yourself
2. Name the file 'gated.yml'
3. Copy the Gated file code from the Rebuild repo into the new file
4. Make any changes specific to your project (build steps, tests used)
5. Start commit > create new branch and start a pull request
### Setting Up Serverless Framework
Next step is to install the Serverless Framework and complete the template. More in-depth guides can be found here: https://www.serverless.com/framework/docs/
1. If running serverless for the first time, install to your machine by running: `npm install -g serverless` or `choco install serverless`
2. Create the template by running : `serverless create --template <template name>`, you can get a list of available templates by running `serverless create --help`
3. Update build.sh and build.cmd files with the correct output location and project location
4. Complete serverless.yml with the details of your project
5. Commit changes

You may experience an issue with executing the build.sh file in the CI or CD builds. To fix this you will need to mark the sh file executable:
1. cd to the sh file location
2. run: `git update-index --add --chmod=+x build.sh`
3. commit: `git commit -m 'Make build.sh executable'`

### CI Build
Next step is to set-up a CI build which will deploy our code to a QA environment, run tests, and merge to the master branch if successful.
1. On the GitHub repo, go to Actions > New workflow > set up a workflow yourself
2. Name the file 'ci.yml'
3. Copy the CI file template code into the new file
4. Make any changes specific to your project (deployment, environment, build steps, tests used)
5. Add any needed secrets (Access Ids and key) to the repo settings.
6. Start commit > create new branch and start a pull request
### Deploy Build
Next step is to set-up a Deploy build which will deploy our code to a Stage environment, run tests, and deploy to product if successful.
1. On the GitHub repo, go to Actions > New workflow > set up a workflow yourself
2. Name the file 'deploy.yml'
3. Copy the Deploy file template code into the new file
4. Make any changes specific to your project (deployment, environment, build steps, tests used)
5. Add any needed secrets (Access Ids and key) to the repo settings.
6. Start commit > create new branch and start a pull request
40 changes: 40 additions & 0 deletions docs/how-tos/file-rebuild-how-it-works.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
title: Rebuild API - Under the Hood

sidebar_label: Under the Hood
---

## Background

The Rebuild API leverages serverless functions to provide the battle hardened capablities of the Rebuild SDK at scale, available through HTTP API calls. With a range of consumption options available from pay-per-request to private cloud deployments, all the code is open sourced in [GitHub](https://github.com/filetrust). The first generation of code is focused on providing the capability using AWS components with Azure components surfacing shortly after first launch.

For our hosted solution we will assemble the components that give us the best levels of performance and resillience which could mean we are operating within multiple cloud providers. For managed and self deployed solutions then it really becomes a pick and choose for which is your cloud provider of choice, the aim here being that Glasswall's engineering team provide the components that fit all the major cloud providers that suit as many use cases as possible.

Glasswall's Rebuild SDK can process a file in-memory with no need to write the file, this means that for most of the resources we offer the file never leaves the serverless function other than the response to the request.

:::caution
Before you integrate Rebuild into your solution, be aware that file in the HTTP body only supports files up to 6MB due to a limitation in the request payload to AWS Lambda. If your use case deals with larger files, please use the input URL based pathway which currently supports files up to 30MB
:::caution

## Solution Overview

The serverless functionality offered by the cloud providers have clear similarities but also differences that shape the functionality that we offer.

### AWS

AWS Lambda provides multiple triggers but the one most appropriate for providing these API resources is the HTTP trigger. This gives a very clean usage pattern of file-in file-out where the body of the HTTP request is the binary of the file within a multipart/form-data content type. Where this usage patten ends is when we exceed the maximum HTTP payload for a Lambda function - currently 6MB.

![Figure1](/img/Rebuild-File-Architecture.png)

To bypass this limitation we have a second usage pattern which is to utilise pre-signed URL's to give the Lambda function authorised access to larger files (at launch files up to 30mb). In this pattern the file is placed in one of AWS's S3 buckets and a URL is generated that contains the authorisation required to access the file, this URL is then passed into the function which can then perform the required processing on the file. If (as is the case with Rebuild) the processing results in a new file, this file is placed in another S3 bucket which can be accessed via another signed URL.

The functionality is then accessed via API Gateway (in itself having a max request size of 10MB) which is secured via a Lambda Authoriser.

### Azure

When it comes to Azure the patterns are the same, it is the platform limits that change. Swapping Lambda for an Azure Function we still have access to an HTTP trigger but where the limit was 6MB - it is now 100MB. This means that the direct HTTP file-in file-out pattern becomes much more usable.

For larger payloads again a route using a pre-signed approach is available with Azure Blob Storage which allows files over 100MB to be processed.



Loading

0 comments on commit fa31dbc

Please sign in to comment.