-
Notifications
You must be signed in to change notification settings - Fork 10
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
82 changed files
with
12,739 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
name: CD | ||
|
||
on: | ||
push: | ||
branches: master | ||
|
||
jobs: | ||
|
||
build: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- uses: actions/setup-node@v1 | ||
|
||
- name: Installation ⏳ | ||
run: yarn | ||
|
||
- name: Build 🧱 | ||
run: yarn build | ||
|
||
- name: Deploy 🚀 | ||
uses: JamesIves/github-pages-deploy-action@releases/v3 | ||
with: | ||
ACCESS_TOKEN: ${{ secrets.ACCESS_TOKEN }} | ||
BRANCH: gh-pages | ||
FOLDER: build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
name: CI | ||
|
||
on: | ||
pull_request: | ||
branches: master | ||
|
||
jobs: | ||
|
||
build: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- uses: actions/checkout@v2 | ||
|
||
- uses: actions/setup-node@v1 | ||
|
||
- name: Installation ⏳ | ||
run: yarn | ||
|
||
- name: Build 🧱 | ||
run: yarn build |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Dependencies | ||
/node_modules | ||
|
||
# Production | ||
/build | ||
|
||
# Generated files | ||
.docusaurus | ||
.cache-loader | ||
|
||
# Misc | ||
.DS_Store | ||
.env.local | ||
.env.development.local | ||
.env.test.local | ||
.env.production.local | ||
|
||
debug.log* | ||
npm-debug.log* | ||
yarn-debug.log* | ||
yarn-error.log* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,202 @@ | ||
--- | ||
id: doc1 | ||
title: Style Guide | ||
sidebar_label: Style Guide | ||
--- | ||
|
||
You can write content using [GitHub-flavored Markdown syntax](https://github.github.com/gfm/). | ||
|
||
## Markdown Syntax | ||
|
||
To serve as an example page when styling markdown based Docusaurus sites. | ||
|
||
## Headers | ||
|
||
# H1 - Create the best documentation | ||
|
||
## H2 - Create the best documentation | ||
|
||
### H3 - Create the best documentation | ||
|
||
#### H4 - Create the best documentation | ||
|
||
##### H5 - Create the best documentation | ||
|
||
###### H6 - Create the best documentation | ||
|
||
--- | ||
|
||
## Emphasis | ||
|
||
Emphasis, aka italics, with _asterisks_ or _underscores_. | ||
|
||
Strong emphasis, aka bold, with **asterisks** or **underscores**. | ||
|
||
Combined emphasis with **asterisks and _underscores_**. | ||
|
||
Strikethrough uses two tildes. ~~Scratch this.~~ | ||
|
||
--- | ||
|
||
## Lists | ||
|
||
1. First ordered list item | ||
1. Another item ⋅⋅\* Unordered sub-list. | ||
1. Actual numbers don't matter, just that it's a number ⋅⋅1. Ordered sub-list | ||
1. And another item. | ||
|
||
⋅⋅⋅You can have properly indented paragraphs within list items. Notice the blank line above, and the leading spaces (at least one, but we'll use three here to also align the raw Markdown). | ||
|
||
⋅⋅⋅To have a line break without a paragraph, you will need to use two trailing spaces.⋅⋅ ⋅⋅⋅Note that this line is separate, but within the same paragraph.⋅⋅ ⋅⋅⋅(This is contrary to the typical GFM line break behaviour, where trailing spaces are not required.) | ||
|
||
- Unordered list can use asterisks | ||
|
||
* Or minuses | ||
|
||
- Or pluses | ||
|
||
--- | ||
|
||
## Links | ||
|
||
[I'm an inline-style link](https://www.google.com) | ||
|
||
[I'm an inline-style link with title](https://www.google.com "Google's Homepage") | ||
|
||
[I'm a reference-style link][arbitrary case-insensitive reference text] | ||
|
||
[I'm a relative reference to a repository file](../blob/master/LICENSE) | ||
|
||
[You can use numbers for reference-style link definitions][1] | ||
|
||
Or leave it empty and use the [link text itself]. | ||
|
||
URLs and URLs in angle brackets will automatically get turned into links. http://www.example.com or <http://www.example.com> and sometimes example.com (but not on Github, for example). | ||
|
||
Some text to show that the reference links can follow later. | ||
|
||
[arbitrary case-insensitive reference text]: https://www.mozilla.org | ||
[1]: http://slashdot.org | ||
[link text itself]: http://www.reddit.com | ||
|
||
--- | ||
|
||
## Images | ||
|
||
Here's our logo (hover to see the title text): | ||
|
||
Inline-style:  | ||
|
||
Reference-style: ![alt text][logo] | ||
|
||
[logo]: https://github.com/adam-p/markdown-here/raw/master/src/common/images/icon48.png 'Logo Title Text 2' | ||
|
||
--- | ||
|
||
## Code | ||
|
||
```javascript | ||
var s = 'JavaScript syntax highlighting'; | ||
alert(s); | ||
``` | ||
|
||
```python | ||
s = "Python syntax highlighting" | ||
print(s) | ||
``` | ||
|
||
``` | ||
No language indicated, so no syntax highlighting. | ||
But let's throw in a <b>tag</b>. | ||
``` | ||
|
||
```js {2} | ||
function highlightMe() { | ||
console.log('This line can be highlighted!'); | ||
} | ||
``` | ||
|
||
--- | ||
|
||
## Tables | ||
|
||
Colons can be used to align columns. | ||
|
||
| Tables | Are | Cool | | ||
| ------------- | :-----------: | -----: | | ||
| col 3 is | right-aligned | \$1600 | | ||
| col 2 is | centered | \$12 | | ||
| zebra stripes | are neat | \$1 | | ||
|
||
There must be at least 3 dashes separating each header cell. The outer pipes (|) are optional, and you don't need to make the raw Markdown line up prettily. You can also use inline Markdown. | ||
|
||
| Markdown | Less | Pretty | | ||
| -------- | --------- | ---------- | | ||
| _Still_ | `renders` | **nicely** | | ||
| 1 | 2 | 3 | | ||
|
||
--- | ||
|
||
## Blockquotes | ||
|
||
> Blockquotes are very handy in email to emulate reply text. This line is part of the same quote. | ||
Quote break. | ||
|
||
> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can _put_ **Markdown** into a blockquote. | ||
--- | ||
|
||
## Inline HTML | ||
|
||
<dl> | ||
<dt>Definition list</dt> | ||
<dd>Is something people use sometimes.</dd> | ||
|
||
<dt>Markdown in HTML</dt> | ||
<dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> | ||
</dl> | ||
|
||
--- | ||
|
||
## Line Breaks | ||
|
||
Here's a line for us to start with. | ||
|
||
This line is separated from the one above by two newlines, so it will be a _separate paragraph_. | ||
|
||
This line is also a separate paragraph, but... This line is only separated by a single newline, so it's a separate line in the _same paragraph_. | ||
|
||
--- | ||
|
||
## Admonitions | ||
|
||
:::note | ||
|
||
This is a note | ||
|
||
::: | ||
|
||
:::tip | ||
|
||
This is a tip | ||
|
||
::: | ||
|
||
:::important | ||
|
||
This is important | ||
|
||
::: | ||
|
||
:::caution | ||
|
||
This is a caution | ||
|
||
::: | ||
|
||
:::warning | ||
|
||
This is a warning | ||
|
||
::: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
title: Documentation and Guides | ||
--- | ||
|
||
The pages here describe the Glasswall Product APIs and how to use them. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
--- | ||
title: Getting Started | ||
--- | ||
|
||
# File Drop | ||
|
||
The perfect introduction is to try out the Glasswall technology with our [File Drop web page](https://file-drop.co.uk/ "Glasswall Filedrop Page"). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
--- | ||
title: Set-up of CI/CD for GitHub Repository | ||
|
||
sidebar_label: CI/CD Setup | ||
--- | ||
|
||
This guide will walk through the steps to set-up and create a CI/CD pipeline for your GitHub Repository | ||
|
||
 | ||
## Steps | ||
### Branch Policy | ||
1. On the GitHub repo, create a new branch from 'master' called 'develop' | ||
2. Go to Settings > Branches, select 'develop' from the drop down list to make it the default branch, click update | ||
3. Under 'Branch protection rules', click 'Add rule' | ||
4. Add 'master' for 'Branch name pattern', check 'Restrict who can push to matching branches' and save. | ||
5. Add new branch protection rule for 'develop', with the following settings: | ||
- Require pull request reviews before merging | ||
- Require status checks to pass before merging | ||
- Require branches to be up to date before merging | ||
### Repo Security | ||
Next we need to add in the 'Glasswall-Github' user as an Admin of the repo so that we can perform branch merges from the actions, and use submodules. | ||
1. On the GitHub repo, go to Settings > Manage access > Invite teams or people | ||
2. Search for 'Glasswall-Github' and select the Admin role. | ||
3. Go to to Settings > Secrets > Add a new secret | ||
4. Name the secret 'TOKEN_GITHUB' and use the Personal Access Token which can be given to you by IT Support, as the value. | ||
### Gated Build | ||
Next step is to set-up a Gated build which will build our code and run tests on pull requests | ||
1. On the GitHub repo, go to Actions > New workflow > set up a workflow yourself | ||
2. Name the file 'gated.yml' | ||
3. Copy the Gated file code from the Rebuild repo into the new file | ||
4. Make any changes specific to your project (build steps, tests used) | ||
5. Start commit > create new branch and start a pull request | ||
### Setting Up Serverless Framework | ||
Next step is to install the Serverless Framework and complete the template. More in-depth guides can be found here: https://www.serverless.com/framework/docs/ | ||
1. If running serverless for the first time, install to your machine by running: `npm install -g serverless` or `choco install serverless` | ||
2. Create the template by running : `serverless create --template <template name>`, you can get a list of available templates by running `serverless create --help` | ||
3. Update build.sh and build.cmd files with the correct output location and project location | ||
4. Complete serverless.yml with the details of your project | ||
5. Commit changes | ||
|
||
You may experience an issue with executing the build.sh file in the CI or CD builds. To fix this you will need to mark the sh file executable: | ||
1. cd to the sh file location | ||
2. run: `git update-index --add --chmod=+x build.sh` | ||
3. commit: `git commit -m 'Make build.sh executable'` | ||
|
||
### CI Build | ||
Next step is to set-up a CI build which will deploy our code to a QA environment, run tests, and merge to the master branch if successful. | ||
1. On the GitHub repo, go to Actions > New workflow > set up a workflow yourself | ||
2. Name the file 'ci.yml' | ||
3. Copy the CI file template code into the new file | ||
4. Make any changes specific to your project (deployment, environment, build steps, tests used) | ||
5. Add any needed secrets (Access Ids and key) to the repo settings. | ||
6. Start commit > create new branch and start a pull request | ||
### Deploy Build | ||
Next step is to set-up a Deploy build which will deploy our code to a Stage environment, run tests, and deploy to product if successful. | ||
1. On the GitHub repo, go to Actions > New workflow > set up a workflow yourself | ||
2. Name the file 'deploy.yml' | ||
3. Copy the Deploy file template code into the new file | ||
4. Make any changes specific to your project (deployment, environment, build steps, tests used) | ||
5. Add any needed secrets (Access Ids and key) to the repo settings. | ||
6. Start commit > create new branch and start a pull request |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
--- | ||
title: Rebuild API - Under the Hood | ||
|
||
sidebar_label: Under the Hood | ||
--- | ||
|
||
## Background | ||
|
||
The Rebuild API leverages serverless functions to provide the battle hardened capablities of the Rebuild SDK at scale, available through HTTP API calls. With a range of consumption options available from pay-per-request to private cloud deployments, all the code is open sourced in [GitHub](https://github.com/filetrust). The first generation of code is focused on providing the capability using AWS components with Azure components surfacing shortly after first launch. | ||
|
||
For our hosted solution we will assemble the components that give us the best levels of performance and resillience which could mean we are operating within multiple cloud providers. For managed and self deployed solutions then it really becomes a pick and choose for which is your cloud provider of choice, the aim here being that Glasswall's engineering team provide the components that fit all the major cloud providers that suit as many use cases as possible. | ||
|
||
Glasswall's Rebuild SDK can process a file in-memory with no need to write the file, this means that for most of the resources we offer the file never leaves the serverless function other than the response to the request. | ||
|
||
:::caution | ||
Before you integrate Rebuild into your solution, be aware that file in the HTTP body only supports files up to 6MB due to a limitation in the request payload to AWS Lambda. If your use case deals with larger files, please use the input URL based pathway which currently supports files up to 30MB | ||
:::caution | ||
|
||
## Solution Overview | ||
|
||
The serverless functionality offered by the cloud providers have clear similarities but also differences that shape the functionality that we offer. | ||
|
||
### AWS | ||
|
||
AWS Lambda provides multiple triggers but the one most appropriate for providing these API resources is the HTTP trigger. This gives a very clean usage pattern of file-in file-out where the body of the HTTP request is the binary of the file within a multipart/form-data content type. Where this usage patten ends is when we exceed the maximum HTTP payload for a Lambda function - currently 6MB. | ||
|
||
 | ||
|
||
To bypass this limitation we have a second usage pattern which is to utilise pre-signed URL's to give the Lambda function authorised access to larger files (at launch files up to 30mb). In this pattern the file is placed in one of AWS's S3 buckets and a URL is generated that contains the authorisation required to access the file, this URL is then passed into the function which can then perform the required processing on the file. If (as is the case with Rebuild) the processing results in a new file, this file is placed in another S3 bucket which can be accessed via another signed URL. | ||
|
||
The functionality is then accessed via API Gateway (in itself having a max request size of 10MB) which is secured via a Lambda Authoriser. | ||
|
||
### Azure | ||
|
||
When it comes to Azure the patterns are the same, it is the platform limits that change. Swapping Lambda for an Azure Function we still have access to an HTTP trigger but where the limit was 6MB - it is now 100MB. This means that the direct HTTP file-in file-out pattern becomes much more usable. | ||
|
||
For larger payloads again a route using a pre-signed approach is available with Azure Blob Storage which allows files over 100MB to be processed. | ||
|
||
|
||
|
Oops, something went wrong.