Skip to content

Commit

Permalink
Merge branch 'main' into develop
Browse files Browse the repository at this point in the history
  • Loading branch information
@msudgh
msudgh committed Aug 18, 2024
2 parents 13c25c7 + a977d24 commit 493764b
Show file tree
Hide file tree
Showing 2 changed files with 191 additions and 122 deletions.
309 changes: 189 additions & 120 deletions docs/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,46 +48,32 @@ <h3> </h3>
<a href="./.github/workflows/ci.yml"><img src="https://github.com/msudgh/sync-cloud-storage/actions/workflows/ci.yml/badge.svg?branch=main" alt="Pipeline Status"></a>
<a href="https://codecov.io/gh/msudgh/sync-cloud-storage"><img src="https://codecov.io/gh/msudgh/sync-cloud-storage/branch/main/graph/badge.svg?token=2BY6063VOY" alt="Codecov Status"></a>
<a href="LICENSE"><img src="https://img.shields.io/github/license/msudgh/sync-cloud-storage" alt="License"></a></p>
<p>Synchronize files and directories between a remote computer and multiple Serverless cloud providers' storages.</p>
<p>Supported cloud providers:</p>
<ul>
<li>[x] AWS S3</li>
</ul>
<p>Synchronize files and directories between a remote machine and a cloud storage via cloud frameworks and stacks consisting of <a href="https://www.serverless.com/">AWS SAM (Serverless)</a> and <a href="https://aws.amazon.com/cdk/">AWS Cloud Development Kit (CDK)</a>. This package supports the following cloud storage providers: <a href="https://aws.amazon.com/s3/">AWS S3</a>.</p>
<h2>Features</h2>
<ul>
<li>Sync multiple storages at once and flexible file matching (single or multiple file/dir sync) by defining patterns of <a href="https://en.wikipedia.org/wiki/Glob_(programming)"><code>glob</code></a> to include or exclude</li>
<li>Supports a set of options as following for each file based on storage: <code>Prefix</code>, <code>Access Control List (ACL)</code>, <code>Tags</code>, <code>Metadata</code></li>
<li>Select a list of specific sync actions for each storage: <code>uploading</code>, <code>deleting</code></li>
<li>Modern and uses the latest official cloud provider's SDK
<ul>
<li>AWS S3: <a href="https://www.npmjs.com/package/@aws-sdk/client-s3"><code>[email protected]</code></a></li>
</ul>
</li>
<li>Sync multiple storages at once and flexible file matching (single or multiple file/dir sync) by just defining patterns of <a href="https://en.wikipedia.org/wiki/Glob_(programming)"><code>glob</code></a> to include or exclude</li>
<li>Set the following configs for each synced file:
<ul>
<li>prefix</li>
<li>access control list (ACL)</li>
<li>tags</li>
<li>metadata</li>
</ul>
</li>
<li>Select a list of specific sync actions for each storage:
<blockquote>
<p><strong>Note</strong>: This plugin is still in development and may not be stable. Use with caution.</p>
</blockquote>
<h2>Installation</h2>
<ul>
<li>upload</li>
<li>delete</li>
<li><a href="https://npmjs.com/"><strong>npm</strong></a>: <code>npm i sync-cloud-storage</code></li>
<li><a href="https://yarnpkg.com/"><strong>yarn</strong></a>: <code>yarn add sync-cloud-storage</code></li>
<li><a href="https://pnpm.io/"><strong>pnpm</strong></a>: <code>pnpm add sync-cloud-storage</code></li>
<li><a href="https://github.com/antfu/ni"><strong>ni</strong></a>: <code>ni sync-cloud-storage</code></li>
</ul>
</li>
<li>Written in <strong>TypeScript</strong> with strong type checking</li>
<li>Integration test with cloud providers</li>
</ul>
<h2>Installation</h2>
<ol>
<li><a href="https://github.com/antfu/ni"><strong>ni</strong></a>: <code>ni sync-cloud-storage -D</code></li>
<li><a href="https://npmjs.com/"><strong>npm</strong></a>: <code>npm i sync-cloud-storage -D</code></li>
<li><a href="https://yarnpkg.com/"><strong>yarn</strong></a>: <code>yarn add sync-cloud-storage -D</code></li>
<li><a href="https://pnpm.io/"><strong>pnpm</strong></a>: <code>pnpm add sync-cloud-storage -D</code></li>
</ol>
<h2>Usage</h2>
<h3>Serverless</h3>
<h4>AWS S3</h4>
<h3>AWS S3</h3>
<h4>Serverless</h4>
<p>Sync storages action as a pre-deploy hook in the <code>serverless.yml</code>:</p>
<pre class="prettyprint source lang-yaml"><code>plugins:
- sync-cloud-storage

Expand All @@ -105,96 +91,179 @@ <h4>AWS S3</h4>
foo: bar
bar: foo
</code></pre>
<h2>Configuration Reference</h2>
<p>This section provides a detailed reference for all configuration options.</p>
<ul>
<li>
<p><code>name</code>:</p>
<ul>
<li>Type: <code>string</code></li>
<li>Required: <code>true</code></li>
<li>Minimum length: 1</li>
<li>Example: <code>name: assets</code></li>
</ul>
</li>
<li>
<p><code>patterns</code>: File patterns to include or exclude during synchronization.</p>
<ul>
<li>Type: <code>array</code> of <code>string</code></li>
<li>Required: <code>true</code></li>
<li>Minimum items: 1</li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>patterns:
- &quot;assets/*&quot;
- &quot;!assets/temp/*&quot;
</code></pre>
</li>
</ul>
</li>
<li>
<p><code>actions</code>: List of actions to perform during synchronization.</p>
<ul>
<li>Type: <code>array</code> of <code>string</code></li>
<li>Required: <code>true</code></li>
<li>Valid Values: <code>upload</code>, <code>delete</code></li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>actions:
- upload
- delete
</code></pre>
</li>
</ul>
</li>
<li>
<p><code>prefix</code>: The prefix to apply to all synced files in the bucket.</p>
<ul>
<li>Type: <code>string</code></li>
<li>Required: <code>false</code></li>
<li>Default: <code>&quot;&quot;</code></li>
<li>Example: <code>prefix: assets</code></li>
</ul>
</li>
<li>
<p><code>enabled</code>: Whether to enable the sync for the storage.</p>
<ul>
<li>Type: <code>boolean</code></li>
<li>Required: <code>false</code></li>
<li>Default: <code>true</code></li>
</ul>
</li>
<li>
<p><code>acl</code>: Access control list setting for the synced files.</p>
<ul>
<li>Type: <code>string</code></li>
<li>Required: <code>false</code></li>
<li>Valid Values: <code>private</code>, <code>public-read</code>, <code>public-read-write</code>, <code>authenticated-read</code></li>
<li>Reference: <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl">AWS S3 Canned ACL</a></li>
<li>Example: <code>acl: public-read</code></li>
</ul>
</li>
<li>
<p><code>metadata</code>: Custom metadata to apply to the synced files.</p>
<ul>
<li>Type: <code>object</code></li>
<li>Required: <code>false</code></li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>metadata:
foo: bar
bar: foo
</code></pre>
</li>
</ul>
</li>
<li>
<p><code>tags</code>: Custom tags to apply to the synced files.</p>
<ul>
<li>Type: <code>object</code></li>
<li>Required: <code>false</code></li>
<li>Default: <code>{}</code></li>
<li>Example:<pre class="prettyprint source lang-yaml"><code>tags:
environment: production
<h4>CDK</h4>
<p>Call sync storages action after setting up a CDK App:</p>
<pre class="prettyprint source lang-typescript"><code>import { Stack, App } from '@aws-cdk/core'
import SyncCloudStorage from 'sync-cloud-storage'

const app = new App()
const stack = new Stack(app, 'MyStack')
const syncCloudStorage = new SyncCloudStorage(stack, {
storages: [
{
name: 'my-bucket',
patterns: ['assets/*'],
actions: ['upload', 'delete'],
prefix: 'assets',
acl: 'public-read',
metadata: {
foo: 'bar',
bar: 'foo',
},
},
],
})

// Sync storages
syncCloudStorage.storages()

// Sync tags
syncCloudStorage.tags()

// Sync metadata
syncCloudStorage.metadata()
</code></pre>
</li>
</ul>
</li>
</ul></article>
<h2>Options</h2>
<h3>General</h3>
<table>
<thead>
<tr>
<th>Option</th>
<th>Notes</th>
<th>Type</th>
<th>Required</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td>storages</td>
<td>List of storages, Minimum items: 1</td>
<td><code>array</code> of <a href="#storage"><code>storage</code></a></td>
<td>true</td>
<td>undefined</td>
</tr>
<tr>
<td>region</td>
<td>Cloud (AWS) region</td>
<td><code>string</code></td>
<td>false</td>
<td>undefined or <code>AWS_REGION</code> environment variable</td>
</tr>
<tr>
<td>endpoint</td>
<td>Cloud (AWS) Endpoint URL</td>
<td><code>string</code></td>
<td>false</td>
<td>undefined or <code>AWS_ENDPOINT_URL</code> environment variable</td>
</tr>
<tr>
<td>offline</td>
<td>Offline mode</td>
<td><code>boolean</code></td>
<td>false</td>
<td>false or <code>IS_OFFLINE</code> environment variable</td>
</tr>
<tr>
<td>disabled</td>
<td>Disable sync</td>
<td><code>boolean</code></td>
<td>false</td>
<td>false</td>
</tr>
<tr>
<td>silent</td>
<td>Silent output logs</td>
<td><code>boolean</code></td>
<td>false</td>
<td>false</td>
</tr>
</tbody>
</table>
<h3>Storage</h3>
<table>
<thead>
<tr>
<th>Option</th>
<th>Notes</th>
<th>Type</th>
<th>Required</th>
<th>Default</th>
</tr>
</thead>
<tbody>
<tr>
<td>name</td>
<td>Name of storage (AWS S3 Bucket), Minimum length: 1</td>
<td><code>string</code></td>
<td>true</td>
<td>undefined</td>
</tr>
<tr>
<td>patterns</td>
<td>Patterns of <a href="https://en.wikipedia.org/wiki/Glob_(programming)"><code>glob</code></a> paths to include or exclude on sync action, Minimum items: 1</td>
<td><code>array</code> of <code>string</code></td>
<td>true</td>
<td>undefined</td>
</tr>
<tr>
<td>actions</td>
<td>Sync actions, Valid values: <code>upload</code>, <code>delete</code></td>
<td><code>array</code> of <code>string</code></td>
<td>false</td>
<td><code>upload</code>, <code>delete</code></td>
</tr>
<tr>
<td>prefix</td>
<td>Prefix for the storage files and folders</td>
<td><code>string</code></td>
<td>false</td>
<td><code>''</code></td>
</tr>
<tr>
<td>enabled</td>
<td>Enable or disable the storage on sync action</td>
<td><code>boolean</code></td>
<td>false</td>
<td><code>true</code></td>
</tr>
<tr>
<td>acl</td>
<td><a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html#canned-acl">AWS S3 Canned ACL</a>, Valid values: <code>private</code>, <code>public-read</code>, <code>public-read-write</code>, <code>authenticated-read</code></td>
<td><code>string</code></td>
<td>false</td>
<td>undefined</td>
</tr>
<tr>
<td>metadata</td>
<td>A set of metadata key/value pair to be set or unset on the object</td>
<td><code>object</code></td>
<td>false</td>
<td>undefined</td>
</tr>
<tr>
<td>tags</td>
<td>A set of tag key/value pair to be set or unset on the object</td>
<td><code>object</code></td>
<td>false</td>
<td><code>{}</code></td>
</tr>
<tr>
<td>gitignore</td>
<td>Use .gitignore file to exclude files and directories</td>
<td><code>boolean</code></td>
<td>false</td>
<td>false</td>
</tr>
<tr>
<td>ignoreFiles</td>
<td>Ignore files and directories to exclude from sync action</td>
<td><code>array</code> of <code>string</code></td>
<td>false</td>
<td>undefined</td>
</tr>
</tbody>
</table></article>
</section>


Expand All @@ -211,10 +280,10 @@ <h2><a href="index.html">Home</a></h2>
<br class="clear">

<footer>
Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.2</a> on Sat Mar 02 2024 23:22:23 GMT+0100 (Central European Standard Time)
Documentation generated by <a href="https://github.com/jsdoc/jsdoc">JSDoc 4.0.2</a> on Sun Aug 18 2024 20:02:56 GMT+0200 (Central European Summer Time)
</footer>

<script> prettyPrint(); </script>
<script src="scripts/linenumber.js"> </script>
</body>
</html>
</html>
4 changes: 2 additions & 2 deletions package.json
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "sync-cloud-storage",
"version": "1.0.0-rc.3",
"version": "1.0.0-rc.4",
"license": "MIT",
"type": "module",
"main": "dist/esm/index.js",
Expand Down Expand Up @@ -34,7 +34,7 @@
"lint:fix": "eslint . --fix",
"docs:build": "pnpm run build && rm -rf ./docs && ./node_modules/.bin/jsdoc -c jsdoc.json",
"docs:start": "pnpm run docs:build && ./node_modules/.bin/http-server -c-1 -o docs",
"preversion": "pnpm run lint && pnpm run build && pnpm run format && pnpm run docs:build",
"preversion": "pnpm run lint",
"clean": "rm -rf ./dist ./docs",
"start": "node dist/esm/index.js"
},
Expand Down

0 comments on commit 493764b

Please sign in to comment.