Skip to content

Commit 7fde18b

Browse files
TillaTheHun0TillaTheHun0
committed
feat(api-reference): add Storage service reference
1 parent fba50bc commit 7fde18b

File tree

3 files changed

+278
-2
lines changed

3 files changed

+278
-2
lines changed

‎src/docs/api-reference/rest/cache.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ curl -X DELETE https://$HOST/cache/$DOMAIN
107107

108108
| Status | Description | Response |
109109
| ------ | :--------------------------------: | ---------------------------: |
110-
| 201 | The `Cache` Service was created | `{ ok: true }` |
110+
| 200 | The `Cache` Service was destroyed | `{ ok: true }` |
111111
| 404 | The `Cache` Service does not exist | `{ ok: false, status: 404 }` |
112112

113113
## Cache a Document

‎src/docs/api-reference/rest/data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ curl -X DELETE https://$HOST/data/$DOMAIN
110110

111111
| Status | Description | Response |
112112
| ------ | :-------------------------------: | ---------------------------: |
113-
| 201 | The `Data` Service was created | `{ ok: true }` |
113+
| 200 | The `Data` Service was destroyed | `{ ok: true }` |
114114
| 404 | The `Data` Service does not exist | `{ ok: false, status: 404 }` |
115115

116116
## Create a Document

‎src/docs/api-reference/rest/storage.md

Lines changed: 276 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,277 @@
11
# hyper `Storage` Service
2+
3+
A hyper `Storage` service is an object storage bucket that can be used to interact with unstructured data like images, videos, and files. Upload, download and remove files.
4+
5+
[[toc]]
6+
7+
## Features
8+
9+
- Upload Images/Videos/Files
10+
- Use resource path to organize objects into sub-folders
11+
- Generate Presigned URLs for uploading or retrieving large objects from the underlying store.
12+
13+
## Why `Storage`?
14+
15+
Most applications will need a way to store assets such as images, videos, or other files. Services like AWS S3, GCP Storage, and MinIO have an absolutely massive API surface, when most folks just need a simple and secure way to upload a couple assets.
16+
17+
A hyper `Storage` Service provides a small and intuitive API for interacting with a Storage bucket.
18+
19+
### Use Cases
20+
21+
#### Profile Images
22+
23+
If your application needs to save images to a User profile, you can use a hyper `Storage` Service to store those images and then retrieve them for display. You can go a step further by placing your `assets` entrypoint behind a CDN, incurring the cost of downloading an asset from your hyper `Storage` Service only when the object is not cached on the CDN.
24+
25+
#### Allow Users to Upload Large Files
26+
27+
If your application requires users to upload large files, for example a large CSV file, you can request a presigned-url from your hyper `Storage` Service. Your user can then upload directly to the underlying store power your hyper `Storage` Service without overloading your application Server or hyper `Server`.
28+
29+
In combination with a hyper `Queue` Service, you can use this pattern to upload large files, then kick off a hyper `Queue` job to fetch that file from hyper `Storage` and process it further ie. ETL.
30+
31+
## Create a `Storage` Service
32+
33+
Create a hyper `Storage` Service in the hyper [`Domain`](/docs/concepts/clean-cloud-architecture#hyper-domain).
34+
35+
::: code-group
36+
37+
```js [node.js]
38+
import { connect } from "hyper-connect";
39+
40+
const { storage } = connect(process.env.HYPER);
41+
42+
await storage.create(); // { ok: true }
43+
```
44+
45+
```sh [curl]
46+
export HOST="hyper.host"
47+
export DOMAIN="foobar"
48+
49+
curl -X PUT https://$HOST/storage/$DOMAIN
50+
```
51+
52+
:::
53+
54+
### Common Responses
55+
56+
| Status | Description | Response |
57+
| ------ | :----------------------------------: | ---------------------------: |
58+
| 201 | The `Storage` Service was created | `{ ok: true }` |
59+
| 409 | The `Storage` Service already exists | `{ ok: false, status: 409 }` |
60+
61+
## Destroy a `Storage` Service
62+
63+
Destroy a hyper `Storage` Service in the hyper [`Domain`](/docs/concepts/clean-cloud-architecture#hyper-domain). This will remove all objects stored in the `Storage` Service.
64+
65+
:::danger
66+
This is a destructive operation that will destroy the `Storage` Service and all objects stored within it. Be really sure you want to do this, before destroying your `Storage` Service.
67+
:::
68+
69+
::: code-group
70+
71+
```js [node.js]
72+
import { connect } from "hyper-connect";
73+
74+
const { storage } = connect(process.env.HYPER);
75+
76+
await storage.destroy(true); // { ok: true }
77+
```
78+
79+
```sh [curl]
80+
export HOST="hyper.host"
81+
export DOMAIN="foobar"
82+
83+
curl -X DELETE https://$HOST/storage/$DOMAIN
84+
```
85+
86+
:::
87+
88+
### Common Responses
89+
90+
| Status | Description | Response |
91+
| ------ | :----------------------------------: | ---------------------------: |
92+
| 200 | The `Storage` Service was destroyed | `{ ok: true }` |
93+
| 404 | The `Storage` Service does not exist | `{ ok: false, status: 404 }` |
94+
95+
## Upload an Object
96+
97+
Upload an object to a hyper `Storage` Service.
98+
99+
:::info
100+
When using `hyper-connect`, `storage.upload` accepts a [**Web** `ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream)
101+
NOT a NodeJS `ReadableStream`.
102+
103+
Starting in `NodeJS` `17`, if you only have a `NodeJS.ReadableStream`, you can use Node's built-in `toWeb`:
104+
105+
```js
106+
import { createReadStream } from "node:fs";
107+
import { Readable } from "node:stream";
108+
109+
// convert to a ReadbleStream from a NodeJS.ReadableStream
110+
await storage.upload("foo.png", Readable.toWeb(createReadStream("foo.png")));
111+
```
112+
113+
:::
114+
115+
::: code-group
116+
117+
```js [node.js]
118+
import { createReadStream } from "node:fs";
119+
import { Readable } from "node:stream";
120+
import { connect } from "hyper-connect";
121+
122+
const { storage } = connect(process.env.HYPER);
123+
124+
const stream = Readable.toWeb(createReadStream("foo.png"));
125+
126+
await storage.upload("/path/in/storage/bucket", stream); // { ok: true }
127+
```
128+
129+
```sh [curl]
130+
export HOST="hyper.host"
131+
export DOMAIN="foobar"
132+
133+
curl -X POST https://$HOST/storage/$DOMAIN/path/in/storage/bucket
134+
--data-binary "@foo.png"
135+
```
136+
137+
:::
138+
139+
### Common Responses
140+
141+
| Status | Description | Response |
142+
| ------ | :------------------------------------: | ---------------------------: |
143+
| 200 | The object was successfully downloaded | `{ ok: true }` |
144+
| 404 | The `Storage` Service does not exist | `{ ok: false, status: 404 }` |
145+
146+
## Download an Object
147+
148+
Download an object from a hyper `Storage` Service.
149+
150+
:::info
151+
When using `hyper-connect`, `storage.download`, returns a [**Web** `ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream)
152+
NOT a NodeJS `ReadableStream`.
153+
154+
Starting in `NodeJS` `17`, if you need a `NodeJS.ReadableStream`, you can use Node's built-in `fromWeb`:
155+
156+
```js
157+
import { createReadStream } from "node:fs";
158+
import { Readable } from "node:stream";
159+
160+
// Convert the ReadableStream to a NodeJS.ReadableStream
161+
await storage.download("foo.png").then((res) => {
162+
if (!res.ok) throw res;
163+
return Readable.fromWeb(res.object);
164+
});
165+
```
166+
167+
:::
168+
169+
::: code-group
170+
171+
```js [node.js]
172+
import { createReadStream } from "node:fs";
173+
import { Readable } from "node:stream";
174+
import { connect } from "hyper-connect";
175+
176+
const { storage } = connect(process.env.HYPER);
177+
178+
await storage.download("/path/in/storage/bucket"); // { ok: true, object: ReadableStream }
179+
```
180+
181+
```sh [curl]
182+
export HOST="hyper.host"
183+
export DOMAIN="foobar"
184+
185+
curl -X GET https://$HOST/storage/$DOMAIN/path/in/storage/bucket
186+
```
187+
188+
:::
189+
190+
### Common Responses
191+
192+
| Status | Description | Response |
193+
| ------ | :------------------------------------: | ---------------------------: |
194+
| 200 | The object was successfully downloaded | `{ ok: true }` |
195+
| 404 | The `Storage` Service does not exist | `{ ok: false, status: 404 }` |
196+
197+
## Retrieve a Pre-signed URL
198+
199+
the hyper Service Framework does not impose a limit to the size of objects uploaded and downloaded from a hyper `Storage` Service. However, many of the platforms used to deploy The hyper Service Framework do. For example, AWS' API Gateway only allows a payload size of 10MB. AWS Lambda only allows 6MB. GCP's Apigee maximum payload size is 10MB.
200+
201+
Because of these imposed limitations, there needs to be a way to sideskirt the hyper `Server` to upload and download large objects. This is where pre-signed urls come in.
202+
203+
:::info
204+
Using a pre-signed url side-skirts your hyper `Server` and exposes the underlying infra powering hyper `Storage`. Be extra careful not to couple your business logic to this layer.
205+
:::
206+
207+
Receive a Pre-signed URLto underlying store powering hyper `Storage`. You can then use the `url` to interact directly with the underlying store, either uploading or downloading objects.
208+
209+
:::warning
210+
Depending on the `Adapter`, and the underlying store it uses to implement the `Storage` Service, it may or may not provide pre-signed URLs. Ensure the `Storage Adapter` used by your hyper `Server` supports pre-signed URLs.
211+
:::
212+
213+
::: code-group
214+
215+
```js [node.js]
216+
import { connect } from "hyper-connect";
217+
218+
const { storage } = connect(process.env.HYPER);
219+
220+
// Retrieve a pre-signed url to perform an upload
221+
await storage.signedUrl("/path/in/storage/bucket", { type: "upload" }); // { ok: true, url: '...' }
222+
223+
// Or retrive a pre-signed url to perform a download
224+
await storage.signedUrl("/path/in/storage/bucket", { type: "download" }); // { ok: true, url: '...' }
225+
```
226+
227+
```sh [curl]
228+
export HOST="hyper.host"
229+
export DOMAIN="foobar"
230+
231+
curl -X POST https://$HOST/storage/$DOMAIN/path/in/storage/bucket?useSignedUrl=true
232+
curl -X GET https://$HOST/storage/$DOMAIN/path/in/storage/bucket?useSignedUrl=true
233+
```
234+
235+
:::
236+
237+
:::info
238+
The `Storage Adapter` implementation will determine the expiration of the URL. Ensure the Adapter's expiration configuration for pre-signed urls fits your use-case
239+
:::
240+
241+
### Common Responses
242+
243+
| Status | Description | Response |
244+
| ------ | :-------------------------------------------: | ---------------------------: |
245+
| 200 | The pre-signed url was successfully generated | `{ ok: true, url }` |
246+
| 404 | The `Storage` Service does not exist | `{ ok: false, status: 404 }` |
247+
248+
## Remove an Object
249+
250+
Remove an Object from a hyper `Storage` Service
251+
252+
::: code-group
253+
254+
```js [node.js]
255+
import { connect } from "hyper-connect";
256+
257+
const { storage } = connect(process.env.HYPER);
258+
259+
// Retrieve a pre-signed url to perform an upload
260+
await storage.remove("/path/in/storage/bucket"); // { ok: true }
261+
```
262+
263+
```sh [curl]
264+
export HOST="hyper.host"
265+
export DOMAIN="foobar"
266+
267+
curl -X DELETE https://$HOST/storage/$DOMAIN/path/in/storage/bucket
268+
```
269+
270+
:::
271+
272+
### Common Responses
273+
274+
| Status | Description | Response |
275+
| ------ | :----------------------------------: | ---------------------------: |
276+
| 200 | The object was successfully removed | `{ ok: true }` |
277+
| 404 | The `Storage` Service does not exist | `{ ok: false, status: 404 }` |

0 commit comments

Comments
 (0)