Skip to content

Commit

Permalink
"advanced" readme section
Browse files Browse the repository at this point in the history
  • Loading branch information
3vorp committed Apr 21, 2024
1 parent 1fd6b92 commit e6872e5
Show file tree
Hide file tree
Showing 4 changed files with 109 additions and 53 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- TypeScript overview to README.md.
- Optional replacement argument for `array-splice` edit fields.
- Optional constructor for the `JSONDatabase` PHP class to reduce repetitive code.
- "Advanced" section to the README for previously undocumented features.

### Changed

Expand Down
147 changes: 101 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ npm install firestorm-db
To install the PHP backend used to actually query data, copy the sample [php](./php/) folder to your hosting platform and edit the configuration files.
More information about configuring Firestorm server-side is given in the [PHP](#php-part) section.

# JavaScript Part
# JavaScript Client

The JavaScript [index.js](./src/index.js) file is simply an [Axios](https://www.npmjs.com/package/axios) wrapper of the PHP API.

Expand All @@ -30,6 +30,7 @@ The JavaScript [index.js](./src/index.js) file is simply an [Axios](https://www.
First, set your API address (and your writing token if needed) using the `address()` and `token()` functions:

```js
// only needed in Node.js, including the script tag in a browser is enough otherwise.
const firestorm = require("firestorm-db");

firestorm.address("http://example.com/path/to/firestorm/root/");
Expand All @@ -39,24 +40,11 @@ firestorm.address("http://example.com/path/to/firestorm/root/");
firestorm.token("my_secret_token_probably_from_an_env_file");
```

Now you can use Firestorm to its full potential:

```js
const firestorm = require("firestorm-db");

// returns a Collection instance
const userCollection = firestorm.collection("users");

// all methods return promises
userCollection
.readRaw()
.then((res) => console.log(res))
.catch((err) => console.error(err));
```
Now you can use Firestorm to its full potential.

## Create your first Collection

A Firestorm collection takes one required argument and one optional argument:
Firestorm is based around the concept of a `Collection`, which is akin to an SQL table or Firestore document. A Firestorm collection takes one required argument and one optional argument:

- The name of the collection as a `string`.
- The method adder, which lets you inject methods to query results. It's implemented similarly to [`Array.prototype.map`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map), taking an outputted element as an argument, modifying the element with methods and data inside a callback, and returning the modified element at the end.
Expand All @@ -65,13 +53,13 @@ A Firestorm collection takes one required argument and one optional argument:
const firestorm = require("firestorm-db");

const userCollection = firestorm.collection("users", (el) => {
// inject a method into the element
// assumes you have a 'users' table with a printable field called 'name'
el.hello = () => `${el.name} says hello!`;
// return the modified element back with the injected method
return el;
});

// assumes you have a 'users' table with a printable field called 'name'
// all methods return promises
const johnDoe = await userCollection.get(123456789);
// gives { name: "John Doe", hello: Function }

Expand All @@ -80,7 +68,7 @@ johnDoe.hello(); // "John Doe says hello!"

Available methods for a collection:

### Read operations
## Read operations

| Name | Parameters | Description |
| ----------------------------- | ------------------------------------------------------------| --------------------------------------------------------------------------------------------------------------------- |
Expand All @@ -97,7 +85,7 @@ The search method can take one or more options to filter entries in a collection

Not all criteria are available depending the field type. There are more options available than the firestore `where` command, allowing you to get better and faster search results.

### All search options available
## All search options available

| Criteria | Types allowed | Description |
| ---------------------- | ----------------------------- | --------------------------------------------------------------------------------- |
Expand All @@ -120,7 +108,7 @@ Not all criteria are available depending the field type. There are more options
| `'array-length-le'` | `number` | Entry field's array size is lower or equal to your value |
| `'array-length-ge'` | `number` | Entry field's array size is greater or equal to your value |

### Write operations
## Write operations

| Name | Parameters | Description |
| ----------------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------- |
Expand All @@ -134,7 +122,7 @@ Not all criteria are available depending the field type. There are more options
| editField(obj) | obj: `EditObject` | Changes one field of a given element in a collection |
| editFieldBulk(objArray) | objArray: `EditObject[]` | Changes one field per element in a collection |

### Edit field operations
## Edit field operations

Edit objects have an `id` of the element, a `field` to edit, an `operation` with what to do to this field, and a possible `value`. Here is a list of operations:

Expand All @@ -152,9 +140,9 @@ Edit objects have an `id` of the element, a `field` to edit, an `operation` with

Various other methods and constants exist in the JavaScript wrapper, which will make more sense once you learn what's actually happening behind the scenes.

# PHP Part
# PHP Backend

The PHP files handle files, read, and writes, through `GET` and `POST` requests sent by the JavaScript wrapper. All JavaScript methods correspond to an equivalent Axios request to the relevant PHP file.
Firestorm's PHP files handle files, read, and writes, through `GET` and `POST` requests sent by the JavaScript wrapper. All JavaScript methods correspond to an equivalent Axios request to the relevant PHP file.

## PHP setup

Expand Down Expand Up @@ -217,7 +205,6 @@ sudo chown -R www-data "/path/to/uploads/"
```

From there, you can use the functions in `firestorm.files` (detailed below) from the JavaScript wrapper.
If your upload folder is accessible from a server URL, you can directly use its address to retrieve the file.

## Upload a file

Expand All @@ -239,6 +226,7 @@ form.append("path", "/quote.txt");
form.append("file", "but your kids are gonna love it.", "quote.txt");
// override is false by default; don't append it if you don't need to
form.append("overwrite", "true");

const uploadPromise = firestorm.files.upload(form);

uploadPromise
Expand All @@ -248,7 +236,7 @@ uploadPromise

## Get a file

`firestorm.files.get` takes a file's direct URL location or its content as its parameter.
`firestorm.files.get` takes a file's direct URL location or its content as its parameter. If your upload folder is accessible from a server URL, you can directly use its address to retrieve the file.

```js
const firestorm = require("firestorm-db");
Expand All @@ -257,13 +245,13 @@ firestorm.address("ADDRESS_VALUE");
const getPromise = firestorm.files.get("/quote.txt");

getPromise
.then((fileContent) => console.log(fileContent)) // 'but your kids are gonna love it.
.then((fileContent) => console.log(fileContent)) // but your kids are gonna love it.
.catch((err) => console.error(err));
```

## Delete a file

`firestorm.files.delete` has the same interface as `firestorm.files.get`, but as the name suggests, it deletes the file rather than gets it.
`firestorm.files.delete` has the same interface as `firestorm.files.get`, but as the name suggests, it deletes the file.

```js
const firestorm = require("firestorm-db");
Expand Down Expand Up @@ -302,7 +290,7 @@ const johnDoe = await userCollection.get(123456789);
// type: { name: string, password: string, pets: string[] }
```

Methods should also be stored in this interface, and will be filtered out from write operations:
Injected methods should also be stored in this interface. They'll get filtered out from write operations to prevent false positives:

```ts
import firestorm from "firestorm-db";
Expand All @@ -314,17 +302,19 @@ interface User {
hello(): string;
}

const johnDoe = await userCollection.get(123456789, (el) => {
const userCollection = firestorm.collection("users", (el) => {
// interface types should agree with injected methods
el.hello = () => `${el.name} says hello!`;
return el;
});

const johnDoe = await userCollection.get(123456789);
const hello = johnDoe.hello(); // type: string

await userCollection.add({
id: "123456788",
name: "Mary Doe",
// error: Object literal may only specify known properties, and 'hello' does not exist in type 'Addable<User>'.
// error: 'hello' does not exist in type 'Addable<User>'.
hello() {
return "Mary Doe says hello!"
}
Expand All @@ -337,38 +327,103 @@ Additional types exist for search criteria options, write method return types, c

```ts
import firestorm from "firestorm-db";
firestorm.address("ADDRESS_VALUE");
const address = firestorm.address("ADDRESS_VALUE");
// type: string

const deleteConfirmation = await firestorm.files.delete("/quote.txt");
// type: firestorm.WriteConfirmation
```

# Memory Warning
# Advanced features

Handling big collections can cause memory allocation issues like:
## `ID_FIELD` and its meaning

```
Fatal error:
Allowed memory size of 134217728 bytes exhausted (tried to allocate 32360168 bytes)
```
There's a constant in Firestorm called `ID_FIELD`, which is a JavaScript-side property added afterwards to each query element.

Its value will always be the key of the element its in, which allows you to use `Object.values` on results without worrying about key names disappearing. Additionally, it can be used in the method adder in the constructor, and is convenient for collections with manual key names.

```js
import firestorm from "firestorm-db";
firestorm.address("ADDRESS_VALUE");

const userCollection = firestorm.collection("users", (el) => {
el.basicInfo = () => `${el.name} (${el[firestorm.ID_FIELD]})`;
return el;
});

const returnedID = await userCollection.add({ name: "Bob", age: 30 });
const returnedUser = await userCollection.get(returnedID);

If you encounter a memory allocation issue, you have to allow more memory through `/etc/php/7.4/apache2/php.ini` with a bigger value:
console.log(returnedID === returnedUser[firestorm.ID_FIELD]); // true

returnedUser.basicInfo(); // Bob (123456789)
```
memory_limit = 256M

As it's entirely a JavaScript construct, `ID_FIELD` values will never be in your collection.

## Combining Collections

Using add methods in the constructor, you can link multiple collections together.

```js
import firestorm from "firestorm-db";
firestorm.address("ADDRESS_VALUE");

const orders = firestorm.collection("orders");

// using the example of a customer having orders
const customers = firestorm.collection("customers", (el) => {
el.getOrders = () => orders.search([
{
field: "customer",
criteria: "==",
// assuming the customers field in the orders collection stores by user ID
value: el[firestorm.ID_FIELD]
}
])
return el;
})

const johnDoe = await customers.get(123456789);

// returns orders where the customer field is John Doe's ID
await johnDoe.getOrders();
```

# API Endpoints
## Manually sending data

Each operation type requests a different file. In the JavaScript wrapper, the corresponding file gets appended onto your base Firestorm address.

If you want to manually send requests without using the JavaScript wrapper, simply make an HTTP request to the relevant PHP file with your content. Read requests are `GET` requests sent to `<your_address_here>/get.php`, write requests are `POST` requests sent to `<your_address_here>/post.php` with provided JSON data, and file requests are requests sent to `<your_address_here>/files.php` with provided form data.
- Read requests are `GET` requests sent to `<your_address_here>/get.php`.
- Write requests are `POST` requests sent to `<your_address_here>/post.php` with JSON data.
- File requests are sent to `<your_address_here>/files.php` with form data.

The first keys in a Firestorm request will always be the same regardless of its type, and further keys will depend on the specific method:

```json
{
"collection": "<collectionName>",
"token": "<writeTokenIfNecessary>",
"command": "<methodName>",
...
"collection": "<collectionName>",
"token": "<writeTokenIfNecessary>",
"command": "<methodName>",
...
}
```

PHP grabs the `JSONDatabase` instance created in `config.php` using the `collection` key in the request as the `$database_list` key name. From there, the `token` is used to validate the request if needed and the `command` is found and executed.

## Memory warning

Handling very large collections can cause memory allocation issues:

```
Fatal error:
Allowed memory size of 134217728 bytes exhausted (tried to allocate 32360168 bytes)
```

If you encounter a memory allocation issue, simply change the memory limit in `/etc/php/7.4/apache2/php.ini` to be bigger:

```
memory_limit = 256M
```

If this doesn't help, considering splitting your collection into smaller collections and linking them together with methods.
12 changes: 6 additions & 6 deletions src/index.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// browser check
try {
if (typeof process === "object") var axios = require("axios").default;
} catch (_error) {}
} catch {}

/**
* @typedef {Object} SearchOption
Expand Down Expand Up @@ -161,8 +162,8 @@ class Collection {
* @private
* @ignore
* @param {string} command - The write command name
* @param {Object} [value] - The value for this command
* @param {boolean} [multiple] - Need to delete multiple
* @param {Object} [value] - The value for the command
* @param {boolean} [multiple] - Used to delete multiple
* @returns {Object} Write data object
*/
__write_data(command, value = undefined, multiple = false) {
Expand Down Expand Up @@ -628,8 +629,7 @@ const firestorm = {
},
};

// browser check
try {
if (typeof process === "object") module.exports = firestorm;
} catch (_error) {
// normal browser
}
} catch {}
2 changes: 1 addition & 1 deletion typings/index.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ export type Settable<T> = Addable<T> & {

/**
* Represents a Firestorm Collection
* @template T Type of collection item
* @template T Type of collection element
*/
declare class Collection<T> {
/** Name of the Firestorm collection */
Expand Down

0 comments on commit e6872e5

Please sign in to comment.