|
1 | 1 | # async-iterators |
2 | 2 | `async-iterators` is a library designed to provide iterator functions that run each loop of the iterator asynchronously. This allows you to run CPU intensive loops without blocking the event loop or Promise-based functions in sequenece. |
3 | 3 |
|
| 4 | +The library implements the proposed ECMAScript async iterators functionality by using a custom iterator where `next()` returns a `Promise` of the value. See [Asynchronous Iterators for JavaScript](https://github.com/tc39/proposal-async-iteration). |
| 5 | + |
| 6 | +This library exposes an async iterator function along with a Symbol that can be used to attach the iterator to any object. |
| 7 | + |
| 8 | +In addition to the iterator, there are `map`, `reduce`, and `transform` functions that utilize the async iterator to break up these normally synchronous operations up by using `setImmediate` around each loop iteration. |
| 9 | +To prevent blocking the event loop, each `iterator.next()` function is called within a `setImmediate`. |
| 10 | + |
| 11 | +## Dependencies |
| 12 | +`async-iterators` has no external dependencies. It is written using ES6 syntax compatible with node 4.x and higher. |
| 13 | + |
| 14 | +## asyncIterator |
| 15 | +`async-Iterators` exports `asyncIterator` function and `asyncIteratorSymbol`, which can be used for custom iteration. |
| 16 | + |
| 17 | +### Example |
| 18 | +```javascript |
| 19 | +const asyncIterator = require('async-iterators'); |
| 20 | +asyncIterator.addToObjectPrototype(); // = Object.prototype[asyncIteratorSymbol] = asyncIterator; |
| 21 | + |
| 22 | +const transformObj = { |
| 23 | + a: 1, |
| 24 | + b: 2, |
| 25 | + c: 3 |
| 26 | +}; |
| 27 | + |
| 28 | + |
| 29 | +function transform(obj) { |
| 30 | + const iterator = obj[asyncIterator.asyncIteratorSymbol](); |
| 31 | + |
| 32 | + return new Promise((resolve, reject) => { |
| 33 | + let sum = 0; |
| 34 | + let keys = ''; |
| 35 | + |
| 36 | + function loop() { |
| 37 | + iterator.next().then(result => { |
| 38 | + if (!result.done) { |
| 39 | + keys += result.value[0]; |
| 40 | + sum += result.value[1]; |
| 41 | + loop(); |
| 42 | + } else { |
| 43 | + resolve({ keys, sum }); |
| 44 | + } |
| 45 | + }) |
| 46 | + .catch(reject); |
| 47 | + } |
| 48 | + |
| 49 | + loop(); |
| 50 | + }); |
| 51 | +} |
| 52 | + |
| 53 | +async function doIt() { |
| 54 | + return await transform(transformObj); |
| 55 | +} |
| 56 | + |
| 57 | +doIt().then(result => console.log({ sum, keys })); |
| 58 | +// { keys: 'abc', sum: 6 } |
| 59 | +``` |
| 60 | + |
| 61 | +### Promises |
| 62 | +`asyncIterator.next()` will return the resolved value of Promises that are values in the iterable. Using the same code as above: |
| 63 | + |
| 64 | +```javascript |
| 65 | +const promises = [ |
| 66 | + Promise.resolve(1), |
| 67 | + Promise.resolve(2), |
| 68 | + Promise.resolve(3) |
| 69 | +]; |
| 70 | + |
| 71 | +transform(promises).then(result => console.log({ sum, keys })); |
| 72 | +// { keys: '012', sum: 6 } |
| 73 | +``` |
| 74 | + |
| 75 | +So, in the `loop` function, we can call `result.value[1]` and get the value rather than the Promise, since `next()` doesn't resolve until the value of the iterator resolves. |
| 76 | + |
| 77 | +An example use case may be calling out to multiple APIs to collect data, but then needing to process the responses in a specific order as part of a transform. You can have all the promises fire off asyncronously, but the transform would still loop in order as each promise resolves. |
| 78 | + |
| 79 | +## Avoid blocking the event loop with: forEach, map, reduce, transform |
| 80 | +The most common use cases for iterating are a simple loop (`forEach`), or doing transforms using `map` or `reduce`. This library provides implementations of these common iteration functions using an `asyncIterator`. These functions support objects and arrays. |
| 81 | + |
| 82 | +These functions use `setImmediate` to fire off each cycle of the loop in order to avoid blocking the event loop. This does add overhead, but it's very useful to avoid blocking the event loop with CPU intensive transformations. They also facilitate using these transforms over arrays or objects of Promises. |
| 83 | + |
| 84 | +These functions automatically use `asyncIterator`, so there's no need to call `asyncIterator.addToObjectPrototype` or add the iterator manually. |
| 85 | + |
| 86 | +### forEach |
| 87 | +`forEach` iterates over an array or object but does not return a value. |
| 88 | + |
| 89 | +Signature: `forEach(obj: Iterable, (value, key, obj) => {}): void` |
| 90 | + |
| 91 | +```javascript |
| 92 | +const forEach = require('async-iterators').forEach; |
| 93 | + |
| 94 | +const promises = [ |
| 95 | + Promise.resolve(1), |
| 96 | + Promise.resolve(2), |
| 97 | + Promise.resolve(3) |
| 98 | +]; |
| 99 | + |
| 100 | +let sum = 0; |
| 101 | +forEach(promises, v => sum += v) |
| 102 | + .then(() => console.log(sum)); |
| 103 | +// 6 |
| 104 | +``` |
| 105 | + |
| 106 | +### map |
| 107 | +`map` iterates over an array or object and returns an array with the result of each iteration of the loop. This is the same as `Array.prototype.map` or `lodash.map`. |
| 108 | + |
| 109 | +Signature: `map(obj: Iterable, (value, key, obj) => {}): Array` |
| 110 | + |
| 111 | +```javascript |
| 112 | +const map = require('async-iterators').map; |
| 113 | + |
| 114 | +const obj = { a: 1, b: 2, c: 3}; |
| 115 | + |
| 116 | +const result = map(obj, (v, k) => v + k) |
| 117 | + .then(arr => console.log(arr)); |
| 118 | +// [ '1a', '2b', '3c' ] |
| 119 | +``` |
| 120 | + |
| 121 | +### reduce |
| 122 | +`reduce` iterates over an array. The `iteratee` function accepts an accumulator value as the first argument and must return a value to be used as the accumulator for the next iteration, or at the end, as the result of the loop. |
| 123 | + |
| 124 | +Signature: `reduce(obj: Iterable, (accumulator, value, key, obj) => {}): any` |
| 125 | + |
| 126 | +```javascript |
| 127 | +const reduce = require('async-iterators').reduce; |
| 128 | + |
| 129 | +const obj = { a: 1, b: 2, c: 3 }; |
| 130 | + |
| 131 | +const result = reduce(obj, (accum, v, k) => accum + v + k, ''). |
| 132 | + then(result => console.log(`result: ${result}`)); |
| 133 | +// '1a2b3c' |
| 134 | +``` |
| 135 | + |
| 136 | +### transform |
| 137 | +`transform` is based on `reduce`, except that the initial accumulator defaults to an empty object, and iteratee function doesn't have to return the object in each loop. The iterator is mutated during the loop and then returned. |
| 138 | + |
| 139 | +Signature: `transform(obj: Iterable, (accumulator, value, key, obj) => {}): object` |
| 140 | + |
| 141 | +```javascript |
| 142 | +const transform = require('async-iterators').transform; |
| 143 | + |
| 144 | +const obj = { a: 1, b: 2, c: 3 }; |
| 145 | + |
| 146 | +const result = transform(obj, (accum, v, k) => v > 1 ? accum[k.toUpperCase()] = v * 2 : undefined). |
| 147 | + then(result => console.log('result:', result)); |
| 148 | +// result: { B: 4, C: 6 } |
| 149 | +``` |
0 commit comments