JavaScript generators part 3: Yet more generators

31 Mar 2019 generators

Welcome to the third and final part of my three-part guide to JavaScript generators. If you're new to generators and haven't yet read the part 1 and part 2, I'd strongly recommend reading those first so the concepts discussed in this article make more sense. I'll wait for you here :)

Generators vs. async/await 🔗

In part 2 we looked in-depth at how generators can hide away asynchronous operations behind their synchronous look.

But ECMAScript 2017 introduced an even easier, more concise way construct to achieve synchronous-looking asynchronicity, however, in the form of the async/await combo.

Armed with this combo we can achieve all the async benefits that generators bring, but without involving iterators. Here's a simple example:

(async () => {
    let rand = await new Promise(res => {
        let num = Math.floor(Math.random() * 10);
        setTimeout(() => res(num), 500);
    });
    alert(rand);
})();

This isn't a tutorial on async/await, so I won't go into too much detail to here. But the key parts are these. async must be used when declaring any function that plans to use await. It ensures that the return always returns a promise - any normal return value it spits out will be implicitly wrapped in a promise, with the value as its resolved value.

async function foo() {
    return 'bar';
}
let val = foo();
console.log(val.constructor.name); //Promise
val.then(a => console.log(a)); //"a"

But it's await that's the interesting part, which is very much like yield. It denotes a pause point in the execution of the code. It differs from yield, however, in two important ways.

Firstly, await is not a two-way conversation and does not yield out anything initially like yield does, because nothing is listening for it in the way we can listen for iterator.next().value from a generator.

Secondly, await expects a promise to the right of it. If it gets a non-promise, it'll convert it to one, and resolve it immediately with the given value.

But await and yield are alike in that, when execution resumes, the resolved (await) or passed-in (generator) value is used in the assignment operation. Thus we can achieve the generator equivalent of the above example with the following. It's easy to see just how much more concise async/await is.

function* myGenerator() {
    let rand = yield setTimeout(() => {
        let num = Math.floor(Math.random() * 10);
        it.next(num);
    }, 500);
    alert(rand);
}
let it = myGenerator();
it.next();

Deepak Gupta has written a more in-depth article on generators vs. async/await if you want to dive in deeper on this issue.

But stop press: It's actually possible to use both generators and async/await, with async generators! We'll take a look at that shortly, but first we need to see how generators can be used as iterables.

Generators as iterators/iterables 🔗

In part 1 we learned that generators, when called, do not execute their function body straight away but rather return an iterator. We've seen evidence of that throughout by calling the next() method to get the next value that the generator can yield.

You can check out my guide to JavaScript iterators and iterables if you're unfamiliar with them.

But it's worth spending a little time just hammering this point home - generators are (or at least return) iterators. This solves a question we so far haven't answered: how do know when to stop asking for yield values?

Because generators return an iterator, this means they conform to JavaScript's so-called iterable protocol. This describes an object structure that should be used to iterate over data, and a mechanism to both retrieve values and know when it's finished.

Generators are an example of built-in iterators, so we don't have to construct this object ourselves - it's created for us. It's wrapped around our yielded values. Suppose we yield out the value "foo". What we get from calling iterator.next() is actually:

{value: "foo", done: true/false}

And it's that done property that's key to knowing when to stop. If there are more yield values after the current yield, it'll be false. If there aren't, it'll be true. We can then look out for this value and decide whether to keep calling next() or quit.

But there's more! Not only do generators return iterators, they are themselves iterable. This means we can use an iteration process such as for-of or spread sytnax to yield out all their values in one fell swoop.

function* myGenerator() {
    yield 1;
    yield 2;
    yield 3;
}
console.log(...myGenerator()); //1 2 3
for (let num of myGenerator()) console.log(num); //1 2 3

Notice we're feeding the generator straight to the iteration process - we don't have to call it to return and iterator as we have been until now, and manually calling next(). That's hidden away as an implementation detail inside spread syntax, which effectively gathers up ("spreads") the yield values into an array.

There's one really important thing to bear in mind when feeding generators to automated iteration processes such as for-of and spread syntax, however: any return statement will be ignored.

function* myGenerator() {
    yield 1;
    yield 2;
    return 3; //note return
}
console.log(...myGenerator()); //1 2 - no 3!

It's not ultimately clear why this is. I'm not sure why the spec designers decided that generators shouldn't spit out the return value when fed to automated iteration processes. Perhaps it's because, when used in this way, we don't have to manually control and track when the generator is considered finished, via a done property like we did before, and so only the yield values count.

This pattern of feeding generators to for-of starts to become real useful where we start to use async generators. Let's meet them now!

Async generators 🔗

Not to be confused with generators that power asynchronous operations, which was the main theme of part 2 of this article series.

Async generators are just like regular generators, but with the crucial difference that the iterator.next() method returns a promise, not a regular object with value and done properties like we've seen and worked with up until now.

To create an async generator we first need to prefix our function definition with async:

async function* myGenerator() {
    let animals = ['dog', 'walrus'];
    for (let i=0; i < animals.length; i++) yield new Promise(resolve =>
            setTimeout(() => resolve(animals[i]), 500)
        );
}

We'll also need to wrap our usage of the generator in an async function (recall that to use await, we must be in a wrapper function prefixed with async):

(async () => {

})();

Now for the weird bit. Remember above I said to use async generators we first needed to look at how generators could be used as iterables? Well, here's why:

(async () => {
    for await (animal of myGenerator()) console.log(animal);
})();

Async generators must be used with the for-of construct. And for-of expects to be served an iterable. Generators are iterables, as we learned above, so this all works fine! Our code generates:

"dog" //after half a second
"walrus" //after a second

Delegated generators 🔗

Another cool feature of generators is you can sort of nest them, delegating specific work out to sub-generators which are then called by a higher-level generator. We do this via the asterisk, just like we use when declaring a generaetor, but this time attached to our yield keyword, followed by the name of a sub-generator.


function* animals() {
    yield 'Here come the animals...';
    yield* canines(); //delegated
    yield* cats(); //and again
    yield 'That\'s it!';
}
function* canines() {
    yield 'dog';
    yield 'dingo';
}
function* cats() {
    yield 'tiger';
}
let it = animals(), next;
while (!(next = it.next()).done)
    alert(next.value);

Notice how, like we discussed above, we check for the done property as a way to know when to stop asking for values? That gives us:

"Here come the animals..."
"dog"
"dingo"
"tiger"
"That's it!"

canines() is our main generator, and starts the output rolling. But the actual animals, our dog and cat species, are output by dedicated sub-generators for each species - cats() and yield* canines().

The code proceeds exactly as it would have had we put all of the yields in one generator, not three. So this is purely a means of managing your code better. Generators have to do their stuff inside special generator functions, but it doesn't have to be in just one function.

So when animals() comes across canines() it understands that control should be given over to the sub-generator animals(), which will yield all its values, and then when finished, return control to the waiting animals(), which then continues. In this way we're delaying with stop-start execution in not just one function but two.

It doesn't have to stop there. You can keep delegating to deeper and deeper levels - the only limit is your code patterns and a desire to keep things manageable.

Summary 🔗

Generators are more verbose than the elegant and more concise async/await that arrived in JavaScript after generators. But they can be used for the same applications.

Where generators retain an advantage is their use as iterators and indeed iterables. Indeed, async/await and generators can work in conjunction with one another in the form of async generators. Instead of an object, the next() method of an async generator's iterator returns a promise which can be used in conjunction with for-of/await.

We're not stuck with one genrator; generator can delegate work to sub-generators via the yield* statement. This means we can have a main generator and sub-generators for smaller task. There is only one execution flow, however - multiple generators do not run in parallel.

And so that's it! I hope you've enjoyed this three-part series informative and helpful.

Did I help you? Feel free to be amazing and buy me a coffee on Ko-fi!