JavaScript generators part 1: Generator basics

25 Feb 2019 generators

When ECMAScript 6 landed in 2015 one of the more exotic arrivals was something called generators. In this three-part series of articles I'll be looking in detail at what they are, how they work and what applications they have.

What are generators? 🔗

Someone one day asking: Wouldn't it be cool if a function could return a value at a certain pont, then resume from that pointer when we call it again later?

Generators were born as the result!

Generators are just functions, but with two key differences over normal functions:

Generators are capable of pausing execution and returning to that point to continue execution later
The function body is not executed when a generator function is called

Generator functions spit out data in a sequential fashion, by what we call "yielding" data (we'll see much more of the yield concept shortly.)

Genereator syntax 🔗

You create a generator the same way you create any function - except it has a * (asterisk) in its declaration.

function* myGenerator(x, y) { ... }

Actually, you can place the asterisk after function, or before myGenerator, or between the two with space either side. ECMAScript is pretty non-opinionated on the matter, and it's thrown up some interesting discussions on best practice.

You can't declare generator functions as arrow functions:

* () => { ... }; //syntax error!

As I mentioned, generators "yield" out data. They do this via the yield keyword. You can sort of think of yield as being a bit like return - except after a return statement, no further code is executed. As we'll see shortly, that's not the case for yield, and it's awesome.

We yield data in the following way:

function* myGenerator() {
    yield 'foo';
}

So how do we run that, to ultimately get our "foo" value? Let's look at how generators are executed.

Generator execution 🔗

I mentioned a minute ago that, unlike normal functions, when we call a generator function its function body - the statements within it - do not run immediately.

Instead, calling a generator function returns an iterator.

If you're familiar with iterators, you can check out my guide to JavaScript Iterators and Iterables. You may be surprised to know that generators are iterators, and are also iterable. More on that in part 3.

Let's take a look at one in action:

function* myGenerator() {
    yield 'foo';
}
let iterator = myGenerator();

Right now, we still don't have our foo value. What we do have is an iterator representing our generator, via which we can iterate over its "yielded" values to ultimately extract them. We do this by calling the iterator's next() method.

Like all iterators, next() returns an object, defining a value and boolean, done, denoting whether iteration is complete (i.e. all values have been spat out).

With manually-created iterators, these value and done properties are optional, but for built-in iterators like those associated with generators, both value and done are always present on the returned object.

iterator.next(); //{"value": "foo", "done": "true"}

Finally, we got a value. What we've done is basically equivalent to something that'll be much more recognisable:

function myFunc() { return 'foo'; }
myFunc(); //foo - albeit as a string, not part of an object

So far, so... not all that exciting. But I said up top that generators' great strength is their ability to spit out multiple values, in sequence, pausing execution and resuming later. And this is where things really get cool.

Generators keep the score 🔗

You've probably put two and two together right now and guessed that, if a generator can return multiple values from repeated calls to it, it supports multiple yields.

function* myGenerator() {
    yield 1;
    yield 2;
    return 3;
}

How do we get each of these values? By making repeated calls to our iterator's next() method.

iterator.next(); //{"value": 1, "done": false}
iterator.next(); //{"value": 2, "done": false}
iterator.next(); //{"value": 3, "done": true}

There's three important things here. Firstly, each yield statement pauses the function's execution, and it's from this point that execution will resume on our next call to next().

That's the power of generators and yield. We couldn't do that with normal functions. If we had:

function myFunc() {
    return 1;
    return 2;
    //etc
}

Each call to myFunc() would always return 1; the rest of the code will never - could never - execute.

Secondly, generator functions, like normal functions, can accommodate return statements. Indeed it's the return statement that tells the iterator that iteration is complete.

However, a generator's return statement will never take effect where a generator is used as an itereable and fed to automated iteration proceses such as for-of. We'll cover this in part 3.

This feeds in to the other thing to notice - namely, how on the third call to next() the done flag was set to true. It's because we returned the value rather than yielded it.

But generators wouldn't be much fun if we had to manually list all our yield/return values. Fortunately, we can use dynamic constructs like loops:

function* myGenerator() {
    let i = 0;
    while (true) yield i++;
}

Wait... did we just do an infinite loop!? Do we want our PC to explode? It's OK, honestly. Madness though that would be in a normal function, the yield keyword makes all the difference.

Remember, as we saw above, each time the function meets yield it pauses the function at that point, yielding (spitting out) the value immediately after it. So all the loop is infinite in potential, it runs for only as long as we keep calling next(). And so we made ourselves an infinite, sequential number generator!

Note also that, with the above example, the done flag will never be set to true in the returned object, because there's no return statement in our function body.

The only way we'd hit infinite-loop armageddon would be to call next() infinitely, too. Don't, er, do that.

Talking back to generators 🔗

So far we've seen how generators can yield/return data to us. But can we feed data into the generator, when we initiate it (create the iterator) or even later on when stepping through it via next()?

Well, yes we can! There's two things here:

Passing data into our generator function, as initialisation data
Passing data into the iterator (when calling next())

We can pass data into our generator function at the point when we initiate it to create the iterator. The generator function can do whatever it needs to do with this data.

Let's modify the previous example to spit out an infinite sequence of numbers, but starting from a number fed in by the outside world:

function* myGenerator(starAt) {
    let i = startAt;
    while (true) yield i++;
}
let iterator = myGenerator(15);
iterator.next(); //{"value": 15, "done": false}
iterator.next(); //{"value": 16, "done": false}

Great - now things are a little more dynamic.

But more interesting still would be to communicate with the iterator between each step, so it's a genuine two-way conversation. We do this by passing data back in via next().

Don't worry if this next part is a little hard to get your head around - the concept is a little weird, and the inner workings of the magic are hidden from view, but it'll make sense in the end.

function* myGenerator() {
    let result = yield 'song';
    alert('Ah, so you like '+result+'!')
}
let it = myGenerator(),
    faveWhat = it.next().value, //"song"
    fave = prompt(`What's your favourite ${faveWhat}?`, '');
if (fave) it.next(); //alert(...

OK, don't worry, I realise that looks seriously odd, but stick with me. What's happening there is as follows.

First, we do some familiar stuff - we create our generator and call it, returning an iterator that we can call next() on to kick things off.

Our generator runs until it finds the first yield keyword, at which point it spits out the string "song", which we can pick up as the value property in the returned object.

Remember, from that point on our generator is paused; if we ever call next() again, it will resume from the point it left off. In fact, it didn't even get to the end of the first line - that semi-colon after "song" has not yet been seen.

That's a key point; the first line has not completely run yet, so when we revisit our generator with another call to next() it will pick up right from the yield point. BUT, the value we pass in becomes the value of that yield (and the part to the right of the yield will be ignored.)

So yield actually serves two jobs in generators:

Where no value was passed into it via next(), it spits out whatever is to the right of it and execution pauses.
Where a value was passed into it, the yield takes on that value, and execution continues until the next yield is found.

In our case, that means that when we make a second call to next(), passing in the result of the prompt() asking the user's favourite song, their answer is passed into and stored in that first yield (the one where execution was paused). That first line completes, and because the yield holds our value that means it's assigned to result. The second line is then free to run and, since there's no further yields, that's the end of our generator function's activity.

Notice that we don't have to declare the incoming value in the generator function definition. That's because we're technically not passing a value into the generator at all - we're passing it to next(), which then magially binds it to the waiting yield. Generator functions can receive incoming arguments, but as we saw above that's when calling them to return the iterator.

Summary 🔗

Generators are a special type of function that can pause and resume their execution at specific points denoted by the yield keyword. Calling a genereator doesn't, in the manner of normal functions, execute the function body, but rather returns an iterator that we can use to step through the generator via calls to next().

Each time a yield keyword is found, the generator pauses and spits out - "yields" - the value to the right of it, in the form of an object that has, like objects returned by all built-in iterators, a value and a done property.

It's possible to feed something back into the generator on the next call to next() by passing it as the only argument. This is fed to the waiting yield, and can be captured by an assignment operation. Execution then resumes, until the next yield or return or, if none, to the end of the function body.

So that's the basics of generators! I hope you found it informative. In Part 2, we'll see how powerful generators can be when combined with asynchronicity. Join me for that one!

Did I help you? Feel free to be amazing and buy me a coffee on Ko-fi!