The yield keyword demystified

I had recently a discussion with a younger developper in C# that was asking question about the usage of the yield keyword.
He was saying he never used and though it was useless. He then confessed me it didn’t really understood wath the keyword was exactly about.
I tryed to explain him what it does and this the material I would have used it if I had it at that time.
I will try with this post to explain what “yield” is all about with simple but concrete examples.

First thing first. Where can we use it?

It should be used in a function that returns an instance that implement IEnumerable or and IEnumerable<> interfaces.
The function must return explicitely one onf those interfaces like the two following functions:

public IEnumerable GetIntegers1()
{
    yield return 1;
    yield return 2;
    yield return 3;
}
 
public IEnumerable<int> GetIntegers2()
{
    yield return 1;
    yield return 2;
    yield return 3;
}

By returning the IEnumerable interfaces those functions become iteratable and can now be used directly from the foreach loop like:

foreach (var i in GetIntegers1())
{
    Console.WriteLine(i.ToString());
}
 
foreach (int i in GetIntegers2())
{
    Console.WriteLine(i.ToString());
}

Ok but why using it?

What is the difference between those two functions and this one?

public IEnumerable GetIntegers1()
{
    return new List{1,2,3}
}

It might not be obvious at first sight as the result is identical but the execution flow is different.
Basically if you debug the program execution you will see the following for the returned list

1. Enter the foreach loop
2. Call the GetIntegers ONCE
3. Write the first number
4. Write the second number
5. Write the third line

And you will see the following when using the yield return

1. Enter the foreach loop
2. Call the GetIntegers but leave at the first return
3. Write the first number
4. Call the GetIntegers but start at the second return and leave just after
5. Write the second number
6. Call the GetIntegers but start at the third return and leave just after
7. Write the third line

That is all. It simply changes the execution flow and allow you to handle each element of the list one by one before the next element is called.

Fantastic! but is this magic?

No it is not. You could have achieve the same result by having implemented yourself the iterator pattern using the interface IEnumerable and IEnumerator and building a dedicated class to handle this like the following code (for simplicity I will only implement IEnumerable but IEnumerable<> could have been implemented as well):

public class IterableList : IEnumerable, IEnumerator
{
    public List numbers;
    public int index;
 
    public IterableList()
    {
        numbers = new List();
        int index = 0;
    }
 
    public IterableList(IEnumerable inputlist): this()
    {
        foreach (var i in inputlist)
        numbers.Add(i);
    }
 
    public IEnumerator GetEnumerator()
    {
        return this;
    }
 
    public bool MoveNext()
    {
        index++;
        if (index > numbers.Count)
            return false;
        return true;
    }
 
    public void Reset()
    {
        index = 0;
    }
 
    public object Current
    {
        get
        {
            if (index == 0)
                return 0;
            return numbers[index - 1];
        }
    }
}

And then define a function:

public static IterableList GetIntegers3()
{
    return new IterableList(new List{1,2,3});
}

Both of the code generated by the compiler will look very similar.
This can be confirmed by looking at the IL code generated by both of our implementation.
We can see that when using yield an extra class is generated for us that implements IEnumerable and IEnumerator (and their generic version).

The Iterable class we have written will look mostly the same (But for the generic versions that we have not implemented)

To summarize!

Basically using the yield will allow us to have the control over the way the items in our IEnumerable result items and their processing happens. And no magic behind.
It is simply an helper that will generate the code for you.