Python Iterators

Python Iterators

An iterator is an object that contains a countable number of values.

In Python, you can find iterators everywhere. They are implemented within for loops, generators, comprehensions etc.

An iterator is an object that can be iterated upon, meaning that you can cross over all the values.

Technically, in Python, an iterator object must implement two special methods, __iter__() and __next__().

Iterator vs Iterable

An object is called iterable if we can generate an iterator from it. They are many iterable objects in python like list, tuple, set, dictionary string.

To return an iterator from iterable objects, we can use the iter() function (which internaly call the __iter__() method).

In the following example, we will return an iterator from a list and print each value using the iter() andnext() method.

my_list = ['kiwi', 'apple', 'blueberry']
my_iter = iter(my_list)

print(next(my_iter))
print(next(my_iter))
print(next(my_iter))

Output

kiwi 
apple 
blueberry

Strings are also iterable objects, containing a sequence of characters. In the following example, we will iterate over a string and print each character.

my_str = 'apple'
my_it = iter(my_str)

print(next(my_it))
print(next(my_it))
print(next(my_it))
print(next(my_it))
print(next(my_it))

Output

a
p
p
l
e

Iterating Through an Iterator

To iterate through all the items of an iterator, we can use the next() function. When the end of an iterable object is reached, and there is no more data to be returned, the StopIteration exception will be raised.

# define a tuple
my_tuple = ('kiwi', 'apple', 'blueberry')

# get an iterator using iter()
my_iter = iter(my_tuple)

# iterate through a tuple using next()

# output: kiwi
print(next(my_iter))

# output: apple
print(next(my_iter))

# next(obj) is same as obj.__next__()

# output: blueberry
print(my_iter.__next__())

# this will raise an error
next(my_iter)

Output

kiwi
apple
blueberry

Traceback (most recent call last)
...
---> 21 next(my_iter)
...
StopIteration: 

The for loops uses an elegant way of automatically iterating. It can iterate over any object that can return an iterator.

my_fruits = ('kiwi', 'apple', 'blueberry')

for fruit in my_fruits:
    print(fruit)

Output

kiwi
apple
blueberry

Working of for loop for Iterators

As discussed above, the for loop can iterate automatically through the tuple.

The for loop can iterate over any iterable. Let us see how the for loop is implemented in Python.

for item in iterable:
    # do something with item

The above for loop is implemented as the following.

# create an iterator from the given iterable 
iter_obj = iter(iterable)

# infinite loop
while True:
    try:
        # get the next item
        item = next(iter_obj)
        # do something with item
    except StopIteration:
        # if StopIteration is raised break from the loop
        break

As we can see above from the implementation of the for loop, it creates an iterator object by calling the iter() method on the iterable.

The for loop is an infinite while loop that keeps calling the next() method to get the next item and executes the body of the for loop with its value. When the iterator arrives at the end of the iterable object, a StopIteration exception is raised, which is internally caught, and the loop ends.

Create an Iterator

Creating an iterator from scratch is easy in Python. We just need to implement the __iter__() and the __next__() methods.

The __iter__() method returns the iterator object itself. It can also be used to perform some initialization.

The __next__() method allows to do some operations and must return the next item in the sequence. When reaching the end of the sequence and after a subsequent call, it must raise the StopIteration exception.

In the following example, we will build an iterator that will give an even number in each iteration. The iterator will start from zero to a user set number.

class Even:
    """Class to implement an iterator to generate even numbers"""

    def __init__(self, max=0):
        self.max = max
        self.result = 0

    def __iter__(self):
        self.result = 0
        self.n = 0
        return self

    def __next__(self):
        if self.n == 0:
            self.n += 1 
            return self.result
        if self.result < self.max:
            self.result += 2
            self.n += 1
            return self.result
        else:
            raise StopIteration

# Create an object
numbers = Even(8)

# Create an iterable from the object
i = iter(numbers)

# using next to get to the next iterator element
print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))

Output

0
2
4
6
8

Traceback (most recent call last)
<ipython-input-17-da0cc2cf8404> in <module>()
..
---> 36 print(next(i))
...
StopIteration: 

We can also use a for loop to iterate over our iterator class.

for n in Even(8):
    print(n)

Output

0
2
4
6
8

Python Infinite Iterators

When creating an iterator object in Python, it is unnecessary for the item in the iterator object to be exhausted. There can be infinite iterators (That never ends). We must take some precautions when handling infinite iterators.

We can build our own infinite iterators. The following iterator will return all the even numbers.

class Even:
    """Class to implement an iterator to generate even numbers"""

    def __iter__(self):
        self.n = 0
        return self

    def __next__(self):
        num = self.n
        self.n += 2 
        return num

# Create an object
numbers = Even()

# Create an iterable from the object
i = iter(numbers)

# using next to get to the next iterator element
print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))
print(next(i))

Output

0
2
4
6
8
10

It is recommended to include a terminating condition when working with infinite iterators.

The significant advantage of using iterators is that they save resources. As we can see above, we could get all the even numbers without storing the entire number system in memory.

There's another way more easy to build iterators in Python. To learn more about it read: Python generators using yield