Iterators in python

In this python tutorial, you will master the elegant concept of iterators in python. You will also build your own iterator in python with the help of __iter__() and __next__() methods with super handy examples.

Before going to the topic let's have a quick recap on what is iteration and what are iterables in python which we have already discussed in our tutorial Python Loop. Iteration is the process of repeating a set of codes for multiple instances or until they meet the specific condition (Eg: for loop and while loop). Iterables refer to an object that can be repeated or iterated over(Eg: String, List, Tuples, etc). 

What is an iterator

An iterator in a python programming language is an object which is used to iterate over iterables in an iteration. An iterator stores the state of the iteration and hence it returns one item at a time. In python, Iterators are intrinsically implemented for loops, comprehension, generators, etc.

How to create an iterator

Technically, an iterator object must employ the iterator protocol which contains two functions __iter__() and __next__(). The iter() function converts the iterables into iterators while the next() function calls the next item in the iterable.

Str_list = ['Yellow','Orange','Red']

it_obj = iter(Str_list)

print(it)

print(next(it_obj))
print(next(it_obj))
print(next(it_obj))
print(next(it_obj)) 

Output:

Yellow
Orange
Red
Traceback (most recent call last):
  File "iter_ex.py", line 17, in 
    print(next(it))
StopIteration

In the above code snippet, you can see the creation of an iterator and its implementation.

• Initially, we have defined a list named Str_list, which contains three string elements ‘Yellow’,’ Orange’, and ‘Red’.
• Secondly, we created the iterator, it_obj, with the help of the iter() function which here converts the list to iterators.
• Thirdly we use the function,print(it) assuming the outcome will be the iterator value i.e, the first element(yellow)in the list. Instead, it prints the object of the iterator.
• So to get the iterator value python has got an in-built function called next(). print(next(it_obj)) prints the first element in the list i.e. Yellow.  
• The next print(next(it_obj)) prints the next element in the list i.e. Orange.
• The later print(next(it_obj)) prints the next element in the list i.e. Red.
• The last and final  print(next(it_obj)) ended up in an exception StopIteration as the list has reached its end and has no more value to return.

From this, we understand that through the use of next() function we manually iterate over items in an iterable.

To make it more convenient we have another elegant way of iteration which is nothing other than for loop iteration. The same example can be implemented automatically with the use of for loop as given below.

Str_list = ['Yellow','Orange','Red']

for i in Str_list:
 print(i) 

Output:

Yellow
Orange
Red

How for loop works internally

Before explaining how FOR loop works internally in Python, just look at the below code fragment and analyze it.

Str_list = ['Yellow','Orange','Red']

it_ob = iter(Str_list)

while True:
 try:
     i = next(it_ob)
     print(i)
 except StopIteration:
     break 

Output:

Yellow
Orange
Red

This is how the for loop is internally implemented. The for loop intrinsically creates an iterator object which then iterates the iterables like list using next() function inside a while loop. When the iterable reaches the end and tries to iterate again, the program will halt and raise an exception.

How to create  your own  iterator in python

We have seen the working of iterators on built-in constructs. Now let’s try to create our own iterator i.e. the user-defined version. One thing we need to keep in mind is that while creating an iterator we must implement the iterator protocol. To be specific we must use iter() and next()
So we let’s create an iterator to produce an outcome with the square of even numbers. The output will take the form.

class Pow:
    def __iter__(self):
    self.a = 0
    return self

  def __next__(self):
     x = self.a**2
     self.a =  self.a + 2
     return x
    
even = Pow()
it_ob = iter(even)

print(next(it_ob))
print(next(it_ob))
print(next(it_ob))
print(next(it_ob))
print(next(it_ob)) 

Output:

0
4
16
36
64

What are infinite iterators

When you observe the above code you can notice that no termination steps were included in the code which implies the iterator will iterate infinitely. To understand it clearly let's use ‘for loop’ to iterate our class Pow.

for i in Pow():
 print(i) 

Output:

0
4
16
36
64
100
.
.
.

Those iterators which iterate infinitely are referred to as infinite iterators. While coding this is not a fair practice and hence we must be careful to include the termination statement in the program.

How to stop iteration

So now we are aware that we need to halt the iteration to prevent the never-ending iteration. To accomplish that we use a StopIteration statement inside the next () function as shown in the below code snippet.

class Pow:
  def __init__(self, max =0):
     self.max = max

  def __iter__(self):
    self.a = 0
    return self

  def __next__(self):
    if self.a<=self.max:
     x = self.a**2
     self.a =  self.a + 2
     return x
    else:
     raise StopIteration

even = Pow(5)
it_ob = iter(even)

print(next(it_ob))
print(next(it_ob))
print(next(it_ob))
print(next(it_ob))
print(next(it_ob)) 

Output:

0
4
16
Traceback (most recent call last):
  File "iter_ex.py", line 47, in 
    print(next(it_ob))
  File "iter_ex.py", line 39, in __next__
    raise StopIteration
StopIteration

The same code implemented with for loop is as follows:

for i in Pow(8):
 print(i) 

Output:

0
4
16
36
64

The key benefit of utilizing iterators in a program is that they save resources which signifies no special memory is needed for getting all the elements except a single memory. Theoretically, it says infinite elements can be stored in finite memory.