Coroutine

Intro

Coroutines are computer program components that generalize subroutines for nonpreemptive multitasking, by allowing multiple entry points for suspending and resuming execution at certain locations. Coroutines are well-suited for implementing more familiar program components such as cooperative tasks, exceptions, event loop, iterators, infinite lists and pipes.

Coroutines are a natural way of expressing many algorithms, such as simulations, games, asynchronous I/O, and other forms of event-driven programming or co-operative multitasking. Python’s generator functions are almost coroutines — but not quite — in that they allow pausing execution to produce a value, but do not provide for values or exceptions to be passed in when execution resumes. They also do not allow execution to be paused within the “try” portion of try/finally blocks, and therefore make it difficult for an aborted coroutine to clean up after itself.

Such applications could then write co-routines to do non-blocking socket I/O by yielding control to an I/O scheduler until data has been sent or becomes available.

Because generator-iterators begin execution at the top of the generator’s function body, there is no yield expression to receive a value when the generator has just been created. Therefore, calling send() with a non-None argument is prohibited when the generator iterator has just started, and a TypeError is raised if this occurs. Thus, before you can communicate with a coroutine you must first call next() or send(None) to advance its execution to the first yield expression.

As with the next() method, the send() method returns the next value yielded by the generator-iterator, or raises SopIteration if the generator exits normally, or has already exited. If the generator raises an uncaught exception, it is propagated to send()’s caller.

迭代器（Iterators）

Python supports a concept of iteration over containers. This is implemented using two distinct methods; these are used to allow user-defined classes to support iteration. Sequences, described below in more detail, always support the iteration methods.

Iteration is a process implying iterables (implementing the __iter__() method) and iterators (implementing the __next__() method). Iterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables.

The iterator objects themselves are required to support the following two methods, which together form the iterator protocol:

iterator.__iter__()

Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements. This method corresponds to the tp_iter slot of the type structure for Python objects in the Python/C API.

iterator.next()

Return the next item from the container. If there are no further items, raise the StopIteration exception. This method corresponds to the tp_iternext slot of the type structure for Python objects in the Python/C API.

#!/usr/bin/env python
# -*- coding:utf-8 -*-
class CountDown(object):
    def __init__(self, start):
    	self.count = start
 
    def __iter__(self):
        return self
 
    def next(self):
        if self.count <= 0:
            raise StopIteration
        r = self.count
        self.count -= 1
        return r

下面是搬自stackoverflow的一个cheat sheet。

sequence
  +
  |
  v
   def __getitem__(self, index: int):
  +    ...
  |    raise IndexError
  |
  |
  |              def __iter__(self):
  |             +     ...
  |             |     return <iterator>
  |             |
  |             |
  +--> or <-----+        def __next__(self):
       +        |       +    ...
       |        |       |    raise StopIteration
       v        |       |
    iterable    |       |
           +    |       |
           |    |       v
           |    +----> and +-------> iterator
           |                               ^
           v                               |
   iter(<iterable>) +----------------------+
                                           |
   def generator():                        |
  +    yield 1                             |
  |                 generator_expression +-+
  |                                        |
  +-> generator() +-> generator_iterator +-+

生成器（Generators）

生成器提供了一种出色的方法，使得需要返回一系列元素的函数所需的代码更加简单、高效。基于yield指令，可以暂停一个函数并返回中间结果。该函数将保存执行环境并且可以在必要时恢复。

#!/usr/bin/env python
# -*- coding:utf-8 -*-

def fibonacci():
    a, b = 0, 1
    while True:
        yield b
        a, b = b, a+b

fib = fibonacci()
print [fib.next() for i in range(10)]
print [fib.next() for i in range(10)]
print map(lambda x,f = lambda x,f:f(x-1,f)+f(x-2,f) if x >1 else x:f(x,f),range(10))

In Python, everything described here as a coroutine is still a generator.

Remember, yield both passes a value to whoever called next(), and saves the “state” of the generator function.

Generators vs. Iterators

A generator function is slightly different than an object that supports iteration

A generator is a one-time operation. You can iterate over the generated data once, but if you want to do it again, you have to call the generator function again.

This is different than a list(which you can iterate over as many times as you want)

When you’re using send to “start” a generator, you must send None.

Generators are used to generate a series of values
Yield is like the return of generator functions
The only other thing yield does is save the “state” of a generator function
A generator is just a special type of iterator
Like iterators, we can get the next value from a generator using next() for gets values by calling next() implicitly.

Generator function vs Generator expression

Generator functions

They are coded as normal def but use yield to return results one at a time, suspending and resuming.

Generator expressions

There are similar to the list comprehensions. But the return an object that produces results on demand instead of building a result list.

Because neither of them constructs a result list all at once, they save memory space and allow computation time to be split by implementing the iteration protocol.

Python引入的与生成器相关的最后一个特性是提供了与next方法调用的代码进行交互的功能。yield将变成一个表达式，而一个值可以通过名为send的新方法来传递。

def psychologist():
    print 'Please tell me your problems'
    while True:
        answer = (yield)
        if answer is not None:
            if answer.endswith('?'):
                print ("Don't ask yourself too much questions")
            elif 'good' in answer:
                print "A that's good, go on"
            elif 'bad' in answer:
                print "Don't be so negative"

free = psychologist()
free.next()
free.send('I feel bad')
free.send("?")
free.send("ok then i should find what is good for me")

Output:
Please tell me your problems
Don't be so negative
Don't ask yourself too much questions
A that's good, go on

send的工作机制与next一样，但是yield将变成能够返回传入的值。因而，这个函数可以根据客户端代码来改变其行为。同时，还添加了throw和close两个函数，以完成该行为。它们将向生成器抛出一个错误：

throw 允许客户端代码传入要抛出的任何类型的异常；

close的工作方式是相同的，但是将会抛出一个特定的异常——GeneratorExit，在这种情况下，生成器函数必须再次抛出GeneratorExit或StopIteration异常。

def my_generator():
    try:
        yield 'something'
    except ValueError:
        yield 'dealing with the exception'
    finally:
        print "ok let's clean"

gen = my_generator()
print gen.next()
print gen.throw(ValueError('mean mean mean'))
gen.close()
print gen.next()

Yield

Yield返回一个生成器。简单地讲，yield的作用就是把一个函数变成一个generator。一个带有 yield 的函数就是一个 generator，它和普通函数不同，生成一个 generator 看起来像函数调用，但不会执行任何函数代码，直到对其调用 next()（在 for 循环中会自动调用 next()）才开始执行。虽然执行流程仍按函数的流程执行，但每执行到一个 yield 语句就会中断，并返回一个迭代值，下次执行时从 yield 的下一个语句继续执行。看起来就好像一个函数在正常执行的过程中被 yield 中断了数次，每次中断都会通过 yield 返回当前的迭代值。

为了精通yield，你必须要理解：当你调用这个函数的时候，函数内部的代码并不立马执行。

yield produces a value, but suspends the function. function resumes on next call to next().

Generators as Pipelines

import time
def follow(thefile):
	thefile.seek(0,2) # Go to the end of the file
 	while True:
 		line = thefile.readline()
        if not line:
 			time.sleep(0.1) # Sleep briefly
 			continue
 		yield line

def grep(pattern,lines):
	for line in lines:
	if pattern in line:
		yield line
# Set up a processing pipe : tail -f | grep python
logfile = open("access-log")
loglines = follow(logfile)
pylines = grep("python",loglines)
# Pull results out of the processing pipeline
for line in pylines:
	print line,

def grep(pattern):
	print "Looking for %s" % pattern
	while True:
		line = (yield)
		if pattern in line:
			print line,

Coroutine

If you use yield more generally, you get a coroutine.

These do more than just generate values

Instead, functions can consume values send to it

>>> g = grep("python")
>>> g.next() # Prime it (explained shortly)
Looking for python
>>> g.send("Yeah, but no, but yeah, but no")
>>> g.send("A series of tubes")
>>> g.send("python generators rock!")
python generators rock!
>>>

# coroutine.py
#
# A decorator function that takes care of starting a coroutine
# automatically on call.

def coroutine(func):
    def start(*args,**kwargs):
        cr = func(*args,**kwargs)
        cr.next()
        return cr
    return start

# Example use
if __name__ == '__main__':
    @coroutine
    def grep(pattern):
        print "Looking for %s" % pattern
        while True:
            line = (yield)
            if pattern in line:
                print line,

    g = grep("python")
    # Notice how you don't need a next() call here
    g.send("Yeah, but no, but yeah, but no")
    g.send("A series of tubes")
    g.send("python generators rock!")