Module Index

Submodules

Module: cogen

This is a library for network oriented, coroutine based programming. The interfaces and events/operations aim to mimic some of the regular thread and socket features.

cogen uses the enhanced generators in python 2.5. These generators are bidirectional: they allow to pass values in and out of the generator. The whole framework is based on this.

The generator yields a Operation instance and will receive the result from that yield when the operation is ready.

Roughly the cogen internals works like this:

+------------------------+
| @coroutine             |
| def foo():             |
|     ...                |             op.process(sched, coro)
|  +->result = yield op--|----------------+------------+
|  |  ...                |                |            |
+--|---------------------+    +---------------+  +---------------------+
   |                          | the operation |  | the operation can't |
  result = op.finalize()      | is ready      |  | complete right now  |
   |                          +------|--------+  +----------|----------+
  scheduler runs foo                 |                      |
   |                                 |                      |
  foo gets in the active             |                      |
  coroutines queue                   |                      |
   |                                 |                      |
   +----------------------<----------+                      |
   |                                                    depening on the op
  op.run()                                               +---------+
   |      socket is ready               add it in        |         |
   +-------------<------------  ......  the reactor   <--+         |
   |                         later                                 |
   +------<-------------------  ......  add it in some other     <-+
    some event decides                  queue for later run
    this op is ready


The scheduler basicaly does 3 things:
 - runs active (coroutine,operations) pairs (calls process on the op)
 - runs the reactor
 - checks for timeouts

The reactor basicaly does 2 things:
 - calls the system to check what descriptors are ready
 - runs the operations that have ready descriptors

The operation does most of the work (via the process, finalize, cleanup,
run methods):
 - adds itself in the reactor (if it's a socket operation)
 - adds itself in some structure to be activated later by some other event
 - adds itself and the coro in the scheduler's active coroutines queue

The coroutine decorator wrappes foo in a Coroutine class that does some
niceties like exception handling, getting the result from finalize() etc.

Modules

common

A module for quick importing the essential core stuff. (coroutine, Scheduler, events, sockets, priority)

core

This module holds the essential stuff.

Programming with this library should be straghtforward. A coroutine is just a generator wrapped in a operation handling class:

@coroutine
def mycoro(bla):
    yield <operation>
    yield <operation>
  • the operation instructs the scheduler what to do with the coroutine: suspend it till someting happens, add another coro in the scheduler, raise a event and so on.
  • the operations are split up in 2 modules: events and sockets
    • the operations from sockets are related to network, like reading and writing, and these are done asynchronously but your code in the coroutine will see them as a regular synchronous or blocking call.
    • the operations from events are related to signals and coroutine/scheduler management.
  • if a operation has a result associated then the yield will return that result (eg. a string or a (connection, address) tuple) otherwise it will return the operation instance.

Typical example:

import sys, os
from cogen.common import *

@coroutine
def server():
    srv = sockets.Socket()
    srv.setblocking(0)
    srv.bind(('localhost',777))
    srv.listen(10)
    while 1:
        print "Listening..."
        conn, addr = yield sockets.Accept(srv)
        print "Connection from %s:%s" % addr
        m.add(handler, conn, addr)

@coroutine
def handler(sock, addr):
    yield sockets.Write(sock, "WELCOME TO ECHO SERVER !\r\n")
    while 1:
        line = yield sockets.ReadLine(sock, 8192)
        if line.strip() == 'exit':
            yield sockets.Write(sock, "GOOD BYE")
            sock.close()
            return

        yield sockets.Write(sock, line)

m = Scheduler()
m.add(server)
m.run()

web

This module holds a wsgi server and future web-related code.