Flow control with Deferreds, by example

by on Jun.26, 2011

[NOTE: If you are reading this post in an RSS reader, the formatting on the code examples may not be optimal; please read this in a web browser with JavaScript enabled for the optimal viewing experience]

I often find myself helping newer Twisted users come to grips with arranging their flow control when they first start writing code that uses Deferreds. While the Deferred Reference does a reasonable job of covering all of the details, it is often difficult to make the intuitive leap from the synchronous patterns one is used to, to their asynchronous equivalents. To that end, I often find that comparing sync and async versions is illustrative; there are some examples of this nature in the Deferred Reference, but some patterns are missing, and I’ve never actually put all of the examples down in one place, so I thought I’d do that in my blog post. Without any further ado, here they are:

EDIT: Added composition example

Call a function, and use the result

# Synchronous version
result = getSomething()
doSomething(result)

# Asynchronous version
d = getSomething()
d.addCallback(doSomething)

Call a function and use the result, catching a particular exception

# Synchronous version
try:
    result = getSomething()
    doSomething(result)
except SomeException as e:
    handleError(e)

# Asynchronous version
def _eb(f):
    f.trap(SomeException)
    handleError(f)

d = getSomething()
d.addCallback(doSomething)
d.addErrback(_eb)

Call a function and use the result, catching any exception

# Synchronous version
try:
    result = getSomething()
    doSomething(result)
except:
    log.err()

# Asynchronous version
d = getSomething()
d.addCallback(doSomething)
d.addErrback(log.err)

Call a function and use the result, catching exceptions raised by that function

# Synchronous version
try:
    result = getSomething()
except:
    log.err()
else:
    doSomething(result)

# Asynchronous version
d = getSomething()
d.addCallbacks(doSomething, log.err)

Call a function and use the result, recovering from a particular exception raised by the function

# Synchronous version
try:
    result = getSomething()
except SomeException:
    result = 42
doSomething(result)

# Asynchronous version
def _eb(f):
    f.trap(SomeException)
    return 42

d = getSomething()
d.addErrback(_eb)
d.addCallback(doSomething)

Call a function and use the result, performing cleanup if an exception occurs

# Synchronous version
try:
    result = getSomething()
    doSomething(result)
finally:
    cleanStuffUp()

# Asynchronous version
d = getSomething()
d.addCallback(doSomething)
d.addBoth(lambda ignored: cleanStuffUp())

Compose several functions

# Synchronous version
result = getSomething()
result2 = doStuff(result)
result3 = doMoreStuff(result2)

# Asynchronous version
d = getSomething()
d.addCallback(doStuff)
d.addCallback(doMoreStuff)

If anyone has any suggestions for other examples I should add to this list, feel free to leave a comment or drop me a note, and I’ll consider updating the post.

:, , , ,

  • http://robertwaters.myopenid.com/ Robert Waters

    This is a great cheat-sheet; you should add some info about inlineCallbacks and then get this into the Twisted docs!

  • Gerrat

    Thanks for the very clear examples.  As for other examples, I (for one) would find it very helpful to see a similar set examples; but interspersed with functions called with the defer.inlineCallbacks decorator.  …so something like a function that yields a deferred…called by another function that yields it, added as a callback in another function…all with proper error trapping/err callbacks. 

Search

Loading