XML versus Binary Document Format Debates

Responding to Sean McGrath on his recent post about XML-vs-binary document formats:

Look, its just like I was trying to say in #python@irc.freenode.net the other day: There is nothing wrong with a format being binary. There is no virtue to be found in every byte of a file being interpreted as a textual character (or part of one) that represents your real data. "There ain't such thing as plain text," says Joel Spolsky! There isn't any difference in interpresting the binary as text than just interpretting it directly as your data.

You can easily have XML formats as undocumented and inconsistant between versions as any binary format, but you get the added benefit of extra processing overhead, bloated filesizes, and limitations on structure and performance (try keeping efficient on-disk indexes into an XML file up-to-date).

I do believe text-based formats and XML has its place, but these places are limited. I would have much perfered an opening and standardization of a relational-based format the way Word documents worked internally before Microsoft was bullied into an XML-based format.

I'm wondering if I actually know Sean and didn't realize it, because his points are exactly those I was arguing against in #python the other day, and so I wonder if someone I was arguing was McGrath by a different name. Knowing your strangers is a great gem of the internet-age, isn't it?

DeferArgs 0.4 (Busy day!)

Continuing discussions in #twisted about the benefits and complaints of my deferargs drove me to add more things and release pretty often today. This is version 0.4. New is tests and ability to move the functionality to the callsite, so you can do this:

def printArgs(*args, **kwargs):
    print args
    print kwargs
deferargs(printArgs)(10, defer.succeed(20))

Also, and I don't know how useful this will be, you can not define specially handled argument types, such as lists that might contain deferreds. These are optional, and not enabled by default. Lists are the only special type handled so far. Use it as follows:

@deferargs([list])
def printList(l):
    print l
printList([1,2, defer.succeed(3)])

I might add dictionaries, sets, and tuples to the next release.

Filed in: programming python twisted

DeferArgs 0.3 by Example

DeferArgs 0.3, just a few hours after 0.2, is now released here. The only update is support for attemp/catch-all blocks without any @catch(type) handlers. In response to some people in #twisted@irc.freenode.net, here are examples that show what can be done more clearly.

To write any function that can accept deferred arguments nicely:

@deferargs
def printArgs(*args, **kwargs):
    print args
    print kwargs

To add psuedo-exception handlers and a psuedo-finally block to a @deferargs decorated function:

@deferargs
def printArgs(*args, **kwargs):
    print args
    print kwargs
    print "This one is important! ", kwargs['important']
@catch(KeyError)
def onKeyError(e):
    print "There was no important kwarg!"
@catch()
def onAnyError(e):
    print "Crap!"
@cleanup()
def _(r):
    print "Every thing is done."

There is also a model for a psuedo-try block, which basically decorates just calls the function immediately when you define it and its catch/cleanup handlers:

@attempt
def this():
    assert False
@cleanup
def _(r)
    print "And thats it."

DeferArgs supports asyncronous try/except/finally constructs

A new version of DeferArgs is available at the cheeseshop.

New in 0.2 are the attempt, catch, and cleanup decorators. These mirror the functionality of try, except, and finally. Any deferargs decorated function can be followed immediately by any number of error handlers decorated as @catch(ErrorType) or @catch() to catch all, with the same semantics as except clauses if they were after a try block that surrounded the code in the function. The catches can all be followed with a cleanup decorated function, which will be called when everything else is done, errors or not.

There is also a convinience decorator called attempt, which automatically calls the function once a @catch() decorated handler is defined (signifying that all error handlers for it are prepared). Of course, calling it only really creates the deferred that fires when its arguments are ready, so you dont have to expect it to run for real untl after the attemp/catch/cleanup is completely defined.

As always, comments are welcome!

DeferArgs lets you write syncronous looking functions that really aren't!

So a little side project for my own uses provided a simple decorator that lets me write a function I can pass deferreds and regular arguments to, and have the function return a deferred that fires when all of its deferred arguments are ready and the function has processed them. Some example usage:

@deferargs
def printArgs(*args, **kwargs):
    print "Positional Arguments: ", ", ".join(args)
    print "Keyword Arguments: ", ", ".join("%r=%s"%(k,v) for (k,v) in kwargs.items())

printArgs("foobar", baz=someNetworkRequest())

Really basic, but it can prove useful for a large portion of Twisted code you might write. I'm planning to add some semi-evil way to do something that looks a lot like a try/except/finally block but is actually (obviously) not, and works with any errbacks from deferreds within the try-like block of code. The reaction has been interesting. I've had some people stand up for the idea, which is similar to

defgen

, and others who think it is a bad idiom that is dangerous to encourage.

Using this kind of abstraction over asyncronous code, you do have to be careful to remember what is asyncronous and take consideration that your code won't run until all deferred arguments are ready, even if some parts could be run with only some of the arguments. In those cases, however, you should just break up the function, and I'd like to note that you can make the same mistakes using deferreds and the like directly, so I don't really see it as an issue.

If you want to do any error handling for the moment, you need to attach errbacks to the deferred from the function call. I want to work in my semi-evil error handling soon, because my goal here is to hide the fact that there are deferreds as much as possible, but for now this is just fine, and I've already had use of it myself.

You can get it from the Cheeseshop, so check it out there now and place any comments about it here.

Filed in: twisted python

It's a Boy!

This is a little late, because I've just been so busy, but my first child, my son, Caelan Mathew Spealman, was born on May 13, 2006 at 11:45, just fifteen minutes before Mother's Day. It was the most amazing expirience in my life, to go from a couple to a family as my son was brought into this world, and honestly, this is just so cool! I can't wait to teach him to program. I might be showing how much geek I am by saying this, but my one-and-a-half week old son already has his own computer.

Early Morning

Woke up at 6 AM, made a pot of coffee (with my patent pending technique of adding sugar and spices directly into the grounds before brewing), and caught up on my 600+ unread messages in GMail. Really made me want keyboard shortcuts for labelling, but the only Greasemonkey userscript I found didn't seem to work. It was supposed to let me hit the 'l' key and type in an autocompleted label name, but I did it the new old fashioned way in the end. Anyone know of such a userscript that does, in fact, work?