Sunday, June 10, 2007

Dynamic Hell in Python Names

There is a question I often see from Python newbies: How to use the contents of a string as a variable name. In other words, to dynamically create a variable based on some runtime-found name. A lot of these users come from more static backgrounds and have heard the benefits of dynamic languages. I have to wonder if these are signs of their over-eagerness to exploit that dynamic nature.

exec "%s = %d" % (raw_input(), input())
Example of dynamic variable names in action. To a pythoner, obviously not pythonic.

d = {}
d[raw_input()] = int(raw_input())
Less bad. This is why dictionaries exist, so we use them. Also, we remove the potentially dangerous and frivolous use of input() in favor of int(raw_input()).

There are a lot of things that different programming languages are good for and things they are bad for. There are times when their negative points can be exploited, of course. There are far more times when their positive points can be exploited, and turned into disaster. Pointers in C, for example, are useful in the cases where they are used properly and smartly, but terrible when wielded by the wrong hand with evil or ignorant intent. The same can be true for a lot of a dynamic language's features.

People trying to avoid knowing the names and number of variables they have, create classes on the fly with runtime information, and build and evaluate python code in strings are looking at the flexibility of the language and really want to use that for their own good. The problem is that great power requires great responsibility, as we all know. What most of us don't realize, is that great responsibility requires great wisdom, and we can't buy, read, or request on IRC for wisdom. That means the great power that lures these new comers to the language is inevitably what will always keep them from fully experiencing the gifts it bestows. This is not a flaw or a unique attribute of the language, of any language, or of anything at all which benefits mankind and requires knowledge. Just don't exert yourself. Don't do anything for the sake of paradigms or languages or buzzwords. Solutions are neutral, so solve them for solution's sake, not implementation's sake.

[foo() for i in range(10)]
An obvious abuse of a list comprehension as a for loop one-liner. (We know this, because the resulting list is thrown away.)

What I find the most odd is the reactions these individuals give when you tell them what they are doing wrong, and what to do instead. If they want dynamic variable names, we tell them to use a dictionary; they complain about the inefficiency of hash table lookups versus variable names (from a C perspective, often). The simple thing I cannot understand with this is how they expect the dynamic variables work, if not a hash table. If they think they work like C stack locals, yet can be dynamic, why would they think that magical technique would not be applied to dictionaries. In other words, if they recognize they achieve the same end, why do they think they are such wildly different routes?

Update: I added some examples the day after initially posting this.

7 comments:

SoYoYo said...

this entries need examples. I think I agree with you but you need to be more specific.

Generalities won't help newbies understand your point, old-timers understand but do not need them.

As bonus, some specific examples could foster some thinking (we are all part newbies).

Anonymous said...

You know, back when I started into programming I found I wanted (although i wasn't sure why) to be able to be able to create variable names programaticaly. Later I found, that if i kept an array of values and a syncronized array of names i could get close to what i wanted. Then i found associative arrays in php, and haven't looked back.

Basically it all boils down to the need for dicts. A data structure we all feel the need for, instinctively.

But as you said, some people from the static camp can't see why what they want is a dictionary.

Yoni said...

reminds me of what happens when people try to use what they know from Java to write Python, only from a C perspective. The classic blog entry on the topic: python is not java

One of the commenters made me think of this part of it:

"Got a switch statement? The Python translation is a hash table, not a bunch of if-then statments. Got a bunch of if-then's that wouldn't be a switch statement in Java because strings are involved? It's still a hash table. The CPython dictionary implementation uses one of the most highly-tuned hashtable implementations in the known universe. No code that you write yourself is going to work better, unless you're the genetically-enhanced love child of Guido, Tim Peters, and Raymond Hettinger."

Doug Napoleone said...

Whenever one of the researchers asks me about doing this instead of using a dictionary, I show them the globlas() and locals() commands which return the dictionaries for the respective closures. "See it's a dictionary, and this specific dictionary, have fun."

Anonymous said...

Had an interview for a Python job. It wasn't deep Python but the first technical question the interviewer asked was "what's a dictionary?"

I suspect if I hadn't known the answer the interview would have been quite short.

A friend teaches computer science. He is appalled when seniors and even masters students come into his classes with zero understanding of dictionaries, also known as hashes, also known as associative arrays.

If you don't use them, it's a good guarantee your code will be obscure, and very likely at least one power of N too slow.

Albert said...

Too right! Dynamic languages like Python and PHP lead to nothing but trouble.

On a related note, even C# has adapted some aspects of dynamic languages - the "var" keyword. I'm currently in an "argument" with some colleagues about disallowing it in production code, but I'm meeting heavy resistance. Do you have any good arguments I can use?

Calvin Spealman said...

Albert, I am a Python developer and wouldn't change that. I'm only advocating that whatever kind of code you are doing, you do it right.

I write here about programming, how to program better, things I think are neat and are related to programming. I might write other things at my personal website.

I am happily employed by the excellent Caktus Group, located in beautiful and friendly Carrboro, NC, where I work with Python, Django, and Javascript.

Blog Archive