Thursday, September 21, 2006

User Expectations for Free Services

"Google, you have no right to track my searches!"

How many people have been saying that, especially since the whole thing with AOL "leaking" those users' search records? "Leak" nothing, they released them carefully as an opportunity for research, and we did nothing but attack them. They could have sold them, names and personal billing information included, without ever telling us a thing. What they did was perfectly within their bounds, just read your agreement with them.

Google seems to almost get more cruft about the whole thing than AOL. Maybe people just can't get surprised when they think they see something a company they never liked doing something they disagree with. People want to love Google, so they will get defensive about them almost to the point of taking it personally. How do you draw a line to the rights of users, when they do not even pay for the services being considered? Can we have any rights without being customers? If I asked you a question, do I have some right to demand that you take that question to the grave?

We are throwing an ever increasing amount of information we consider our own into free services across thousands of miles over unsecured channels, and we expect some kind of privacy and set of rights about how that information is dealt with. We have more than search history to deal with, because we're starting to replace desktop office apps with websites to entrust with our unpublished novels and family finance records. We obviously have a large trust in the services, but why do you expect this? Do we even have a right to expect that trust, or are we just delusional?

I do not care if Google tracks every site I browse to, analyzes it, locates patterns, adjusts my searches accordingly, shows me adds based on my browsing habits, or anything else they might be doing with the information. They are an ad driven company. They make money on advertisement clicks and I won't click on something I don't want to buy, so they have to do everything with getting advertisements in my face that I actually want to click, and that means it must be something I'm actually interested in (why else would I click?), so I'm getting as much out of it as they are. Google makes a quarter, and I finally found that great flatscreen TV deal that saves me a grand. Who is the winner here, and all I had to do was drop my pretension notions and enjoy a free service for its hidden cost: a little anonymous (outside Google) exposure and a few pixels of screen on the result pages. Wow, I am so winning in this relationship.

Amazon has been tracking your use of their site for years and doing amazing things with the way they cater to individual users, and there has been little fuss about it. No one complains, because they are too busy watching the movies Amazon suggested they buy. We have rejected the advertising the drives the economy for decades, but when it starts to actually be things we want to buy, do we even see an advertisement, or just a cool new album by your favorite artist? Advertising is meant to convince you to buy some product, and the best way to convince you is to find you already wanting to buy what they have to sell. You can't convince someone who doesn't want or need a product very well, you can just annoy them. That means getting rid of useless advertising is great for the consumer, and I welcome my personally tailored advertising.

Now, if only I could figure out why these amazing, personally specific advertisements were trying to sell outdoor equipment on my programming blog.

Wednesday, September 20, 2006

Thank God for Standards! Or, Thank Someone Else?


Millions of dollars went into the development of the Digital Video Disc format, its licenses, encryption schemes, patents, bushiness deals to ensure its success, and all the work that went into getting rid of those troublesome plastic cases with yard after yard of magnetic tape. We can all be thankful for that!


We have gone through many forms of video, and we can take a lesson from the history of recorded audio. In 1878, Edison invented the phonograph, which we usually think of today as a cylindrical record. The lateral-disc records were technically invented in 1888, as a direct adaptation from Edison's design, but remained covered by patents and used as novelty talking toy technology for another thirty years, until the patents ran out in 1918, forty years after Edison's original invention. It would be another 56 years before the introduction of the cassette tape in Germany. The trend ended there, and we became frustrated with cassette limitations, seeing the introduction of the Compact Disc (CD) in 1982, just 18 years later. It would be only 15 more
years before the first MP3 player hit the market, although the technology of compressed digital audio had been seeing wide use in other circles for at least half a decade.

We have seen similar transition from reels, to beta, to VHS, to DVD, and now to both HD-DVD and Blu-Ray DVD. When I say to both, I really mean it! Warner Brothers is developing a triple-formatted disc, which has a DVD on one side, HD-DVD on the other, and Blu-Ray on a two-way mirror-like surface over the HD-DVD layer, allowing a single disc to work in traditional DVD players, HD-DVD, and Blu-Ray players.

Seriously, folks, what point is there to all these competing formats, when the end result is to just use all of them? Before you know it, everyone will license this WB patent and all the players we buy will support both HD-DVD and Blu-Ray, and we won't even know which format will be used because our movies and players will be playing split-personality the whole way.

The madness probably won't end until we move away from physical distribution methods, and even then there will be encoding, compression, and encryption wars, but at least our smart players and laptops will be able to just download any drivers they need automatically. Yeah, there still won't be a point to the varying formats, but it will be a little bit easier to ignore that there are competing formats, as we always will inevitably do.

Doesn't that tell anyone something?

I can't believe I can use the term "Traditional DVD", already.

Tuesday, September 19, 2006

Concurrency and Stabilty. More Zen for Python.

I was planning on writing something about the current conversations on the Python-3000 list about concurrency, which just doesn't stop being brought up and never gets resolved to any point that anyone is happy with. In an unrelated action, I went to check on my older blog, which I thought of resurrecting for some non-development articles, and I found a draft from a previous time the topic came up. I read it and was surprised to find that it proposes pretty much exactly what is being put on the table over this past weekend, with reference to walling off multiple interpreters in a single process and controlling messages past between them. The same technique scales for multiple cores, multiple processors, or multiple machines.

I've decided to take the easy way out and just post the original draft with minor editing. I enjoy how spot on I ended up being with what is currently becomming an acceptable solution, it might seem. The original was written nearly a year ago. Does this mean my predictions are worth something? Decide for yourself!

Concurrency is a hot topic on the Python mailing lists lately. There is a strong push to get some kind of native concurrency into Python, as the 3.0 branch is a great opportunity to do things that we can't otherwise do, as they would break old code. If we don't get something in now, particularly, something that can scale to hundreds of thousands of tasks and take advantage of multiple processors, we may not get a chance at what could be the best improvement of the language, until the next major version 4.0.

A large part of the problems stems from how horrible an idea threads really are, as they are typically implemented. Threads, as the basic level, are just multiple pre-emptive tasks running with access to the same memory space. Doing this can be a boost for performance, but is hell to control properly. The threads must be syncronized to access their shared resources without clobbering each other. This can be done, but it is very error prone and very difficult to debug.

The solutions seem to lean toward two ends of the spectrum: cooperative tasks and processes. With cooperative tasks, each concurrent unit runs until it says "OK, I'll let someone else run now", and so there is no explicit syncronization needed, because nothing happens without you knowing it, idealy.

Processes, on the other hand, are basically threads without shared memory. These are the same way the concept of processes are implemted at an OS level. Some, including Guido van Russom, even think that is how we should go: multiple system processes communicating via pipes and sockets.

What I propose is a process implementation with python itself. This would offer a lightweight process execution, where each process would consist of a thread of execution and an "object space". Each thread would only be able to access objects within this space. Communication would occure through channels between processes, which can be used like generators (gen.send(10), for example). I've created a basic implemenation, available on my webserver, here.

With my basic demo, you can create object spaces, each with their own global and local dictionaries. There is no real protection, but it shows how it would work and offers an idea of how it would be used if we had real protected object spaces. You can run a function in a space with the run method, which takes a resonse function first, which is called when the the function returns and will be passed the called functions return value. If a function is already executing, the request is queued until its turn.


I need to look around and find that demo/prototype code. I barely remember writing it, but I remember being pleased with the results. I'll look around and resurrect it. Perhaps it will serve as an interesting proof of concept for a possible solution to some of our concurrency problems. I wish I could put more work into a solution now, but at the moment I have little practicle use for concurrency of this type. Maybe in a while I can find justification to spend time on it.

Monday, September 18, 2006

Responding to Mencia About "Outdated" Technology

I wasn't planning to write this, but a quick segment on Mind of Mencia made me want to say something about his piece on tonight's show. It was a rerun, of course, but I don't get to watch the show regularly, although I am a fan.

Carlos Mencia blasted some technologies he deemed as outdated and berated anyone who still uses them. I normally enjoy anything he does, having been a fan since before his currently popular show. However, I just had to say something how much I disagree with him on all but one of these technologies.

  • Corded Phones
    Yeah, I can't go without a cordless phone in a large-ish two story home, but sometimes you can't beat a five dollar phone that you can't loose between the couch cushions.
  • Nintendo 64
    Retro gaming. I don't have to say more.

  • Fax Machine
    I hate fax machines, but the gas company won't let me e-mail them forms because, like it or not, its still easier than e-mail to get from deadtree to deadtree across long distances in a few minutes.
  • Pager
    I've actually considered getting a pager instead of a cell phone. There are times you must be reachable, but must not be interupted. A device that takes away the desire to answer it, because you can't, can be a limitation from more than technology (its limitations original source) to a limitation for convinience. Technology can be a burden, and sometimes we must dial back to reclaim our lives.
  • Cassette Tape
    The one thing I'll agree on. I probably have a few tapes left laying around, but if you dont replace or migrate the content soon, they'll degrade and be lost anyway, so its time to leave them behind.
Technology moves forward at ever-quickening paces, but if we turn our back on it, we loose more than just brick-sized car phones.

Sunday, September 17, 2006

Developing is the new Programming

I've been saying this increasingly in debates and discussions over in #python, where I frequent. It has to be something considered before, by more intelligable people than myself, but I don't remember hearing such a statement from anyone else.


The focus on the software world has drifted over the years from a focus on programming to a focus on developing. The difference is import and subtle. We can see this in the trend of popular software related books, the evolution of practices and languages, and changing patterns in the industry markets.

Books, Blogs, and Bantering

The landscape has changed on the kind of information pushed across the board to techie types. Where as reading the official specification of the C language was once a good software book, the best of today have no mention or dependance on any particular language. The emphasis is on books on development as can be applied broadly and generally, such as the excellent Prefactoring.

This can also be seen on the blogscene and what was made available online the most vigourously in the past. Resources are less and less often references to the boring syntaxes and APIs of programming languages and their libraries. More and more often, resources found talk about testing practices, organizational details, mindsets, and the best coffee to start your day coding.

Everything 37 Signals has to say is usually worth putting some thought into absorbing, even though they use a language I dislike for various reasons. They are the source of Ruby's recent spike in popularity, yet it is rare to see them mentioning anything about on Signal vs Noise. Instead, they opt for a kind of content that sometimes has nothing to do with development at all, yet can be applied directly to every line of code written in any language.

Is this an artifact of my personal interest and information source drifting, or a wider trend of focus across the board?

Backposted to meet my personal post-a-day deadline (but only by 34 minutes!)

Saturday, September 16, 2006

3 1/2 out of 10 Languages You Should Learn Right Now

Yes, I do believe a good developer has to know more than a handful of languages. Ten sure is an arbitrary number to pick, but whatever!


Read the list, I don't want to recap it.

Of the ten languages listed, I don't find three and a half of them compelling choices.
  • C#
  • Python
  • JavaScript
  • AJAX (this is the half)
Some people might find that an odd list. If you know a little about any of the languages, you might find it even odder. If you know a lot about the languages, you'll have no trouble agreeing, I feel confident.

C# is largely what Java should have been. Python is, of course, a fantastic language for most purposes. JavaScript is essential for web work, and web work is essential for getting the rent paid, these days. AJAX, much the same.

C# and JavaScript get some crap from the OSS communities. People look down their noses at C#, having such a distaste for all things wearing the brand of Gates. JavaScript can be an ugly, incompatable-with-anything mess, but it can be elegant and powerful, as well. Both languages can play very nicely with Python, via good web services and toolkits.

Python gets some cruft from the communities who normally work with C# and JavaScript. C# developers are often not open source fans, and they don't know what these weird hippies are talking about, because Visual Basic is just fine. JavaScript users often work exclusively in the web industry, where languages like PHP reign supreme. Python and PHP do not clash, so the JavaScript world hasn't had a great past with respect to Python. A lot of this is changing with the work being done at Mozilla getting Python support into the platform, adopting Python features for JavaScript 2.0, and PyPy being able to compile a python interpreter to JavaScript (crazy!).

And, quickly, why the other 6 1/2 languages just don't make the cut, or under what circumstances you might justify learning PHP, for example.
  • PHP
    So many of us hate it, but we work with it day in and day out. Sometimes you hate what you do, but you the bills it pays. Do many people enjoy PHP? No, but many of those people get paid for it.
  • Perl
    Very little perl these days is used off the web, and if you don't have the freedom to use something fun like Python, then PHP is even a more fitting choice than Perl.
  • Ruby and Ruby on Rails
  • Ruby on Rails isn't really a language, unless you buy into the idea that Ruby allows for use as a DSL engine. Ruby was around for a long time with no fan real fanbase, until Rails stole the scene and we are largely just waiting for the tidal wave to subside. Ruby isn't a terrible language, but I find it odd and there is little weight in a language that is effective carried by a single product.
  • Java
    The promise this language once showed has waned just in time to see Sun almost get the picture before its flame is extinguished. The niche Java fit is filled nicely with languages like Python and Ruby.
    There is a long held stubborn view in the Java world that all things must be Java. Look at their toolkits written entirely in Java, and even prefering low-level work done in Java, rather than wrap existing libraries. Much of this is in the name of portability, but in the end it just means that I have to wait longer for Eclipse to load than it took my machine to boot up in the first place.
  • VB.Net
    I have not used the .Net versions of Visual Basic, but its ancestor is not a fun thing to live with. Even given its improvements (so I hear), it looses a lot of its weight anymore.
    These statements should be taken with salt, because I haven't worked with VB.Net, but here they are anyway. Being a .Net language, it shouldn't matter what language you use, so long as it utilizes the CLR. That means you could even use C# or Python to write the code for Excel and Access, which was one day the only reason anyone had to touch Visual Basic. So with the advent of IronPython, and the myriad of languages that can take VB's place in any given project, where is the need for this language anymore?
As Johnny Storm says, "Flame on!"

PS - That was a punny reference to flamewars. ... get it? ahem. I'm leaving.

The Internet Flashed Me

IGN has a case of split personality. On the one hand, I don't have a new enough version of Flash installed for Firefox on Linux to play their videos.
On the other hand, if I want to watch some advertising, they wouldn't dare to stop me from doing what I wanna do!
Why do so many sites think I don't have a new enough version? Is newer Flash better? No, but the company pays for new versions of Flash Studio anyway, and it exports by default to a newer version, so why bother to change it? Thats like a couple mouse clicks or something! Jeez.

Friday, September 15, 2006

But, didn't PHP break the Web in the first place?

Rasmus Lerdorf is an opinionated man.

However, so am I, so I have some things to say about these particular opinions. Lerdorf is claiming the web is broken. I do not disagree. Lerdorf is claiming PHP is the cure, and I couldn't disagree more if he had written that statement on a shard of tin and jammed it in my stomach.

That is quite a strong disagreement.

I mean, didn't PHP break the web in the first place?

Right off the bat, I should note that I do believe PHP can be used well. Any language (almost) can be used properly enough that it can be a decent environment to use, so long as you follow strict rules. PHP is a great example of a language that promotes ignoring any rules. Following a good set of policies, one can develop well structured and elegant applications with PHP, but the fact of the matter is that the language does very little to promote anything in the way of good use of itself.

PHP might not promote bad coding, but it simply does so little in the way of promoting good code that it might as well provide facilities for plaintext passwords in querystrings built in at the global level. There are too many aspiring developers finding their way to PHP, being drawn by the crowd, rather than the quality of the language. There is a critical mass of bad information about all the wrong ways we can do things in PHP, and none of them tell you its the wrong way. Are they evil? No, they just don't know any better either.

PHP is not evil. PHP broke the web with nothing but good intentions. PHP still broke the web, and only with a massively backwards-compatibility breaking (and that means no options to enable it back!) revisions to the langauge would anything be remedied. Even in that case, either everyone will migrate to other languages or the language would fork, because the only way to fix PHP is to become like Perl 6 and not exist at all.

I am beginning to sound like a language biggot.

Why Johnny Can Code

This is in response to the article Why Johnny Can't Code , over at Salon Tech.

I have heard several people already agreeing with the arguments against this article, so I know I am not alone. Although I completely agree that it is a very good thing for kids to have a quick and easy way to program on their computers, should they have the curiosity, I do not believe the author made very compelling points on the use of BASIC, or anything resembling it. Particularly, I think the author has far too high a reverence for BASIC and fails completely to see the damage the language can do to an aspiring developer, which I won't go into in this article. Conversely he seems to find languages that could fill the gap he says, namely Python, as somehow wrong for a job, which is entirely an incorrect idea. This is pointing from multiple vantages as being written by an unenlightened developer.

Yes, a fundamental understanding of how software works can be a good starting point for a life of technology as a hobby, career, or even just what burns the weather forecast into your toast. Love it or use it, technology with a background is more meaningful. In the past, this was intrinsically bound with BASIC, but that doesn't and should not hold true today. We have moved on out of more than disdain for our old friend and enemy. We have learned better.

Very quickly does the author of the original article mention and pass over Python and Perl as alternatives that don't mean some non-mentioned requirements for the job that BASIC filled in the past. Mentioning of putting his son to C++ after finishing with BASIC make me think that he is not a fan of this class of languages. There is some missing opportunity here, obviously. If he had brought his son to Python, rather than buying an old Commodore 64 out of frustration for finding a good BASIC interpreter for a modern OS, his son could have moved directly to more complex programs in a "basic" language, instead of learning to write the same "basic" programs in a complex language. Python is a fantastic educational language, but shines equally well when scaled to professional usage, which eliminates the bridges needed to cross from one language to the next through the levels of one's programming education.

Some shred of truth found, however, is the problems with bringing a language to the forefront of education in the was BASIC once was. There are too many languages that would argue they are superior. I will not deny that I think none of them old a flame to Python, especially in this context. However, I know I will be debated on this point. The fact remains that BASIC was a bad language, which served a purpose well, but has seen its day and is obsolete for a good reason. We have better ways of doing things, and that doesn't have to mean more complex languages, simply better languages.

David Brin , thanks for wrapping up Foundation , but look into the gaps in your logic about educational programming please, for your son's sake.

Friday, September 08, 2006

IronPython and a New Era

As many people have written about, IronPython 1.0 was recently released. There has been a lot said about it, and most interesting has been the response from many groups with little or no connection to the Python community. The fact that the language runs on .Net is more important than most Pythoners realize yet, and there is a predicted large number of future users of Python coming in from this new area of exploration. Many developers using C# or Visual Basic will be glad to move to a fun language like Python for at least part of their projects, and the integration with the CLR makes it flawless to mix with the rest of the components. It all reminds me of a line from Victor Vernado, the black albino comedian: its all the fun of dating a black man, without the disappointing look from your father. IronPython opens the door for a lot people who were unable to move into the community, due to personal or corporate barriers to leave the safety of the Microsoft Umbrella. IronPython is a bridge between two very strong and thriving camps of software development. It will be interesting to see who and what crosses this bridge and what new connections between these two arenas will rise.

I'm personally interested in IronPython for tapping some new ground in the commercial realm, but I won't speak of it much yet. I am also keen on seeing how well IronPython will work with and be succesful in areas like game development with the new XNA technologies, Mono on non-Windows machines, PyPy related compiling functionality, and Twisted on .Net platforms.

Its surely one hell of a Brave New World.

Tuesday, September 05, 2006

Give it a REST

I'm wrapping up a REST layer to the service backend I've been developing for my still-unnamed-employer (find out when we launch, real soon now!). I had never developed a service under the "REST" acronym before, so my boss gave me a crash course, I read some things, I thought I got it. REST, a buzzword in its own right, is like stapling smoke to water when you try to define it. That isn't because its vague, its because most of the people who talk about it don't know what they're talking about.

Maybe I'm one of them and I shouldn't be posting this.

REST is Not:
  • HTTP
  • XML
  • RPC
  • A protocol, format, or even much of a specification
Rest is:
  • An idea(l)
  • Agnostic on just about every specification associated with it
Requests on a REST Service are:
  • Atomic. No request relies on any other made before or after it.
  • Self Authenticating. Every request must include any credentials. See point 1.
  • Self Describing. This is most commonly XML, and sometimes people think it must be, but it can be anything. We use JSON.
Some of the most interesting things I've learned working with a REST service are the things that do not fit the model well. No model fits every need perfectly, and REST doesn't escape that fact, I'm afraid.

In particular, you are not always transfering a state. There is a distinct difference between state transfer and a request to perform some operation upon a state. Unfortunately, any ways around some of the problems posed are directly rejected by the rules of REST.

For example, say you want to provide as a service a simple counter. You expose PUT on /counter/foobar to register a new counter, and then GET on /counter/foobar will provide the current level of the counter. Following the rules of REST, how do you provide an interface to safely increment such a counter? We can not perform a GET and a PUT, because it violates that each request be self contained, and it will break when any other client of the service is incrementing at the same time. We need a single operation to alter the state, without performing a state transfer.

The best thing you can do is use POST on a resource, and transfer a request to increment. It seems to violate the tenents of REST that the resource you POST will not actually reside at some permenant location, as they are throw-away requests. You either have to live with a not-exactly-REST interface (but, isn't that it works the important thing?) or actually keep requests around for some time. Maybe put them at some location, where they can be checked for review of their status.

I don't know if this is helpful to anyone else writing REST services, but the information around isn't always accurate, so why should I worry if I am?
I write here about programming, how to program better, things I think are neat and are related to programming. I might write other things at my personal website.

I am happily employed by the excellent Caktus Group, located in beautiful and friendly Carrboro, NC, where I work with Python, Django, and Javascript.

Blog Archive