Developing Upwards

I couldn't think of a better title that fit my "How To ..." pattern. The point is, I wanted to make a response to the 37signals post that I found a little harsh. Sure, if you were able to build your company up without investors, that's a great thing! It doesn't make it a terrible thing to get a boost in the early stages or give you a license to insult people trying to pay the bills and put children through college. Making great products is something a lot of us aspire to. Frankly, that simply isn't all of us and there really are good developers out there who are still only in it for the money. I don't know that is the case with Mint.com, but neither does anyone over at 37signals. Belittling them for taking a quick-cash option assumes a lot that may just be completely wrong about the intentions. Now with a chunk of change, maybe the founders are planning to jump ship in a couple years and self-fund their real dreams. On the matter of start-up investment it

I want to say a few things for my own benefit. Maybe that's the only thing I do here. As always, I hope something I have is useful to someone else. In this case, if you're in any position to deal with a big surge on a small site, you might get something useful, or at least enjoy, what I have to from my experience getting a bump from some guy named Mike Arrington with a little blog called TechCrunch. This is about reaction and what was the right and wrong way to react to the impact of a weeks traffic in a couple hours. Should natural means have brought our typical traffic to these levels (time will bring this) the means to handle it on a day to day basis would have been put in place. The sudden increase began to timeout our FastCGI processes and this was alerted to me quickly. I confirmed this and my first response was to initiate a restart cycle, restarting each process in turn, which did nothing to help. I brought up a new instance on EC2 and prepared to roll it out as a new p

When I complained about the problem, I promptly outlined some ideas about solving it, vaguely. Now, I want to narrow that outline into systems I actually use. I do most of my work with Django, some hobby time is spent with App Engine and Twisted, and I enjoy Amazon Web Services, so I'm thinking from these perspectives when I approach this. Parts one and two were broad, but some of this might only apply to fewer of you. Either ignore those or adapt to whatever you use. Django's cache layer sucks. Simply stated and simply true. Any time I decide I can cache something, I should ask myself if I could have built it before I even had the request in the first place. Doing that with the template caches simply isn't possible. It should be possible and it should be the first path you take, instead of forcing us to go out of our way to do the better thing. Anything I might want to cache, I also might want to be sure I'm not doing in more place than once, and forcing them inline i

Distributed version control is a good thing. I've started wondering, abstractly, removing the middle word of that phrase. In other words, how are we being affected by "distributed control" and how will the landscape of free software politics change as it becomes more predominate and we all become more comfortable with it? Even centralized version control began the distribution of control. At least, it made it easier for more than one person to control the changes of a codebase. In old days of e-mailing patches around, it was pretty much a requirement that a single person be responsible for the merging of patches into any single codebase (or any section of that codebase). Source control allowed multiple developers to commit changes and began to put less burden and less power in any one person's hands. Anything that makes the submission of new code easier is going to thin that power even more. When anyone can come along and submit changes to change functionality or ad

I did my best to outline the problem in Part 1. Now I have to stand up and propose some kind of solution. Otherwise, I'm just complaining and contributing nothing of real value. Our frameworks make certain things easier. They don't provide tools to help us with other things. For some other set of activities, they may actually prohibit. The problem here is a combination. Django makes it easy to query your database and wrap functionality up into re-usable template tags. While I'm thankful for that, I am also realizing that ease of one thing can prohibit another. When one path is made easier it creates the perception of greater difficulty in other paths. I think this is why, when our web frameworks give us all these tools to response to a web request, we completely lack in everything we could do aside from that request. How can we make it easier to work outside the web request? We need some idea of what working outside the web request means. We also need to define these in ter

Something is bothering me this past week. I've been taking some stabs at reducing the maximum render time of a site, when the caches are all empty. I cache certain components and queries, but when the caches are primed the render time is under 500ms and I think that's pretty good. That worst case senario, however, is just not acceptable. Worse than a couple seconds. That isn't time that should be taken. I dug in and found a really bad pattern. It isn't hard to make a page faster, but the default is to be as slow as possible. We have to understand this pattern. I am looking at this in relation to Django, but I have a feeling there are similar patterns other places. The common tagging application is a good example. It makes it really easy to tag objects, count them, query by them, and build those clever little clouds. You're given lots of new wrappers for all the common tag-related queries you'd need to do. This may be a source of the problem. We've gotten

As the web becomes more "2.0" we're collapsing collections of pages into fewer and even single, more dynamic pages. Instead of ten pages of event listings, we might have page that loads further items dynamically as you scroll. The state that was once static to a page is now loose and can alter in the lifetime of a page, which grows longer every day. Parameters of the page state have always sat in either the path, in URLs like http://myblog.us/post/how-are-you or the querystring in cases like http://www.coolstore.com/view.html?product=4356 . Neither approach works when those parameters are changing for the life of the page, and where a single URL needs to be able to represent these multiple parameter values at any time. In most uses so far, the bullet is simply bitten. The user can browse to your site and click around, and if they bookmark or send the link to a friend, they'll always come back to the front page, because the state of the page is no longer held in that

I had some down time today to relax, and in true obsessive fashion I spent it coding for the hell of it. I got something in my head and whipped up a demo of the idea. Do you ever need to let someone select a range of things? Maybe they need to pick which and how many items to show in a search result or which letters of names they want to see from an address book? I wanted to allow selection of both "what" and "how much" in one click. click for demo The range being selected from can be anything: numbers, letters, weeks of the year, etc. Users can click among that like a list of a page numbers, like they would expect. I think this would work well in situations where you don't need the entry to be exact, although it can be used for precise entries. Multiple quick selections would also be easy here, maybe quickly changing the range you're viewing in an analytics app. I'd also like to look at adding a "zoom" feature, so that one selection fills the