Friday, November 30, 2007

How To Predict The Solid Web

Developers from both Opera and Mozilla have recently blogged about 3D rendering contexts for the canvas element, confirming my year-old predictions. Of course, the news is a bit saddened by the decision from Opera to support a new, non-GL-based API. I understand the desire for something more high level, but putting well known GL functions underneath is a perfectly acceptable idea. One side or a third party needs to provide a compatibility layer, or they need to decide on one of these APIs. I really hope OpenGL ES makes the win here. This also ties into the OpenGL APIs on Android, accessable through WebKit, so it only makes sense that Firefox, Opera, and all the WebKit-based browsers should standardize, before Redmond releases DirectX 1.0 Web Edition Premium.

For Users This Means...

We're going to see some fun web applications taking advantage of this, but there isn't a lot we'll see that we didn't have with Flash, for years now. I think some of the most interesting effects will come when we can use a canvas as a 3D texture and can render DOM elements into the canvas. When we reach this, we'll see lots of page effects, from folding elements to crumpled elements being deleted to rotating text and interface units.

We're going to see a lot of ugly abuse.

For Developers This Means...

Just one more thing to wait years before specialists are expected, and again you need to be a jack of all trades. Now you need to understand some code, a little database theory, CSS styling, artistic design, business layout, and 3D modeling and texturing. Have fun with it.

Wednesday, November 28, 2007

How To Work a Sigmoid

Software Development in Really Big Steps
  1. How To Work a Sigmoid
  2. How To Work a Sigmoid - Part Two

I've written about my use of FogBugz, driven by their great time tracking and estimation features. Using these, I've come across what I think is probably common and should be a goal for estimating the time of a project.

There are two estimations of a project. When you start, you can make some wild guess, pulled from the ether, weeks or months ahead of when you think it will be complete. This is the number that is notoriously and unequivically wrong. This kind of prediction is simply an invitation to make a smart person look dumb, since so few of us realize that he never was able to make that estimate. The larger the project, the greater the exponent on your chances of being able to make this estimate. This is not new to any of us.

The second estimate is the running estimate, compiled from the tasks the project has been broken down into. Now, the pro of this running estimate is that it is bound to be more accurate than the wild guess you started with, especiallly if computed with some of the fancy number working FogBugz does to account for how good different developers actually are at estimating their time. However, to every pro is a con and this one has a big one: the running estimate, although more accurate, is incomplete. You can only estmate for the tasks you've broken the project into and that is a fluxing set of tasks. As you develop you break larger tasks into smaller ones, learn new things you need to do, change requirements, find bugs in the work you've done already or the dependencies you use, and continue to iron out the design. This is even more true if you use agile techniques, so you didn't design a lot of upfront, but design on the go. Not to say this isn't a good thing, but it is a thing to be aware of.

The project starts at 0.0 and it ends at 1.0. Your guess is somewhere below or above 1.0, but mathematically cannot be equal to it (because, you can't guess!). As you accelerate your collecting of tasks to do the running estimate begins to increase toward 1.0 very quickly, until you start to level out and complete more tasks than you create. We work on the running estimate of a sigmoid curve, winding up from nothing and leveling off at the best real estimate that can be given with the real data at hand. Now, I grabbed this image from some place and I didn't add the flat line that represents your initial guess. This is both because I didn't have the time and because that guess is completely useless.

Great, so we work a sigmoid. So what?

The world is flooded with useless information and I don't want to contribute, so this is the part where I make my revelation somewhat useful. At least, theoretically. A good estimation system, like the Evidenced-Based Scheduling from Fog Creek, is really great. But, what if we included estimation of estimations? Oh, that sounds recursive, for sure. Suppose that, in addition to computing the weighted estimates and the running estimates of release after compiling all the information that can be taken usefully into account, we also track the running estimates as they change over time. If we graph these, I suspect they would roughly follow the curve of the sigmoid. If we find this or any other pattern to be true, we can estimate the estimations. The further along a project goes, we can estimate the future of the curve and make moderately intelligent guesses about where the estimation will go in the future. Weighting for how different teams and individual developers estimate, the system can train itself for accuracy.

I'm already into my current FogBugz tracked project, but my next will be setup to grab the estimate data periodically and I'm just itching to test out my theories. We can't predict when a project is going to be complete, if it ever is, but we can damn sure do better than pulling numbers out of the air.

How To Combine Your Own Efforts

I came across the invite-only beta for Onaswarm.com and I've had some good conversations since having my invite request responded by one of the developers. The upcoming plans are looking really interesting, and I like being able to combine my feeds. I'm still wanting some aggregation for things like bookmark feeds, and it would be nice if trivials like Twitter or Jaiku didn't put full posts into my final feed, but annotated the existing posts with my status. There are a lot of ways this could all go and I'm really interested in this.

You can view my Onaswarm feed. I'm going to continue keeping an ear on things over there and try to get in on this band wagon. This is more my kind of social network, because it deals with the things I already care about. What am I talk about, where have I surfed, and what am I talking about? It lets me pull all this from what I am already doing. I think this can turn out very well.

I have a related surprise to announce here, probably after the holidays. This is a developing side project of mine, which goes in line with things like this and is also quite a different beast. I'm really excited.

Monday, November 26, 2007

How To Have Too Much To Do

I've got a lot of things I'd like to tackle and I just wanted to layout some of the things on my mind lately. Many of them are small, so maybe I'll even complete a few before 2008.
  • Launch a small, free service that uses a del.icio.us account to take social bookmarks and forward them to Twitter or Jaiku or Onaswarm automatically.
  • Learn how to write a Firefox sidebar application and replace the crappy TwitBin. I want to only see the most recent status from each user and to remember my preferences better.
  • Develop a small desktop tool to grab my bugs from FogBugz and let me track time offline. This will come in handy when I travel around the holidays.
  • Get reacquainted with Nevow and Athena for a few small games, like TicTacToe and Squares.
  • And, as I write this, I want something that will take highlighted text and replace it with a link to a Google search for the text. Easier than looking up the links to everything I just mentioned!
It really seems like an OK plate, now that I have it written out.

How To Enjoy a Week of FogBugz

I have been on an eternal struggle to find the rights tools to keep me organized and on track with my projects. Flying blind is just not something I can do, with such a wandering mind. I especially like time tracking tools, because if I am tracking my time in a task, I am far more likely to focus on it until it is complete. Distractions make a lier out of me. When Joel Spolsky blogged about the Evidence-Based Scheduling in the newest release of his FogBugz product, I finally decided to try the service out for a new project I am starting on over the holiday. It has been about a week and I already have some really good impressions.

As far as bug tracking goes, FogBugz seems to be bare a good deal of similarities with Bugzilla, but is still very familiar to a Trac fan. They've even added a Wiki, although I've not used it. I'm working in solo on my FogBugz trial, right now. (More on that later.) I do wish for dependancy field on cases, instead of just linking to them in the comments. Overall I don't have many wishing for the case tracking itself, and I'm barely using the features available.

The listing is very customizable and I've taken advantage of a few different configurations already, so I can definitely see myself finding more that are useful. There have been some things I haven't found the right fit for. Notably, areas and releases have been a little awkward. Many things cross over different areas of work, so I don't have a clear separation there. I kind of wish for tags, instead. As for releases, there simply are not good uses for those when a project is so internal. I can just make a release for when we decided it is done, but then the field is as useful as not existing at all. I tried to make pseudo-releases for different milestones of functionality, but I am not sure if that is a proper fit.

Time tracking, the very thing that drove me to try FogBugz, is possibly my favorite part. Seeing what you guess and what you actually take is revealing. I seem to guess over, usually, but I wonder if I'll see my task estimates actually getting better as I use this over a period of time. The feedback may train me. I've even found, so far, that the release estimations seem to be pretty well calculated and I've hit the dates its estimated pretty well. I want to write more about my thoughts on estimation and how well you can estimate what you can't design until you've done much of the things you need to estimate in the first place.

I really am loving it, but I know I need to wait out my trial before making any final call. I think it is well worth the cost over the free Trac and others, even for personal use.

Saturday, November 24, 2007

How To Really Want To Love Flock 1.0

Everyone is abuzz with the release of Flock 1.0 and I so wanted to get on the bandwagon. I just couldn't make the jump. I tried to like it, very hard. Twitbin trumps Flock's Twitter integration and the del.icio.us bookmarks extension does a better job of helping me find my bookmarks. I find it very annoying that I wanted Flock for integration with online services, yet I couldn't remove the local bookmarks I care nothing for from the bookmark sidebar.

Now, there were some things I really liked. I loved that logging into a supported service configured it automatically. I want to see more of that.

Thursday, November 08, 2007

How To Demo With Zero Barrier

When one browses to MindMeister and looks at the nicely designed page, the user will notice a nice screenshot of the service. This is not a screenshot, but an anonymous, live embedding of the actual mind mapping service. Right at the first page, you get to start messing around with things. I think all Web 2.0 apps need to provide this kind of immediate use. We can provide such a low barrier to use, with no installation, but we've really lowered the bar, so to speak. The users won't jump very high for us these days. Let them trip and fall right into our arms.

For Web 2.0 This Means...

Web-based applications need to provide an anonymous access to their application on the front page of the website. If you have a to-do application, let the user start interacting before they do anything. Even registration is a barrier to entry. Of course, if you take what they did anonymously and migrate it when they register, you get a gold star. You get two gold stars if you also keep their anonymous data around when they return with a cookie.

For Development Frameworks This Means...

Frameworks need to provide the infrastructure to do this easily. We build things in the context of a user, so sometimes there is a barrier we have to cross ourselves to provide this. Built-in ways to create anonymous uses, promote them with credentials, and expiration anonymous accounts: all will let developers provide this Siren call to users at little cost.

For Users This Means...

More choices, because I can try more things. I don't need to give out my information and remember credentials just to try out yet another twitter clone, to-do app, or mind mapping software.

Tuesday, November 06, 2007

How To Git Away From CVS

James went along with the idea of moving away from CVS quicker than I thought and we put the plan into action last week. I put in the time to the project and started off with the default CVS replacement: Subversion. I really was looking forward to using it at work, until a friend made a subtle suggestion to look closer at the git project, which Linus Torvalds is heading as the version control system of choice for some little thing he's writing called Linux. Needless to say, I was skeptical, given the track record of the developer.

Quicker than I realized, I was falling head over heals for the examples of git use I was seeing. I cooked up a latest stable for OS X, as the installer I found was 130MB from the 1MB source tarball. Universal binaries on a project that generates about 145 distinct executables is a real bitch. I whipped up a little script to name git and toss into $PATH, while keeping all the git-* executables and other files tucked away in /usr/local/git/. I've cleaned that up and released it for anyone else interested in a clean git install on OS X. I may be releasing more git related work in a the near future, if you read on.

For Development Sanity This Means...


As a team grows out of a few developers and reaches nearly a handful, you need to start thinking more about development processes. Fixing a bug quick while in the middle of a half-finished change is a real problem, especially if you've commited already. Multiple developers working on separate projects can also be difficult to manage, if you don't think about things. These were among our concerns. Also among these concerns was the benefit of making the switch before bringing anyone else on board.

We had seven CVS modules and I developed a script that imported each of them to a new git repository and them merged them in the same layout as we had by checking out all our modules into the same directory. Keeping them together was a good move, and we kept our whole history.

Branching was one of the core driving motivators and I was thrown back when I found branching not doing what I expected in git. Branches in git are local, and although you may often push or pull between branches named the same on different repositories, they are not really related. I got to work on developing a set of branching tools, and I'm very happy with what I did in a small amount of time. The functionality is pretty complete. We can create, share, merge, and switch branches easily. I've even implemented automatic stashing and restoring of working copy changes that haven't been commited when switching between branches!

For Released Work This Means...


The tools are proving fun to work on when some issue comes up where they could be improved. They began life as a set of shell scripts, but they will very likely evolve into Python scripts to facilitate their future improvement. They are also very likely to leave the repository which they themselves are in control of. That was a headache to get proper.

When they are converted to Python I am going to release them. I really want to actually publish the git repository, but I need to figure out how to do that and if we can do it securely from work somehow. It would be nice to do that, such that the repository is always up to date with what we have. I'll probably mirror them to http://repo.or.cz/.

I've also placed the git binaries for OS X in two bzip2 compressed tarballs. git-1.5.3-osx-intel.tar.bz2 is everything you really need, placing only a single git executable into the path. You can grab git-1.5.3-remote-osx-intel.tar.bz2 to expose the tools needed to access your local repository remotely, via ssh.

For SocialServe This Means...


The three current of us are now branching for our development happily. We've done hot fixes in the middle of larger branches, without disrupting our work. We've pulled bug fixes from the master branch into our own things, shared branches, merged them into the production line, and are generally having a really good experience.

I think that, in the end, this really means more productivity. I'm able to be more flexable with my project, because I don't need to keep every commit in a state that can be pushed into production if James suddenly needs to fix something unrelated. Branch based development is great, and our scripts help manage them very well. I can't wait to release them.
I write here about programming, how to program better, things I think are neat and are related to programming. I might write other things at my personal website.

I am happily employed by the excellent Caktus Group, located in beautiful and friendly Carrboro, NC, where I work with Python, Django, and Javascript.

Blog Archive