Skip to main content


Showing posts from August, 2008

How to Understand AppEngine Datastore Under the Hood: Part 2 - The Raw Datastore API

If you haven't yet read the first part of this series, feel free to start from the beginning with Part 1 - An Overview of the Underview

Every AppEngine developer is familiar with the module. In Part 1 I introduced what goes on under the hood of this API, to give everyone a better understanding of what they are taking advantage of. Now, in Part 2, I'm going to detail the actual API that is used to utilize the raw entities behind our Model instances. At this time I am unsure if anything in this API is suspect to change, but I doubt anything is subject to drastic flux and I'm fairly confident everything here is safe for actual use, as much as anything else in AppEngine.

Module: google.appengine.api.datastore

Our main focus here is the Entity class. Everything supports it, from the Get, Put, and Delete functions to the Query class. Their uses are obvious. As previous exposed, each entity is essential a property bag and will take any given properties to the datastore for storage, …

How to Understand AppEngine Datastore Under the Hood: Part 1 - An Overview of the Underview

There are a lot of wrong perceptions about the datastore in Google AppEngine. People both familiar and foreign with AppEngine don't really understand what the datastore is. There is a deeper system underneath the nice API we are given. Understanding the guts can help us understand the skin. We may also find there are times when we must shed the skin for new clothing.

The biggest misconception about the datastore is the assumption that "kinds" are anything like "tables". You could use a set of entity kinds similar to the way you would use a set of tables, but they simply are different beasts, entirely. A table controls a strict requirement on the structure of its rows. Every entity, on the other hand, is free to hold any properties of allowed types. The published Model API is all an abstraction provided to give us a nice interface on top of an otherwise much looser foundation.

Many people would be very surprised to learn that a given kind doesn't actually requ…

How to Bubble the Good of Twitter to the Top

The aftermath of the quakes in California saw a lot of talk about Twitter getting the word spread, from the trenches, very quickly. Chris O'Brien heralded it as a sign that NextNewsRoom is doing something right. A lot of people were talking about it. Twitter carried the news before any news agency. First is one thing, but quality control is something else. The flood of messages reached a point that its almost assured no one read every quake tweet that was sent. There were just too many of them. Can anyone imagine the flood that would have been seen if Twitter existed and was popular on the morning of 9/11? It would have been maddening.

We can take this situation and ask two questions. How can we form something better from the flood of tiny messages? Do we even want to? Can we find some way of filtering both relevant and "good" posts and could we pull some larger picture from all the little pieces? Of course, doing so would take resources, and those are either iron, eyes, …