Author Archives: Doug

Flex/Flash/Actionscript

I just decompiled a decompiler so I could use it to decompile itself

Update: Turns out that the code in Nemo 440 is actually just the code from the abcdump.as file in the Tamarin project. It looks like the abcdump.as file was written by Dan Schaffer from Adobe. So it turns out I could have just grabbed that file and not decompiled Nemo 440 at all, oh well 🙂 Shigeru Nakagakai has also used that same abcdump.as file to create his own AIR app that lets you compare SWFs and inspect classes and packages.

My brain almost exploded with meta-geekiness. If you haven’t seen the Nemo 440 AIR app, it’s a SWF decompiler written in AS3 as an AIR application, written by Vadim Melnik. It lets you load SWF files (or SWCs) and it will show you a list of all classes in the SWF, as well as the detailed ABC bytecode for each class. Technically I guess using the word “decompiler” isn’t correct, Nemo 440 is a SWF disassembler that produces ActionScript Byte Code (ABC). This isn’t the same as a true decompiler that produces real ActionScript code (like the Sothink Decompiler).

So if you download and play with Nemo 440 you’ll see what it can do. Basically you can see all the classes in any SWF, and all the methods within those classes. You also get to see some stuff like class-level and static variables. Then if you want to uber geek out you can see the ActionScript Byte Code of the SWF. That means you get shit that looks like this:


    0       getlocal0     	
    1       pushscope     	
    2       getlex        	http://www.adobe.com/2006/flex/mx/internal::layoutObject
    5       getlocal1     	
    6       setproperty   	direction
    9       findpropstrict	invalidateSize

That’s far from being nice beautiful AS3 code that you can actually use, but if you spend enough time understanding ABC code you can start to make some sense of it. But even just seeing the packages, classes used, and the methods of all the classes is pretty awesome.

So after playing with Nemo 440 a bit I encountered the glaringly obvious question. What happens if I use Nemo 440 to decompile the Nemo 440 application itself? Excited in my geekery, I quickly extracted the SWF file from the AIR app and loaded it in, only to be sorely disappointed. When I decompiled the app I could see all the Flex classes and other libraries that were used (even some components from Flexlib!), but I couldn’t see any classes that were used to do the decompiling. Hmm, I thought, how odd.

I figured that it wasn’t technically possible that the dissassembler would work so well on all SWFs except itself, something was fishy. I mean, that just doesn’t make sense. So I decided to load the SWF into the Sothink Decompiler to have a look. Then I came across this little gem within the Nemo 440 source code:


private function _checkName(param1:String) : Boolean
{
   if (param1 != null)
   {
      if (param1.indexOf("docsultant") >= 0)
      {
         return false;
      }
      if (param1.indexOf("nemo440") >= 0)
      {
         return false;
      }
   }
   return true;
}

That code specifically checks if the package name matches one of the packages used in the Nemo 440 source code and excludes it from being processed! Mother fucker!

So I decompiled the code using the Sothink Decompiler, and then decided I wanted to put it back together into a running app. It took me about 6 hours or so of work, and I had to consult the SWF specification document and the AVM 2 spec document a lot during the process, but I got it working.

So then I had my own AIR app that used the code from the Nemo 440 app to load SWFs and disassemble them. The first thing I did, obviously, was load up the Nemo 440 SWF file and give it a whirl, and I was instantly looking at the full class/package structure and the disassembled ABC bytecode of all the classes.

Ahh, sometimes it feels so good to geek out so much.

For those interested in learning more about decompiling, disassembling, ABC bytecode, and all that good stuff, I’ll be talking about this whole thing and a bunch more stuff in my presentation at Flash on the Beach in Brighton, England on October 1st. I won’t be posting the code that I decompiled and put back together, unless I get the original author’s permission (which I’ll be asking for, but haven’t yet done). Who knows, maybe the guy will be cool with open sourcing it 🙂 but maybe not. And if not, that’s cool too, he put a lot of work into it (I have a great appreciation for the amount of work after reading through much of the SWF spec and seeing the kind of code he had to write).

I just thought the concept of decompiling the decompiler to decompile the decompiler was too cool. God, I’m such a geek.

Standard
Flex/Flash/Actionscript

Followup for Using BitmapData for Array manipulation: using a hashmap is (a little) faster

UPDATE: The test app has been updated to include a third test that uses a ByteArray approach. Looks like the ByteArray approach is the fastest, followed by the hash table, followed by the bitmap method. They’re all pretty close when you’re working with 10k or 100k records, but once you get into a million or more items then the ByteArray approach pulls ahead. Thanks to Alex Bustin for sharing his code for the ByteArray test.

After I posted my technique for using the Flash bitmap drawing methods to perform calculations on arrays, Tom Carden started posting a few test of his own that tried to optimize some code-only algorithms. In his test he got an algorithm that used a hash table lookup method to be almost as fast as the bitmap method (but in his tests the hash table method was still slower than using bitmaps). I’ve created a slightly modified version of a hashmap algorithm that squeaks out just ahead of the bitmap test.

The algorithm basically just uses a very simple Array that holds boolean values that represent whether or not a particular item exists in the filtered array. So this is just like using the bitmap approach, except instead of creating different colored pixels, you just set the proper index in the array to true or false. Then once you have those two array of true/false values for both your filtered arrays, it’s just a matter of looping over the full array and checking the hash table arrays to see if a particular item exists in one, both, or neither of the filtered arrays.

And it turns out this is faster (but not by that much) than the bitmap method. But if you’re looking for the fastest way to take two arrays and find the unique values in either or the intersection or union, using this hash table method is the fastest way I’ve found so far. Too bad the bitmap stuff didn’t win, I was rooting for it… oh well, so it goes.

I’ve updated the test application (source) to include this new algorithm too, and here are the performance test results on my machine These stats were generated by taking the average of about 10 runs of each algorithm, running in the standalone Flash Player (which seems significantly faster than in any browser):

# of items Hashmap test Bitmap test
10 items 3 ms 4 ms
100 items 4 ms 4 ms
1,000 items 5 ms 6 ms
10,000 items 26 ms 26 ms
100,000 items 225 ms 237 ms
1 million items 2340 ms 2418 ms
2 million items 4874 ms 5705 ms
5 million items 13389 ms 14220 ms

And here’s the full class that I created for testing this has table algorithm:


package
{
import flash.utils.Dictionary;

public class ArrayHashProcessor
{
private var fullArray:Array;
public function setFullArray(value:Array):void {
	fullArray = value;
	reset();
}

public function reset():void {
	hashes = new Dictionary(true);
}

private var hashes:Dictionary;

public function filter(filterFunction:Function):Array {
	var newHash:Array = [];
	
	var filtered:Array = fullArray.filter( filterWithHash(filterFunction, newHash) );
	hashes[filtered] = newHash;
	
	return filtered;
}

private function filterWithHash(filterFunction:Function, hash:Object):Function
{
	return function(item:Object, index:int, array:Array):Boolean {
		var b:Boolean = filterFunction(item, index, array);
		hash[index] = b;
		return b;
	};
}

public function process(array1:Array, array2:Array):Array {

    var unique1:Array = [];
    var unique2:Array = [];
    var intersection:Array = [];
    var union:Array;

	var hash1:Array = hashes[array1];
	var hash2:Array = hashes[array2];
	
    fullArray.forEach(function(item:Object, i:int, a:Array):void {
        if (hash1[i]) {
            if (hash2[i]) {
                intersection.push(item);
            }
            else {
                unique1.push(item);
            }
        }
        else if (hash2[i]) {
            unique2.push(item);
        }
    });
                
    union = unique1.concat(unique2).concat(intersection);
    return [unique1, unique2, intersection, union];
}        

}
}

There’s nothing real complicated here (and certainly nothing as visually interesting as using bitmaps). Basically you need to take advantage of using the filtering of the array as the place where you store the true/false value in the hash table (using the bitmap method this was where we called setPixel() as well). Doing that extra step in the filter function doesn’t add very much time to the filter function (but sticking a boolean into an Array is faster than calling setPixel on a bitmap). Then when you want to compare the arrays we just loop over each item in the full array and check the two hash tables.

So I’m a little sad to see my bitmap method gets surpassed by something so simple, but hey, that’s just how it works out sometimes. I suppose I should’ve tried this method first 🙂 but then I wouldn’t have gotten to play with bitmaps.

If you know of an even faster way to accomplish the same thing, let me know in the comments. I’m thinking about trying out using a ByteArray as an alternative to storing the hash lookups in a normal Array, but my gut tells me that using writeByte and readByte will be slower than simply setting array[index] = true, but who knows.

Overall I’m happy I tried out the bitmap approach (even if I won’t be using it). I think it’s a pretty interesting way to approach the problem and it helps illustrate that sometimes thinking outside the box can have some pretty cool results.

Standard
Flex/Flash/Actionscript

Using BitmapData for Array manipulation in AS3

UPDATE: The test app has been updated to include two more tests: one for a hash table based approach, and one that uses a ByteArray approach. Looks like the ByteArray approach is the fastest, followed by the hash table, followed by the bitmap method. They’re all pretty close when you’re working with 10k or 100k records, but once you get into a million or more items then the ByteArray approach pulls ahead. Thanks to Alex Bustin for sharing his code for the ByteArray test.

Recently I was thinking about how to do some complex Array manipulation, and how to do it fast. I needed to take a large source Array and filter it two different ways, producing two new filtered arrays, each of which is made up of a subset of the values in the source array. Then I wanted to know which values of the first filtered array were unique to that array (did not appear in filtered array #2) and vice versa. I came up with a technique that uses the bitmap manipulation of the Flash player to accomplish these goals.

An example of the problem:
Let’s say I have a source array that looks like this:

Source Array
Item 1
Item 2
Item 3
Item 4
Item 5

Then I filter that array to produce a new array, Filtered Array #1, that contains some but not all of the items. I do the same thing again, except this time using a different filter that produces Filtered Array #2. Now my data looks like this:

Source Array Filtered Array #1 Filtered Array #2
Item 1 Item 2 Item 1
Item 2 Item 3 Item 2
Item 3 Item 5 Item 3
Item 4 Item 4
Item 5

I want to know the unique values in Filtered Array #1 that do not appear in #2 (Item 5) and the unique values in Filtered Array #2 that do not appear in #1 (Item 1, Item 4). An added bonus would be some extra stuff like being able to know the union of both arrays (items that appear in either array 1 or 2) or the intersection (items that only appear in both filtered arrays).

The slow-ass way
So the first thing that comes to mind when trying to solve this is that you can simply do a for loop over the source array and check if each item appears in Filtered Array #1 and Filtered Array #2. This means you have one full loop over every item in the data set, and then you need to do a check for whether the item exists in each of the filtered arrays. The most basic way would be something like this:


var unique1:Array = [];
var unique2:Array = [];
				
for(var i:int=0; i

So that's the easiest way and it certainly works. I'm sure there are countless ways to do this in a more optimized way, like not looping over the source array but instead looping over the first filtered array, somehow marking all the items in the second filtered array if they were found in the first filtered array, then looping over the remaining items in the second filtered array. I'm sure someone will tell me that I should use a binary tree search since I know the source index, or that I should use a hashtable to lookup the existence of items faster. I bet all those different approaches would beat this simple example speed-wise, but I was still worried no matter how optimized an approach like this can get, it would still suck. (Side note: If you have an alternative approach that will handle many items with really fast performance, especially if you have an AS3 implementation, I'd love to hear about it.) So anyway, I decided I wanted to try something different.

Using bitmaps
Check this out. Say you have a source array of 10,000 items. Then you filter that down to a filtered array that might have a subset of about 4,000 of those items. The relationship of the items in the filtered array to the source array can be drawn using a bitmap like this:
screenshot024.png

This image is a 100 pixel by 100 pixel square, and each pixel within that square represents whether or not the item in the source array appears in the filtered array. So if the pixel at 0,0 is black that means that the first item in the source array is also in the filtered array. If a pixel is red, that means that that item does not appear in the filtered array.

Why did I choose to make a square image? I originally just did a single line 1px high. But Flash has a bitmap limitation of a max width or height of 2,880 pixels. So a single line means that you would be limited to arrays with only 2,880 items, and that's not the kind of problem I was trying to solve. The performance need arose from having to do this with a shitload of items. So if we use a square instead (easy to calculate the dimensions, just take the square root of the # of items), then we have a theoretical maximum number of items of 2,880 x 2,880 = about 8.3 million. I'm not going to try to filter more than 8 million items client-side, so I'm ok with that limitation.

OK, cool. Now we do the same thing for the second filtered array. Below is the generated image of both filtered arrays side by side. The red image represents Filtered Array #1 and the green represents Filtered Array #2:
screenshot026.png

But how do you create these images in an optimized way? Since the arrays are already being filtered, that means I'm using Array.filter() with a callback function to decide whether or not the item passes the filter test. This filter function is going to be run on every item in the array. So I use that function to also do my bitmap drawing. And check this out, the callback for a filterFunction tells you the index of the item in the array! So that gives you all the information you need to draw these images. You know that the function will get called one time for each item, and it tells you the index. So just do a simple BitmapData.setPixelAt() and once your filtering is complete your image has been generated. Dope.

So now we start with the real bitmap magic. We're going to use the compare() method of the BitmapData class to generate a third image that represents a pixel by pixel comparison of these two images. To use the compare() method you call it on one BitmapData and pass in a second BitmapData. The function will return a new BitmapData object that contains pixel comparisons. If the pixel in bitmap 1 is the same as the pixel in bitmap 2 then the new comparison bitmap will have a transparent pixel (0x00000000). If they are not the same, then it will return the "difference pixel", which contains the difference between each channel (RGB). So what happens when we run compare() on these two images? We get a new image something like this:
screenshot027.png

There are only a few possible pixel values now, and each unique pixel value tells us something special. So by looping over the pixels we can know whether the item occurs in both arrays, one array or the other, or neither array. And just like that, we got our answer.

Try out the demo application I created here. And you can view the full source here. Be forgiving, the source code is dirty as hell... no comments, no nothin'... I basically just cranked this out and decided to put it out there without taking the time to clean it up because a) I think it's a pretty cool approach and b) it's dinner time and I'm hungry.

Performance
So what about some benchmarks? I've put together a little demo app that you can play with to try out this method. It lets you run two algorithms, one that is a very simple for loop that loops over each item and calls array1.indexOf(item) and array2.indexOf(item) to see if the item is present in both filtered arrays. Then you can also run the bitmap test that I've described. Here are the results that I had on my machine when running both techniques on various numbers of items:

# of items For loop text Bitmap test
10 items 4 ms 4 ms
100 items 4 ms 4 ms
1,000 items 19 ms 7 ms
10,000 items 1300 ms 32 ms
100,000 items timeout 350 ms
1 million items timeout 3700 ms

These aren't meant to be scientific results. There are some pretty big factors that might influence speed. For the tests I have a filter function that just chooses at random (with 50% probability) which items to include in the two filtered arrays. If you change the number of items in either array the speeds will fluctuate. But the basic idea is that doing a for loop and checking for an item using indexOf fails miserably when you start getting over the tens of thousands of items. The bitmap method works even on a million records, albeit slowly at 3-4 seconds, but at least it works without timing out or crashing the browser.

And like I said before, I'm sure there are other algorithms that would be faster than looping over every item in the full array and checking using indexOf. If you have some implementations that solve the same problem but with better performance, please let me know in the comments.

So what?
I don't really know if I'm going to actually use this technique anywhere, but I might. I'm also thinking about performing other calculations on large datasets using similar methods. Essentially if you have two or more datasets, you can use this graphical technique to perform all kinda of other calculations. I'm only using the compare() function to test for a few very distinct possibilities, but you could imagine doing something much more detailed, like performing calculations based on the aggregate color difference of multiple array images. The bitmap manipulation in Flash is so damn fast, that it opens a whole new door if you try to think about using those methods for number crunching and not just pretty graphics.

P.S. I know it's been a long time since I did a blog post with code. I think this one geeks out to a pretty extreme level, so for anyone reading who was getting fed up with my non-technical ramblings, I hope this makes up for it.

Standard
Maps, SpatialKey

SpatialKey featured on the cover of ComputerWorld

The most recent issue of ComputerWorld magazine features a cover story called “Can Web 2.0 Save B.I.?” that features a case study on SpatialKey. In the article they interview Chief Jon Greiner of the Ogden Police Department in Utah. Ogden is the first installation of the enterprise version of SpatialKey Law Enforcement Dashboard (see the press release), and Universal Mind has been working closely with the Ogden PD to use the SpatialKey platform to develop what we think is a game-changing crime mapping product.

Here’s an excerpt from the article talking about SpatialKey:

Today, the officers are using the new BI tools to perform geographic profiling of crimes and analysis of police data “in seconds,” he says. Before, it could take days for the department’s single crime analyst to fulfill a report request. An added bonus is that experienced police officers with extensive street experience are now able to apply their firsthand knowledge to crime analysis.

“You have practitioners asking the what-if questions, which has changed the way we police,” Greiner says.

And here’s the cover of this month’s print edition of the magazine:

SpatialKey on cover of ComputerWorld magazine

That’s our heatmap! Yeah!

Standard
Flex/Flash/Actionscript

Code samples from Adobe Flex 3.0 For Dummies

The compiled list of code samples from our Flex for Dummies book have been posted on the dedicated Flex for Dummies page of my blog. Each chapter (other than chapters 1, 4, 19, and 20) contains source code samples, and you can browse through them all online or download the code to run in Flex Builder yourself. We’ve double and triple checked everything to make sure it’s all error-free.

Download the code here.

screenshot071.jpgYou’ll see that each chapter contains two links, one to view the source code online and one to download a zip file. If you view the code online you can browse through each of the code listings for the chapter right in your web browser (and cut and paste to try running the sample yourself in Flex Builder). If you download the zip file you can get all the source files on your computer and load them right into Flex Builder yourself. Each chapter was created as a separate Flex project, so you can just create a new project for whichever chapter you like and copy over the files that are in the zip.

We’re still pulling together the last of the samples, so the code for chapters 2, 8, and 12 is on its way. We hope to have those chapters posted by the end of the week. My sincere apologies for not having the code samples online immediately when the book shipped. Wrapping up the creation and production of a book like this was a whirlwind of activity, and we certainly slipped a little in getting the code online.

Standard
Flex/Flash/Actionscript

Flex spotted on TV

I came across a video of a segment that aired on Fox Business News about HotPads.com, which is an online housing search website (rentals, sales, etc). As I was watching the clip I noticed that beautiful little preloader we’ve all come to know and love 🙂 HotPads uses a custom map engine they developed in Flex (after porting an older Flash app).

Here’s the screenshot that caught my eye:
screenshot070.jpg

You can watch the full video here:

And if you haven’t checked out hotpads before, it’s pretty dope. In addition to seeing rental listings on a map, they let you see some cool heatmaps of various things like foreclosure rates, income and age levels, etc. And it’s Flex, so show some love people.

Disclaimer or whatever: The guys who founded and run hotpads are friends of mine, I even did some work for them back a few years ago. I’ve got a little investment in the company that will one day either make me rich or hopefully at least buy me a six pack of beer.

Standard
Maps, SpatialKey

Announcing SpatialKey – Geographic Information Without Limits

SpatialKey LogoToday I’m proud to announce the launch of SpatialKey, the geospatial information visualization product I’ve been working on with our fantastic team at Universal Mind. I’ll make a bold statement that I stick by: this is the best web-based mapping product in existence. Today we’re releasing a “technology preview” that gives you a little glimpse at what we’ve been working on (just to whet your appetite until we release the full product).

Quick links
Before I explain what SpatialKey is I wanted to give a few quick links because I know a lot of you are going to have your ADD act up before you read the rest of the post.

  • SpatialKey Gallery – lists a few dataset/template pairs that we think tell great stories. Read the descriptions of the datasets and then launch the app to play with the data yourself.
  • screenshot067.jpg
    San Antonio Prostitution hotspots

    San Antonio Prostitution Crimes – This link will jump you straight into exploring the prostitution crimes in San Antonio from Jan 2006 – July 2007. Check out how clearly the heatmap points out the corners that are the hotspots in the city.

  • Growth of Walmart – This link will load the Walmart dataset into a playback template that lets you click play and watch Walmart take over America.



Beyond points on a map

screenshot068.jpg
Overwhelmed with markers

We’ve been seeing the same tired approach to web-based mapping for years now. Everyone throws markers on a map. You want to track crime? Throw a bunch of markers on a map. Little pin markers work fine if you’re showing a few data points. Want to see the location of Starbucks within a 3 block radius of your house? Use markers. But what if you want to see the total sales of all Starbucks worldwide? Or all crimes for the past 10 years? For the whole country?

SpatialKey uses some of the most advanced visualization renderings for geospatial data that have ever been seen on the web. The focus here is on aggregate renderings: heatmaps, thematic grids, graduated circles. 1,000 markers all piled on top of each other doesn’t help anyone. What you want to see is density or sum total value. SpatialKey focuses on rendering aggregate data in meaningful ways. We can show you a heatmap of the entire country and let you visualize any number of data fields. You want to see the heatmap represent total sales of all stores in the region? No problem. You want to see average house price over the past 10 years? We can do that.

Heat Map Heat Grid
SpatialKey Heat Maps SpatialKey Heat Grids

We haven’t seen innovative technology in this industry since Google let you drag the map. (I actually vividly remember that moment when I first dragged a Google map and my mouth started to water). It’s time to move beyond points on a map.

Your data doesn’t have limits
Try adding 10,000 data points to a Google Map. I dare you. What happens? If you’re using the “My Maps” feature of Google Maps, you’re limited to only show 200 points at a time, then you have to page through your data. And to top it all off you’re limited to a whopping total of 1,000 data points in the entire data set. So you get to page through 5 pages of data and only see 1/10th of your total data set anyway. If you create your own application with the Google Maps AJAX API you’re going to have serious performance problems when you get up into a few hundred markers. We think that’s ridiculous.

SpatialKey breaks through the limits of previous mapping technology in two ways. First, we’re simply faster. Flash can process and render data far faster than JavaScript. We can render 10,000 data points in a matter of milliseconds. You simply can’t do that with any JavaScript API out there. Second, we’re smarter. We aggregate data to produce heatmaps instead of just trying to overlay markers one on top of the other. Fundamentally, a massive dataset is an information visualization problem, not a technical one. You need better renderings to convey massive amounts of data, and that’s what SpatialKey delivers.

This is just the beginning
This is a technology preview. That basically means we’re showing you some cool stuff, but we’ve got way more up our sleeve. We’re looking for feedback on what we’ve got, and we’re hoping to get you excited about what we’ll be rolling out. We’ll be releasing new versions of SpatialKey Personal that will let you easily import your own data (if you’ve got an Excel file with addresses you can drop it right in). We’re also going to be releasing SpatialKey Enterprise, which lets you load a data set of any size (millions and millions of points). And then we’ve got a third product that we’re launching called SpatialKey Law Enforcement Dashboard, which is an enterprise version of SpatialKey specifically targeted toward police departments (includes special law enforcement reporting templates). And in the meantime we’ll be rolling out some more example datasets for you to play with, so keep an eye on the SpatialKey blog.

So go check out the SpatialKey Gallery and play with some data. We’re looking for feedback during this phase, so if you have any suggestions or (god forbid!) you run across bugs, please let us know by emailing feedback@spatialkey.com.

Standard
Uncategorized

Why did I stop blogging?

I haven’t been blogging like I used to. It’s been a month since my last post (and that was just a silly cartoon). I’ve been thinking about why this is, and there are some typical expected reasons (busy work schedule, extra non-work commitments, a recent move, etc etc). This is all part of why I’ve fallen off the blogging wagon, but it’s not the whole story. This post is a bit of a dive into my psychology, trying to get at why I blog and where I’m trying to get to in my life. This isn’t a technical post and it’s not really even about Flex. It’s about me and my brain and my feelings. So if you don’t like mushy crap, you can stop reading.

First, the standard stuff:

  • Intense work
    I took a full-time gig with Universal Mind and we’re about to launch the product that I’ve been working on for a very long time. It’s one thing to work on a consulting gig where you walk in, do your shit, and get out. This is different. This product has so much of my time and energy invested in it. It’s my baby, and because I’m so invested, it consumes more of my mental time than any consulting project ever did. It’s not like I work 100 hour weeks for UM, I work normal hours. But my head doesn’t shut off when I commit my night’s changes and walk away from the computer. So that means I’m thinking about Flex for a huge portion of my non-work hours.
  • Writing Flex for Dummies
    Flex For Dummies basically took over my weekends for 7 months of my life. I wrote from Friday night to Sunday night (our deadlines were Monday and I usually turned in a chapter around 1 am). I didn’t go out, I didn’t go on weekend trips. And the last thing I wanted to do was blog. But that process is over and the book’s about to come out! I now have my weekends back, but I also have a new appreciation for my free time. I used to do a lot of blog posts over the weekends, now sitting in front of the computer is the last thing I want to do.
  • Moving in with my girlfriend
    It’s been about a month since I moved from San Francisco to Berkeley and moved in with my girlfriend. So I was dealing with the whole packing and moving thing, and also adjusting to the new living situation. Things have been fantastic since moving in, and this is another reason I’m not blogging as much. After a day of work I’d much rather go to dinner with my girlfriend and spend time hanging out with her instead of sitting in front of my monitor writing code.

So that’s all the stuff that I’d tell you if you asked me why I haven’t been blogging. “Man, I’ve just been so busy, I can’t find the time.” There’s a lot of truth to that, I certainly don’t want it to sound like I haven’t been busy. But largely the “I’ve been busy” excuse is bullshit. So why haven’t I been blogging?

I’m not hungry anymore.

Now we’re getting to the meat of this. I’ve been thinking about what drives me to blog and reflecting on the past year and a half. I’ve had a crazy ride. I went from being an unknown kid learning ActionScript and Flex to being a semi-famous-within-a-very-specific-geek-circle, book-writing, highly-paid, influential expert (and yeah, clearly you can tell that it’s all gone to my head). And it all happened in a year. My salary multiplied, I was speaking at conferences, I had a book deal. That’s some seriously crazy shit. That wasn’t entirely my plan from day one, although as things started picking up steam I started aiming higher and higher. I wanted to be famous. I wanted people to think I was good. I wanted to be in demand, to be sought after.

And so now what? What’s the next step? I’ve got a stellar job, I make bank, my book’s about to come out, I’ve got a reputation I’m proud of, and I’m being flown halfway around the world to speak at a conference. The people that I thought were rockstars, that I learned so much from while I was getting started (and still do every day); they all know who I am now (some of them are even good friends).

I used to be so hungry to prove myself. If someone had a coding problem they couldn’t solve I wanted to be the guy who showed everyone how to do it. I wanted people to stop and say “woah, where the fuck did this guy come from?” And now that I got that I guess I feel a bit drained. That fire to be the best, to be known, is dampened. I’m not saying I’m the best, I’m just saying I no longer care like I used to.

The Tech
There’s also a technology aspect to this. I’m starting to feel like I’ve taken Flex as far as it will go. I started proving small things; how to make little components, or how to hack some piece of the Flex framework. And I rode that until I knew Flex inside and out. I hit every brick wall (I climbed over or busted through some). And it was all so challenging and rewarding. I was unlocking secrets, uncovering things people didn’t know. But in terms of intellectual curiosity I’ve exhausted Flex 2 and 3. Of course there’s always new stuff in Flash Player 10 and Flex 4 that is going to be really interesting. But my interest has been shifting. I don’t want to solve small problems with a component or with a framework. I want big problems. I want new problems. I want real problems. I want to solve something that’s not just a technical detail.

I’m not ditching Flex or ActionScript. On the contrary, I feel like I’ve gotten to the point with the technology that I’m supposed to be: Flex and ActionScript are just tools to solve problems, not problems in themselves. When I started blogging Flex was a problem I was chipping away at, hacking together examples and tweaking the framework to get it to serve my needs. Now I want to move beyond that, and take on problems that are information problems on their own, and Flex just happens to be the best tool I have at my disposal.

The Next Step
I’ve been reading a good bit about information visualization as a field of research and soaking in as much as I can. The product I’ve been working on for Universal Mind takes geographic information visualization to a level that nobody’s done yet on the web (I put the “on the web” qualifier there, but I hope we’re working on stuff that pushes limits for any software, online or offline). And that’s just scratching the surface. There’s a ton room for innovation in the geo space. It’s a field that has so much potential but is stifled by a few big players and their ideas about how things are supposed to work. So I’m hoping I can help shake that up a bit. But geospatial information is just one small piece of the whole pie. I want to visualize everything. I want to create new ways of seeing data. And this is a field without limits. There is no end to the possible innovation.

So in the months ahead I hope to use this blog as an experimental proving ground for my forays into data visualization. I’m doing a lot of this for work, which is awesome because that means I get paid to do interesting work, but it has the downside that I often can’t share code. I imagine there will be more posts that are examples of what we’ve done (videos or live apps) that don’t show exactly how we did it. That sucks, but I think there’s still a lot to be said for showing off inspirational stuff. I’m also going to be trying to refocus on blogging small experiments in data visualization.

So if you’ve made it this far I’m impressed you read through my psycho-babble ramblings (or maybe you just skipped to the end looking for something worthwhile). From here on out I’ll probably keep the “dive into Doug’s head” type of posts like this to a minimum, and get back to blogging cool tech.

Standard