After I posted my technique for using the Flash bitmap drawing methods to perform calculations on arrays, Tom Carden started posting a few test of his own that tried to optimize some code-only algorithms. In his test he got an algorithm that used a hash table lookup method to be almost as fast as the bitmap method (but in his tests the hash table method was still slower than using bitmaps). I’ve created a slightly modified version of a hashmap algorithm that squeaks out just ahead of the bitmap test.

The algorithm basically just uses a very simple Array that holds boolean values that represent whether or not a particular item exists in the filtered array. So this is just like using the bitmap approach, except instead of creating different colored pixels, you just set the proper index in the array to true or false. Then once you have those two array of true/false values for both your filtered arrays, it’s just a matter of looping over the full array and checking the hash table arrays to see if a particular item exists in one, both, or neither of the filtered arrays.

And it turns out this is faster (but not by that much) than the bitmap method. But if you’re looking for the fastest way to take two arrays and find the unique values in either or the intersection or union, using this hash table method is the fastest way I’ve found so far. Too bad the bitmap stuff didn’t win, I was rooting for it… oh well, so it goes.

I’ve updated the test application (source) to include this new algorithm too, and here are the performance test results on my machine These stats were generated by taking the average of about 10 runs of each algorithm, running in the standalone Flash Player (which seems significantly faster than in any browser):

And here’s the full class that I created for testing this has table algorithm:

package
{
import flash.utils.Dictionary;

public class ArrayHashProcessor
{
private var fullArray:Array;
public function setFullArray(value:Array):void {
	fullArray = value;
	reset();
}

public function reset():void {
	hashes = new Dictionary(true);
}

private var hashes:Dictionary;

public function filter(filterFunction:Function):Array {
	var newHash:Array = [];

	var filtered:Array = fullArray.filter( filterWithHash(filterFunction, newHash) );
	hashes[filtered] = newHash;

	return filtered;
}

private function filterWithHash(filterFunction:Function, hash:Object):Function
{
	return function(item:Object, index:int, array:Array):Boolean {
		var b:Boolean = filterFunction(item, index, array);
		hash[index] = b;
		return b;
	};
}

public function process(array1:Array, array2:Array):Array {

    var unique1:Array = [];
    var unique2:Array = [];
    var intersection:Array = [];
    var union:Array;

	var hash1:Array = hashes[array1];
	var hash2:Array = hashes[array2];

    fullArray.forEach(function(item:Object, i:int, a:Array):void {
        if (hash1[i]) {
            if (hash2[i]) {
                intersection.push(item);
            }
            else {
                unique1.push(item);
            }
        }
        else if (hash2[i]) {
            unique2.push(item);
        }
    });

    union = unique1.concat(unique2).concat(intersection);
    return [unique1, unique2, intersection, union];
}        

}
}

There’s nothing real complicated here (and certainly nothing as visually interesting as using bitmaps). Basically you need to take advantage of using the filtering of the array as the place where you store the true/false value in the hash table (using the bitmap method this was where we called setPixel() as well). Doing that extra step in the filter function doesn’t add very much time to the filter function (but sticking a boolean into an Array is faster than calling setPixel on a bitmap). Then when you want to compare the arrays we just loop over each item in the full array and check the two hash tables.

So I’m a little sad to see my bitmap method gets surpassed by something so simple, but hey, that’s just how it works out sometimes. I suppose I should’ve tried this method first but then I wouldn’t have gotten to play with bitmaps.

If you know of an even faster way to accomplish the same thing, let me know in the comments. I’m thinking about trying out using a ByteArray as an alternative to storing the hash lookups in a normal Array, but my gut tells me that using writeByte and readByte will be slower than simply setting array[index] = true, but who knows.

Overall I’m happy I tried out the bitmap approach (even if I won’t be using it). I think it’s a pretty interesting way to approach the problem and it helps illustrate that sometimes thinking outside the box can have some pretty cool results.

Recently I was thinking about how to do some complex Array manipulation, and how to do it fast. I needed to take a large source Array and filter it two different ways, producing two new filtered arrays, each of which is made up of a subset of the values in the source array. Then I wanted to know which values of the first filtered array were unique to that array (did not appear in filtered array #2) and vice versa. I came up with a technique that uses the bitmap manipulation of the Flash player to accomplish these goals.

An example of the problem:
Let’s say I have a source array that looks like this:

Then I filter that array to produce a new array, Filtered Array #1, that contains some but not all of the items. I do the same thing again, except this time using a different filter that produces Filtered Array #2. Now my data looks like this:

I want to know the unique values in Filtered Array #1 that do not appear in #2 (Item 5) and the unique values in Filtered Array #2 that do not appear in #1 (Item 1, Item 4). An added bonus would be some extra stuff like being able to know the union of both arrays (items that appear in either array 1 or 2) or the intersection (items that only appear in both filtered arrays).

The slow-ass way
So the first thing that comes to mind when trying to solve this is that you can simply do a for loop over the source array and check if each item appears in Filtered Array #1 and Filtered Array #2. This means you have one full loop over every item in the data set, and then you need to do a check for whether the item exists in each of the filtered arrays. The most basic way would be something like this:

var unique1:Array = [];
var unique2:Array = [];

for(var i:int=0; i<sourceArray.length; i++) {
	var item:Object = sourceArray[i];
	var inArray1:Boolean = array1.indexOf(item) != -1;
	var inArray2:Boolean = array2.indexOf(item) != -1;

	if(inArray1 && !inArray2)
		unique1.push(item);
	else if(inArray2 && !inArray1)
		unique2.push(item);
}

So that’s the easiest way and it certainly works. I’m sure there are countless ways to do this in a more optimized way, like not looping over the source array but instead looping over the first filtered array, somehow marking all the items in the second filtered array if they were found in the first filtered array, then looping over the remaining items in the second filtered array. I’m sure someone will tell me that I should use a binary tree search since I know the source index, or that I should use a hashtable to lookup the existence of items faster. I bet all those different approaches would beat this simple example speed-wise, but I was still worried no matter how optimized an approach like this can get, it would still suck. (Side note: If you have an alternative approach that will handle many items with really fast performance, especially if you have an AS3 implementation, I’d love to hear about it.) So anyway, I decided I wanted to try something different.

Using bitmaps
Check this out. Say you have a source array of 10,000 items. Then you filter that down to a filtered array that might have a subset of about 4,000 of those items. The relationship of the items in the filtered array to the source array can be drawn using a bitmap like this:

This image is a 100 pixel by 100 pixel square, and each pixel within that square represents whether or not the item in the source array appears in the filtered array. So if the pixel at 0,0 is black that means that the first item in the source array is also in the filtered array. If a pixel is red, that means that that item does not appear in the filtered array.

Why did I choose to make a square image? I originally just did a single line 1px high. But Flash has a bitmap limitation of a max width or height of 2,880 pixels. So a single line means that you would be limited to arrays with only 2,880 items, and that’s not the kind of problem I was trying to solve. The performance need arose from having to do this with a shitload of items. So if we use a square instead (easy to calculate the dimensions, just take the square root of the # of items), then we have a theoretical maximum number of items of 2,880 x 2,880 = about 8.3 million. I’m not going to try to filter more than 8 million items client-side, so I’m ok with that limitation.

OK, cool. Now we do the same thing for the second filtered array. Below is the generated image of both filtered arrays side by side. The red image represents Filtered Array #1 and the green represents Filtered Array #2:

But how do you create these images in an optimized way? Since the arrays are already being filtered, that means I’m using Array.filter() with a callback function to decide whether or not the item passes the filter test. This filter function is going to be run on every item in the array. So I use that function to also do my bitmap drawing. And check this out, the callback for a filterFunction tells you the index of the item in the array! So that gives you all the information you need to draw these images. You know that the function will get called one time for each item, and it tells you the index. So just do a simple BitmapData.setPixelAt() and once your filtering is complete your image has been generated. Dope.

So now we start with the real bitmap magic. We’re going to use the compare() method of the BitmapData class to generate a third image that represents a pixel by pixel comparison of these two images. To use the compare() method you call it on one BitmapData and pass in a second BitmapData. The function will return a new BitmapData object that contains pixel comparisons. If the pixel in bitmap 1 is the same as the pixel in bitmap 2 then the new comparison bitmap will have a transparent pixel (0×00000000). If they are not the same, then it will return the “difference pixel”, which contains the difference between each channel (RGB). So what happens when we run compare() on these two images? We get a new image something like this:

There are only a few possible pixel values now, and each unique pixel value tells us something special. So by looping over the pixels we can know whether the item occurs in both arrays, one array or the other, or neither array. And just like that, we got our answer.

Try out the demo application I created here. And you can view the full source here. Be forgiving, the source code is dirty as hell… no comments, no nothin’… I basically just cranked this out and decided to put it out there without taking the time to clean it up because a) I think it’s a pretty cool approach and b) it’s dinner time and I’m hungry.

Performance
So what about some benchmarks? I’ve put together a little demo app that you can play with to try out this method. It lets you run two algorithms, one that is a very simple for loop that loops over each item and calls array1.indexOf(item) and array2.indexOf(item) to see if the item is present in both filtered arrays. Then you can also run the bitmap test that I’ve described. Here are the results that I had on my machine when running both techniques on various numbers of items:

These aren’t meant to be scientific results. There are some pretty big factors that might influence speed. For the tests I have a filter function that just chooses at random (with 50% probability) which items to include in the two filtered arrays. If you change the number of items in either array the speeds will fluctuate. But the basic idea is that doing a for loop and checking for an item using indexOf fails miserably when you start getting over the tens of thousands of items. The bitmap method works even on a million records, albeit slowly at 3-4 seconds, but at least it works without timing out or crashing the browser.

And like I said before, I’m sure there are other algorithms that would be faster than looping over every item in the full array and checking using indexOf. If you have some implementations that solve the same problem but with better performance, please let me know in the comments.

So what?
I don’t really know if I’m going to actually use this technique anywhere, but I might. I’m also thinking about performing other calculations on large datasets using similar methods. Essentially if you have two or more datasets, you can use this graphical technique to perform all kinda of other calculations. I’m only using the compare() function to test for a few very distinct possibilities, but you could imagine doing something much more detailed, like performing calculations based on the aggregate color difference of multiple array images. The bitmap manipulation in Flash is so damn fast, that it opens a whole new door if you try to think about using those methods for number crunching and not just pretty graphics.

P.S. I know it’s been a long time since I did a blog post with code. I think this one geeks out to a pretty extreme level, so for anyone reading who was getting fed up with my non-technical ramblings, I hope this makes up for it.

I came across a video of a segment that aired on Fox Business News about HotPads.com, which is an online housing search website (rentals, sales, etc). As I was watching the clip I noticed that beautiful little preloader we’ve all come to know and love HotPads uses a custom map engine they developed in Flex (after porting an older Flash app).

Here’s the screenshot that caught my eye:

You can watch the full video here:

And if you haven’t checked out hotpads before, it’s pretty dope. In addition to seeing rental listings on a map, they let you see some cool heatmaps of various things like foreclosure rates, income and age levels, etc. And it’s Flex, so show some love people.

Disclaimer or whatever: The guys who founded and run hotpads are friends of mine, I even did some work for them back a few years ago. I’ve got a little investment in the company that will one day either make me rich or hopefully at least buy me a six pack of beer.

I’ve just posted the title and description of the session I’ll be giving at Flash on the Beach, which is happening September 28-Oct 1 in Brighton, England. My session is titled: Decompiling Flex and Flash. Here’s the full description (which you can also find on the FOTB site):

In this session we’ll learn how to decompile ActionScript 3 SWF files and peek inside other people’s code. Decompiling a SWF is often seen as an evil tactic that should be punishable by death, but regardless of your moral opinion, every SWF you create can be decompiled into often beautifully readable source code. If you’ve produced something cool, chances are someone has decompiled it (hell, chances are I’ve decompiled it myself).

In this session you’ll learn what you get when you decompile a SWF and what you don’t. We’ll cover how far you can get piecing a decompiled application back together and I’ll share a few real-world stories of how decompiling has proven invaluable in my development career.

This session will focus on ripping apart some large-scale Flex applications and diving into the source (we’ll see if I can get sued by the end of the session). I’ll cover some Flex-specifics that are important when you decompile a Flex app (Flex framework classes, generated MXML code, data binding code, etc). But decompiling AS3 SWFs is just as applicable for SWFs produced in Flash Authoring as well, so there should be plenty of information for everyone.

And for all the paranoid folks out there, in addition to decompiling code, I’ll also cover a few techniques to protect your source code to make it harder for people to steal.

I hope to see some of you in England!

This is a document that I wrote back in October 2007. I forgot about it until just now. My original idea was to define my “dream job” within Adobe, explain exactly what I wanted to do, and tell them why they should hire me. For a variety of reasons I never sent this document. This was during a period in my life when I was considering a few different options for my “career”. I was doing the Flex consulting thing and billing my time hourly (which is pretty sweet). I was talking to Universal Mind about making the shift to a full time employee, and during that decision I considered what other full-time options I had.

When I first started this whole Flex blogging thing my original goal was to get a job at Adobe. No joke, I basically started blogging thinking “I’m going to blog, get good, show people I can do good shit, then ask for my dream job at Adobe.” The Adobe SF office is a 5 minute drive or 15 minute bike ride from my house in San Francisco, and I figured it would rock to work there doing something Flex related. Then my eyes were opened to the world of Flex consulting (the money, the women…) But I always had a desire to go work for Adobe. So when I was considering full-time options, I obviously thought of boarding the mothership. So I wrote up a job proposal that describes the kind of job that I wanted. Yes, this is completely arrogant and egotistical. There were no job postings that I liked, so I decided to make one up. This was basically my way of saying: “You guys should want to hire me, but your job openings suck, so not only should you hire me, you should also make this sweet position just for me.”

I never sent this. In the end I decided that the position I was offered at Universal Mind was too good to pass up, and I was being ridiculous for even thinking anything otherwise (which I still believe). But I just read over this proposal again and I think that a lot of the ideas I bring up are still good ones. I would love to see this kind of experimental position created. So without further ado, here’s the job proposal I wrote. This is the main part of the document, I only took out the last page or so that talked about my relevant experience. If you can take this and get Adobe to create this job for you, go for it. I think this job would be sweet and it would benefit the Flex community and the Flex product as a whole.

——————————————-

JOB PROPOSAL
I am interested in a position within Adobe that would allow me to create experimental user interfaces and data visualization prototypes using Flex. The purpose of this proposal is to clearly explain the ideal position I have in mind, why such a position would benefit Adobe, and why I would be right for the job.

Job description
I propose an experimental software developer position to explore alternative and inspiring user interfaces created with Flex, focusing on alternative input methods and advanced data visualization techniques. Work would be heavily focused on Actionscript 3 and, more specifically, utilizing the Flex framework. It is important to emphasize the experimental nature of this work. This work is meant to move AS3 and Flex onto the cutting edge of research and development in user interfaces. The end goal of such work would be to inspire and teach the Flex developer community by publishing the open-source research results and blogging explanatory tutorials.

Reasons for being within Adobe
An important question that should be discussed is the benefit of bringing experimental community-focused development in-house in Adobe. I am already involved in open-source Flex projects externally, and I have done a fair amount of experimental work using AS3 and Flex that has been published on my blog. Regardless of where I am employed, I plan on continuing this type of work. However, being within Adobe will provide some serious advantages to all parties involved.

Experimental projects benefit three distinct stakeholders: me personally, the Flex developer community, and Adobe. However, nearly all the experimental work that I do is done outside of paid working hours, forcing me to decide between work that gets me paid and work that gets me excited (which can often overlap, but seldom do completely). A traditional employer does not have the resources to allow me to work on experimental projects that may or may not have a commercial use. But even if an experimental project is completely useless in the commercial software space, it still benefits me (through personal growth and education), the Flex community (through published source code and inspiration), and Adobe (by generating buzz and interest in the technology). Adobe is the only commercial organization that benefits financially from work that might have no commercial purpose.

While Adobe benefits from projects with no commercial viability, Adobe would also be in a position to benefit from possible commercial success of research projects. All projects would be reviewed by Adobe, which would allow the company to decide if a research project has interesting commercial possibilities. If so, a project could continue as an internal product development effort. If Adobe decides it does not want to pursue a project internally then the project still benefits the Flex community when it is released open-source.

Research Projects
The decision to take on research projects would take multiple factors into account: awesomeness, pragmatic feasibility within a limited timeframe, potential benefit to the developer community, and potential commercial application within Adobe. The first and most important factor is the wow factor of each project. I plan on making experiments that blow people away and show off the strengths of Flash Player. Flex/Flash Player allows us to make interfaces and data visualization prototypes unlike anything ever seen on the web (or on the desktop for that matter). I want to make things that amaze people.

Projects will also need to have short timeframes, at least in terms of creating something impressive that can be shared with the community. Future development of experimental projects might continue after the initial project timeline, either within Adobe or in the developer community, but each project would have an initial exploration phase of no longer than a few months. I’ve found that setting such time constraints leads to more focused development and more impressive results.

A major goal of all projects would be to benefit the developer community through the release of source code. Each project would include blog posts and technical articles explaining how the project was developed. This will encourage the developer community to use the source code in other projects, and will also provide inspiration that will lead to the development of even more impressive projects within the community. I have considerable experience blogging about custom Flex component development and AS3 experiments. I understand the importance of useful resources with well-written source code and explanations. As a developer, I think the most important resource for learning advanced techniques is the wealth of information found in blogs. I know that personally I’ve probably learned more from Ely Greenfield’s code than from any other single source (not to mention the amount of inspiration his examples provide). But now you’ve got Ely off architecting instead of writing code. It’s my personal opinion that Adobe should get back into technically-focused blogging, as opposed to the non-technical evangelism that seems to be the current priority.

The final consideration, which is purposefully listed last, is the commercial possibility within Adobe. A research project may have serious commercial possibilities, and given further development might turn into an official Adobe product. This seems all the more likely given Adobe’s movement into software services that are based on Flex (Buzzword, CoCoMo, Pacifica, Photoshop Express, Premier Express). It’s important to note that I would hope potential research projects would be brainstormed with possible commercial viability as an afterthought. Personally, I believe the most inspirational work is done without first considering the commercial possibilities.

BRAINSTORMING: possible directions for research
Be aware that the list of potential research projects below is an off-the-top-of-my-head list. The number of awesome possibilities that show off Flex/AS3 is nearly infinite. I have provided a list of current research interests, but the intent of this document is not to propose specific projects, but instead articulate why this type of experimental developer position is important.

Eye-tracking
There are various open-source projects for eye tracking using normal web cameras. This software could theoretically communicate to a Flash/Flex application using a local socket connection. This could allow Flex applications to receive data about where the user is looking within the interface. The possible uses of such data are amazing. We could make a Flex analytics app that would let developers visually analyze how users are interacting with their software. We could track not only mouse click interactions and mouse movement, but now also the actual eye movements. Companies that develop Flex applications would love the ability to accurately track their user’s focus. And with video we could track where people are looking as the video plays, which could be commercially valuable for online advertising companies to run studies, especially now that video-overlay advertisements are becoming popular. Imagine a heatmap that plays live over a video indicating the hotspots that most people view.

Multi-touch
I’ve been experimenting with some home-brew multi-touch hardware and an early AS3 api that allows me to incorporate a multi-touch interface in a Flex application. Multi-touch is currently a hot topic because of the iPhone and the Microsoft Surface. In terms of availability to developers, the Flash options for working with multi-touch are the most advanced and powerful. The AS3 API needs a lot of work to be ready for mass use and the hardware is still in its infancy, but the demos that we could build would blow people away.

Alternative menu systems
I’m interested in exploring alternatives to traditional menu systems within Flex applications. I’ve had great success implementing a radial menu in Flex in my TileUI prototype application, I would like to explore menus that size their items based on past usage, three dimensional menu systems, powerful shortcut-key navigation with Flex, and zooming menu interfaces.

3D interfaces
I have implemented a 3D interface for an experimental project called TileUI. This project used many available open-source AS3 libraries (most importantly PaperVision 3D and Actionscript Physics Engine). The result is an impressive prototype that shows off the amazing visual capabilities of the Flash Player. I would like to take the concept of 3D interface design more generally, and explore alternative 3D possibilities that would really demonstrate how we can create experiences far beyond traditional user interfaces.

I’m speaking at the Flash on the Beach conference in Brighton, England in September! I’m really excited about this conference because it’s not a Flex conference, it’s a Flash conference (but of course there will be plenty of Flex represented). I’ve got a little notoriety among you Flex developers, but Flash guys are like, “Doug McWho?”, so that’ll be pretty dope to get out in front of a different audience.

I’m going to be presenting on a yet-to-be-determined Flex topic, and I’ll likely ask for some suggestions as I’m refining my presentation ideas. Here are my current potential topics:

• super-sweet data visualization in Flex
  I’ve been reading lots of Edward Tufte books and other information visualization books. These are fantastic resources to help you think about effective ways of visualizing information, and I’m thinking about taking many of the principles and applying them to the Flex framework. This would involve the creation of a bunch of altered charting controls (new axis renderers, different chart types, etc). And I’d analyze what’s good and bad about the current set of data visualization components available (in the Flex SDK, the iLog component set, and the open source community). I’d probably throw in some brief mapping examples too.
• Maps, maps, maps
  I’ve been doing a ton of Flex mapping applications for work. Unfortunately a lot of that stuff can’t be shared (I certainly can’t show much of the code, and I can’t even show demos of some of it). But I’m thinking about doing a session that goes over all the different options for AS3 map components (we now have ESRI, MapQuest, Google, Yahoo, UMap, Poly9). A year ago or so ESRI was pretty much the only game in town, now there are a bunch. So I’m thinking about showing some demos of how to use the different map components, and some data visualization techniques for working with geospatial data.
• advanced open source Flex projects
  My presentation at the last 360|Flex conference in Atlanta was on using open source projects in your Flex apps. I’m thinking about doing a part 2 of that topic anhd going into more depth with a smaller number of projects. My last preso was sort of the shotgun approach (ie show many projects with only cursory information). This preso would be focusing on creating two or three demos that combine a handful of projects and dive deeper into the code and how I went about creating the demos.
• Decompiling Flex applications for fun and profit
  I’ve been playing a lot with SWF decompiling tools and ripping apart lots of Flex applications. There’s a lot you can learn from this (it’s not an inherently bad thing to decompile someone’s code). I’m thinking about stepping through the process of decompiling an app and talk about what you can learn, what decompiling actually gets you, how you can piece things back together, etc. I figured I could rip apart one or two main example apps (like Photoshop Express or Buzzword). I toyed with the idea of submitting this as a topic for MAX, just to see if Adobe would let me rip apart the Buzzword code base and show it off to everyone. But I figure at FOTB it’s all fair game.
• Using Flash 10 features in your Flex apps
  I’m considering a presentation that focuses on taking advantage of all the new features of Flash Player 10 (specifically in relation to the Flex SDK). I’m considering this topic because it would force me to stay on top of the new technology and know my shit. I haven’t yet played with FP10, so this would force me to do some cool stuff.

I’ll be holding down the Flex front with a few other guys, I know that Ben Stucki and Tink are both going to present on Flex topics. I’m also excited to meet some Flash peeps I’ve admired for a long time.

So if you’re looking for a European vacation near the end of September, then come to FOTB and hang out in Brighton! It’s gonna be sweet.

This is a quick post to about a few of the conferences I’m either speaking at, going to, or regretting not being able to attend.

360|Flex
When: August 18-20, 2008
Where: San Jose, CA

I can’t go! Nooo! I’ll be in Sweden. I might make it back in time to come and hang out with people at one of the last parties, we’ll see. Tom tried to convince me to fly from Sweden to Cali and give a presentation the next day and that just sounded like a horrible idea, so sadly I won’t be presenting at this 360|Flex. I really wanted to, I think 360|Flex is one of the best conferences, and it’s certainly where I’ve had the most fun. I’ve attended the first three 360|Flex conferences (all the ones in North America) and I spoke at the last two. And since I love the conference so much (but can’t present a session this time), I’ve made sure I’ll still make my mark on all attendees (more on that in a post tomorrow).

Flash on the Beach
When: September 28-October 1, 2008
Where: Brighton, UK

I’m presenting! w00t! I’m looking forward to this one, I get to fly over and kick it in England and present cool Flex shit to a bunch of Flashers. I’ll write a full post about FOTB shortly…

Adobe MAX
When: November 16-19, 2008
Where: San Francisco, CA

I’ll be attending MAX, but not presenting. I thought about submitting a session topic, seeing as MAX is literally in my backyard this year. But I’ve heard some horror stories about multiple required submissions months and months ahead of time, and that’s not really how I roll. I’ve been pretty overwhelmed with writing Flex for Dummies, so the MAX session topic submission just sort of felt like an additional ordeal I didn’t want to deal with. That said, I’ll definitely be there and we’re going to rock the shit out of San Francisco, so come and play.

I was contacted by Justin Ouellette, the creator of Muxtape, who asked that I remove the download functionality from MuxMaster. I’m torn about this issue, on one hand Muxtape is a service based on technology that makes distributing MP3s extremely easy. Justin has chosen to hide this capability because he’s appropriately scared of the consequences of having Muxtape turn into a file-swapping website. But essentially Muxtape is a massive dropbox for MP3s. On the other hand, I really do like Muxtape, and I don’t really want to be the one partly responsible for getting the site shut down.

Note that this isn’t a decision about legality (not for the legality of MuxMaster anyway) and I wasn’t coerced or threatened or anything like that. Justin just sent me an email saying he liked the app (and is cool with people making apps like this) but that the download feature endangers Muxtape. I agree, although I still don’t know whether I should feel obligated to remove the download feature.

MuxMaster icon
On a lighter note, Juan Sanchez whipped up a little icon for MuxMaster, which is now included in the latest version of the application. Pretty sweet huh? Thanks Juan!

You can still download MuxMaster (Lite) without the download functionality:

1. Make sure you have the AIR runtime, if you need to download it from Adobe
2. Download and run the MuxMaster installer

MuxMaster is an open-source desktop player to browse, play, and download Muxtape mixes. It is an AIR application that lets you stream music from Muxtape, explore different mixes to find new music, and even download tracks and entire playlists to your computer with a single click.

1. Make sure you have the AIR runtime, if you need to download it from Adobe
2. Download and run the MuxMaster installer

Here are a few screenshots:

Why did I make this?
I like Muxtape. Like a lot of software/web guys, I saw the clean interface and the barren source code and was immediately impressed. I mean, the dude did a fantastic job. It was just so damn clean.

Then a few days ago people went ga-ga over this video of some dude showing muxtape mixes using the Coverflow visualization to browse through them. OK, so that’s cool, except it uses a Mac-only application called Fluid to basically add normal Mac desktop functionality to web apps. That shit got dugg like 750 times. Now I’m not saying it’s not cool, cause it is. But I figured I could probably whip up a Flex application pretty quickly using the Coverflow component I released before and get a real desktop application that anyone could download and use. Oh, and I thought it would be neat to add downloading functionality too.

So I threw MuxMaster together over the past 3 nights. I like to think of it as a testament to the badassness of Flex and the open-source Flex/Flash community. The fact that I can grab some open source stuff and throw this together in almost no time is pretty cool.

What kind of an application is this?
MuxMaster is an Adobe AIR application, built with Flex. That means it’s a desktop application that can run on Windows, Mac, Linux, whatever. Click the “Install Now” link above to install the app. You might need to go and download the AIR runtime first. Hopefully it’ll tell you that and do everything automagically.

The Muxtape API
Muxtape has no API. But the HTML code is so sleek and beautifully clean. Go look at the source code for the HTML pages on muxtape.com. It’s ridiculous how sparse it is. And of course it validates as valid XML. So what? Well, nice XHTML like that is just as good as the best XML API you can ask for. You want to get a listing of some random Muxtapes? Just load up the XHTML on the Muxtape home page and parse out the list of users. Using E4X you can get a list of users with this single expression: body.div.div.ul.li.a. It’s almost too easy. So then when you get a username you just load that user’s HTML page and you can easily parse the list of songs (body.div.ul.li) or related users (body.div.div.div.ul.li.a). There’s a little trickery involved to figure out exactly how to construct the URLs for the MP3s, but it’s pretty easy to figure out. After a little parsing you can get a list of MP3 URLs and the data about each song, like artist, track title, duration, etc.

Downloading songs

The thing about Muxtape is that all the songs are just MP3s stored on Amazon S3. If you know the URL you can download the file. The main Muxtape player doesn’t expose this functionality, but if you take a look at the files that the web-based player is loading you can just copy/paste the URL and load it into a web browser to save the file. So I just made that a bit easier. MuxMaster has a Download link for each song underneath the song name. One click and the song will download in the background (you can specify where you want songs to download to). Additionally, each playlist also has a Download All link, so with a single click you can download all the songs in the playlist.

Is this bad?
There’s a whole ethical dilemma that we can spend forever discussing. Is this application against everything that Muxtape stands for (clean, simple, bare-bones UI)? Is adding download functionality quickening the shutdown of the service? Is this effectively taking a step toward destroying the service that I actually really enjoy using? Yeah, the answer to all those might be yes.

Yes, the excessive use of 3D and especially the over-used Coverflow visualization are exactly opposite to the perfectly clean design concept of Muxtape. The truth is, I hate Coverflow. But people gravitate toward it and seem to love it, so I figured it’d be fun to whip something up and see the response. And it was just so damn easy.

And yes, being able to download an entire mix with a single click may very well be a horrible thing for Muxtape as a whole. I assume the service is already on the radar of the lawyer-crazy music execs, and if it’s not now I assume it will be if it gets big enough. But the technology underneath the service was just so simple (I mean, really? just let people throw a bunch of MP3s up on Amazon S3?) that someone would have shown how to download all those songs soon enough. So I’ll definitely feel bad if this app has a negative effect on Muxtape, but come on, it’s something I whipped up in a few days, someone was bound to do it.

Source code
I’m releasing all the source code for the application. When you install the AIR application you can right click and select “View source” to see all the code. Or you can view the source here. It’s small (16 files, 898 total lines of code). And the code was influenced by the minimalist design of the Muxtap app itself, so I tried to keep things extremely tidy and clean.

I used a few open-source flex libraries. The most obvious one is the Coverflow Flex component that I created and released on my blog. That’s licensed under the MIT license, so you can take that component and add this type of Coverflow visualization to your Flex apps. The Coverflow visualization uses the Papervision 3D library, which is a 3D engine for Flash. It’s badass and free (MIT license). The other Flex library I used is FlexLib, which was used for the FlowBox container. That component can be seen when you download songs. It’s the container that lays out the little download rectangles. FlexLib is also licensed under the MIT license (is there a pattern forming?).

The source code for MuxMaster, like all code released on my blog, is also licensed under the MIT license, which you can read here. Basically you can do whatever you want with the code.

What’s next?
I don’t know if I’m going to do anything more with this application. The app does rely on the format of the HTML pages on Muxtape, so if the creator of Muxtape decides to try to break MuxMaster he probably can, and I’d have to update it to get it to work again. I might do updates like that to keep it working. If people have feature request I’d love to hear them, but I’d encourage other developers to take my source code and add the stuff that they want. If you have feedback shoot me a message using my contact form or leave comments here. I don’t have any big plans, this was fun to make.

Legal nonsense
This application doesn’t contain any music files whatsoever. I am not storing any music files on my server or any server that I am associated with. I am not providing a list of any music files stored anywhere on the Internet. I don’t have any information about how to find music files stored in any database or anywhere in the entire galaxy. If you’re a lawyer looking to scratch that soul-destroying litigious itch that you have, I’m the wrong guy to talk to.

Here’s a simple motion detection utility that you can use to do motion detection on a webcam, or video, or any other component in your Flex application. It uses a bitmap technique to compare image snapshots and calculate the number of pixels that were changed between snapshots. This gives you a percentage of the total pixels that were changed, which is a crude way to figure out motion. Basically what you do is draw one frame onto the previous frame using the “difference” blend mode. Then you threshold the image and you can figure out the number of pixels that had any change.

See the full example after the jump.
(more…)