In this post I’m going to dive into details about the stats of the FlexCoders mailing list usage over the past 5 and a half years. It’s full of graphs of various fun statistics, like who’s most active on the list, when people post, and the overall traffic over time. It’s a bit of a trip down memory lane, and I apologize if I ramble, I like data and pretty pictures, and I have a soft spot in my heart for FlexCoders, so bear with me and hopefully for those of you on the mailing list it will be a fun trip.

Background
I’ve been on the FlexCoders mailing for a few years now (my first post was back in September 2006). As the Flex community grew, the list grew, some would say it grew to unmanageable levels. It’s certainly a lot of mail, I currently have 22,100 unread flexcoders emails in GMail. At one point we even debated furiously whether the list should be split up into multiple more focused lists, or if the whole thing was going to die. Regardless of that outcome the flexcoders list remained as it has been for years. One thing did change though: Adobe replaced their official forum (which was literally God’s worst forum software) with a new one. And the Adobe employees definitely seemed to be pushing people there, which isn’t to say they stopped answering flexcoders questions, but the community was certainly now split between two lists.

I subscribe to both flexcoders and the Adobe Flex forums (which you can setup to receive emails from). I started noticing a trend. Take a look at this picture of my inbox (only flexcoders and Adobe forums emails) as of right now:

The orange label is used to tag the posts from flexcoders and the green label is posts from the Adobe forums. I started noticing that the number of posts from the forums were more than on flexcoders. That obviously made me wonder if the overall traffic on flexcoders was in decline. I’ve been inactive on the list for quite a while (been quiet for most of 2009). So I didn’t exactly have my finger on the pulse of flexcoders.

The Data
So I wanted to download the entire Yahoo group dataset to start playing with it. Turns out Yahoo doesn’t make this easy, but I found a sweet program called PG Offline that I used to pull down the entire list. It took me a few days to get all 148,826 messages (as of about 7pm tonight). But PG Offline worked incredibly well and I then had an Access database file with all the emails (it was about 1.5 gigs). I then used another program called MDB Converter to convert that to a text CSV file.

If you want to play with the data yourself you can download the CSV file (11 megs). It includes columns for the sender, date, and subject. I did not include the full-text of the emails, since that would make it a gig and a half.

Analyzing the Trends
I pulled the data into SpatialKey (which is what I work on for my day job) and started digging into the data. Here’s the report setup I created in SpatialKey to play around and filter down the data (click for a larger view):

So we can start seeing the overall trend in the main timeline, which shows the rise and fall in traffic.

So there certainly has been a decline in traffic to the list. The most active month ever in the list’s history was March 2008 with 3,834 posts. And then it’s been a fairly steady decline since that peak.

Some other interesting high level stats are the hours of the most activity. This chart shows the number of posts by hour of day. Hour of day is Pacific time.

You can clearly see the work-day hours there. 8, 9, and 10am are the most active, and then it slows down as the work-day finishes up (earlier for east coast people), and then there’s another small bump around 9pm in the evening.

Who’s Most Active?
Anyone who reads FlexCoders knows that Alex Harui (from Adobe) is the king. Here are two charts showing the top 10 posters of all time and the top 10 from just 2009.

Alex certainly still holds the number one spot overall, but Tracy has him beat for this past year.

Diving into Individual Activity
It’s also pretty interesting to look at how different individuals use flexcoders, and how their usage has changed. Here are just a few selected people that I was curious about:

Alex wasn’t always the king. He had a few messages back in 2005, but his heavy involvement on the list actually started relatively late, in March of 2007 (which is also when he started blogging coincidentally).

Tracy Spratt, on the other hand, has been on the list since its very beginning:

Matt Chotin (Flex product manager) has also been active since the list started:

Actually, Matt Chotin and Tracy Spratt are the only two people who have posted to the list at least once every single month since the very beginning (from April 2004 to now). They get the FlexCoders Lifetime Achievement Award!

Some people were around in the early days but then dropped out. Here’s Jesse Warden’s activity:

Some people get sucked into the list fast and then fizzle out. Josh McDonald was the third most prolific poster of 2008, but then stopped posting as quickly as he started:

And some people stop posting when it’s no longer part of their job, like Roger Gonzalez who worked for Adobe and left in March 2007 (which was also the last time he posted to the list):

Ely Greenfield (Principal Architect at Adobe working on Flex 4) used to be fairly active back in 2006/2007, but hasn’t said a word in the past two years:

And what about me? I was fairly active on the list from about 2007 through the beginning of 2009, then pretty much radio silence:

And some people don’t live in the USA and post at completely different times. Here’s Tom Chiverton’s (4th most prolific poster of all time) usage pattern by hour of day and day of week. It groups the posts by the combination of what day and what hour they occur on.

At first glance it looks like Tom emails the list in the middle of the night, until you realize that he lives in England

I’ve had a lot of fun drilling into the history of this list. It’s really cool what kinds of trends you can find (probably another post in more detail on that later).

Want to play with the data?
You can download the complete CSV file and use it if you want. I’d love to see people turn it into much more interesting visualizations. This dataset goes up until November 2 2009. Since it’s a bit of a pain to keep it updated I probably won’t update it very often, but if there is interest I might do so once a month or so.

Notes on privacy
All this data is public, you can see it all by going to the Yahoo group and searching. There are no email addresses in this data (unless perhaps if someone used their email address as their name as well). Any names in this data are there because the person knowingly emailed the public flexcoders email list. This CSV download is obviously a much easier format to work with all the data, and it can certainly be mined for interesting trends. I just ask that people play nice with the data. We’re a community, and this is data that represents our lives (or at least one small sliver of our lives) for the past 5 years.

Happy Halloween! This is the dorkiest pumpkin I’ve ever carved. For those of you into data visualization or mapping, maybe you can recognize it:

This is a pumpkin representation of Charle’s Minard’s visualization of Napoleon’s march into Russia in 1812. This graphic is considered by some (ie Edward Tufte) to be the “best statistical graphic ever drawn.” The graph shows the size of Napoleon’s army as they marched to and from Moscow. You can see how the army shrank as they approached Moscow. Once they reached Moscow they found the city had been abandoned and burned. Then they marched back home, except it was through a brutal Russian winter and nearly killed the remaining army. By the time they return home you can see the size of the army is just a small trickle.

Beyond just the two charts of the march to and from Moscow, the graphic also serves as a map, with the paths indicating where the troops were geographically. And below the map is a temperature chart that visualizes how severe the winter weather was, which correlates with some of the major drops in troops on the way home.

The carved pumpkin ended up being very hard to take a photo of because the graph wraps around over half the pumpkin’s circumference. So I tried to take a few pictures to get the different sides. I carved the march to and from Moscow, as well as the temperature chart along the bottom.

Hope everyone has a great Halloween tonight!

We just posted a new example of using SpatialKey to visualize crime in San Francisco. We load in 90 days of crime data from the city, then filter down to only include sales of heroin, crack cocaine, and methamphetamine within 1,000 feet of a school. Why those particular crimes around schools? The SFPD just launched a new initiative called “Operation Safe Schools” that specifically targets these drug crimes. If you’re caught dealing crack, heroin, or meth around a school while the school is in session you can get extra prison time.

Check out the video below and read the full article on the SpatialKey blog.

Read the whole article on the SpatialKey blog to see how we put this together and learn more about the SFPD’s “Operation Safe Schools.” You can also watch the full resolution video on YouTube

User Experience Design (or UX for short) has exploded on the software scene carrying a blazing torch of freedom and promising to guide us to the holy land. We are now dedicating time and resources specifically to user experience design. This emphasis on actually designing user interactions and the experience of our applications is a fantastic effort and I applaud all those that wave the UX flag and sing its praises. But I have one problem: UX is making me a worse software developer.

An unfortunate byproduct of the UX revolution is the misinformation that developers simply can’t design good experiences. We see article after article explaining what happens when you leave developers in charge of UX design. People laugh at how stupid developers can be, that they just don’t “get it” when it comes to designing things well. These articles should highlight the importance of explicit thought when designing interactions in applications, they should not strip developers of their confidence and creativity. And yet that’s exactly what I’ve noticed happening to me.

Let me relate a brief hypothetical example. On our software project I am the lead clientside developer. We have a UX Specialist (aka UX Guy) that is responsible for the design of wireframes and mockups that explain how the user navigates through the application. During the UX pass, our UX Guy listens to us (developers and business guys) explain what the problem is and what we think we want the software to do. Then UX Guy goes off and comes back with a set of wireframes for how the application will work. We discuss, bring up problems, send back the wireframes for further iteration. Repeat that process until everyone is satisfied with the wireframes, then move on to actual development of features.

But then during development I have found myself getting the following question from my manager: “Hey Doug, how come you can’t do Feature X in the app?” and I have found myself actually saying this (shudder): “Oh, I guess that wasn’t included in UX Guy’s wireframes, you should ask him.” That’s a bullshit response. I’m not a neutered incompetent mindless developer. If something doesn’t work or was forgotten, I can figure out how to fix it. That doesn’t mean throw the code in without thinking about design. That means pausing my code writing, thinking through the problem carefully, designing a solution that will be sleek and elegant, and then continuing on with development.

And yet in my mind I have the unconscious prejudice that I, as a developer, cannot be allowed to make “UX” decisions. Fuck that. We are all responsible for designing the experience of our software. We are all responsible for carefully thinking through every interaction. We are all responsible for making things beautiful, making things simple, making things elegant. And more than that, we are all capable.

Tom Ortega and John Wilker, who organize the kick ass 360|Flex conference, are teaming of with Joshua Cyr, who organized the first RIAdventure cruise. They’re planning a new event that will be a 7-day cruise that features a 2-day conference. The cruise, called RIAdventure 360, is going to leave from New Orleans and hit Mexico, Guatemala, and Belize.

The conference will take place December 6-13, 2009.

Only new, forward-looking content
The speakers are required to prepare new content specifically for RIAdventure 360. That means no canned presentations that you saw 5 times already at past MAX conferences. The content has to be fresh, it has to be cutting edge, and it has to be made specifically for this event. This requirement is awesome. I think that all presenters at all conferences should prepare fresh content every time. Sure, it’s more work, but if you can’t prepare fresh shit for a conference then you should present less often. But of course that’s just a personal opinion with no disrespect meant to anyone on the speaker circuit

The speakers for this fine event will be: Ted Patrick, Ryan Stewart, Faisal Abid, Samuel Rivello, and… me!

Focus on the future of RIA
This is a conference about the future, not the present. I’ve been formulating some ideas about what I’d like to present, but I haven’t quite solidified my plan yet (hell, if the conference is about the future I can’t plan it too far ahead, everything will change!). I’m currently playing with some ideas about the future of data visualization in application development. We’re reaching the point where data storage and bandwidth have reached levels capable of transmitting massive data sets, but we haven’t yet understood the implications this has for application developers. What if you had access to the location data of all the cell phones in the country? Or your entire genome? Or the record of every dollar ever spent by the government? The data is coming soon (much of it is here already), but if we, as RIA developers, don’t have the skills necessary to make sense of it all we’ll miss the opportunities to truly innovate and impact the world.

One big party!
And of course this is a cruise after all! We’re going to be partying it up like it’s nobody’s business. The conference also includes dinner discussions about our industry, which I think roughly translates to getting Ryan and Ted drunk enough to spill some Adobe secrets.

Get a free 360|Flex ticket!
And if all that wasn’t enough Tom and John are also throwing in a 2-for-1 deal. If you sign up for RIAdventure you also get a free ticket to the next 360|Flex in February. So if you want to have a hell of a good time while geeking out, go sign up for the RIAdventure cruise.

Here are the slides and the code samples I presented at CFUnited.

You can also download the PDF of the slides if you want.

Adobe Open Source Media Framework + FlexLingo
This demo uses Adobe’s OSMF project to synchronize subtitles with video. This was the demo that I used to create the Hitler Goes to CFUnited video I posted before. You can seek around the video and OSMF makes sure to show the right subtitles. Then you can click on the different flag icons in the demo to use the FlexLingo library to translate the subtitle text to other languages (using Google translation API).

View the source
This demo is made with Flex 4, so to use the source code you’ll need to be able to build a Flex 4 project.

Axiis + Tweetr
The second demo I showed used the Axiis framework to create a chart of trending Twitter topics that have associated pictures on TwitPic. Unfortunately there’s no crossdomain file in place for TwitPic images, so I can’t run a live version of this app (but running locally works fine).

View the source
This demo is also made with Flex 4, so to use the source code you’ll need to be able to build a Flex 4 project.

Safe Sexting
The third demo I showed was the “Safe Sexting” app that uses the Marilena library for facial detection and uses Pixel Bender to automatically blur out your face. This demo was also shown in 360|Flex Indianapolis. You can read a bit more about this demo in a previous post about the 360|Flex presentation.

Run the demo | View the source

Here’s the video that I showed at CFunited yesterday of Hitler (as a Flex developer) finding out he’s speaking at a ColdFusion conference.

A little context for those who didn’t attend my session: I was doing a demo of how to use the Adobe Open Source Media Framework that involved creating a Flex app that added subtitles to a video. The demo also showed tying in another open source library to automatically translate the subtitles into other languages (so the demo wasn’t just completely silly). Also, the demo I showed after this one was using Axiis for data visualization (hence the reference to the data viz demo in the video).

And for those of you who didn’t attend CFUnited, I assume this won’t be nearly as funny, but maybe you’ll still get a kick out of it.

This Friday, August 14, I’ll be speaking at the CFUnited conference in Virginia. My session is titled Riding Coattails to the Top: Using Open Source Flex and ActionScript Projects (yeah, I know my session titles are getting too verbose) and will be all about using the thousands of open-source Flex projects that are out there in the community. I’ll be covering projects like Axiis, Adobe’s Open Source Media Framework, computer vision libraries, and a host of other cool projects that you should know about.

I’m preparing a handful of demo applications that I’ll be showing off as examples of integrating some of the coolest open-source work out there. The goal of each example is to show how to integrate at least two open source projects together to create a sweet application. The screenshots below are from a few of the apps that I’ve been working on, just to give a little taste. If you’re going to be attending CFUnited be sure to stop by my session on Friday at 1:30.

If you’re going to be at CFUnited be sure to flag me down and say hi or find me at the bar on Friday night!

James Ward and Jon Rose just published the latest episode of Drunk on Software that features members of the SpatialKey team, including myself, Tom Link (CTO of Universal Mind), and Brandon Purcell (Director of Technology for UM).

As a brief disclaimer in case I slur any words near the end: I was in Denver for a short trip and we squeezed in a time to meet with Jon and James right before I had to head to the airport to fly home. The only problem was that we had to meet at about 10am in the morning. And since the show is called Drunk on Software we obviously had to be drinking. So by the time I got on my flight I was probably 6 beers down

A big thanks to Jon and James for making the time to have us over (that’s the living room of Jon’s house). And thanks for the beer guys!

Another 360|Flex has come and gone and I’ve returned home with my liver and dignity partially intact. This post contains the slides from my presentation as well as all the code for the examples that I showed during the presentation. The slides are embedded below or you can download a PDF of them here.

The problem with these slides, however, is that if you read them out of context a lot of it probably won’t make that much sense (and some of it may very well be misunderstood completely). So I’m planning on doing a follow up post shortly after this that will try to put my slides in context by providing some notes about what I was talking about when I was showing each slide. So don’t look at the slides like the one that says “Fuck Flex” and jump to any conclusions. I’m not ditching Flex development, I wasn’t angry, and I wasn’t just trying to shock people with swear words on slides. If you saw my presentation then I hope it all made sense and I hope some of it resonated with you. If you didn’t make it to my session then I’ll be posting a few followup posts about the main topics that I covered. I’ll be doing some posts that go into detail about steganography, head tracking (including detailing some of the optimizations I came up with), and augmented reality (hopefully with some good video).

Slides
UPDATE: Turns out SlideShare’s embeddable player is throwing errors, so you can view the presentation slides directly on SlideShare here.

Or download the PDF of the slides.

Code
Here’s all the code of all the examples that I showed.

• STEGAsaurus
  This is an AIR app that hides secret data (either text or files) in PNG images. It uses the PNGEncoder class in the Flex framework and the PNGDecoder class written by Heriet. When you run the app it allows you to drag and drop an image file to either encode secret stuff or to decode secret stuff that has already been encoded in the image (it can only decode PNGs created with the app itself).
  Download the AIR installer | Download the source
• Modified Marilena head tracking library
  The Marilena library is part of the libspark project and it is a port of a facial detection algorithm from OpenCV that uses a technique called Haar Cascades. I used an optimized version by Mario Klingemann as the base that I started with, and then I made further optimizations from there. The main optimizations that I made have to do with not having to rescan the entire image every pass (since we’re concerned with live webcam tracking) and also checking for different rotations of the face to allow the user to turn his or her face at an angle.
  Download the source
• Head Tracking Targets
  This was a simple demo that uses Flash Player 10 3D stuff (no PaperVision or 3D library used). I was trying to recreate the fantastic demo that Johnny Lee did that uses the Wiimote to detect head movement. This demo should detect your face and move the targets as you move your head around.
  Run the demo | View the source
• Crappy Top Gun game (controlled with your head)
  This is another demo that uses head tracking. This one is a little Flash game that recreates part of the original Top Gun NES game. You control the plane by moving your head in the direction you want to fly and the objective is to avoid getting hit by rockets. It’s a really crappy game and isn’t very fun to play, but it was intended to just be a proof of concept.
  Run the demo | Download the source
• Safe Sexting
  This is another head tracking demo that can automatically blur out your face. It’s sort of a Photo Booth type of application that lets you take pictures of yourself. The general concept is that it’s for all those young teenagers out there who are getting in trouble for sexting. Being the good citizen that I am, I wanted to come to their aid. This app will let you take dirty pictures while concealing your identity!
  Run the demo | View the source
• Augmented reality demo with fake chests (male and female)
  And for the finale of the presentation I pulled off my shirt to expose an augmented reality marker on my chest. I then “augmented” myself with a 3D muscular male chest that made me look like a bodybuilder, and then to top it off I threw some fake 3D breasts on my chest (all while simultaneously blurring my face with the Safe Sexting app). If you want to try the augmented reality stuff out you’ll need to print out this marker and hold it up the the camera (or tape it to your chest for the full effect).
  Run the demo | View the source

So that’s all the slides and code. I’ll be putting together a series of blog posts that go over each of the individual topics I covered in much more detail. And like I mentioned, I’ll be doing a post that puts the slides in context for those of you who didn’t get to see the presentation live. I think there might also be some video floating around of most of the presentation, so if I can get my hands on that then I’ll be sure to post it.

Links
In addition to the slides, here are the direct links to all the videos or demos that I referenced in my slides:

• Lee Brimelow’s dead drop (and the explanation here)
• Johnny Lee’s Wii experiments
• Boffswana head tracking demo
• Mr. doob’s head tracking demo
• Funny video review of Top Gun for NES
• Grant Skinner’s webcam experiments with snow and fire
• Boffswana FLARToolkit demo
• JackLinks Sasquatch AR example
• GE SmartGrid AR example
• Topps AR baseball cards (video here)
• AR game with characters shooting each other (video)
• Weird AR dolls from Japan

Were you there?
I’d love to get some feedback on how you thought the session went. Was it engaging? Was the technical content good? Too technical/not technical enough? Let me know in the comments or feel free to shoot me an email: doug.