Maps, SpatialKey

Crime Maps on the Guardian Powered by SpatialKey

I’m happy to announce a new crime mapping application I’ve been working on that just went live on the Guardian DataBlog. The app lets you compare different cities in England to see where crimes of different types are distributed. You can either compare two cities side by side, or two different crime types in the same city. So if you’ve ever wondered which areas of London have high amounts of violent crime but low amounts of burglary, now you can find out.

This custom app was built upon SpatialKey, which made cranking it out only take a matter of days (the whole thing start to finish took about 4 days).

Standard
Art, Data Visualization

Iraq Death Dots – Visualizing Each Death in the Wikileaks Iraq War Logs

What would 108,394 deaths look like?

I’ve been combing through the Wikileaks Iraq War Logs dataset and experimenting with different visualizations. This new one shows each individual death logged in the data. A single death is drawn as a single dot. The color of the dot indicates who was killed: either a civilian, a coalition soldier, an Iraqi soldier, or an enemy combatant. These classifications are taken directly from the military records, I did not categorize the data myself in any way. The dataset documents exactly 108,394 deaths, so exactly 108,394 dots are drawn.

Coalition soldiers are white dots, Iraqi forces are gray dots, enemy forces are blue dots, and civilians are red dots. At a glance you can see the shift from the heavy blue in the early days of the war to the overwhelming red. Let that soak in for a second. Every red dot is a civilian life.

The live visualization is embedded below. Or view a larger standalone version.

This movie requires Flash Player 9.

Explore the visualization by selecting a different tab along the top (Years, Months, Incident Type, Category, Casualty Type) or by using the plus and minus buttons to zoom in and out of the visualization. I encourage you to experience the visualization in full screen (use the full-screen button on the bottom-right).

The data

This dataset uses the dataset produced by the Guardian, which filtered the full WIkileaks dataset to only include records with one or more deaths logged. It contains 52,048 records that document 108,394 deaths.

Please note that this data only contains incidents documented by Multi-National Force – Iraq and presents only a partial, incomplete record of the war. Please see this article about issues with this dataset.

Inspiration

This work was inspired by Kamel Makhloufi, who created some fantastic images that colored individual pixels by the type of casualty.

Standard
Art, Maps, SpatialKey

Night Vision Maps of the WikiLeaks Iraq Casualty Data




In 1990 I was an eight year-old kid. And like most eight year-olds I spent a lot of time in front of my TV. But the summer of 1990 was different. Instead of cartoons I was watching the first Gulf War.

The television media coverage of the war was everywhere. Except these weren’t the gruesome images of the Vietnam era. These were images that looked more like videogames. We had cameras attached to bombs that used night vision and targeting scopes as they dove into buildings. All the images were a bit fuzzy, a bit grainy, either tones of gray or green, and overall void of emotion.

But we were watching people die.

The disconnect between the emotionless images shown on TV and the reality that they represented has always stuck with me. The fact that we could (and still do) present something so horrible in such a clinical, disconnected way makes my head spin.

WikiLeaks Iraq data

I’ve been experimenting with mapping the recently released data from WIkileaks that documents deaths in Iraq. All told the data documents 108,365 deaths, which we assume are just a fraction of the true casualty count from this war. Of those deaths, 65,641 were civilians.

I’ve used SpatialKey to produce some heatmaps of these deaths by recreating the aesthetic of the night vision images we’ve grown so used to seeing. I downloaded the data from the compiled spreadsheet published by the Guardian. Each image has a high resolution version available (2,474 pixels by 1,419 pixels).

A view of the entire country


High resolution version

A closer look at the area of Baghdad


High resolution version

More details of Baghdad


High resolution version

Why?

These images are meant to be a bit provocative. Every tiny blurred dot represents someone dying. And yet it’s all presented in a way that everyone is comfortable with. When you glance at these images you don’t immediately think of killing. We’re so used to seeing emotionless, blurry images of rockets exploding and precision bombs targeting buildings that we disconnect the image from the reality. These are images of death. And the fact that we’re comfortable looking at them should give us pause.

Standard
Data Visualization, Maps, SpatialKey

Take the Tangent – Video of my 360|Flex Keynote

I was honored to be asked to give a keynote presentation at 360|Flex in DC last month. All the sessions were recorded, and John Wilker was gracious enough to let me post the full video of my keynote.

This keynote was a bit different. I went out on a limb a bit and talked about the experimental projects that I’ve been working on, and my belief in the importance of pursuing fun experiments to stay invigorated and passionate about our work. It covers a number of mapping and data visualization projects I’ve been playing with, but the point was really that we all need to pursue what we’re passionate about. For me that happens to be maps right now, but everyone has their own unique areas of interest.

If you’re interested in mapping work then the projects I talk about should be right up your alley. But even if you’re not a map geek, I think the presentation is still interesting and (I hope!) inspirational.

You can also see the slide deck on its own, but I think the video gives much better context to the slides.

Standard
SpatialKey

SpatialKey on ABC News in Salt Lake City

I just found out a local news story in Salt Lake City featured SpatialKey and the work we’re doing with the Ogden Police Department. Pretty sweet seeing your code come to life on TV 🙂

Here’s the video:

Standard
Art, Data Visualization, Maps

If San Francisco Crime were Elevation

I’ve been playing with different ways of representing data (see my previous night lights example) and I decided to venture into 3D representations. I’ve used a full year of crime data for San Francisco from 2009 to create these maps. The full dataset can be download from the city’s DataSF website.

A view from above

This view shows different types of crime in San Francisco viewed directly from above. The sun is shining from the east, as it would during sunrise.

top_500

I love how some of the features in these maps are pretty consistent across all the crime types, like the mountain ridge along Mission St., and how some of the features only crop up in one or two of the maps. The most unique map by far is the one for prostitution (more on that further down).

An alternate view

Here’s the same data but from a different angle, which helps show some of the differences.

UPDATE: Whoops, I screwed up originally and had a duplicate image. The original graphic showed the same map for Vandalism and Assault (both were the Vandalism map). This updated graphic has the correct map for Assault.

right_500_2

Many of the maps have peaks in the Tenderloin, which is that high area sort of in the north-east center area of the city. Some are extremely concentrated (narcotics) and some are far more spread out (vehicle theft).

My favorite map is the one for prostitution (maybe “favorite” is the wrong choice of words there). Nearly all the arrests for prostitution in San Francisco occur along what I’m calling the “Mission Mountain Ridge”, which runs up Mission St between 24th and 16th.

EDIT: I’ve been corrected. Upon closer inspection the prostitution arrests are peaking on Shotwell St. at the intersections of 19th and 17th. I’m sure the number of colorful euphemisms you can come up with that include the words “shot” and “well” are endless.

I love the way the mountain range casts a shadow over much of the city. There’s also a second peak in the Tenderloin (which I’m dubbing Mt. Loin).

prostitution_500

Drug crimes are also interesting to look at, since so much of the drug activity in San Francisco is centered in a few distinct areas. We can see Mt. Loin rising high above all the other small peaks. The second highest peak is the 16th St. BART peak.

drugs_500

There are other consistent features in these maps, in addition to Mt. Loin and the Mission Range. There’s a valley that separates the peaks in the Mission and the peaks in the Tenderloin, which is where the freeway runs (Valley 101). You’ll also notice a division in many of the maps that separates the southeast corner. That’s the Hunter’s Point Riverbed (aka the 280 freeway).

Disclaimer

These maps were generated from real data, but please don’t take them as being accurate. The data was aggregated geographically and artistically rendered. This is meant more as an art piece than an informative visualization.

Standard
Uncategorized

Nate Beck’s Birthday Surprise at 360|Flex

Payback’s a Queen!

Nate got a special surprise in the middle of his session at 360|Flex. This should teach you a) don’t fuck with me and b) don’t do a presentation on your birthday.

Happy birthday Nate!

P.S. Apologies for the shaky camera work, there were plenty of other video cameras in the room recording (including the official tripod camera), so there will be a bunch of copies of this video up soon I assume.

Standard
Uncategorized

What do you do with a giant head?

You make it vacuum your floor of course.

Inspired by this fantastic piece of work by Eric Testroete, my friend and I created my very own paper craft giant head (my buddy Todd did all the hard work of the 3D modeling and texturing). Of course, once you have such an amazing giant head, you need to figure out what the hell to do with it. And so boredom on a Friday night plus a few beers plus a giant head plus a roomba equals a magical vacuuming head!

A few more pictures of the head in action:
giant_head

gianthead2
(*not my baby)

As I figure out more shenanigans to get into with my giant head I’m sure I’ll post more ridiculous photos and videos.

Standard
Data Visualization

The New York Times Without Flash

I’m in love with the New York Times data visualization/infographics division. They consistently put out some of the most amazing visualization pieces (both in print and online) that I’ve ever seen. Their recently geographic analysis of Netflix ratings was absolutely superb. And we all probably saw their election maps (either for 2008 or 2004). They produce stunning displays that convey amazing amounts of information in a way that only interactive graphics can do. And they’re all done in Flash.

nyt_netflix
A Peek Into Netflix Queues





And for even more check out the NYT’s selected infographics list or simply do a Google search for “interactive graphic” on the New York Times website.

flashSo when you see images showing the missing plugin icon on the New York Times website on the iPad or iPhone, that’s not just some annoying ad that’s not playing or a streaming video. That’s some of the most cutting edge visualization work that’s being produced today. And without Flash it simply doesn’t exist.

Sure, you might be able to recreate some of these without using Flash (I’d argue that many you simply would never be able to do, but that’s for another debate). But the point isn’t whether or not you could eventually do it without Flash. The point is that the New York Times does them all with flash. So we need to ask why. It’s not an accident or an arbitrary technology choice. Newspapers operate on a schedule and a budget (and one that is getting tighter and tighter). The simple truth is, creating amazing visualizations like you see on the NYT website is possible and easy with Flash. They use the tools that get the job done most efficiently and produce the best end result. This isn’t an argument about whether it’s theoretically possible to create these types of visualizations without Flash, it’s about whether it’s being done. And save for handfuls of examples, it’s not (for every one good JavaScript visualization I’ll show you ten good Flash ones). Taking away the New York Times’ ability to use Flash is setting their data visualization department back 5 or 10 years. And it would mean that we, as readers and citizens, would be missing out on some of the most important journalism being produced today.

The New York Times (like all newspapers) is in crisis. They are trying to reinvent themselves in an online form. And as a news organization they are one of the most progressive and experimental out there. They are embracing the new medium by doing some of the best damn interactive graphic work I’ve ever seen. They make things that convey news and information in ways that draw people in and keep them coming back for more.

But without Flash they’re just a newspaper. And we all know newspapers are dying.

Standard
Data Visualization

Recreating William Playfair’s Import/Export Charts

I’ve been reading William Playfair’s Commercial and Political Atlas, in which he invented the line chart. In the book, Playfair examines the imports and exports between Britain and various countries. To illustrate these trade relationships, Playfair created the first ever line charts that show the change in trade over time.

The Inspiration

Each section of the book covered a different country, and each one contained a chart that showed the imports and exports like this:

playfair_north_america_trade2
playfair_ireland
Two line series are shown, one for imports and one for exports, and shading is used to show when there was a “balance in favor of England” (when there were more exports than imports).

My Recreation

I’ve been captivated by these charts and wanted to recreate them, but with modern data. You can find tons of US trade data at the US Census Bureau’s website, including a spreadsheet that has all the data in one place. I downloaded that data and put together a little application to create Playfair-esque charts.

Click this screenshot to play with the app yourself:
playfair_app_screenshot
View source is enabled.

The app displays all the countries that the US has trade data for, month by month going back as far as 1985. Each country is displayed in the list on the left with a sparkline chart of the trade data. A red fill indicates we are importing from a given country more than we are exporting, and a light green fill indicates we are exporting more than we are importing.

Exploring the data

The charts tell some really interesting stories. Some of the charts show a nearly identical relationship of imports to exports, both growing at the same rates, like these charts of the UK and Guatemala.
united_kingdom
guatemala

While some other charts show different relationships. Notice how exports to Hong Kong have been steadily increasing, but imports from Hong Kong have been declining.
hong_kong

Or we can see what imposing sanctions on a country looks like, as illustrated by sanctions on Burma that were put into place in 2003:
burma

Or what a coup in Haiti looks like:
haiti

Or what a massive tsunami can do to a place like the Maldives:
maldives

We can see the massive growth of China (and notice how interestingly seasonal each year is, peaking in October):
china

And one final one that I find very interesting, isn’t a country, but the import and export of what is classified as “Advanced Technology Products“, which includes things like biotech and advanced electronics products. Notice how up until the early 2000s we were exporting more of these products than we were importing, but by 2002 that balance shifted and the gap continues to increase:
advance_tech_products

I had fun creating this app, but one thing I didn’t expect was how much fun researching the charts was going to be. The charts that stuck out with trends that were abnormal all had interesting stories to tell about the history of the country.

In closing, I’ll end with a quote from Playfair in which he describes the concept of displaying numeric values in a line chart (remember, he was the first person to actually do this):

As the eye is the best judge of proportion, being able to estimate it with more quickness and accuracy than any other of our organs, it follows, that wherever relative quantities are in question … this mode of representing it is peculiarly applicable; it gives a simple, accurate, and permanent idea, by giving form and shape to a number of separate ideas, which are otherwise abstract and unconnected.

Well said, Mr. Playfair, well said. Your charts are just as effective nearly 200 years later.

Standard