I just found out a local news story in Salt Lake City featured SpatialKey and the work we’re doing with the Ogden Police Department. Pretty sweet seeing your code come to life on TV

Here’s the video:

Images courtesy of the Image Science & Analysis Laboratory, NASA Johnson Space Center

As I was flying back home into San Francisco airport I was watching the city lights out the window and got struck by a bit of inspiration. I find cities beautiful, from the graffiti to the neon signs to the line of headlights on the highway. A city viewed from above at night is captivating. I wanted to try to recreate that same look, but by visualizing data (in one sense you can say that the real view of a city from above is already a visualization of population data).

I started searching for images of cities at night, and found these amazing images from NASA. All those images were taken from a space shuttle orbiting the earth. These images tell you a lot about the city, the layout, urban density, planning (or lack thereof). I wanted to take other meaningful data and create similar images.

All the visualizations below have been created with SpatialKey. However, this is some experimental work I’ve been playing with to generate the “night light” images, so it’s not released (and might not ever be). Basically this is a peak behind some of the R&D work I do for fun (yes, for a dataviz dork like me making fake “cities at night” images is my idea of fun).

Crime in San Francisco
This image is all crime in San Francisco for a 3-month period. You can see some of the same features that you can see in the NASA space image, such as Golgen Gate Park and the Presidio (the area on the north-west edge of the city). All in all it’s interesting how similar the crime image looks compared to the NASA image. Downtown is the brightest spot in both images, which means that it’s literally the brightest area of the city (the most streetlights), and also has the most crime.

And here are breakdowns for a few different crime types. Notice how different the distributions are. Narcotics crimes are heavily clustered and can be found downtown (in the Tenderloin), in the Mission (near the 16th St BART station), and along Haight Street near Golden Gate Park. Whereas vehicle theft is scattered fairly evenly throughout the city.

Narcotics

Theft

Vehicle Theft

Burglary

Graffiti Reports in San Francisco and New York
Both San Francisco and New York publish their 311 data, which is when citizens call for city services. One category of 311 calls is to report graffiti. Graffiti is interesting in that it often follows specific city streets. When we look at the graffiti data for both cities we see specific streets that have far more graffiti than others. I love these images (particularly the one of SF) because they really look like a view of street lights from a plane.

Trees planted in San Francisco
Another one of my favorites of this set is data for all the trees that the city of San Francisco has planted since 1990 (all this SF data is available at datasf.org). You can see the heavy planting along Market St (which cuts diagonally through downtown), as well as along streets like Sunset Blvd (the street running north/south on the western side of the city).

Street lights (or SF as a giant lite-brite)
One final image of San Francisco we have is the locations of every street light in the city. I liked this image because it reminded me of playing with a Lite-Brite when I was a kid. It almost makes city planning feel light a grown-up version of playing with little plastic lights.

We just posted a new example of using SpatialKey to visualize crime in San Francisco. We load in 90 days of crime data from the city, then filter down to only include sales of heroin, crack cocaine, and methamphetamine within 1,000 feet of a school. Why those particular crimes around schools? The SFPD just launched a new initiative called “Operation Safe Schools” that specifically targets these drug crimes. If you’re caught dealing crack, heroin, or meth around a school while the school is in session you can get extra prison time.

Check out the video below and read the full article on the SpatialKey blog.

Read the whole article on the SpatialKey blog to see how we put this together and learn more about the SFPD’s “Operation Safe Schools.” You can also watch the full resolution video on YouTube

James Ward and Jon Rose just published the latest episode of Drunk on Software that features members of the SpatialKey team, including myself, Tom Link (CTO of Universal Mind), and Brandon Purcell (Director of Technology for UM).

As a brief disclaimer in case I slur any words near the end: I was in Denver for a short trip and we squeezed in a time to meet with Jon and James right before I had to head to the airport to fly home. The only problem was that we had to meet at about 10am in the morning. And since the show is called Drunk on Software we obviously had to be drinking. So by the time I got on my flight I was probably 6 beers down

A big thanks to Jon and James for making the time to have us over (that’s the living room of Jon’s house). And thanks for the beer guys!

All of us at Universal Mind who have been hard at work on SpatialKey are incredibly proud to announce that we have begun a private beta program.

To really launch this product in style, we’ve put together this video to give an overview of the SpatialKey system. Even after watching it a dozen times I get blown away every time I watch it. Make sure to check it out in all its HD full-screen glory!

We’ve also got a whole series of videos that go into much more detail of all the different features. If you spend a few minutes watching some of those we’re betting you’ll start brainstorming ideas about how you can use SpatialKey with your own data.

This private beta period means that you fill out the form requesting access and we will be adding accounts on a first come first served basis over the next few weeks. We will add accounts as fast as we can over the following weeks and then after this private beta period we will open the system up to a full public beta, at which point anyone will be able to sign up for a trial account. SpatialKey will be a software as a service product that you’ll pay for with a monthly subscription based on your usage and data needs (pricing details will be coming very shortly, for now all accounts are a free trial with a cap of 10,000 rows per dataset).

I’ll be posting much more about SpatialKey and what you can do with the software, but until then go check it out and let us know what you think!

SpatialKey has been a long time coming and it wouldn’t be possible without the fantastic work of everyone involved. A personal thank you (in no particular order) goes out to every talented member of the SpatialKey family: Darron, Ben, Andy, Zach, Reggie, John, Tom G, Anthony, Francisco, Robert, Mike, Brandon, and Tom L.

The most recent issue of ComputerWorld magazine features a cover story called “Can Web 2.0 Save B.I.?” that features a case study on SpatialKey. In the article they interview Chief Jon Greiner of the Ogden Police Department in Utah. Ogden is the first installation of the enterprise version of SpatialKey Law Enforcement Dashboard (see the press release), and Universal Mind has been working closely with the Ogden PD to use the SpatialKey platform to develop what we think is a game-changing crime mapping product.

Here’s an excerpt from the article talking about SpatialKey:

Today, the officers are using the new BI tools to perform geographic profiling of crimes and analysis of police data “in seconds,” he says. Before, it could take days for the department’s single crime analyst to fulfill a report request. An added bonus is that experienced police officers with extensive street experience are now able to apply their firsthand knowledge to crime analysis.

“You have practitioners asking the what-if questions, which has changed the way we police,” Greiner says.

And here’s the cover of this month’s print edition of the magazine:

That’s our heatmap! Yeah!

Today I’m proud to announce the launch of SpatialKey, the geospatial information visualization product I’ve been working on with our fantastic team at Universal Mind. I’ll make a bold statement that I stick by: this is the best web-based mapping product in existence. Today we’re releasing a “technology preview” that gives you a little glimpse at what we’ve been working on (just to whet your appetite until we release the full product).

Quick links
Before I explain what SpatialKey is I wanted to give a few quick links because I know a lot of you are going to have your ADD act up before you read the rest of the post.

• SpatialKey Gallery – lists a few dataset/template pairs that we think tell great stories. Read the descriptions of the datasets and then launch the app to play with the data yourself.
• 
  San Antonio Prostitution hotspots

  San Antonio Prostitution Crimes – This link will jump you straight into exploring the prostitution crimes in San Antonio from Jan 2006 – July 2007. Check out how clearly the heatmap points out the corners that are the hotspots in the city.

• Growth of Walmart – This link will load the Walmart dataset into a playback template that lets you click play and watch Walmart take over America.

Beyond points on a map

Overwhelmed with markers

We’ve been seeing the same tired approach to web-based mapping for years now. Everyone throws markers on a map. You want to track crime? Throw a bunch of markers on a map. Little pin markers work fine if you’re showing a few data points. Want to see the location of Starbucks within a 3 block radius of your house? Use markers. But what if you want to see the total sales of all Starbucks worldwide? Or all crimes for the past 10 years? For the whole country?

SpatialKey uses some of the most advanced visualization renderings for geospatial data that have ever been seen on the web. The focus here is on aggregate renderings: heatmaps, thematic grids, graduated circles. 1,000 markers all piled on top of each other doesn’t help anyone. What you want to see is density or sum total value. SpatialKey focuses on rendering aggregate data in meaningful ways. We can show you a heatmap of the entire country and let you visualize any number of data fields. You want to see the heatmap represent total sales of all stores in the region? No problem. You want to see average house price over the past 10 years? We can do that.

Heat Map	Heat Grid

We haven’t seen innovative technology in this industry since Google let you drag the map. (I actually vividly remember that moment when I first dragged a Google map and my mouth started to water). It’s time to move beyond points on a map.

Your data doesn’t have limits
Try adding 10,000 data points to a Google Map. I dare you. What happens? If you’re using the “My Maps” feature of Google Maps, you’re limited to only show 200 points at a time, then you have to page through your data. And to top it all off you’re limited to a whopping total of 1,000 data points in the entire data set. So you get to page through 5 pages of data and only see 1/10th of your total data set anyway. If you create your own application with the Google Maps AJAX API you’re going to have serious performance problems when you get up into a few hundred markers. We think that’s ridiculous.

SpatialKey breaks through the limits of previous mapping technology in two ways. First, we’re simply faster. Flash can process and render data far faster than JavaScript. We can render 10,000 data points in a matter of milliseconds. You simply can’t do that with any JavaScript API out there. Second, we’re smarter. We aggregate data to produce heatmaps instead of just trying to overlay markers one on top of the other. Fundamentally, a massive dataset is an information visualization problem, not a technical one. You need better renderings to convey massive amounts of data, and that’s what SpatialKey delivers.

This is just the beginning
This is a technology preview. That basically means we’re showing you some cool stuff, but we’ve got way more up our sleeve. We’re looking for feedback on what we’ve got, and we’re hoping to get you excited about what we’ll be rolling out. We’ll be releasing new versions of SpatialKey Personal that will let you easily import your own data (if you’ve got an Excel file with addresses you can drop it right in). We’re also going to be releasing SpatialKey Enterprise, which lets you load a data set of any size (millions and millions of points). And then we’ve got a third product that we’re launching called SpatialKey Law Enforcement Dashboard, which is an enterprise version of SpatialKey specifically targeted toward police departments (includes special law enforcement reporting templates). And in the meantime we’ll be rolling out some more example datasets for you to play with, so keep an eye on the SpatialKey blog.

So go check out the SpatialKey Gallery and play with some data. We’re looking for feedback during this phase, so if you have any suggestions or (god forbid!) you run across bugs, please let us know by emailing feedback.