Hobu, Inc.

Dear ESRI,

Please listen to us. We’re your cold, poor, hungry, and sometimes taken advantage of third party developer community. We don’t get to sell $56 million of services and software to the federal government each year. We live with much smaller budgets, much shorter timelines, and much higher expectations. Like those of any software company, you have some products that are fantastic, some of them that are mediocre, and some of them should have never been sold to the public. One of the products that I think is really fantastic is a piece of spatial database middleware you bought a while back called ArcSDE. Lots of your customers use this product to do lots of really neat things. And I would like to use it to do allow my customers to do some really neat things with the software they bought from you.

I’m part of a group of developers of an open source library called GDAL (Geospatial Data Abstraction Library). I like to call it the libc/libc++ of geospatial software. It’s used everywhere, it takes care of the dirty job of raster data translation, and it allows people to write fantastic software.

I would like to write a driver for GDAL that can read ArcSDE raster data so people can do fantastic things with the petabytes of raster data that is stored in ArcSDE throughout the world. To use this driver, people would still have to purchase an ArcSDE license from you. In fact, the ability to use ArcSDE raster data from GDAL might allow you to sell ArcSDE licenses you might not otherwise have sold.

With a shortage of funds and no shortage of id, I set to writing a proposal for implementing the driver for my client. I had heard about the EDN program last summer, and it was my understanding that I would be able to get a temporary (one year) license to be able to use ArcSDE for this type of development. After some investigation, I came across some show-stopping issues with the EDN license that cause a lot of concern.

The first part of the license (which for some reason has PDF copy protection and doesn’t allow me to copy-paste out of it) says:

Licensee shall not reverse engineer, decompile, or disassemble the Software, Data, Web Services, or Documentation, except to the extent that such activity is expressly permitted by applicable law notwithstanding this restriction.

The license doesn’t say if I’m ever released from this restriction. If I sign this agreement, does it mean that I can not ever participate in reverse engineering any ESRI software in perpetuity? A lot of my problem with this restriction is that it depends on what’s being called “reverse engineering,” and in my experience a good portion of software development *is* reverse engineering in the sense that you are trying to figure out what another developer did, why a piece of software works the way it does, and how you as a developer can work with, around, and through it.

The second part of the license, the EDN-specific part, has a few things that cause me to pause:

Licensee may grant access to server applications developed using the EDN Software Library to Licensee’s customers and internal users for acceptance testing purposes only; provided that Licensee’s customers and internal users will not perform any debugging, configuration, or maintenance.

…

Publicly disclose results of benchmark testing except with prior written permission of ESRI.

What does this first part mean? For something like the ArcSDE raster driver, my client is other software developers. The only way they can reliably do acceptance testing *is* to jump in and debug what it is doing. I understand not using EDN software for configuration and maintenance (well, maybe), because you don’t want folks using the cut-rate EDN software to do “real work.” My problem with a restriction like this is that it appears to prevent someone like me from using EDN to do *any* work.

With respect to benchmark testing, why are you afraid to have unabashed results of how your software performs out in public?

Broad, complicated (I had to spend an inordinate amount of time tracing through multiple layers of footnotes) licensing agreements like the ESRI general and ESRI EDN ones scare your developers away. Third party developers provide you with value, momentum, and a pool of potential talent to poach from. A large wall of legalese causes casual folks to not bother. If you’re ever looking to garner the momentum, energy, and hype of the GYM neogeographers and masheruppers, you’ll never get it by making them sign to twelve pages of stuff like this. It won’t matter if your software is fantastic or not.

Hopefully, someone from your company will stumble across this weblog post and be able to answer a question for me. I don’t have any licensing or maintenance agreements with you, but there is some financial incentive for you to do so because, like most of your third-party products, I think an ArcSDE raster driver for GDAL will sell you more licenses of your software.

As an independent developer, is it possible for me to use an EDN seat to develop a raster driver for GDAL?

Sincerely,

Howard Butler,

Hobu, Inc.

If you’ve done GIS for a long time, the stuff on Google Maps is old hat. They were doing this back in the 60s, you might say. The concept of a pre-rendered tile index is old hat too (see TerraServer for a fantastic, prior-art implementation), but Google trades CPU for storage in a way that only Google can. As one prominent #mapserver user said, “Everyone wants to do it like Google, but not everyone has Google’s resources.” Google Maps is the perfect mix of “good enough” data, responsiveness, and a slick UI that is amenable to non-GIS types.

But there’s something else that Google Maps has going for it that the other web mapping giants (MapQuest, Yahoo, ESRI, Topozone, and TerraServer) don’t have — hackability. Google positioned their product to be hackable in the “O’Reilly Hacks” sense, giving free reign to web developers with little or no geography background to embed location in their applications.

I get the feeling that those in the GIS arena are watching what ESRI might do next. I have a hunch that they will attempt to grab some mindshare by opening up ArcWeb services in some limited way to developers, but this might be a bit late. From my involvement in the MapServer community, I know that there are typically two audiences for web map development. Those that are coming to it from the GIS side (and understand all of the data issues), and those that are coming to it from the web development side (and understand scale, deployment, and usability). In MapServer, the ratio has gone from mostly GIS folks to mostly web developers learning GIS as they develop an application.

In my opinion, ESRI doesn’t carry the web developer perspective with them into ArcWeb Services. Even if the pricing structure of ArcWeb Services becomes more attractive, the developer uptake might be kind of slow because of the “enterprise” overhead of SOAP (and SOAP’s poor fit with dynamic, hackable languages like Python, PHP, Perl, and JavaScript) and the depth of GIS knowledge required to use the services effectively. Maybe it is easier than I think, but it has been such a walled garden for so long there is no way for me to know. ESRI has correctly seen that the future is back on the server again, but are they in a position to do it the Google Way ™ — so large, so big, and so bulletproof that everyone gloms on and starts using it?

Reading the Tea Leaves
I’ll finish by describing some behaviors of ESRI that illustrate for me that they won’t be able to make the jump. First, they are being dragged kicking and screaming (mostly through the Geospatial Onestop requirement of being OGC interoperable) into supporting OGC interop. Even when they say they are OGC compatible, it’s not fully there (see ESRI’s OGC page here). ArcIMS cannot cascade WMS services (which is the whole point of WMS, IMO), and we’ve heard forever that they will be in the next version (wait for the next version is the mantra of all monopolies).

Bryan Baker (who’s position in ESRI is unknown to me) made some inflammatory postings on a Directions Magazine article that were later retracted (see here for the article, where most of his comments were in-lined in other comments and preserved). There was a great hub-bub when the comments were retracted and David Maguire stepped in and made a kind of a blah post about Open Source.

Anyway, the point is that Mr. Baker’s comments, whether sanctioned or not, illustrate ESRI’s fears about openness. Like another monopoly we all know and love, ESRI has made good hay on being the software provider who’s software develops the data. Open Source lowers those barriers (and hence their fears), but Google made the whole point moot. The data most people care about are their GPS tracks, house locations, and addresses, not environmental modeling data. People or businesses can now generate their own data (or take what they might already have in their databases in the form of addresses and the like) and plop it into Google Maps.

And that’s the bit that big providers like ESRI should worry about. The size of the market I described above (people plotting their own points) is orders of magnitude larger than people doing environmental data development. But don’t completely count ESRI out. When they were up shit creek without a GUI (think ArcView 3.x and Arc/Info days), they turned on a dime and jumped headlong into COM and Windows. Maybe they still have a few tricks up their sleeves. We’ll find out next week at the User’s Conference if they react or stand there and play the fiddle.