I have caught myself cursing at ArcObjects numerous times for not being performant for some of the spatial operations I had done using it in the past. I have also heard the same about it from other developers I have come across in the past. But what I was always curious about was, how does ArcObjects compare in performance to the other open-source spatial libraries like PostGIS? Since I have always been swimming in the ESRI and .NET ocean, I have little or no knowledge on how the PostGIS and Java worlds performed. So, it was interesting to listen to John Coryat speak this thoughts on how using the PostGIS library (that sits on top of the PostgreSQL database) makes spatial operations up to 10 -100 times slower than if those operations where performed using the spatial operations built into Postgresql.
Both ArcObjects and PostGIS suffer from the same drawback that make them significantly slower, they both are spatial libraries that run outside of the process space of the database where the spatial data is stored. Please see comments from Paul Ramsey below explaining that PostGIS actually runs within the process space of the database and how recents changes have made it considerably faster. ArcObjects runs outside of the process space of the database where the spatial data is stored.And that takes a major toll on the performance. Most databases these days including MS SQL Server 2008 pack a powerful set of spatial operations built right now that can satisfy the requirements for some cases but definitely not all of them. The spatial operations that are built right into the databases should always be prefered over using external spatial libraries since they run magnitudes faster. But the problem is that spatial operations included in those databases are not comprehensive and they all don’t pack the same geometry operations either. Most of them do support the SQL functions outlined by the OGC like SQL Server 2008 but some don’t. The most notable memberwould be MySQL.
To overcome or to add more capability to the spatial operations in these databases, an external spatial library become a necessity. Expecting the database vendors to develop spatial operations and to have to them keep up with the developments in the industry would not be an attractive option. This puts us, the spatial developers, in a catch-22 situation wherein we need to give up on performance to gain more powerful spatial capabilities. So, we may not be able to FIX the performance issues with spatial operations but just patch it as needed.
Even though John Coryat talks about his experience with PostGIS and PostgreSQL performance comparisions, that still doesn’t give a comparision between ArcObjects and PostGIS performance. It would be interesting to see some performance benchmarks between the two though.
I have to say that I am intrigued by this announcement about improvements in 9.3.1
Especially the coming improvements in the dynamic map publishing below and also the Map Server Document (MSD) format.
High-Performance Dynamic Map Publishing
- New faster rendering engine
- Outperforms equivalent ArcIMS services.
- Produces significantly better-looking maps.
- Shortens map caching time.
- Quicker, smoother zoom and pan.
- New Map Optimization toolbar in ArcMap helps you tune-up your map documents before publishing to ArcGIS Server.
- Review and respond to errors, unsupported content, and warnings about items that will slow down your dynamic map services.
- Preview your optimized map document and estimated rendering time.
- Save your optimized map document to the new Map Server Document (MSD) format.
Before I start, I need to say that the link to the code below was provided to me by Pamela Fox. The sample below illustrates how to create a light weight custom marker GOverlay for Google maps that is REALLY FAST. And it allows for a multitude of customizations as you see fit. It was just too good not to share. Please find the the tweak I made to the code to allow for it to do a little bit of thematic mapping in just minutes. The demo below also allows you to check out the performance of the custom overlay with large number of markers loaded.
Here are the reasons provided by Pamela Fox for the default marker overlay in Google Maps to be slower than the custom marker overlay below.
The default marker class (GMarker) is designed so that it works well across-browsers, has hot spot shape, can be draggable, has a shadow, prints well in any browser, etc. It is not optimized for performance.
Here is the custom marker overlay code sample.
I came across an interesting blog post by the Redfin developers about their experience with moving from the VirtualEarth API to the Google Maps API. That got me thinking about my experiences with both the APIs. I am not going to be talking about whose imagery looks better or has a higher resolution here. But, as a developer, I should say that I actually prefer the Google Maps API over the VirtualEarth API. For most cases, both the APIs are really to use and are comparable in their feature sets. But here are something in the Google Maps API that I miss in the VirtualEarth API.
- The TileOverlay feature in Google Maps is present in the form of a custom tile layer in VirtualEarth. The GroundOverlay feature in the Google Maps API is absent in VirtualEarth. Wouldn’t this feature come in handy?
- The Google Maps API exposes their shape drawing and editing API but VirtualEarth doesn’t. This is a BIG one for me. Almost a MUST have. I don’t understand why the shape drawing and editing functionality available in live maps is not exposed in the API. I also love the vertex editing functionality and its usability especially how easy it is to insert a new vertex in a polygon.
- The latitude/longitude encoding feature in the Google Maps API is a nice to have feature and will help save bandwidth. The GPolyline factory method helps create geometry overlays straight from the encoded string. Try out the encoding here.
- The getLength and getArea functions on GPolyline and GPolygon are good enough for most cases where accuracy is not real important.
- Google maps API provides static methods to handle XML documents and fragments (GXml) and methods to apply Xslt to Xml (GXslt). The VirtualEarth API doesn’t contain any such equivalents.
- The geo-coding functionality in Google Maps also has added functionality like
- A useful set of utility libraries for the Google maps API.
- The StreetView is currently only available in google maps
Did I miss anything else either with the Google Maps API or the VirtualEarth API?
I have written about some of the advantages that a REST API can bring to exposing GIS data. This is not just for delivering map images but also for delivering and managing vector data. We have been building a RESTful API to not only expose vector GIS data, but to perform all of the CRUD operations on the data as required. Since most of our mapping applications are built using the ESRI JS API, the REST API makes it a breeze to access those functionalities through URL endpoints.
Not to revisit the topic again but the idea behind REST is to use HTTP as an ‘application protocol’ rather than as a transport protocol like SOAP web services do. One protocol that uses HTTP the way it is meant to be used is the AtomPub protocol. I am not going to go into details about the AtomPub protocol here since there a lot of documentation available online that do a good job of explaining the AtomPub protocol. But in brief terms,
AtomPub is both a format and a protocol. And I think that the AtomPub is a natural way to expose data especially also GIS data. I am going to go over how in my opinion GIS data exposed as map services seamlessly falls into the AtomPub data container structure. The figure below illustrates a simplified data container structure in AtomPub.
As you can see from the above figure, an AtomPub service document contains a list of ‘Workspaces’ which in turn contains a list of ‘Collections’. The ‘Collections’ are represented by Atom Feed documents. The Atom Feed document contain a list of ‘Entrys’. In my opinion, here is how the ‘Service Document’ in the AtomPub protocol relates to a map service. (I am only referring to ESRI Map services below).
Service Document – The Map Service itself – The entry point that describes the location and capabilities of collections
Workspace – Data Frames or Group Layers – A logical grouping of collections
Collection/Atom Feeds – Feature Layers – A Resource that contains a set of Member Resources. Collections are represented as Atom Feeds.
Entry – Feature – Represents an individual piece of content/Data
Categories Document/Categories Element – Feature Type – Provides metadata to describe an Atom entry. Categories Document describe Categories that are allowed in collections.
The AtomPub ‘Category’ element can also be used to represent feature types like ‘LinearRing’, ‘Polygon’, ‘LineString’, ‘MultiLineString’, ‘MultiPoint’ and be attached to Atom entries to provide metadata. They can also be used to represent the domain entity represented by the Feature Layer like National Forest etc…
Service Document Sample:
<?xml version=“1.0” encoding=‘utf-8’?>
<collection href=“http://vishcio.us/NationalHighways” >
<categories href=“http://vishcio.us/NationalHighways” />
<collection href=“http://vishcio.us/Rivers” >
<collection href=“http://vishcio.us/Lakes” >
<atom:category scheme=“http://vishcio.us/geometry/polygon” term=“polygon” />
<atom:category scheme=“http://vishcio.us/Lakes” term=“Lakes” />
Atom Feed Document:
<?xml version=“1.0” encoding=“utf-8”?>
<link href=“http://example.org/feed/” rel=“self”/>
<title>Atom-Powered Robots Run Amok</title>
<?xml version=“1.0” ?>
<atom:category term=“animal” />
<atom:category term=“vegetable” />
<atom:category term=“mineral” />
The above is just my take on how the AtomPub elements can relate to map services. I think this is the next logical step in making map services more resource oriented and will allow for dynamic discoverability and ease in application development. ESRI map services already have the concept of a service document that can be accessed at the links below.
Supporting the AtomPub output format for that service document should be pretty simple.
I am going to attempt to highlight the difference between URLs and URIs the way I see it. I am sure the purist might disagree with my take on it. So here goes nothing…
URI – Uniform Resource Identifier – This identifies a resource uniquely. The identifier used could optionally contain the location and the name or not. Ok, you have read that before…what does it really mean? Here is how I see it.
Take ISBN number for instance. It helps you identify the book uniquely. Let’s assume you are interested in the book with the ISBN number ‘ISBN-10: 0596519206′. Using an ISBN number you can confirm as to whether any book you are looking at is the book you are interested in. But using the number, you cannot get access to the book, you can only confirm whether you have found the book or not. This doesn’t mean that an URI cannot tell you how to access the resource also or not, but it SHOULD at least let you differentiate that resource from all others.
URL – Uniform Resource Locator – This lets you know how to get access to that particular resource. In most cases, URI is contained as a sub-part within the URL. URL contains not just the resource identifying information (URI) but also the resource locating information. Well, take for example, the above book resource, a URL for the book could be ‘http://www.amazon.com/dp/0596519206/‘. Here you can see that the ISBN number (URI) is contained as a part of the URL. But the URL also provides you more information that lets you gain access to it. Like, it lets you know that the resource can be accessed on the ‘World Wide Web’ via the HTTP protocol at the Amazon address using the ISBN number.
That said, I would also like to add that the two terms URL and URI are used so interchangeably these days that the subtle difference really doesn’t have a lot of significance.