Vishful thinking…

Should you have multiple maps (or dataframes) in your MapServer MXD for ArcGIS Server

Posted in ArcGIS, ESRI, GIS, Uncategorized by viswaug on March 22, 2008

At the ESRI Dev Summit 2008, I managed to get some insights into questions that have been bouncing around in my head from the ESRI folks building ArcGIS server. One such design question was whether it was reasonable to create MXDs with multiple maps in them that can be accessed as different map resources or just create different map services for multiple maps. By creating a map service with multiple data frames in them you can serve both maps with just a single instance of a SOC. When creating different map services for different maps you end up with different SOCs for each map service.

One scenario where you might ask yourself this question is when deciding whether the overview map should be in a data frame on the same map service or have a different map service for the overview map. Obviously, the overview map only has some of the layers from the map itself.

The REST API in ArcGIS Server 9.3 will only support the current data frame in the MXD.

Now to the answers, it is not a bad idea to have multiple data frames in a single MXD and consume them as separate resources as long as the multiple data frames (or maps) in the MXD do not get used simultaneously all the time. When used simultaneously, a single SOC process is going to process the requests for both the maps and thus the concurrent requests are going to slow down the service. If the maps are going to be used concurrently, then it is better to create different map services for each of the maps, this way different process can process the requests and can return faster. That being said, for most cases it is better to create separate map services for each map.

Little known bottlenecks for ArcGIS Server Map cache generation performance

Posted in ArcGIS, ESRI, GIS, Uncategorized by viswaug on March 22, 2008

I had a chance to ask some of the ESRI developers the reason why the time required for our map cache generation didn’t reduce by very much when we started running SOCs on multiple machines (more powerful machines with Dual Quad core processors) with increased SOC instances. The bottleneck in these cases happen to be the following

  • ArcSDE
  • Disk speed (RPM) of the cache destination

Ok, now for the reasons as to why they are bottlenecks and some suggestions on how they can be worked around. The map service for which we were generating caches for, had almost all of its layers coming in from a single ArcSDE instance. When a large number of SOC instances (or processes) start to generate the map cache, all of them are accessing the same ArcSDE instance concurrently to generate the map cache. This multiple simultaneous access of the ArcSDE could slow down access to the map data and thus slow down map cache generation. One way this bottleneck could be worked around is by copying the ArcSDE data to multiple file geodatabases on different machines and referencing different layers in the map service from different file geodatabases. This way access to the map data is not throttled at just a single machine and is split amongst different machines.

All the SOC instances are also outputting their cache files to the same output destination. This means that this could also be a bottleneck if the disk speed of the output destination is not fast enough. One way that I can think of to work around this issue is by running different map cache generation processes for different scale levels and have their outputs going to different machines. Once done, the caches for the different scale levels can be copied to a single destination(ESRI uses SecureCopy to copy map caches between disks for their ArcGIS Online services). But, I don’t believe that this technique would save any time at all and would be too much trouble for not enough returns. So, just spend some money on a really fast hard disk.

Larger tile sizes for map caches take lesser disk space and generating them take lesser time. ArcGIS Online uses a 512X512 tile size whereas VirtualEarth and Google Maps use a 256X256 tile size.

ESRI Dev Summit 2008 – The real skinny

Posted in ArcGIS, ESRI, GIS, Uncategorized by viswaug on March 22, 2008

I am going to try to keep this post clear of chatter about how ESRI is doing things and my opinions on them. I would like to thank all the ESRI developers who took the time to answer some of my questions. So, here are some important tech notes to take away from the conference.

REST API

  • The REST API does not support curves to be returned.
  • When curves are encountered, they are densified automatically and returned as polylines.
  • Support for curves is being planned for in the future versions.
  • They were also open to the idea of GeoJSON converters in the JavaScript API for the future.
  • The shapes returned by the REST API are not GeoJSON compliant. But the ESRI developers seem open to support GeoJSON when it is finally ratified by the standards committee.
  • The REST API does all its work by using the ArcGIS Server SOAP API. See the notes below for the reasons why.

JavaScript API

  • The ESRI developers chose to use the DOJO toolkit primarily because its graphics capabilities.
  • The DOJO client-side libraries provide support for performing some simple spatial operations like ‘intersection’ on the client-side.
  • The intellisense for the Javascript API currently has some problems in Visual Studio 2008.
  • The client-side layers provides support for attributes on the client-side features. Attributes on client-side features are not even supported in VE and Google Maps.
  • The client-side layers support events.
  • Currently, only one client-side layer is supported. The reason for this being (according to the ESRI developers) there are problems with listening to events on features that are not on the top-most layer. ESRI is currently researching the problem and is trying to find ways to make multiple client-side layers work.
  • The client-side JavaScript for the .NET ADF has been completely re-built in 9.3.
  • Changes are already planned for the JavaScript API between the Beta version and the final release.
  • An object model diagram for the JavaScript API might also be available.

Map cache

  • ArcGIS Server 9.3 supports the following tiling scheme
  • The Map cache generated using the VirtualEarth tiling scheme does not generate tiles using the quadkey naming scheme for the file names generated but only the tiling scheme is the scheme.
  • The REST API has built-in support for handling quadkey naming scheme, it maps the quadkeys to the file names generated by the ArcGIS Server cache.
  • The WMS Service of the MapServer object will start taking advantage of the Map cache if available starting in 9.3.

Geodatabase

  • Sounds like SDE will support storing shapes in SQL Server 2008 as SQL CLR types also. This should enable other applications to access and use spatial data in the SDE without ArcObjects.

Note: It was decided to use SOAP over DCOM for the REST handlers to talk to ArcGIS Server because ESRI realized after running some metrics that SOAP was more performant than DCOM to access ArcGIS Server functionalities. The chatty nature of DCOM reduces its performance over SOAP. When using DCOM, the client uses a proxy to communicate with the server object. That means every time the client accesses a property on the server object, it is accessed over the network. But when using SOAP, an object representation is completely serialized and sent to the client so that property access doesn’t need to be over the network.

Even though it is not completely documented, it is possible to access Server Object Extensions over SOAP directly with some custom programming.