Little known bottlenecks for ArcGIS Server Map cache generation performance
I had a chance to ask some of the ESRI developers the reason why the time required for our map cache generation didn’t reduce by very much when we started running SOCs on multiple machines (more powerful machines with Dual Quad core processors) with increased SOC instances. The bottleneck in these cases happen to be the following
- Disk speed (RPM) of the cache destination
Ok, now for the reasons as to why they are bottlenecks and some suggestions on how they can be worked around. The map service for which we were generating caches for, had almost all of its layers coming in from a single ArcSDE instance. When a large number of SOC instances (or processes) start to generate the map cache, all of them are accessing the same ArcSDE instance concurrently to generate the map cache. This multiple simultaneous access of the ArcSDE could slow down access to the map data and thus slow down map cache generation. One way this bottleneck could be worked around is by copying the ArcSDE data to multiple file geodatabases on different machines and referencing different layers in the map service from different file geodatabases. This way access to the map data is not throttled at just a single machine and is split amongst different machines.
All the SOC instances are also outputting their cache files to the same output destination. This means that this could also be a bottleneck if the disk speed of the output destination is not fast enough. One way that I can think of to work around this issue is by running different map cache generation processes for different scale levels and have their outputs going to different machines. Once done, the caches for the different scale levels can be copied to a single destination(ESRI uses SecureCopy to copy map caches between disks for their ArcGIS Online services). But, I don’t believe that this technique would save any time at all and would be too much trouble for not enough returns. So, just spend some money on a really fast hard disk.