Vishful thinking…

Black magic behind the MSD Map Services?

Posted in ArcGIS, ESRI by viswaug on September 29, 2009

Much has been said about the performance boost that can be achieved by running MSD map services over MXD map services. Here I will talk a little bit about the other side of the MSD map services which trips up developers like me. The only information that I was able to find about the capabilities/short-comings of the MSD map services from ESRI was on their blog post here. Since, I develop and use a few custom Server Object Extensions (SOEs), I noticed that some of the SOEs that I had written earlier had stopped working when registered with MSD map services. And the following piece of information from the blog post mentioned earlier seemed to indicate the cause of the problem.

What capabilities (server object extensions) are supported for MSD-based map services?

The WMS and KML capabilities are available with MSD-based services, as well as the default Mapping capability. Custom server object extensions can also work with MSD-based services but they cannot access fine-grained ArcObjects; they must use the coarse-grained methods on MapServer and its related classes.

The important part of the above statement is tha ‘custom SOEs cannot access fine-grained ArcObjects’. Well, what does that really mean? It kinda means that you cannot access the fine-grained ArcObjects features that are exposed through the IMapServerObjects interface on the MapServer server object. But being from the .NET world, I had trouble wrapping my head around how the same MapServer server object class can serve both the MXD map services and also the MSD map services and behave differently for each service type. How can the same MapServer class offer the IMapServerObjects features for MXD map services but not for MSD map services? I tried casting(QI) the MapServer server object obtained from a MSD map service to an IMapServerObjects interface which actually resulted in an exception being thrown. But the same cast worked on the MapServer object obtained from a MSD map service. My first thought when I noticed this behavior was that maybe the MSD map services are actually served by a different new server object class and not the MapServer class that serves the MXD map services. The ESRI  9.3.1 documentation doesn’t mention anything about a new server object. So, either ESRI didn’t document it or the same MapServer has developed multiple-personalities in AGS 9.3.1. To test it out, I obtained the CLSID of the server object that served MSD map services and the MXD map services. The were the exact SAME. That probably means that the same MapServer class is just behaving differently in different usage scenarios by leveraging some COM black magic which I don’t have too much knowledge about.

Since, we can’t get at the rich features in the IMapServerObjects interface, the custom SOEs on MSD map services really become only a little more useful than custom COM utility objects. So, some of the custom server object extensions that developers wrote earlier for MXD map services will not work with MSD map services. And that is a real bummer for me. One way (that I really really hate) to overcome this limitation is to run another MXD map service for the MSD map service. The MSD map service makes the maps and the MXD map service is only used to obtain and use the features of the  custom server object extension. I do really hate the idea of running a another expensive map service process on the AGS machine only for the purpose of using the features of the custom server object extensions. Although, I do have some ideas on a better ways of working around these limitations. The most frustrating part about this whole thing is that I think internally ESRI does have access to fine grained ArcObjects for the MSD map services and they just don’t let us know how to gain access to the fine grained ArcObjects. Why/How do I say that? The reason for my suspicions that the above is the case is because, the WMS and KML server object extensions in-fact DO work with MSD map services also. Which means that those server object extensions are able to access fine-grained ArcObjects to do what they need to do.

Update : Another change that relates to SOEs in AGS 9.3.1 is regarding it registration process. As many might have noticed, their SOEs shows (or doesn’t showup :)) up without any label in the ArcCatalog->Service Properties->Capabilities tab. It took me a while to realize what was happening. I had initially assumed that the SOE wasn’t showing up in the list. But later, luckily I discovered that the SOE was in-fact displaying but there was just no label for it in the Capabilities list box. In 9.3, the IServerObjectExtensionType interface has been superseded by the IServerObjectExtensionType2 interface ArcObjects style. So, when you register the SOE with ArcGIS Server by calling the CreateExtensionType, the returned object should be casted to the type IServerExtensionType2 and the ‘DisplayName‘ property should be set on it before calling AddExtensionType. This ‘DisplayName‘ property is what shows up in the capabilities tab and if it is not set, the entry for the SOE in the capabilities tab will be empty. Earlier, the extension’s Name property used to show up in the capabilities tab. But no longer, it is the DisplayName property that is used to label the entry in the capabilities tab list box. I hope ESRI finds some time to update their documentation about this over here.

Tagged with: ,

18 Responses

Subscribe to comments with RSS.

  1. Stefan said, on September 29, 2009 at 12:07 pm

    Hi,
    what I find interesting is, that registered SOEs (mapserver) won’t even appear in ArcCatalog ‘s service dialog if you are dealing with MSD services. If you put in the SOE configuration properties to [mapservicename].MapServer.cfg the service will initialize properly as long as you don’t access the suspicious classes.
    Vish, do you get the SOE’s listed with ArcCatalog?
    Cheers
    Stefan

    • viswaug said, on September 29, 2009 at 10:51 pm

      Hey Stephan,

      see update to the post above.

      Vish

      • jeff said, on September 29, 2009 at 11:01 pm

        Or the quick and dirty route, just edit the SOE extension types config file directly @ …\ArcGIS\server\system\ServerTypesExt.dat. Edit the DisplayName element there.

        StormWatchesSOE
        Storm Watches SOE
        Navajo.ArcGisServer.SOEs.StormWatchesSOE
        Navajo

      • jeff said, on September 29, 2009 at 11:04 pm

        Oops. XML didn’t render. try again

        <ExtensionType>
        <Name>StormWatchesSOE</Name>
        <DisplayName>Storm Watches SOE</DisplayName>
        <CLSID>Navajo.ArcGisServer.SOEs.StormWatchesSOE</CLSID>
        <Description>Navajo</Description>
        </ExtensionType>

      • viswaug said, on September 29, 2009 at 11:08 pm

        Yes. Jeff. That will definitely work also.

        Vish

    • Stefan said, on September 30, 2009 at 4:46 am

      Hi Vish, hi Jeff,
      yes, putting in the xml will work, but sadly that’s not the way we like to tell our customers how to manage the custom SOE.

      I double-checked the issue with the SOEs in ArcCatalog (I already recognized the new interface as well): using a *.mxd shows the SOE with ArcCatalog, using the same map with a *.msd won’t. Do you have other experiences with that? If I do not register the SOE using the new interface, I’ll get at least the checkbox to enable the SOE.

      I assume ESRI does some internal filtering with ArcCatalog like
      if (soe.name == ‘WMSServer’ || soe.name == ‘KMLServer’){display()}
      else {}

      //great!

      Stefan

  2. jeff said, on September 29, 2009 at 1:53 pm

    I find it frustrating that ESRI has not been forthright about this shortcoming in the MSD design and have provided virtually no insight… but for the two sentences in the blog post you refer to. The additional map service workaround you mention for those SOEs requiring fine-grained ArcObjects is a bit “hacky” but will have to do.

    I’ll have to verify what Stefan’s seeing. That too would blow!

  3. kudos ltda. said, on September 29, 2009 at 5:53 pm

    This article is really helpful!!
    I’m having the same issue using arcgis server with java.

  4. Mark Prins said, on October 1, 2009 at 8:12 am

    this makes using msd’s rather pointless AFAIC

  5. Michael said, on February 26, 2010 at 6:33 pm

    Do MSD’s address query performance at all? Or are they only supposed to improve the time it takes to render the map to the browser? We are looking to leverage any approach that might increase the performance of the IQueryFuntionality.Query method and the time it takes to add its featuregraphics and attribute results to the map. Can you confirm that IQueryFuntionality.Query is not even an option with MSDs? This seems to be the case — my tests return null when using this method. I had no idea that these much ballyhooed MSDs were going to be so limited.

    “this makes using msd’s rather pointless AFAIC”
    +1

    Many thanks.

  6. Jim Sibbald said, on March 27, 2010 at 1:40 pm

    I am a dabbler in ArcObjects but I have found (and successfully reported as a bug to ESRI last week) that when an MSD is published through the ArcMap interface as a MapServer it has a Property of {FileName} with the msd filename only. When published through ArcCatalog it uses the correct property of {FilePath} with the full filepath. {FilePath} is what is in the ArcGIS Help (http://webhelp.esri.com/arcgisserver/9.3.1/dotNet/index.htm#cfg_file_service.htm ) for MapServer.

    Would this be contributing to the SOE issue?

    • viswaug said, on March 29, 2010 at 11:37 am

      Hi Jim,

      I am not sure that it is contributing the SOE issue. But I did learn at the devsummit last week that ESRI has opened access to the layers in MSD services thru a new interface (IMAPDataAccess)…

      Thank You,
      Vish

      • jeff said, on March 29, 2010 at 12:16 pm

        So Vish – did any ESRI folks mention future (AGS v10?) SOE support for MSD services?

      • viswaug said, on March 29, 2010 at 3:45 pm

        Hi Jeff,

        Yeah, like I mentioned above the new IMapDataAccess interface allows access to layers in MSD from SOEs. Hurray !!!

        Thank You,
        Vish

  7. tomlux said, on May 3, 2010 at 6:50 am

    Hy,
    do you have a link to the documentation of the IMapDataAccess.

    I didn’t found it.

    Thanks.
    Tom

  8. Víctor Velarde Gutiérrez said, on May 7, 2010 at 2:14 pm

    Do you guys think that is possible to use a SOE extension to substitute a geoprocessing service returning a raster layer inside an associated map service? Do you know any resource / example similar to that?

    Thanks,
    Víctor

    • viswaug said, on May 7, 2010 at 3:02 pm

      Hi Victor,

      Your question is vague. So, can’t say anything definite but sounds like what you want to do might require a big effort if at all possible🙂

      Thank You,
      Vish

      • Víctor Velarde Gutiérrez said, on May 7, 2010 at 4:29 pm

        Thank you for answering me Vish,

        I already have a geoprocessing service doing some raster processing with a python script and I was considering other alternatives to improve it. As the process has to return a raster result, among other statistics, and I want to consume it in a webpage with the Javascript-API, my only path is to associate it with a mapservice, so that it renders it for me over my map. Reading some posts as yours and dbouwman’s made me think about this alternative. I was thinking in some kind of all-arcobjects-server-side-geoprocessing-routine inside a SOE that does the process and adds a raster layer to its associated mapservice, trying to get the best performance.

        Thank you again and congratulations for your useful blog.
        Have a good weekend🙂

        Victor


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: