ESRI’s Generalize function / MS SQL Server 2008’s Reduce function

We were trying to load some watershed polygon data into VirtualEarth but the number of vertices on those polygons was large enough to affect the performance of the map. Since, the data was being used to pull up documents for each watershed and not for getting accurate map data, we decided to generalize or reduce the vertices on the watershed polygons by an acceptable distance factor and limit the scale on the map to not let the user view the data at low scales. We knew that ESRI’s Generalize(…) function on IPath did exactly that and takes a tolerance. It does so using the “Douglas-Peucker algorithm“, although ESRI’s documentation calls it “Douglas-Poiker algorithm”. But when I was tooling around MS SQL Server 2008, I found the Reduce(…) function in it does the same. This made me curious to see if Microsoft and ESRI’s versions of the algorithm performed the same. And they do… Both of the functions reduced the number of vertices on one of watersheds polygon from 14245 to 450 when using a tolerance of 30 meters.

Thinking about JSON… Schema definition for JSON? RESTful presentation & transformation services for JSON?

Since I have been working with JSON, I have had a lot of ideas and questions regarding its use, validation and presentation. JSON is overwhelmingly being preferred over XML these days for most applications since most of them are web-based. Given its quick acceptance for various applications, do we need other supporting standards and technologies for JSON like XML’s XSD and XSLT technologies. When I was pouring over the GeoJSON standard when working on the GeoJSON.NET library, I couldn’t help but wonder if the GeoJSON standard should be expressed in a schema definition language for JSON like the XSD for XML. I don’t believe validating JSON data against such a schema definition would be practical right now, given that the validation will be processor and memory intensive and may not be a great idea on the client-side browsers. I did find Kwalify and Cerny which seem to take a stab at it. It will be interesting to see whether such technologies will develop further.

Another interesting area that is slowly progressing with JSON is technologies to convert JSON data into a user presentable format i.e. a XSLT like technology to convert JSON data to HTML. Some of my thoughts and performance metrics on some examples can be found here. Given that transforming large JSON data into HTML using JavaScript may not be as fast as required on the client-side, I believe this and other reasons will open up server-side RESTful services that do the job of transforming JSON into a user presentable format. To perform the transformation on the server-side, the amount of data being transformed should be substantially big. These RESTful JSON presentation services could look like the following

http://www.XXX.com/Presentation/?outputFormat=[HTML]&jsonDATA=[jsonURL]

In the above URL, the ‘[jsonURL]‘ represents the URL that GETs the JSON data and ‘[HTML]‘ tells the service the format to which the JSON data needs to be transformed to. I don’t think this kind of RESTful presentation services for data is anything new. There are similar services available right now even for GIS but not exactly JSON data. The service I am referring to is one of VirtualEarth’s services to view KML data(not GeoJSON, but hopefully that is coming soon). It should be a breeze to build a similar service for GeoJSON using OpenLayers since it can consume GeoJSON data directly as a layer. One example of an OpenLayers app consuming GeoJSON data can be found here. The GeoJSON data is obtained from here, which is a FeatureCollection of the boundary data for all countries.

Anyway, both these services take data in one format (KML and GeoJSON) and present it on a map.

URL FORMAT http://maps.live.com/?mapurl=[insert url here]

http://maps.live.com/?mapurl=http://dsc.discovery.com/utilities/googleearth/nationalparks/nationalparks.kml

GeoJSON.NET - A .NET library to produce GeoJSON output

I have been working on a .NET library to produce GeoJSON output from GIS data. To have the GeoJSON serialization and rehydration be independent of any paid software, I decided to build the framework to produce the GeoJSON output from the geometry types in the NetTopologySuite open source library. Also, for the JSON serialization in .NET, I am using the JayRock open source library. Although, I had to make a little change to the JayRock build script to have the assemblies be strongly named.

The library for the GeoJSON serialization is also open source and can be found here (at Assembla, which I love by the way). The library has also been unit tested using the MbUnit framework.

Are multi-layer caches worth it?

The requirements for one our projects had eliminated the use of single fused cache for the web application. I was doing some research into whether using a multi-layer cache would help the performance of the application given the extensive requirements that we are trying to tackle. I was trying to find some broad guidelines to justify using multi-layer caches for our application. One of the helpful guys at ESRI pointed me to this link in the ArcGIS 9.3 documentation page (I couldn’t find it in the 9.2 documentation) that does just that. http://webhelp.esri.com/arcgisserver/9.3/dotNet/index.htm#choosing_cache_properties.htm (See the “Advanced options (cache type)” section). I think you need to be in the 9.3 Beta program to access the link. If you don’t have access to the link, here is what you need to know.

If you do use a multi-layer cache, ArcMap is the recommended client for working with the cache. Using a multi-layer cache in a Web application provides little or no advantage over using a non-cached map service. The Zoom Level control that appears with fused caches does not appear when you use a multi-layer cache in a Web Mapping Application.

The issue with the zoom level control is supposed to be fixed in 9.3. But the recommendation on using ‘ArcMap’ as the client still should hold true. So, if you are thinking about using multi-layer caches in your application, keep that in the back of your mind. But this does make me wonder as to how many people are actually using multi-layer caches with any kind of success in their web applications…

Should you care about the "UseMimeData" property on the .NET ADF Map control?

The ‘UseMimeData’ property tells the ADF whether to request a stream of bytes or a URL. This property is set to TRUE by default. In this case, the ADF is going to store the stream of bytes for the map images into the session storage. If the property is set to false, the image is written to disk in the virtual directory specified and passed back to the browser as a URL. Setting the ‘UseMimeData’ property to TRUE and streaming images from the session is fine when the in-process session storage is used. But other providers like SQL Server is used for storing the session, it is wise to set this property to FALSE and return URLs. This will save the added overhead of serializing everything into the SQL database on each map draw or other draw operations. Setting this property to FALSE will force the MapServer to use its output directory to write out the images and the associated URL will be passed on to the browser. When specifying the path for the virtual directory property on the map control, specify a relative path as an absolute path doesn’t seem to work.

Adding a map resource dynamically to the map using the .NET ADF

I just completed a feature in our current project allowing the user to be able to select the map resource to view from a list box and have the map loaded with the selected resource. To accomplish this, we had to add map resources to the map dynamically without it being preset on the MapResourceManager using the Visual Studio designer. Here is a code snippet to insert a “ArcGIS Server Local” map resource into the map.

private MapResourceItem AddMapResource(Map map, AGSLocalMapService mapService, int index)

        {

            MapResourceManager mrm = map.MapResourceManagerInstance;

            MapResourceItem mri = new MapResourceItem();

            GISResourceItemDefinition definition = new GISResourceItemDefinition();

            mri.Name = mapService.Name;

            definition.DataSourceDefinition = mapService.MapServerName;

            definition.DataSourceType = “ArcGIS Server Local”;

            definition.ResourceDefinition = mapService.DataFrame + “@” + mapService.MapService;

            definition.DataSourceShared = true;

            mri.Parent = mrm;

            mri.Definition = definition;

            DisplaySettings displaySettings = new DisplaySettings();

            displaySettings.DisplayInTableOfContents = true;

            displaySettings.Visible = true;

            displaySettings.ImageDescriptor.ImageFormat = ImageFormat.PNG24;

            mri.DisplaySettings = displaySettings;

 

            mrm.ResourceItems.Insert(index, mri);

            mrm.CreateResource(mri);

 

            return mri;

        }

and the snippet to insert a graphics resource item.

public static MapResourceItem AddGraphicsMapResource(Map map, string graphicsResourceName)

        {

            if (map != null)

            {

                MapResourceItem mapResourceItem = new MapResourceItem();

                GISResourceItemDefinition graphicsDef = new GISResourceItemDefinition();

                graphicsDef.DataSourceDefinition = “In Memory”;

                graphicsDef.DataSourceType = “GraphicsLayer”;

                mapResourceItem.Definition = graphicsDef;

                DisplaySettings graphicsDisplay = new DisplaySettings();

                graphicsDisplay.DisplayInTableOfContents = false;

                //graphicsDisplay.Transparency = 50.0F;

                graphicsDisplay.Visible = true;

                mapResourceItem.DisplaySettings = graphicsDisplay;

                mapResourceItem.Name = graphicsResourceName;

                map.MapResourceManagerInstance.Initialize();

                map.MapResourceManagerInstance.ResourceItems.Insert(0, mapResourceItem);

                map.MapResourceManagerInstance.CreateResource(mapResourceItem);

                return mapResourceItem;

            }

            return null;

        }

Debugging embedded JavaScript files in Visual Studio

The credit for this post goes to Rachel who recently informed me how to debug and step through embedded JavaScript files in Visual Studio when using IE. Before today, whenever I had to debug through embedded JavaScript files I resorted to using FireFox even though the application I was building needed to be delivered on IE. What is an embedded JavaScript file? These are the JavaScript files that are embedded inside assemblies and retrieved by the web server using the WebResource.axd handler. Anyway, I have been using them extensively when developing custom tasks (ASP.NET server controls) for the ArcGIS Server .NET ADF. This way all the dependencies for the server controls are contained right within the server control assembly and now the server control can be easily redistributed and used by clients by a simple drag and drop.

The Solution, a “hidden” feature in Visual Studio called the “Script Explorer”.

In visual studio, right click anywhere and toolbar, Customize > Commands Tab > Debug Category > Drag and drop Script Explorer. It is greyed out unless you are in debug mode. In IE, open options, make sure you’ve enabled script debugging. Start debugging your web application, while running, click on script explorer window.  you’ll see all the URL’s to the embedded resources, double click them to bring them up in VS window. You should see the embedded JS code, set breakpoints, mouse over as desired.

The proper way to initialize a resource in the .NET ADF

If you have ever tried to use the MapResourceManager on a page without a Map Control or use the functionalities directly through the ResourceItems on the MapResourceManager, you must have found that the functionalities do not work as expected. The reason the functionalities do not work as expected is because the resources underlying the resource items should be initialized before use. Reading over the ESRI’s .NET ADF library, there are 2 methods that suggest that they will accomplish the task of initializing the resource, the Initialize method on the IGISResource interface and the InitializeResource method on the GISResourceItem.

But the Initialize method on the IGISResource interface does not do the job right. In order to initialize the resource right use the InitializeResource method on the GISResourceItem. This is not documented anywhere as far as I know. So, if the Query method on the IQueryFunctionality interface returns null for no reason when you expect results, trying initializing the resource using the InitializeResource method on the GISResourceItem.

Update: Heard from some ESRI folks on this issue. IGISResource.Initialize() initializes the resource only. IGISResourceItem.InitializeResource() initializes both the resource and the datasource of the resource. Always to use InitiializeResource(), since there are more logic here for handling initialization errors, validation stuff etc, whereas, with  Initialize() it’s dependent on the individual resource’s implementation.

Quick note on KML support in the ArcGIS Server 9.3 REST API

ArcGIS Server 9.3 supports multiple output formats in its REST API

• HTML
• JSON
• KMZ
• image
• help
• lyr, nmf(ArcGIS Explorer document), jsapi(Javascript API), gmaps(Goolge Maps), ve(VirtualEarth)

Good news on the KML support though. The output is actually an KMZ output with includes all symbology for the features from the Map Service. The symbols are outputted as images and zipped to a KMX along with the KML file. This is great news because it will help reduce a whole lot of redundant work for developers. Obviously the other formats like JSON that do not support symbology do not include the symbology in them.

Should you have multiple maps (or dataframes) in your MapServer MXD for ArcGIS Server

At the ESRI Dev Summit 2008, I managed to get some insights into questions that have been bouncing around in my head from the ESRI folks building ArcGIS server. One such design question was whether it was reasonable to create MXDs with multiple maps in them that can be accessed as different map resources or just create different map services for multiple maps. By creating a map service with multiple data frames in them you can serve both maps with just a single instance of a SOC. When creating different map services for different maps you end up with different SOCs for each map service.

One scenario where you might ask yourself this question is when deciding whether the overview map should be in a data frame on the same map service or have a different map service for the overview map. Obviously, the overview map only has some of the layers from the map itself.

The REST API in ArcGIS Server 9.3 will only support the current data frame in the MXD.

Now to the answers, it is not a bad idea to have multiple data frames in a single MXD and consume them as separate resources as long as the multiple data frames (or maps) in the MXD do not get used simultaneously all the time. When used simultaneously, a single SOC process is going to process the requests for both the maps and thus the concurrent requests are going to slow down the service. If the maps are going to be used concurrently, then it is better to create different map services for each of the maps, this way different process can process the requests and can return faster. That being said, for most cases it is better to create separate map services for each map.