Debugging embedded JavaScript files in Visual Studio

The credit for this post goes to Rachel who recently informed me how to debug and step through embedded JavaScript files in Visual Studio when using IE. Before today, whenever I had to debug through embedded JavaScript files I resorted to using FireFox even though the application I was building needed to be delivered on IE. What is an embedded JavaScript file? These are the JavaScript files that are embedded inside assemblies and retrieved by the web server using the WebResource.axd handler. Anyway, I have been using them extensively when developing custom tasks (ASP.NET server controls) for the ArcGIS Server .NET ADF. This way all the dependencies for the server controls are contained right within the server control assembly and now the server control can be easily redistributed and used by clients by a simple drag and drop.

The Solution, a “hidden” feature in Visual Studio called the “Script Explorer”.

In visual studio, right click anywhere and toolbar, Customize > Commands Tab > Debug Category > Drag and drop Script Explorer. It is greyed out unless you are in debug mode. In IE, open options, make sure you’ve enabled script debugging. Start debugging your web application, while running, click on script explorer window.  you’ll see all the URL’s to the embedded resources, double click them to bring them up in VS window. You should see the embedded JS code, set breakpoints, mouse over as desired.

Advertisement

The proper way to initialize a resource in the .NET ADF

If you have ever tried to use the MapResourceManager on a page without a Map Control or use the functionalities directly through the ResourceItems on the MapResourceManager, you must have found that the functionalities do not work as expected. The reason the functionalities do not work as expected is because the resources underlying the resource items should be initialized before use. Reading over the ESRI’s .NET ADF library, there are 2 methods that suggest that they will accomplish the task of initializing the resource, the Initialize method on the IGISResource interface and the InitializeResource method on the GISResourceItem.

But the Initialize method on the IGISResource interface does not do the job right. In order to initialize the resource right use the InitializeResource method on the GISResourceItem. This is not documented anywhere as far as I know. So, if the Query method on the IQueryFunctionality interface returns null for no reason when you expect results, trying initializing the resource using the InitializeResource method on the GISResourceItem.

Update: Heard from some ESRI folks on this issue. IGISResource.Initialize() initializes the resource only. IGISResourceItem.InitializeResource() initializes both the resource and the datasource of the resource. Always to use InitiializeResource(), since there are more logic here for handling initialization errors, validation stuff etc, whereas, with  Initialize() it’s dependent on the individual resource’s implementation.

Quick note on KML support in the ArcGIS Server 9.3 REST API

ArcGIS Server 9.3 supports multiple output formats in its REST API

• HTML
• JSON
• KMZ
• image
• help
• lyr, nmf(ArcGIS Explorer document), jsapi(Javascript API), gmaps(Goolge Maps), ve(VirtualEarth)

Good news on the KML support though. The output is actually an KMZ output with includes all symbology for the features from the Map Service. The symbols are outputted as images and zipped to a KMX along with the KML file. This is great news because it will help reduce a whole lot of redundant work for developers. Obviously the other formats like JSON that do not support symbology do not include the symbology in them.

Should you have multiple maps (or dataframes) in your MapServer MXD for ArcGIS Server

At the ESRI Dev Summit 2008, I managed to get some insights into questions that have been bouncing around in my head from the ESRI folks building ArcGIS server. One such design question was whether it was reasonable to create MXDs with multiple maps in them that can be accessed as different map resources or just create different map services for multiple maps. By creating a map service with multiple data frames in them you can serve both maps with just a single instance of a SOC. When creating different map services for different maps you end up with different SOCs for each map service.

One scenario where you might ask yourself this question is when deciding whether the overview map should be in a data frame on the same map service or have a different map service for the overview map. Obviously, the overview map only has some of the layers from the map itself.

The REST API in ArcGIS Server 9.3 will only support the current data frame in the MXD.

Now to the answers, it is not a bad idea to have multiple data frames in a single MXD and consume them as separate resources as long as the multiple data frames (or maps) in the MXD do not get used simultaneously all the time. When used simultaneously, a single SOC process is going to process the requests for both the maps and thus the concurrent requests are going to slow down the service. If the maps are going to be used concurrently, then it is better to create different map services for each of the maps, this way different process can process the requests and can return faster. That being said, for most cases it is better to create separate map services for each map.

Little known bottlenecks for ArcGIS Server Map cache generation performance

I had a chance to ask some of the ESRI developers the reason why the time required for our map cache generation didn’t reduce by very much when we started running SOCs on multiple machines (more powerful machines with Dual Quad core processors) with increased SOC instances. The bottleneck in these cases happen to be the following

• ArcSDE
• Disk speed (RPM) of the cache destination

Ok, now for the reasons as to why they are bottlenecks and some suggestions on how they can be worked around. The map service for which we were generating caches for, had almost all of its layers coming in from a single ArcSDE instance. When a large number of SOC instances (or processes) start to generate the map cache, all of them are accessing the same ArcSDE instance concurrently to generate the map cache. This multiple simultaneous access of the ArcSDE could slow down access to the map data and thus slow down map cache generation. One way this bottleneck could be worked around is by copying the ArcSDE data to multiple file geodatabases on different machines and referencing different layers in the map service from different file geodatabases. This way access to the map data is not throttled at just a single machine and is split amongst different machines.

All the SOC instances are also outputting their cache files to the same output destination. This means that this could also be a bottleneck if the disk speed of the output destination is not fast enough. One way that I can think of to work around this issue is by running different map cache generation processes for different scale levels and have their outputs going to different machines. Once done, the caches for the different scale levels can be copied to a single destination(ESRI uses SecureCopy to copy map caches between disks for their ArcGIS Online services). But, I don’t believe that this technique would save any time at all and would be too much trouble for not enough returns. So, just spend some money on a really fast hard disk.

Larger tile sizes for map caches take lesser disk space and generating them take lesser time. ArcGIS Online uses a 512X512 tile size whereas VirtualEarth and Google Maps use a 256X256 tile size.

ESRI Dev Summit 2008 – The real skinny

I am going to try to keep this post clear of chatter about how ESRI is doing things and my opinions on them. I would like to thank all the ESRI developers who took the time to answer some of my questions. So, here are some important tech notes to take away from the conference.

REST API

• The REST API does not support curves to be returned.
• When curves are encountered, they are densified automatically and returned as polylines.
• Support for curves is being planned for in the future versions.
• They were also open to the idea of GeoJSON converters in the JavaScript API for the future.
• The shapes returned by the REST API are not GeoJSON compliant. But the ESRI developers seem open to support GeoJSON when it is finally ratified by the standards committee.
• The REST API does all its work by using the ArcGIS Server SOAP API. See the notes below for the reasons why.

JavaScript API

• The ESRI developers chose to use the DOJO toolkit primarily because its graphics capabilities.
• The DOJO client-side libraries provide support for performing some simple spatial operations like ‘intersection’ on the client-side.
• The intellisense for the Javascript API currently has some problems in Visual Studio 2008.
• The client-side layers provides support for attributes on the client-side features. Attributes on client-side features are not even supported in VE and Google Maps.
• The client-side layers support events.
• Currently, only one client-side layer is supported. The reason for this being (according to the ESRI developers) there are problems with listening to events on features that are not on the top-most layer. ESRI is currently researching the problem and is trying to find ways to make multiple client-side layers work.
• The client-side JavaScript for the .NET ADF has been completely re-built in 9.3.
• Changes are already planned for the JavaScript API between the Beta version and the final release.
• An object model diagram for the JavaScript API might also be available.

Map cache

• ArcGIS Server 9.3 supports the following tiling scheme
  • VirtualEarth (256X256)
  • Google Maps (256X256)
  • ArcGIS Online (512X512)
  • Custom
• The Map cache generated using the VirtualEarth tiling scheme does not generate tiles using the quadkey naming scheme for the file names generated but only the tiling scheme is the scheme.
• The REST API has built-in support for handling quadkey naming scheme, it maps the quadkeys to the file names generated by the ArcGIS Server cache.
• The WMS Service of the MapServer object will start taking advantage of the Map cache if available starting in 9.3.

Geodatabase

• Sounds like SDE will support storing shapes in SQL Server 2008 as SQL CLR types also. This should enable other applications to access and use spatial data in the SDE without ArcObjects.

Note: It was decided to use SOAP over DCOM for the REST handlers to talk to ArcGIS Server because ESRI realized after running some metrics that SOAP was more performant than DCOM to access ArcGIS Server functionalities. The chatty nature of DCOM reduces its performance over SOAP. When using DCOM, the client uses a proxy to communicate with the server object. That means every time the client accesses a property on the server object, it is accessed over the network. But when using SOAP, an object representation is completely serialized and sent to the client so that property access doesn’t need to be over the network.

Even though it is not completely documented, it is possible to access Server Object Extensions over SOAP directly with some custom programming.

Dynamic legend generation through the ADF

I have uploaded my code snippet to generate legend images through the ADF here. The purpose for this grew out of being able to print multiple map resources on the map through the ADF. I had blogged about it before here. I have made some changes to the code snippet after input from some other people trying to do the same thing. The features of the code snippet are:

• Generates images at the required DPI
• Customizable legend title
• Handles group layers
• Handles symbol groups
• Creates a legend image of the desired height and width. If the legend entries overflow the height specified, they will be omitted. If all legend entries are required in the images, it should only involve a small change in code to remove the check for overflow.

Here is an example of a legend generated by the snippet.

Link to the "ArcGIS Server REST API"

http://sampleserver1.arcgisonline.com/ArcGIS/SDK/REST/index.html (not sure how long this is going to be up for). ArcGIS 9.3 Beta is supposed to have started shipping to the Beta users.

The REST API, Google Maps and VirtualEarth adapters in ArcGIS server 9.3 does hold a lot of promise. These technologies should enable accessing GIS services from the browser a breeze. Unlike Flex the release of Silverlight should also allow all the .NET developers to easily build more usable web-based GIS applications. Handling and processing large datasets through Silverlight should also be more performant than manipulating the DOM using javascript. It should also allow for performing basic and simple geo-processing(intersections etc) at the browser and eliminate round trips to the browser. Unlike Flex, I think the power of Silverlight lies in the fact that it is going to open up RIA development to all the .NET developers.

Creating an installer for ArcGIS Server Object Extensions

If you have been using Server Object Extensions (SOE) with ArcGIS Server, you might have had the pleasure of manually registering the various DLLs involved when deploying to the ArcGIS Server machine. I use a setup and deployment project to create an MSI installer that I can easily distribute and deploy. Many of the developers using SOEs probably have done this already. I thought I would share my version of the installer project so that it can help developers who are not already using it and to get feedback on the way I am doing it from developers who have mastered setup and deployment projects. So, If you have suggestions for improvements, I would love to hear it. That said, here is how I went about creating my MSI installer.

• I am assuming that the SOE being deployed is composed of two or possibly a third project as below
• A SOE definition project – This project contains the interface definition containing the methods exposed by the SOE.
• A SOE implementation project – This project contains the implementation of the SOE interface in the above project.
• An optional SOE property page project – This project contains the user interface elements that will be used to edit the SOE configuration properties via ArcCatalog.
• Add a new installer class to the SOE definition project and copy paste the code snippet here into it. The custom action in this installer class just registers the assembly equivalent to doing a “regasm.exe” on it.
• 
• Add a new installer class to the SOE implementation project and copy paste the code snippet here into it. The custom action in this installer class registers the services in the assembly equivalent to running a “regsvcs.exe” on it..
• Create a new “Setup Project” in Visual Studio
• 
• Bring up the “File System Editor”
• 
• Right click on the Application folder and select “Add->Project output”.
• 
• Add the “Primary Output” from the project containing the SOE definition and the implementation.
• 
• Right click and “Exclude” all ESRI DLLs under the “Detected Dependencies”. Leave only the “Primary Output” from the projects and any assemblies that they depend on included.
• 
• Right click on the project and select “View->Custom Actions”.
• 
• Right click the “Install” folder and select “Add Custom Action”.
• 
• In the “Select Item in Project” dialog, select the “Primary output” from our SOE definition and implementation projects.
• 
• A similar setup project can be created for installing the SOE property page project using the same custom installer class used for the SOE definition project.
• Just a final reminder to make things clear, the setup project is for installing the SOE on the ArcGIS Server machine and not on the web server machine.

Printing at 300 DPI with ArcGIS Server purely through the ADF

I just finished up a task for the current project I am working on where the client needed to be able print all the resources in the map at multiple resolutions and paper sizes. There are a lot of code snippets on ArcScripts that let you print maps using ArcGIS Server using the IMapServerLayout interface on the MapServer object. But using that approach did not solve 2 major requirements I had to satisfy

• Ability to print multiple map resources that might be loaded on the map
• Ability to print the GraphicsLayer from the web tier

Both these requirements would prevent using the IMapServerLayout interface for the printing task since IMapServerLayout works only on single map server objects and it cannot include the GraphicsLayer easily. So, I had just decided to use the DrawExtent method on the IMapFunctionality of every resource on the map after setting the required values on the DisplaySettings property and merge them together to get the complete map. Since, I had to print at 300 DPI, the size of the images I was requesting from ArcGIS Server was pretty huge. For printing a map on a A4 size paper, I was requesting an image size of ( ~ 3000 X 2000). But, whenever I made a call to DrawExtent for an image that size, I kept getting an image of size (2048 X 2048). After, banging my head against the walls for a few hours and using SOEXplorer, I figured out that MapServer objects have “MaxImageWidth” and “MaxImageHeight” settings that default to 2048. Once, I bumped up that number I was getting back images of the size that I requested. You can also bump up the “MaxRecordCount” property to enable ArcGIS Server to return more than 500 records for large queries. But before you bump this number up, take a minute to think about the consequences of handling such large number of records on the client side.

I was also concerned about the performance of ArcGIS Server when requesting images this size but the performance was acceptable and ArcGIS Server kept chugging along pretty good after several such requests. The GraphicsLayer also didn’t cause any problems for images that size (~ 3000 X 2000).

I moved on to generating ANSI D (plotter size maps) which resulted in size of images requested being more than (~ 10000 X 10000). To my pleasant surprise, ArcGIS Server didn’t throw a fit when requesting images that size. But, the GraphicsLayer didn’t like the request that size and decided to throw an exception saying ‘Parameter is not valid”. So, printing on a ANSI D size paper at 300 DPI should be possible if you don’t need that GraphicsLayer in the printout.

The GraphicsLayer decided to misbehave again when I was requesting images whose aspect ratio did not match the aspect ratio of the map on the browser. The ArcGIS Server had again played nice and returned map images with the aspect ratio requested. So, with the two resources not behaving consistently, I ended up with the final merged map images looking like below. You can see that the polygon on the GraphicsLayer was getting stretched vertically instead nicely overlaying the polygon from ArcGIS Server below.

But, this problem above I was able to work around. I was able to pre-calculate the extents that will be returned by the server and request the same size from the GraphicsLayer.

After all this, the printing functionality was not complete, the ADF had no easy way of obtaining a legend for all the resources on the map. You would think that the ADF would have the ability to generate a legend for the map already, but no luck on that yet. Hopefully in 9.3? But we couldn’t wait and hope for ESRI to include that feature. So, I went ahead and built the legend for the map dynamically. It wasn’t really that hard to do. I was able to request the swatches for the legend from the IMapTocFunctionality and create an image of the legend dynamically. Please contact me if you interested in the piece of code that generates the legend. I will be happy to share it with you. An example of the dynamically generated legend image is shown below.

With all the ingredients in place, I was able to generate a PDF of the map print out (looking like below) using the iTextSharp library.