Vishful thinking…

Notes from Silverlight unit testing with the ESRI Silverlight API

Posted in ESRI, Silverlight by viswaug on February 1, 2010

I have been using the ‘Silverlight Unit Test Framework’ to do unit testing for our Silverlight applications. Unit testing applications using the ESRI Silverlight API throws a monkey wrench into the Silverlight unit testing process. The Silverlight unit testing libraries can normally be run with the automatically generate HTML page option in VisualStudio 2008. But unit tests with the ESRI Silverlight API map control will not run on the auto-generated HTML pages since the map control cannot load its layers when run from a web page on disk. So, to test the mapping applications, the unit tests should be run with a web application project which hosts the page over HTTP.

Silverlight_Unit_Tests

Well, that wasn’t a very painful solution was it? But the pain enters the picture when we have to run these Silverlight mapping unit tests on the build server. We don’t necessarily want to publish a web application just to run the Silverlight mapping unit tests. Ideally, we would just like to run the unit tests in the XAP file by themselves without publishing a web application with IIS on the build machine. In fact, our build machine does not run IIS. the ‘Silverlight Unit Test Framework’ doesn’t include any built-in support for such use cases.

But we can solve the problem above using the Statlight Silverlight testing automation tool. Statlight also provides support for integration with TeamCity, our build server of choice. The Statlight tool runs the test library XAP file over HTTP using a web server that it hosts when running. The tool is pretty simple to use and more details can be found at the ‘Getting Started’ page. I did run into some issues with running my tests over Statlight because of the fact that the Statlight tool actually creates it own XAP file when running the tests and copies over only the DLLs from the XAP file being tested. I had made use of some ‘Content’ files in my unit tests that didn’t get copied over and caused the issue. But I was able to solve the problem easily by switching the ‘Content’ files into embedded file in the DLL. Statlight’s creator was also kind enough to give me more information about the inner workings of the Statlight tool. Here it goes

 

    1. Creates a webserver in the background. (hosted in the console)
    2. Creates a browser in the background. (WinForm thread with a WebBrowser control)
    3. The browser loads up StatLight silverlight application. (http://localhost:<port discovered>/…)
    4. The silverlight application connects to the webserver and retrieves the xap to test via REST.
    5. The SL app loads and exectues the tests in the xap.
    6. The SL app reports results back to the webserver via WCF.
    7. The webserver reports results back to the Console / CI server.

So, with Statlight, we can avoid unnecessary complexities with running Silverlight unit tests on the build server and report Silverlight mapping unit test results with the rest of the test results. If you are looking for solutions to running your Silverlight unit tests on your build server, i recommend looking into Statlight.

Quick lesson learnt implementing continuous zoom operations for the ESRI Silverlight API

Posted in ArcGIS, ESRI, GIS, Uncategorized by viswaug on February 1, 2010

One of the nicer navigation features that enhance suser experience on mapping applications is the continuous zoom in & out and pan operations. Also, users that use Google and Bing maps frequently expect the continuous zoom and pan experience in all the mapping applications 🙂 The ESRI Silverlight API Toolkit does not include support for continuous zoom and pan operations. But implementing the continuous navigation operations for the ESRI Silverlight API is real easy with the DispatcherTimer class that was introduced with Silverlight version 2.0. The DispatcherTimer class calls the ‘Tick’ event listeners on the UI thread so developers don’t have to worry about accessing UI controls from across threads. Implementing it is as simple as setting the map’s extent very frequently with new extent calculated for zoom in/out/pan operations. To provide a good experience, I had to reset the map’s extent every 10 milliseconds.

I had initially overlooked the fact that every time I reset the map’s extent every 10 milliseconds a map image request was being made for every DynamicLayer that was loaded on the map. The problem doesn’t occur with the TiledMapServiceLayer because requests for tile images only occur after the zoom/pan operation has caused the map’s extent to exceed the extent of the tile map images already loaded on to the map and the browser also caches the tiles client-side. The excessive number of map image requests (1 request every 10 milliseconds) generated by the continuous zoom operation would bring any GIS server (and did bring our server) to its knees pretty soon. There is no perfect solution to the problem here in my humble opinion. But an acceptable solution to the problem is to hide (change visibility to false) all the DynamicLayer(s) on the map when the continuous navigation operation begins and to reset the layers back to their previous state when the continuous navigation operation ends. Since the base map layers are TiledMapServiceLayer(s), they don’t pose the problem with the excessive number of requests and we can leave them visible. This actually provides an acceptable user experience during the continuous navigation operations.

The code snippet below illustrates the implementation of a sample continuous zoom in & out operations.

Note – In Silverlight any control that inherits from ButtonBase like Button, CheckBox, RadioButton etc will not raise the ‘MouseLeftButtonDown’ and ‘MouseLeftButtonUp’ events even though the events are available on the classes.

public partial class MainPage : UserControl

{

    private DispatcherTimer _timer;

    Dictionary<Layer, bool> _mapLayersVisibilityState = null;

 

    public MainPage()

    {

        InitializeComponent();

 

        IDisposable subscription = null;

        subscription = myMap.Layers.GetLayersLoadCompleted().Subscribe

            (

                ( args ) =>

                {

                    txtStatus.Text = "All layers have loaded.";

                    subscription.Dispose();

                }

            );

        txtStatus.Text = "Loading Layers onto map …";

    }

 

    void zoomOut_MouseLeftButtonUp( object sender, MouseButtonEventArgs e )

    {

        if( _timer != null )

            _timer.Stop();

        EndContinuousOperation();

    }

 

    void zoomOut_MouseLeftButtonDown( object sender, MouseButtonEventArgs e )

    {

        BeginContinuousOperation();

        _timer = new DispatcherTimer();

        _timer.Interval = new TimeSpan( 0, 0, 0, 0, 10 );

        _timer.Tick += _zoomOutTimer_Tick;

        _timer.Start();

    }

 

    void zoomIn_MouseLeftButtonUp( object sender, MouseButtonEventArgs e )

    {

        if( _timer != null )

            _timer.Stop();

        EndContinuousOperation();

    }

 

    void zoomIn_MouseLeftButtonDown( object sender, MouseButtonEventArgs e )

    {

        BeginContinuousOperation();

        _timer = new DispatcherTimer();

        _timer.Interval = new TimeSpan( 0, 0, 0, 0, 10 );

        _timer.Tick += _zoomInTimer_Tick;

        _timer.Start();

    }

 

    void _zoomInTimer_Tick( object sender, EventArgs e )

    {

        ZoomMapByFactor( 0.99 );

    }

 

 

    void _zoomOutTimer_Tick( object sender, EventArgs e )

    {

        ZoomMapByFactor( 1.01 );

    }

 

    private void BeginContinuousOperation()

    {

        _mapLayersVisibilityState = GetMapLayersVisibilityState();

        HideAllDynamicMapLayers();

    }

 

    private void EndContinuousOperation()

    {

        SetMapLayersVisibilityState( _mapLayersVisibilityState );

    }

 

    private Dictionary<Layer, bool> GetMapLayersVisibilityState()

    {

        Dictionary<Layer, bool> visibilityStates = new Dictionary<Layer, bool>();

        myMap.Layers.ForEach<Layer>( lyr => visibilityStates.Add( lyr, lyr.Visible ) );

        return visibilityStates;

    }

 

    private void SetMapLayersVisibilityState( Dictionary<Layer, bool> visibilityStates )

    {

        if( visibilityStates == null )

            return;

 

        foreach( var entry in visibilityStates )

        {

            entry.Key.Visible = entry.Value;

        }

    }

 

    private void HideAllDynamicMapLayers()

    {

        myMap.Layers.ForEach<Layer>( lyr =>

        {

            if( lyr is DynamicLayer )

                lyr.Visible = false;

        } );

    }

 

    void ZoomMapByFactor( double factor )

    {

        if( myMap != null )

        {

            Envelope env = myMap.Extent;

            env.Expand( factor, myMap.RenderSize );

            myMap.Extent = env;

        }

    }

}

The code above makes use of the ‘Expand’ extension method for the ‘Envelope’ type provided below.

public static void Expand( this Envelope extent, double factor, Size mapSize )

{

 

    if( extent == null )

        throw new ArgumentNullException( "extent" );

    if( mapSize.Width <= 0 )

        throw new ArgumentOutOfRangeException( "mapSize.Width" );

    if( mapSize.Height <= 0 )

        throw new ArgumentOutOfRangeException( "mapSize.Height" );

 

    if( extent.XMax == extent.XMin )

    {

        double resolution = extent.Width / mapSize.Width;

        double xVal = extent.XMin;

        //using a 10 pixel buffer

        extent.XMin = xVal 10 * resolution;

        extent.XMax = xVal + 10 * resolution;

    }

 

    if( extent.YMax == extent.YMin )

    {

        double resolution = extent.Height / mapSize.Height;

        double yVal = extent.YMin;

        extent.YMin = yVal 10 * resolution;

        extent.YMax = yVal + 10 * resolution;

    }

 

    MapPoint center = extent.GetCenter();

    double dX = ( extent.Width / 2 ) * factor;

    double dY = ( extent.Height / 2 ) * factor;

 

    extent.XMin = center.X dX;

    extent.XMax = center.X + dX;

    extent.YMin = center.Y dY;

    extent.YMax = center.Y + dY;

}

Enhance your development experience with the ESRI Silverlight API using the Microsoft Reactive Extensions

Posted in .NET, ArcGIS, C#, ESRI, Uncategorized by viswaug on January 31, 2010

In my humble opinion, Reactive Extensions is one of the most exciting libraries to come out of Microsoft lately. And that is saying a lot since there have been quite a bit of good libraries coming out of Redmond recently. The Rx for .NET takes a lot of pain out of asynchronous programming and has drastically reduced the amount of code required to handle and orchestrate responses from multiple web services. Given that Silverlight is a client-side technology, most of the Silverlight applications make calls to web services and display results to the user as needed. A common application scenario is where the Silverlight client needs to make multiple calls to multiple web services and orchestrate calls to other web services based on responses from previous web service calls. Handling this scenario can get quite a bit complicated when you have to maintain contexts to store response states from the various web services. This is where Rx for .NET comes in real handy. The following scenario and the solutions should illustrate how Rx for .NET can help when programming with the ESRI Silverlight API for example.

When adding ArcGIS Map service layers to the Map control in the ESRI Silverlight API, the map control makes a HTTP request for every layer added to request the metadata for the map service. Depending on whether the request to the service metadata succeeds or fails the layer object would raise a ‘Initialized’ event or a ‘InitializationFailed’. So, a layer ‘load’ can be considered to be complete when either the ‘Initialized’ or the ‘InitializationFailed’ has fired. If we want to initiate another sequence of operations when all the ‘load’ of all the layers on the map have completed, like say populate a legend/ Table of Contents, then we will have to wait for all the layer ‘load’ to be complete (Success/Failure). Making a request to the legend web service in the above scenario can get quite cumbersome. But Rx for .NET can simplify things quite a bit. Follow the code sample below to accomplish the task.

The two extension methods below will help us watch for the ‘load’ complete for a layer and to watch for all the layers in a LayerCollection to complete. As you can see, we use the ‘Merge’, ‘Take’ and the ‘ForkJoin’ observable extension methods on the IObservable to watch for the right sequence of events.

public static class ExtensionMethods

{

    public static IObservable<IEvent<EventArgs>> GetLayerLoadCompleted( this Layer layer )

    {

        if( layer == null )

            throw new ArgumentNullException( "layer" );

 

        if( layer.IsInitialized == true )

        {

            IEvent<EventArgs>[] events = new IEvent<EventArgs>[ 1 ] { null };

            events.ToObservable<IEvent<EventArgs>>();

        }

 

        IObservable<IEvent<EventArgs>> oSuccess = Observable.FromEvent<EventArgs>

            (

                ev => layer.Initialized += ev,

                ev => layer.Initialized += ev

            );

 

        IObservable<IEvent<EventArgs>> oFailed = Observable.FromEvent<EventArgs>

            (

                ev => layer.InitializationFailed += ev,

                ev => layer.InitializationFailed += ev

            );

 

        IObservable<IEvent<EventArgs>> obsCompleted = oSuccess.Merge<IEvent<EventArgs>>( oFailed );

 

        IObservable<IEvent<EventArgs>> oFirst = obsCompleted.Take<IEvent<EventArgs>>(1);

 

        return oFirst;

    }

 

    public static IObservable<IEvent<EventArgs>[]> GetLayersLoadCompleted( this IEnumerable<Layer> layers )

    {

        if( layers == null )

            throw new ArgumentNullException( "layers" );

 

        return Observable.ForkJoin<IEvent<EventArgs>>

            (

                from item in layers

                select item.GetLayerLoadCompleted()

            );

    }

}

And now, we can add a handler to be executed when all the layers have loaded like below.

public MainPage()

{

    InitializeComponent();

 

    IDisposable subscription = null;

    subscription = myMap.Layers.GetLayersLoadCompleted().Subscribe

        (

            ( args ) =>

            {

                HtmlPage.Window.Alert( "All layers have loaded." );

                subscription.Dispose();

            }

        );

}

Making HTTP Post with Silverlight a little easier

Posted in C#, Silverlight by viswaug on September 29, 2009

In my earlier post, I talked about how to go about making an HTTP Post in Silverlight. The whole process is a little complicated to say the least. Silverlight does not come with an utility class like WebClient that makes making a HTTP Get a little easier. So, I created a class called ‘HTTPPostHelper’ that does for HTTP Post what WebClient does for HTTP Get. The ‘HTTPPostHelper’ class listed below, makes the whole process a lot easier. To use the class, do the following

  • Create an instance of the HTTPPostHelper class with the type of the input class (T) and the type of the returned class (K) as the generic arguments.
  • Listen to the ‘PostCompleted’ event that gets called when the Post operation completes successfully. The EventArgs of the event contains an instance of the return class type.
  • Listen to the ‘PostFailed’ event that gets called when the Post operation fails. The EventArgs of the event contains the exception information.
  • Call the ‘Post’ method on the HTTPPostHelper with the input object and the URL address to make the Post operation happen.
  • The input object gets serialized by default using the DataContractSerializer. But this can be overridden by setting the Serializer property of the HTTPPostHelper with an appropriate delagate that serializes the input object to a stream.
  • The output from the HTTP Post operation is deserialized by default using the DataContractSerializer. But this too can be overridden by the Deserializer property of the HTTPPostHelper.
  • For example, if you want to send and receive JSON data to the HTTP endpoint, just override the default Serializer & Deserializer with delegates that use DataContractJsonSerializer instead or your own custom binary serializer if need be.
  • The ‘ContentType’ used for the Post operation can also set before calling the Post method.
  • The class handles all the thread hopping internally. The ‘PostCompleted’ and the ‘PostFailed’ events get raised on the same thread from which they were called from.

 

public class HTTPPostHelper<T, K>

{

    SynchronizationContext syncContext;

    HttpWebRequest _request;

 

    public string ContentType

    {

        get;

        set;

    }

 

    public event EventHandler<EventArgs<K>> PostCompleted;

 

    public event EventHandler<ExceptionEventArgs> PostFailed;

 

    public Action<T, Stream> Serializer = null;

 

    public Func<Stream, K> Deserializer = null;

 

    public HTTPPostHelper()

    {

        Serializer = DefaultSerializer;

        Deserializer = DefaultDeserializer;

        ContentType = “application/stream”;

    }

 

    void DefaultSerializer(T input, Stream output)

    {

        DataContractSerializer serializer = new DataContractSerializer(typeof(T));

        serializer.WriteObject(output, input);

        output.Flush();

    }

 

    K DefaultDeserializer(Stream input)

    {

        DataContractSerializer serializer = new DataContractSerializer(typeof(K));

        K result = (K)serializer.ReadObject(input);

        return result;

    }

 

    public void Post(Uri address, T input)

    {

        syncContext = SynchronizationContext.Current;

 

        _request = (HttpWebRequest)WebRequest.Create(address);

        _request.Method = “POST”;

        _request.ContentType = “application/stream”;

 

        _request.BeginGetRequestStream(new AsyncCallback(PostRequestReady), new HttpWebRequestData<T>()

        {

            Request = _request,

            Data = input

        });

    }

 

    private void PostRequestReady(IAsyncResult asyncResult)

    {

        HttpWebRequestData<T> requestData = asyncResult.AsyncState as HttpWebRequestData<T>;

        HttpWebRequest request = requestData.Request;

        Stream requestStream = request.EndGetRequestStream(asyncResult);

        Serializer(requestData.Data, requestStream);

        requestStream.Close();

 

        request.BeginGetResponse(new AsyncCallback(PostResponseCallback), request);

    }

 

    private void PostResponseCallback(IAsyncResult ar)

    {

        try

        {

            HttpWebRequest request = ar.AsyncState as HttpWebRequest;

            HttpWebResponse response = request.EndGetResponse(ar) as HttpWebResponse;

            Stream responseStream = response.GetResponseStream();

            K urls = DefaultDeserializer(responseStream);

 

            syncContext.Post(PostExtractResponse, new HttpWebResponseData<K>()

            {

                Response = response,

                Data = urls

            });

        }

        catch (Exception ex)

        {

            syncContext.Post(RaiseException, ex);

        }

    }

 

    private void RaiseException(object state)

    {

        Exception ex = state as Exception;

        if (PostFailed != null)

            PostFailed(this, new ExceptionEventArgs(ex));

    }

 

    private void PostExtractResponse(object state)

    {

        HttpWebResponseData<K> responseData = state as HttpWebResponseData<K>;

        if (PostCompleted != null)

            PostCompleted(this, new EventArgs<K>(responseData.Data));

    }

}

 

public class ExceptionEventArgs : EventArgs

{

    public Exception ExceptionInfo { get; set; }

 

    public ExceptionEventArgs(Exception ex)

    {

        ExceptionInfo = ex;

    }

}

 

 

 

public class EventArgs<T> : EventArgs

{

    private T eventData;

 

    public EventArgs(T eventData)

    {

        this.eventData = eventData;

    }

 

    public T EventData

    {

        get

        {

            return eventData;

        }

    }

}

Black magic behind the MSD Map Services?

Posted in ArcGIS, ESRI by viswaug on September 29, 2009

Much has been said about the performance boost that can be achieved by running MSD map services over MXD map services. Here I will talk a little bit about the other side of the MSD map services which trips up developers like me. The only information that I was able to find about the capabilities/short-comings of the MSD map services from ESRI was on their blog post here. Since, I develop and use a few custom Server Object Extensions (SOEs), I noticed that some of the SOEs that I had written earlier had stopped working when registered with MSD map services. And the following piece of information from the blog post mentioned earlier seemed to indicate the cause of the problem.

What capabilities (server object extensions) are supported for MSD-based map services?

The WMS and KML capabilities are available with MSD-based services, as well as the default Mapping capability. Custom server object extensions can also work with MSD-based services but they cannot access fine-grained ArcObjects; they must use the coarse-grained methods on MapServer and its related classes.

The important part of the above statement is tha ‘custom SOEs cannot access fine-grained ArcObjects’. Well, what does that really mean? It kinda means that you cannot access the fine-grained ArcObjects features that are exposed through the IMapServerObjects interface on the MapServer server object. But being from the .NET world, I had trouble wrapping my head around how the same MapServer server object class can serve both the MXD map services and also the MSD map services and behave differently for each service type. How can the same MapServer class offer the IMapServerObjects features for MXD map services but not for MSD map services? I tried casting(QI) the MapServer server object obtained from a MSD map service to an IMapServerObjects interface which actually resulted in an exception being thrown. But the same cast worked on the MapServer object obtained from a MSD map service. My first thought when I noticed this behavior was that maybe the MSD map services are actually served by a different new server object class and not the MapServer class that serves the MXD map services. The ESRI  9.3.1 documentation doesn’t mention anything about a new server object. So, either ESRI didn’t document it or the same MapServer has developed multiple-personalities in AGS 9.3.1. To test it out, I obtained the CLSID of the server object that served MSD map services and the MXD map services. The were the exact SAME. That probably means that the same MapServer class is just behaving differently in different usage scenarios by leveraging some COM black magic which I don’t have too much knowledge about.

Since, we can’t get at the rich features in the IMapServerObjects interface, the custom SOEs on MSD map services really become only a little more useful than custom COM utility objects. So, some of the custom server object extensions that developers wrote earlier for MXD map services will not work with MSD map services. And that is a real bummer for me. One way (that I really really hate) to overcome this limitation is to run another MXD map service for the MSD map service. The MSD map service makes the maps and the MXD map service is only used to obtain and use the features of the  custom server object extension. I do really hate the idea of running a another expensive map service process on the AGS machine only for the purpose of using the features of the custom server object extensions. Although, I do have some ideas on a better ways of working around these limitations. The most frustrating part about this whole thing is that I think internally ESRI does have access to fine grained ArcObjects for the MSD map services and they just don’t let us know how to gain access to the fine grained ArcObjects. Why/How do I say that? The reason for my suspicions that the above is the case is because, the WMS and KML server object extensions in-fact DO work with MSD map services also. Which means that those server object extensions are able to access fine-grained ArcObjects to do what they need to do.

Update : Another change that relates to SOEs in AGS 9.3.1 is regarding it registration process. As many might have noticed, their SOEs shows (or doesn’t showup :)) up without any label in the ArcCatalog->Service Properties->Capabilities tab. It took me a while to realize what was happening. I had initially assumed that the SOE wasn’t showing up in the list. But later, luckily I discovered that the SOE was in-fact displaying but there was just no label for it in the Capabilities list box. In 9.3, the IServerObjectExtensionType interface has been superseded by the IServerObjectExtensionType2 interface ArcObjects style. So, when you register the SOE with ArcGIS Server by calling the CreateExtensionType, the returned object should be casted to the type IServerExtensionType2 and the ‘DisplayName‘ property should be set on it before calling AddExtensionType. This ‘DisplayName‘ property is what shows up in the capabilities tab and if it is not set, the entry for the SOE in the capabilities tab will be empty. Earlier, the extension’s Name property used to show up in the capabilities tab. But no longer, it is the DisplayName property that is used to label the entry in the capabilities tab list box. I hope ESRI finds some time to update their documentation about this over here.

Tagged with: ,

Thoughts on making the ESRI REST API a little more scalable

Posted in ESRI by viswaug on September 22, 2009

These are just a couple of my thoughts on how the ESRI REST API can be made a little more scalable. The ESRI REST API has been a boon to me and probably many other GIS developers out there who had to get knee deep in the ADF and build usable apps with it. The ADF has gotten a lot better in 9.3 and the ADF guys at ESRI have done a good job of improving the framework by leaps and bounds. But still, I stay away from the ADF partly because it is still not easy to use and debug. Definitely not as easy to use as the REST API with JS/Flex/Silverlight clients. The REST API inherently leverages the power of ‘HTTP goodness’ and makes the scalability of the apps that are built with the light-weight clients much more scalable. The scalability is achieved because of the fact that the data is cached as necessary both the REST server and the client browser.

One of the concerns I (and others) have with the current architecture of the REST server in AGS 9.3 is with the way how the cached map services are served out by the server. Even though, all the tiles have already been cooked and saved on disk. The default way the tiles are served out by the server is through the AGS REST server. This means that the tiles/images on the disk are not directly served out by IIS itself but the AGS REST handlers (implemented as IHTTPHandler in the library) need to get involved. Every tile request coming into the handler is served out really fast but IIS does have to call into the ASP.NET module that spins up a new thread for this request and this thread is blocked until the image request has been completed. Again, let me re-iterate. This image request-response by the REST handler is very fast. But, why do we need to be satisfied with fast when we can be FASTER. If the tile requests are made directly to just the tile images on a web server (doesn’t need to have AGS) by default, the request doesn’t even need to get to the AGS (which now doesn’t need to be externally accessible and can be behind your firewall if need be) and this tile request can be satisfied by just the web server without having AGS involved. This way cached layers can be served to the light-weight clients by just the web server which could in the cloud or the images can be stored and served directly from Amazon storage services. These things are currently possible with the light-weight clients but ESRI can definitely do their part to make that process much easier. All that’s fine but why is the performance going to be any better? Well, if the web server (IIS) is just serving out tile images from the disk, we can enable kernel-mode caching in IIS to server out these tiles which is magnitudes faster than serving it through the handler. When kernel-mode caching is enabled in IIS, the request is satisfied purely by IIS without the request having to enter the ASP.NET and be served by the IIS worker process. So, yeah it is MUCH MUCH faster with kernel-mode caching.

Also, in my snooping around the REST API, I found that the handlers in the REST API are implemented as IHTTPHandlers. The scalability of the API can be improved a little if these handled are written as IHTTPAsyncHandlers. When using IHTTPAsyncHandlers the threads serving these requests are not blocked until the request is served. Since, the serialization to JSON etc currently happen on the REST web server in AGS 9.3, the time to serve the requests might be non-trivial. So, this makes a higher number of threads available in the thread pool for the IIS worker process to use. This will definitely increase the scalability of the ESRI REST API and your web application. Given, there are things that we need to pay attention to when using this async programming model since, the number of threads could explode when serving very large number of requests and the process might just spend a lot time setting up the threads and cleaning up after them.

Those are just two thoughts I had on the subject today and I managed to put in words. I am also all ears to other ideas/criticisms on the subject.

I am also hoping that in the near future, ESRI’s SOAP and REST API have the exact same service contracts that are just exposed at different endpoints. Currently, the SOAP API seems to more feature-filled and the REST API seems to be playing catch up. Aren’t SOAP and REST just different ways of accessing the same exact GIS feature set in ArcGIS Server? Alright, I will end the rant right here. 🙂

Making a HTTP Post in Silverlight a.k.a thread hopping

Posted in C#, Silverlight by viswaug on September 17, 2009

Making a HTTP GET request is as easy as pie using the WebClient class that is available in the Silverlight library. The DownloadStringCompleted event callback of the WebClient class is invoked on the main UI thread thus allowing the developer to access UI elements in the callback method itself. If the web service call resulted in a failure, then that condition can be easily determined in the DownloadStringCompleted event callback method by checking if the Error property on the DownloadStringCompletedEventArgs argument passed into the callback method is NULL. The code snippet below illustrates one way to use the WebClient class.

    WebClient webClient = new WebClient();

    webClient.DownloadStringCompleted += (sender, e) =>

    {   

        if (e.Error == null)   

        {       

            //Process web service result here       

            txtBox.Text = e.Result;   

        }    else   

        {       

            //Process web service failure here       

            txtBox.Text = e.Error.Message;   

        }    callback();

    };

    webClient.DownloadStringAsync( new Uri( “URL address that needs to be fetched” ) );

 

But when it comes to making a HTTP POST in Silverlight, things aren’t so simple. Silverlight doesn’t come with any helper classes like the WebClient (for making HTTP GET requests) to help take the hassle out of making a HTTP POST. So, to make a HTTP POST request, we must jump through two threads before we get back on the main UI thread where we will be able to access the UI elements and update them as needed. Here is a simple illustration of the process that we have to go through to make a HTTP POST.

SilverlightHTTPPost

Before we get into code snippets for actually accomplishing the above, let’s get into a little bit of why the above needs to happen. When the user initiates an UI event (button click etc) upon which we must make the HTTP request, we need to ask Silverlight/browser to open a HTTP request pipe to the specified URL. We get this request stream in a different non-UI thread. The reason for this being that Silverlight makes HTTP requests through the browser stack (Silverlight 3 can use its own HTTP stack if needed, but uses the browser stack by default). As you might know, browsers limit the number of outgoing HTTP requests to a given domain name. This limit used to be 2 but modern day browsers have increased this limit (8 in IE8 etc). So, given this limit on the outgoing HTTP requests, there is no guarantee that the browser can open a HTTP request pipe to the URL when demanded by Silverlight. So, we end up getting the request stream in a non-UI thread. We can write/post the necessary data to this request pipe and wait to get a response from the web service. The results from the web service is received on another different non-UI thread. Once we read all the information from the response stream, we can’t start updating the UI elements since we are still on a non-UI thread. So, we need to pass this response from the web service back to the UI thread before we can start updating the UI elements.

Since, we need to pass data between threads as explained above, I had written these tiny little classes to help us through the process of making the HTTP POST request.

 

    public class HttpWebResponseData<T>

    {

        public HttpWebResponse Response { get; set; }

        public T Data { get; set; }

    }

 

    public class HttpWebRequestData<T>

    {

        public HttpWebRequest Request { get; set; }

        public T Data { get; set; }

    }

 

    public class ExceptionEventArgs : EventArgs

    {

        public Exception ExceptionInfo { get; set; }

 

        public ExceptionEventArgs(Exception ex)

        {

            ExceptionInfo = ex;

        }

    }

 

    public class EventArgs<T> : EventArgs

    {

        private T eventData;

 

        public EventArgs(T eventData)

        {

            this.eventData = eventData;

        }

 

        public T EventData

        {

            get

            {

                return eventData;

            }

        }

    }

 

We need to store away the SynchronizationContext of the UI thread before we begin making the HTTP POST request and start jumping threads. This is so that we can get back on the UI thread when the HTTP POST web service call has returned. So, create a variable to hold the in the SynchronizationContext Silverlight UserControl.

        SynchronizationContext syncContext;

 

The following code snippet illustrates how to use the HTTPWebRequest and the HTTPWebResponse classes in Silverlight to perform the HTTP POST.

    //This method will called on user action (button click etc); We are on the UI thread

    private void DoHTTPPost(List<string> stuffToPost)

    {

        //Obtain and store away the SynchronizationContext of the UI thread in the private variable declared above

        syncContext = SynchronizationContext.Current;

        HttpWebRequest request = ( HttpWebRequest ) WebRequest.Create( “URL address to POST to” ) );

        request.Method = “POST”;

        request.ContentType = “text/xml”;

        request.BeginGetRequestStream( new AsyncCallback( DoHTTPPostRequestReady ), new HttpWebRequestData<List<string>>()

        {

            Request = request,

            Data = stuffToPost

        } );

    }

 

    //This method will be called on the non-UI request thread

    private void DoHTTPPostRequestReady(IAsyncResult asyncResult)

    {

        HttpWebRequestData<List<string>> requestData = asyncResult.AsyncState as HttpWebRequestData<List<string>>;

        HttpWebRequest request = requestData.Request;

 

        Stream requestStream = request.EndGetRequestStream(asyncResult);

        DataContractSerializer serializer = new DataContractSerializer(typeof(List<string>));

 

        //Write the string list as XML to the request stream

        StreamWriter sw = new StreamWriter(requestStream);

        serializer.WriteObject(requestStream, requestData.Data);

        sw.Flush();

        requestStream.Close();

 

        request.BeginGetResponse(new AsyncCallback(DoHTTPPostResponseCallback), request);

    }

 

    //This methods will be called on the non-UI response thread

    private void DoHTTPPostResponseCallback(IAsyncResult ar)

    {

        //Ignoring error handling here to keep things simple

        HttpWebRequest request = ar.AsyncState as HttpWebRequest;

        HttpWebResponse response = request.EndGetResponse(ar) as HttpWebResponse;

        Stream responseStream = response.GetResponseStream();

        DataContractSerializer serializer = new DataContractSerializer(typeof(string));

        string result = serializer.ReadObject(responseStream) as string;

 

        //Post the response from the web service back to the UI thread using the SynchronizationContext obtained when initiating the HTTP POST request

        syncContext.Post(DoHTTPPostExtractResponse, new HttpWebResponseData<string>()

        {

            Response = response,

            Data = result

        });

    }

 

    //We are back on the UI thread when this method is called

    private void DoHTTPPostExtractResponse(object state)

    {

        string response = (state as HttpWebResponseData<string>).Data;

        //We can update the UI elements with the response here since we are back on the UI thread

        txtBox.Text = response;

    }

 

I will be posting more helper methods for the above pretty soon. Also, I will delve more into making HTTP POST requests to WCF REST endpoints in future posts.

GeoJSON & GPX support added to the ESRI Silverlight API Contrib

Posted in ArcGIS, ESRI, GIS, Silverlight by viswaug on August 18, 2009

I’ve added support for the GeoJSON and GPX layer types to the ESRI Silverlight API Contrib project on CodePlex. Adding GeoJSON or GPX layer to the ESRI Silverlight API map is real easy and you would add it just like adding a FeatureLayer to the map.

Adding a GeoJSON layer to the map

<esri:Map x:Name="MyMap" Grid.Row="1">
            <esri:Map.Layers>
                <esri:ArcGISTiledMapServiceLayer ID="StreetMapLayer"
Url="http://server.arcgisonline.com/ArcGIS/rest/services/ESRI_Imagery_World_2D/MapServer"/>
                <slContrib:GeoJSONLayer ID="geoJSONLayer"
                     URL="http://localhost/SLMaps.Web/Data/JSON.txt" />
            </esri:Map.Layers>
</esri:Map>

Adding a GPX layer to the map

<esri:Map x:Name="MyMap" Grid.Row="1">
            <esri:Map.Layers>
                <esri:ArcGISTiledMapServiceLayer ID="StreetMapLayer"
Url="http://server.arcgisonline.com/ArcGIS/rest/services/ESRI_Imagery_World_2D/MapServer"/>
                <slContrib:GPXLayer ID="gpxLayer"
                 URL="http://localhost/SLMaps.Web/Data/fells_loop.txt" />
            </esri:Map.Layers>
</esri:Map>

This adds to the GeoRSS layer support already present in the library.

Other than just adding these data sources as layers to the map, The GeoJSONReader & GPXReader classes can also be used to just parse the data source and obtain the geometry and attribute values of the features in the data source if/as required. This should allow for the users to upload GeoJSON & GPX data from their GPS units directly from their computers to the Silverlight map control without the need to upload files to the server. Allowing the workflow to be smoother, faster and simple both on the server-side & client-side.

Bing map issues with IE6 & Silverlight 2 / ESRI Silverlight API 1.0

Posted in ArcGIS, ESRI, Silverlight by viswaug on August 17, 2009

I have been having troubles for a while now getting the Bing map layers to show in IE6 with the Silverlight 2 installed. Unfortunately, when I got down to the bottom of it, it turned out to be a bug in Silverlight 2 that actually prevents the Bing map layers to show in the ESRI Silverlight API 1.0 map control. Initially, I wasn’t sure why the bing map layers where not showing up on just the IE6 browsers but displayed and worked just fine on IE7+ and Firefox. To check the issue for yourself, if you are using an IE6 browser with the Silverlight 2 plugin, just browse over the Bing maps demo site setup by ESRI to check it out for yourself. you will see the error message shown below.

Part of the error message displayed above in the alert window (“Make sure you are generating a VE token from the same token service as the VE Layer ServerType: Staging or Production”) is actually not accurate. The demo site simply handles the ‘InitializationFailed‘ event on the Bing TileLayer and displays the message above not matter what the error actually was. This had mislead me initially and had sent me hunting for clues in the wrong direction. One other weird I noted here in the ESRI Silverlight API 1.0 is that the event arguments for the ‘InitializationFailed‘ actually doesn’t provide you with the error information. But the error information is actually available in the ‘InitializationFailure‘ property of the layer which I thought was a little confusing. Hoping this will change in the next versions of the API.

After spending a lot of time on fiddler, snooping around the requests made to the Bing servers & searching the web, I finally figured out the reason why the Bing layers do not show up on the map. The reason turned out to be a bug in the Silverlight 2 plug-in in IE6. The error had to do with Silverlight 2 plug-in in IE6 having troubles handling compressed content returned from the server. More details about the bug can be found here. Since the compressed content in this case comes from the Bing servers, I had not control over it at all and thus it eventually turned out that the problem couldn’t be solved in a good way. But there was an advanced setting in IE6 that I could turn off to get the bing layers to show up in the map. Under the ‘Tools’ menu in IE6, select the ‘Internet Options’ item. In the ‘Internet Options’ dialog, click on the ‘Advanced’ tab and in the settings disable the ‘Use HTTP 1.1’ option.

But this workaround does have its own drawbacks of not being able to use some of the advanced HTTP 1.1 capabilities like compression etc. So, please be mindful of it before adopting the solution.

Some salvaging facts,

  • This bug has been fixed in Silverlight 3. So, users who are still using IE6 but have upgraded to the Silverlight 3 plug-in will not have this problem.
  • IE6 is losing market share – (though a good chunk of the users on the web still use IE6)
  • The Silverlight 2 plug-in is no longer available for download from Microsoft.
  • Windows update upgrades user with Silverlight 2 to Silverlight 3.

But, if you are writing a public application using the ESRI Silverlight API 1.0 that needs to support Silverlight 2 plug-in on IE6, you definitely might want to consider the above issue and display a message to you users asking them to upgrade or turn off the ‘Use HTTP 1.1’ option in IE6.

Deployment hassles with the ‘black box’ MXD file format

Posted in ArcGIS, ESRI by viswaug on July 18, 2009

The MXD file format has always remained a ‘black box’ with its proprietary binary file format which cannot be opened or modified without ArcMap / ArcCatalog / ArcObjects. It’s even got a voodoo doctor that magically treats all its ailments, the ‘MXD Doctor’ software utility. This is fine in the most of the desktop world where the user most probably has access to ArcMap / ArcCatalog. But in the world of server applications that use ArcGIS Server, it does become quite a hassle. The problem occurs during the deployment phase. During the development and the demo phase, we might have prepared the MXD file with all the layers referencing FeatureClasses in the development / demo  geodatabase. But when you move the MXD to the production environment, the layers should now be modified to reference the FeatureClasses from the production geodatabase. Well, to switch the FeatureClass data source reference for the layers in the MXD, you will need access to ArcMap / ArcCatalog. IMHO, I don’t believe that we should be installing ArcMap / ArcCatalog in the production server environment, especially for a reason like the one mentioned above. Installing a full-blown desktop GIS software on a server environment might not make the IT folks very happy and might open up / expose security vulnerabilities. Given the binary nature of the MXD file, I can’t open up the file in a text editor and switch the data sources. And AFAIK, I don’t believe that there is a tool / script that will help make the switch of the data sources without the need for ArcMAP / ArcCatalog. Also, it is not just the MXD file format, the other ESRI file format like ‘.sde’ and ‘.gds’ etc all are in a binary format which makes deployment more painful that it should really be.