MBTilesViewer

I also created this bare bones MBTiles cache viewer to view the tile cache in MBTiles format. This application does not do much, just display the tilecache on a map. To view a MBTiles file, just fire up the viewer and start dragging and dropping MBTiles cache files on to it. You can drag and drop multiple MBTiles cache files at the same time if needed. And also, you can create MBTile caches with the TileCutter. The viewer was created using the ESRI WPF map control. If you would like to see this viewer do more, let me know 🙂

MBTilesViewer can be downloaded here.

Advertisement

TileCutter update – With support for OSM and WMS map services

I just added support for creating MBTiles caches for WMS map services and also to download OSM tiles into the MBTiles format. MBTiles cache for WMS map services would improve map rendering performance, but why did i add support for OSM tile sets? Well, they will come in handy for disconnected/offline use cases. So, here are some usage examples

ArcGIS Dynamic Map Service:

TileCutter.exe -z=7 -Z=9 -x=-95.844727 -y=35.978006 -X=-88.989258 -Y=40.563895 -o=”C:\LocalCache\ags.s3db” -t=agsd -m=”http://server.arcgisonline.com/ArcGIS/rest/services/I3_Imagery_Prime_World_2D/MapServer”

WMS Map Service 1.1.1:

TileCutter.exe -z=7 -Z=9 -x=-95.844727 -y=35.978006 -X=-88.989258 -Y=40.563895 -o=”C:\LocalCache\wms111.s3db” -t=wms1.1.1 -m=”http://sampleserver1-350487546.us-east-1.elb.amazonaws.com/ArcGIS/services/Specialty/ESRI_StateCityHighway_USA/MapServer/WMSServer”

WMS Map Service 1.3.0:

TileCutter.exe -z=7 -Z=9 -x=-95.844727 -y=35.978006 -X=-88.989258 -Y=40.563895 -o=”C:\LocalCache\wms130.s3db” -t=wms1.3.0 -m=”http://sampleserver1-350487546.us-east-1.elb.amazonaws.com/ArcGIS/services/Specialty/ESRI_StateCityHighway_USA/MapServer/WMSServer”

OSM:

TileCutter.exe -z=7 -Z=9 -x=-95.844727 -y=35.978006 -X=-88.989258 -Y=40.563895 -o=”C:\LocalCache\osm.s3db” -t=osm -m=”http://tile.openstreetmap.org”

And always just type “TileCutter -h” for usage information.

Want to customize the parameters with which the maps are being generated? Just use the “-s” command line option ans specify the setting in a query string format.

TileCutter.exe -z=7 -Z=9 -x=-95.844727 -y=35.978006 -X=-88.989258 -Y=40.563895 -o=”C:\LocalCache\ags.s3db” -t=agsd -m=”http://server.arcgisonline.com/ArcGIS/rest/services/I3_Imagery_Prime_World_2D/MapServer” -s=”transparent=true&format=jpeg”

Also, if some of the tile requests result in errors, the level, column, row and error message information would be logged into a text file in the same directory as the MBTiles cache.

And now, for the best new feature of TileCutter, the program does not store duplicate tiles. That is, if the area you are caching has a lot of empty tiles in the ocean etc, the MBTiles cache created by TileCutter will only store one tile for all those duplicated tile images. Should help save disk space 🙂

TileCutter can be downloaded here

Oracle spatial and the web mercator spatial reference system

We have been attempting to streamline the way we store and process spatial data in Oracle. Normally, we would create our spatial tables as FeatureClasses from ArcCatalog and create plain old Oracle tables for our business/attribute data that would be joined with the spatial data in the FeatureClasses through a shared key field to create a SDE spatial view. This works just fine, but there are some drawbacks to this approach

• SDE creates unnecessary fields on the FeatureClass table in Oracle, like the field cad annotation that just don’t need to be there.
• Creating FeatureClasses manually from ArcCatalog is a process that cannot be automated as a part of the build process. Ideally, we like to execute a command from our build scripts that will delete our entire model from the database and run another command to recreate the entire model to start off with a clean slate for testing our every build. These scripts will also be used during the deployment process when the database needs to be recreated in another environment.
• FeatureClasses will need to be edited through ArcObjects (versioned/unversioned) or ArcMap(versioned) which would need an ESRI license on any client machine wanting to edit the spatial data. In a web environment, this would mean a ESRI license on the web server.

We wanted our setup to try and work around some of the drawbacks above. So, we wrote SQL scripts that would

1. Create the spatial table
2. Insert metadata about the spatial table into the Oracle geometry metadata table USER_SDO_GEOM_METADATA
3. Create a spatial index on the table
4. Register the Oracle spatial table with ArcSDE as a FeatureClass using the ‘sdelayer -0 register’ command

Here is some sample SQL scripts to perform the first three steps with Oracle

CREATE TABLE customers (
  customer_id NUMBER,
  last_name VARCHAR2(30),
  first_name VARCHAR2(30),
  street_address VARCHAR2(40),
  city VARCHAR2(30),
  state_province_code VARCHAR2(2),
  postal_code VARCHAR2(9),
  cust_geo_location SDO_GEOMETRY);
 
INSERT INTO USER_SDO_GEOM_METADATA (TABLE_NAME, COLUMN_NAME, DIMINFO, SRID)
   VALUES ('CUSTOMERS', 'CUST_GEO_LOCATION',
   SDO_DIM_ARRAY
     (SDO_DIM_ELEMENT('LONG', -180.0, 180.0, 0.5),
     SDO_DIM_ELEMENT('LAT', -90.0, 90.0, 0.5)),
   8307);

CREATE INDEX customers_sidx ON customers(cust_geo_location)
  INDEXTYPE IS mdsys.spatial_index;

The sample sde command to register the table as a multi-polygon FeatureClass with ArcSDE specifying the bounds

sdelayer -o register -l CUSTOMERS,CUST_GEO_LOCATION -e a+M -C CUSTOMER_ID -i sde:oracle11g -s SERVER_NAME -u XXX -p YYY@orcl -t SDO_GEOMETRY -P High -x -180,-90,11132000

As you can see from the commands above the spatial table is registered with Oracle to have an SRID (spatial reference id) of 8307 which denotes the WGS 84 Geographic Coordinate Systems. Oracle maintains its own CS_SRS table where it maintains a list of Oracle supported spatial reference systems. The SRID specified when adding the geometry metadata with Oracle is expected to be present in the CS_SRS table, else Oracle will insert the metadata into its tables. The bounds specified when registering the spatial table as a FeatureClass with ArcSDE also reflects the WGS 84 GCS.

So far so good. The above solution works like a charm. Now that the ArcGIS Online base maps have moved to the Web Mercator projection system used by google and bing, we also wanted to maintain our data in Web Mercator so that the GIS server doesn’t have to reproject the data when rendering maps and also so that we could serve out spatial data as GeoJSON/ArcJSON from our custom web services to consumed by web clients. Here is where trouble started. Oracle does not have a SRID for the web mercator projection system. We ran through the CS_SRS table to check for it maintained under a different id but with no luck. The process of registering a new SRID with Oracle is not documented anywhere as far as I can tell. Apparently, it is not as simple as adding an entry into the CS_SRS table which we tried unsuccessfully.

So, to work around this issue, we registered our spatial table with the Oracle geometry metadata table without an SRID. So, Oracle thinks the SRID for the spatial table in NULL. This becomes a problem when we try to insert geometries with ESRI’s SRID 102113 into the spatial table. Oracle doesn’t let us insert geometries into the spatial table whose is not NULL to match the entry in its geometry metadata tables. So, we are forced to insert geometries with a NULL SRID into the table. But ArcSDE needs to know that the spatial table is in the Web Mercator projection system. To do this, we registered the spatial table using the ‘sdelayer -o register’ specifying the projection file “WGS 1984 Web Mercator (Auxiliary Sphere).prj” from ESRI. See the sample below

sdelayer -o register -l CUSTOMERS,CUST_GEO_LOCATION -e a+M -C CUSTOMER_ID -i sde:oracle11g -s SERVER_NAME -u XXX -p YYY@orcl -t SDO_GEOMETRY -P High -G file=”C:\TEMP\WGS 1984 Web Mercator (Auxiliary Sphere).prj”

The insert statement for the Oracle geometry metadata tables look like this

INSERT INTO USER_SDO_GEOM_METADATA (TABLE_NAME, COLUMN_NAME, DIMINFO)

VALUES (‘CUSTOMERS’, ‘CUST_GEO_LOCATION’,

SDO_DIM_ARRAY

(SDO_DIM_ELEMENT(‘X’, -20037700, 20037700, 0.1),

SDO_DIM_ELEMENT(‘Y’, -20037700, 20037700, 0.1)));

This is ugly, but it works. We were able to view and edit the data in ArcMap just fine.

Here are some more things discovered along the way

• Oracle makes all the table names and the field names all upper case by default even when you specify the names in lower case in the SQL statements. We can force Oracle to use lower cases alphabets in the table/field names by specifying the table/field names in the SQL statements by enclosing them in quotes.
• If you use the technique above to use table/field names in lower case for the spatial tables, you will not be able to register the geometry metadata for the table with Oracle. This is because Oracle expects the spatial table/field name to be in all upper case for insertion into the geometry metadata table. This is just a crazy crazy thing and i can’t imagine this requirement being intentional.
• Oracle spatial SQL syntax is ugly. Very ugly. They can definitely learn from the sweet SQL syntax in MS SQL SERVER 2008.
• The free version of Oracle called Oracle XE does support the spatial data types. That is, in Oracle XE we can store columns whose data type is MDSYS.SDO_GEOMETRY. Oracle XE also allows us to perform some spatial operations on these spatial columns but not all spatial operations that are available in the enterprise edition of Oracle. The spatial features that are available in the XE edition is not documented anywhere as far as I can tell. Oracle spatial extensions bring more spatial features to Oracle enterprise edition like raster support etc.
• The MDSYS.Oracle ST_GEOMETRY is different from the ESRI ST_GEOMETRY and is a wrapper around MDSYS.SDO_GEOMETRY

If you are aware of a better way to do things with Oracle and SDE for the web mercator projection, please let me know, I am all ears. I hope this post save some other poor soul some pain and suffering.

Issues encountered and solved while building a comprehensive web-based map printing solution

Web-based map printing has been one of those problems that, so far, doesn’t have a COMPLETE solution that meets all the needs of the different users out there. We had created various solutions in the past to meet web-based printing needs on a per-project basis. But, we didn’t have one single comprehensive solution that was capable enough to meet all requirements regardless of the unique complexities involved in each of them. A little while ago, we set out to build one such comprehensive web-based map printing solution and ran into some issues along the way that I thought might be worth sharing here. I am not going to go into what we built here, but just the issues/oddities we encountered…

Here are the issues we faced while developing the printing component and some details into how we worked-around them.

• Printing token secure layers – This is a problem that we initially didn’t see coming because we were using the Silverlight client api to print. When using token layers, the client (which is the browser) requests a token from the GIS server using it’s IP address (or a web address) as the optional ClientID. When using a token generated with a ClientID, the AGS server checks for the origin of the request to confirm identity. So, when we tried to use the token generated by the client browser with it’s IP address as the ClientID in the server-side printing component, the requests were denied by AGS as it rightfully should since the server’s IP address doesn’t match the one in the token. We did not initially see this problem with Silverlight clients because, Silverlight clients currently request tokens without the optional ClientID. To work around this, we had to request token without the ClientID or had to spoof the Referrer in the HTTP request for the image.
• Max Image Size constraints – The size of map image requests that need to be made can get quite large depending on the size of the map on the print layout and also on the DPI required on the map print output. AGS has default max image size limits set to 2048 X 2048. Bing maps maximum image size is around ~800. Increasing the maximum image size limit in AGS will only take you so far. Eventually, your image size requests can be big enough (think plotter size) to either cause AGS to crash or just take an unacceptably long time to return. So, to work around this limitation, we had to resort to cutting the big image requests into a series reasonably sized tile requests. Once all the tiles to cover the big area arrive, the tiles can be stitched back together using GDI+ to produce a seamless big image that the map print layout needs.
• Bing logo – The above solution to split big image requests into works for AGS MapServices and WMS services, but Bing map layers add another twist to the problem. Image responses from Bing contain the Bing logo on the bottom right corner of the image. This caused the Bing logo to appear multiple times on the map when the numerous smaller tile images were stitched together. To solve this issue, we had to get special permission from Bing to access their tile images directly which do not have the Bing logo on them and stitch those together to produce the seamless image required.
• Custom legends – The swatches for legend image for an AGS MapService can be obtained pretty easily using the AGS SOAP API. But more custom work is needed to stitch together the swatch and legend text information from multiple map services. Also, to add to it, there was no easy way to generate swatches for graphics layers. So, we ended up writing a custom Server Object Extension (we call it LegendServer) exposed over SOAP that takes in the information needed to produce swatches for the graphics layers and produces swatch images. The legend service consumed the swatches information from the AGS MapService and the custom SOE and stitched them together into one legend image handling the font styles etc and wrapping as necessary. We still have the issues here that ArcObjects is not able to generate the swatches at the required DPI. For e.g. we can’t request for swatches in 300 DPI etc.
• Missing legend symbol markers in AGS – When writing the custom Server Object Extension described above to produce legend swatches, we discovered that ArcObjects doesn’t support triangle markers. But triangle markers were supported on the client-side APIs. So, to overcome that limitation, we can handle just the triangle markers as picture marker symbols and handle it with a special image service that produces a triangle image in the required dimensions, fill and border color.
• Overflowing legends – Sometimes the legend for a map just can’t fit on a single page. In those cases, we had to make sure that we build the legend in parts that can fit on the page and stick the overflowing legend into new pages as needed. The trick here is to not build one single legend image and chop it to overflow to the next page. Because, we decide to chop off the legend at an arbitrary height, we might end up chopping the text or swatch on the legend. So, we will have to build the legend in parts and then assemble them into the different pages.
• Printing Graphics layers – Printing graphics layers on the map turned out to be a little tricker than expected. We went down the path of rendering the graphic layers as PDF graphics on top of the map. It seemed to do everything we needed until we had to print polygons with holes in it. Then we used the AGS SOAP API to generate an image for the graphics on the map and overlay it on top of the map. We ended up pulling back that solution because that technique did not support transparency in graphics. So, eventually, we ended up writing a custom Server Object Extension (we call it GraphicsServer) that produces images from the graphics layer geometries and symbology respecting their transparency.
• Overview Map – Printing overview map doesn’t sound too complicated until you consider the fact that we might have a totally different set of layers on the overview map than we do on the map itself. Also, the overview map can be static or dynamic, meaning it can always be at the same extent (world extent for example) or it may have it’s extent set at levels that closely follow the extent of the map itself. Also, keep in mind that the overview map will also need to have a small rectangle graphic inside it that highlights to current extent of the map.
• Print Rendering – In most cases, we will want to have the option of being able to render to PDF or an image as the user requires. When implementing these renderers, please keep in mind that the co-ordinates axis for the PDF and the image GDI graphics are reversed. PDF is bottom-up and image GDI graphics is top-down.

I am pretty sure that I am leaving out some more. I will add them to the list above as and when I remember them. But I am happy to say that we did solve/work-around all the problems that came our way and have been re-using the printing component in various projects with great success. The printing component is also being used by all Silverlight, Flex and Javascript clients.

Please let me know, if you ran across other issues when you implemented your print feature or if you solved any of the issues above in a different way.

One overlooked feature that is coming in ArcGIS Server 10

At the ESRI devsummit last week, there were a lot of new features in ArcGIS Server (AGS) version 10 that were getting a lot of attention. There was a lot of fan-dare around geodatabase editing over the web with the new Feature Service in AGS 10 and the flashy demos with all the new RIA clients for editing with the REST API. There are also a lot of articles & blog posts that have been written to cover those features in details. So, I thought I would stay off the beaten path and direct your attention towards a new feature in AGS 10 that is getting little to no attention. That new feature in AGS v10 is support for exporting PNG32 (32 bit PNG) map images from the REST API and other APIs offered by AGS. This should provide the complete support 8-bit transparency in PNGs and should help us avoid issues like the ones here. Printing maps functionality should also benefit from this new feature both in terms of quality and to avoid transparency issues when layering images.

Just check out the ‘Image Format’ drop-down here for proof of 32 bit PNG support . And as you can see here, AGS 9.3.1 only supported PNG24. Hope that helps someone… 🙂

Working with ESRI token secure services

At the ESRI developer summit this past week, I ran into some people that were either having a hard time with using the ESRI token authentication or were leaving their systems vulnerable to hacks given their use/abuse of long lived tokens. I thought it might be useful to share one way that we have been using ESRI token secured services in our web mapping applications.

Token secure services require the client to request a token with their username & password which should then be used/included in all other future requests to access the services. The token provided to the user by AGS is also valid only for the time period requested by the user. The AGS server also applies a upper limit to how long the token can be valid.

One of the main reasons for troubles with using such token secure services in a web mapping application is that the user logs into the web application that he is using and not actually the AGS server(s) that the web application is using map services from. So, in order to use the map services in the web application, the user has to log-in (again) to the AGS server also. Having the user log in again after they have already logged into the web application is highly undesirable. To prevent the user from having to enter in the credentials to access AGS services again, some may decide to use a long lived token and hard-code the token into the web application or hard-code the username & password to access AGS services in the mapping client application. I don’t think I need to explain why hard-coding the username & password in the client web mapping application is dangerous. But this still leaves the application highly vulnerable to hacks since anybody who can read the URL being used to access the services have access to the long-lived token. Using the long-lived token, anybody can obtain access to the AGS services since the only defense is the ClientID (or the HTTP Referrer header) and that can be spoofed easily since it is never verified. Also, the long-lived token doesn’t expire often and leaves the hackers a lot of time to get the token and access the secure AGS services

To get around this, there is an easy way to setup the web application to use and better secure the AGS services. We might have two main ways of sharing username & password between the web application and the web application. The first way is to have AGS and the web application share the membership/permission/roles datastore. In this case, the web application can use the same username & password combination to obtain a token from the AGS server. The second way is to have all users of the web application use the same name username & password to access the AGS services. The second way could work because the user has already been authenticated by the web application and so he can be trusted to access the AGS services also. In this case, the username & password that will be used to log-in all web application authenticated users can be stored in the web application configuration file (web.config). This credential can be used to obtain a token from AGS. This is generally how Bing map services are also handled. The Bing credentials are stored in the web.config and used to obtain a Bing token when the page with the map is loaded.

So, once the user logs into the web application, the username & password from the shared datastore/web.config can be used to make a request to the AGS ‘GetToken’ URL endpoint and obtain a short-lived token for AGS access. This token can then be sent down to the client as a part of the HTML / ASPX page. Another technique is to write a HTTPHandler that accepts a GET request without a username & password and uses credentials from the shared datastore/web.config to obtain a token to access AGS services and sends the token down to the web application client. Is method is secure because the HTTPHandler itself can be secured by either windows/forms authentication of the host web application.

Another thing to note about AGS tokens is that AGS does NOT require a ‘Referrer’ (IP address/ Site URL) to generate a short-lived token (long-lived tokens do require them). If you are generating a token from the AGS web page to generate a token, you will have an option to not specify the ‘Referrer’ (ClientID), but if you are just making a HTTP request to the GetTokens endpoint, you can obtain a short-lived token without the ClientID. When using short-lived tokens obtained without the ClientID, AGS does NOT enforce checks on where the calls are originating from. Actually, this is the reason why Silverlight clients are currently able to consume token secure map / AGS services. Silverlight 3 & under clients do not include the ‘Referrer’ HTTP Header for all outgoing HTTP requests, so ClientID origin checks are not enforced on Silverlight API clients. This issue has been fixed in Silverlight 4.

Unfortunately, the authentication tokens generated by ASP.NET to secure web applications and the ones generated by AGS are generated using different techniques. The key used to generate the token is different, ASP.NET uses the machineKey from web.config and AGS token uses a key from the AGS configuration file. If this wasn’t the case, we could technically have the ASP.NET web application and AGS share the same token…

Quick lesson learnt implementing continuous zoom operations for the ESRI Silverlight API

One of the nicer navigation features that enhance suser experience on mapping applications is the continuous zoom in & out and pan operations. Also, users that use Google and Bing maps frequently expect the continuous zoom and pan experience in all the mapping applications 🙂 The ESRI Silverlight API Toolkit does not include support for continuous zoom and pan operations. But implementing the continuous navigation operations for the ESRI Silverlight API is real easy with the DispatcherTimer class that was introduced with Silverlight version 2.0. The DispatcherTimer class calls the ‘Tick’ event listeners on the UI thread so developers don’t have to worry about accessing UI controls from across threads. Implementing it is as simple as setting the map’s extent very frequently with new extent calculated for zoom in/out/pan operations. To provide a good experience, I had to reset the map’s extent every 10 milliseconds.

I had initially overlooked the fact that every time I reset the map’s extent every 10 milliseconds a map image request was being made for every DynamicLayer that was loaded on the map. The problem doesn’t occur with the TiledMapServiceLayer because requests for tile images only occur after the zoom/pan operation has caused the map’s extent to exceed the extent of the tile map images already loaded on to the map and the browser also caches the tiles client-side. The excessive number of map image requests (1 request every 10 milliseconds) generated by the continuous zoom operation would bring any GIS server (and did bring our server) to its knees pretty soon. There is no perfect solution to the problem here in my humble opinion. But an acceptable solution to the problem is to hide (change visibility to false) all the DynamicLayer(s) on the map when the continuous navigation operation begins and to reset the layers back to their previous state when the continuous navigation operation ends. Since the base map layers are TiledMapServiceLayer(s), they don’t pose the problem with the excessive number of requests and we can leave them visible. This actually provides an acceptable user experience during the continuous navigation operations.

The code snippet below illustrates the implementation of a sample continuous zoom in & out operations.

Note – In Silverlight any control that inherits from ButtonBase like Button, CheckBox, RadioButton etc will not raise the ‘MouseLeftButtonDown’ and ‘MouseLeftButtonUp’ events even though the events are available on the classes.

public partial class MainPage : UserControl

{

    private DispatcherTimer _timer;

    Dictionary<Layer, bool> _mapLayersVisibilityState = null;

 

    public MainPage()

    {

        InitializeComponent();

 

        IDisposable subscription = null;

        subscription = myMap.Layers.GetLayersLoadCompleted().Subscribe

            (

                ( args ) =>

                {

                    txtStatus.Text = "All layers have loaded.";

                    subscription.Dispose();

                }

            );

        txtStatus.Text = "Loading Layers onto map …";

    }

 

    void zoomOut_MouseLeftButtonUp( object sender, MouseButtonEventArgs e )

    {

        if( _timer != null )

            _timer.Stop();

        EndContinuousOperation();

    }

 

    void zoomOut_MouseLeftButtonDown( object sender, MouseButtonEventArgs e )

    {

        BeginContinuousOperation();

        _timer = new DispatcherTimer();

        _timer.Interval = new TimeSpan( 0, 0, 0, 0, 10 );

        _timer.Tick += _zoomOutTimer_Tick;

        _timer.Start();

    }

 

    void zoomIn_MouseLeftButtonUp( object sender, MouseButtonEventArgs e )

    {

        if( _timer != null )

            _timer.Stop();

        EndContinuousOperation();

    }

 

    void zoomIn_MouseLeftButtonDown( object sender, MouseButtonEventArgs e )

    {

        BeginContinuousOperation();

        _timer = new DispatcherTimer();

        _timer.Interval = new TimeSpan( 0, 0, 0, 0, 10 );

        _timer.Tick += _zoomInTimer_Tick;

        _timer.Start();

    }

 

    void _zoomInTimer_Tick( object sender, EventArgs e )

    {

        ZoomMapByFactor( 0.99 );

    }

 

 

    void _zoomOutTimer_Tick( object sender, EventArgs e )

    {

        ZoomMapByFactor( 1.01 );

    }

 

    private void BeginContinuousOperation()

    {

        _mapLayersVisibilityState = GetMapLayersVisibilityState();

        HideAllDynamicMapLayers();

    }

 

    private void EndContinuousOperation()

    {

        SetMapLayersVisibilityState( _mapLayersVisibilityState );

    }

 

    private Dictionary<Layer, bool> GetMapLayersVisibilityState()

    {

        Dictionary<Layer, bool> visibilityStates = new Dictionary<Layer, bool>();

        myMap.Layers.ForEach<Layer>( lyr => visibilityStates.Add( lyr, lyr.Visible ) );

        return visibilityStates;

    }

 

    private void SetMapLayersVisibilityState( Dictionary<Layer, bool> visibilityStates )

    {

        if( visibilityStates == null )

            return;

 

        foreach( var entry in visibilityStates )

        {

            entry.Key.Visible = entry.Value;

        }

    }

 

    private void HideAllDynamicMapLayers()

    {

        myMap.Layers.ForEach<Layer>( lyr =>

        {

            if( lyr is DynamicLayer )

                lyr.Visible = false;

        } );

    }

 

    void ZoomMapByFactor( double factor )

    {

        if( myMap != null )

        {

            Envelope env = myMap.Extent;

            env.Expand( factor, myMap.RenderSize );

            myMap.Extent = env;

        }

    }

}

The code above makes use of the ‘Expand’ extension method for the ‘Envelope’ type provided below.

public static void Expand( this Envelope extent, double factor, Size mapSize )

{

 

    if( extent == null )

        throw new ArgumentNullException( "extent" );

    if( mapSize.Width <= 0 )

        throw new ArgumentOutOfRangeException( "mapSize.Width" );

    if( mapSize.Height <= 0 )

        throw new ArgumentOutOfRangeException( "mapSize.Height" );

 

    if( extent.XMax == extent.XMin )

    {

        double resolution = extent.Width / mapSize.Width;

        double xVal = extent.XMin;

        //using a 10 pixel buffer

        extent.XMin = xVal – 10 * resolution;

        extent.XMax = xVal + 10 * resolution;

    }

 

    if( extent.YMax == extent.YMin )

    {

        double resolution = extent.Height / mapSize.Height;

        double yVal = extent.YMin;

        extent.YMin = yVal – 10 * resolution;

        extent.YMax = yVal + 10 * resolution;

    }

 

    MapPoint center = extent.GetCenter();

    double dX = ( extent.Width / 2 ) * factor;

    double dY = ( extent.Height / 2 ) * factor;

 

    extent.XMin = center.X – dX;

    extent.XMax = center.X + dX;

    extent.YMin = center.Y – dY;

    extent.YMax = center.Y + dY;

}

Enhance your development experience with the ESRI Silverlight API using the Microsoft Reactive Extensions

In my humble opinion, Reactive Extensions is one of the most exciting libraries to come out of Microsoft lately. And that is saying a lot since there have been quite a bit of good libraries coming out of Redmond recently. The Rx for .NET takes a lot of pain out of asynchronous programming and has drastically reduced the amount of code required to handle and orchestrate responses from multiple web services. Given that Silverlight is a client-side technology, most of the Silverlight applications make calls to web services and display results to the user as needed. A common application scenario is where the Silverlight client needs to make multiple calls to multiple web services and orchestrate calls to other web services based on responses from previous web service calls. Handling this scenario can get quite a bit complicated when you have to maintain contexts to store response states from the various web services. This is where Rx for .NET comes in real handy. The following scenario and the solutions should illustrate how Rx for .NET can help when programming with the ESRI Silverlight API for example.

When adding ArcGIS Map service layers to the Map control in the ESRI Silverlight API, the map control makes a HTTP request for every layer added to request the metadata for the map service. Depending on whether the request to the service metadata succeeds or fails the layer object would raise a ‘Initialized’ event or a ‘InitializationFailed’. So, a layer ‘load’ can be considered to be complete when either the ‘Initialized’ or the ‘InitializationFailed’ has fired. If we want to initiate another sequence of operations when all the ‘load’ of all the layers on the map have completed, like say populate a legend/ Table of Contents, then we will have to wait for all the layer ‘load’ to be complete (Success/Failure). Making a request to the legend web service in the above scenario can get quite cumbersome. But Rx for .NET can simplify things quite a bit. Follow the code sample below to accomplish the task.

The two extension methods below will help us watch for the ‘load’ complete for a layer and to watch for all the layers in a LayerCollection to complete. As you can see, we use the ‘Merge’, ‘Take’ and the ‘ForkJoin’ observable extension methods on the IObservable to watch for the right sequence of events.

public static class ExtensionMethods

{

    public static IObservable<IEvent<EventArgs>> GetLayerLoadCompleted( this Layer layer )

    {

        if( layer == null )

            throw new ArgumentNullException( "layer" );

 

        if( layer.IsInitialized == true )

        {

            IEvent<EventArgs>[] events = new IEvent<EventArgs>[ 1 ] { null };

            events.ToObservable<IEvent<EventArgs>>();

        }

 

        IObservable<IEvent<EventArgs>> oSuccess = Observable.FromEvent<EventArgs>

            (

                ev => layer.Initialized += ev,

                ev => layer.Initialized += ev

            );

 

        IObservable<IEvent<EventArgs>> oFailed = Observable.FromEvent<EventArgs>

            (

                ev => layer.InitializationFailed += ev,

                ev => layer.InitializationFailed += ev

            );

 

        IObservable<IEvent<EventArgs>> obsCompleted = oSuccess.Merge<IEvent<EventArgs>>( oFailed );

 

        IObservable<IEvent<EventArgs>> oFirst = obsCompleted.Take<IEvent<EventArgs>>(1);

 

        return oFirst;

    }

 

    public static IObservable<IEvent<EventArgs>[]> GetLayersLoadCompleted( this IEnumerable<Layer> layers )

    {

        if( layers == null )

            throw new ArgumentNullException( "layers" );

 

        return Observable.ForkJoin<IEvent<EventArgs>>

            (

                from item in layers

                select item.GetLayerLoadCompleted()

            );

    }

}

And now, we can add a handler to be executed when all the layers have loaded like below.

public MainPage()

{

    InitializeComponent();

 

    IDisposable subscription = null;

    subscription = myMap.Layers.GetLayersLoadCompleted().Subscribe

        (

            ( args ) =>

            {

                HtmlPage.Window.Alert( "All layers have loaded." );

                subscription.Dispose();

            }

        );

}

Black magic behind the MSD Map Services?

Much has been said about the performance boost that can be achieved by running MSD map services over MXD map services. Here I will talk a little bit about the other side of the MSD map services which trips up developers like me. The only information that I was able to find about the capabilities/short-comings of the MSD map services from ESRI was on their blog post here. Since, I develop and use a few custom Server Object Extensions (SOEs), I noticed that some of the SOEs that I had written earlier had stopped working when registered with MSD map services. And the following piece of information from the blog post mentioned earlier seemed to indicate the cause of the problem.

What capabilities (server object extensions) are supported for MSD-based map services?

The WMS and KML capabilities are available with MSD-based services, as well as the default Mapping capability. Custom server object extensions can also work with MSD-based services but they cannot access fine-grained ArcObjects; they must use the coarse-grained methods on MapServer and its related classes.

The important part of the above statement is tha ‘custom SOEs cannot access fine-grained ArcObjects’. Well, what does that really mean? It kinda means that you cannot access the fine-grained ArcObjects features that are exposed through the IMapServerObjects interface on the MapServer server object. But being from the .NET world, I had trouble wrapping my head around how the same MapServer server object class can serve both the MXD map services and also the MSD map services and behave differently for each service type. How can the same MapServer class offer the IMapServerObjects features for MXD map services but not for MSD map services? I tried casting(QI) the MapServer server object obtained from a MSD map service to an IMapServerObjects interface which actually resulted in an exception being thrown. But the same cast worked on the MapServer object obtained from a MSD map service. My first thought when I noticed this behavior was that maybe the MSD map services are actually served by a different new server object class and not the MapServer class that serves the MXD map services. The ESRI  9.3.1 documentation doesn’t mention anything about a new server object. So, either ESRI didn’t document it or the same MapServer has developed multiple-personalities in AGS 9.3.1. To test it out, I obtained the CLSID of the server object that served MSD map services and the MXD map services. The were the exact SAME. That probably means that the same MapServer class is just behaving differently in different usage scenarios by leveraging some COM black magic which I don’t have too much knowledge about.

Since, we can’t get at the rich features in the IMapServerObjects interface, the custom SOEs on MSD map services really become only a little more useful than custom COM utility objects. So, some of the custom server object extensions that developers wrote earlier for MXD map services will not work with MSD map services. And that is a real bummer for me. One way (that I really really hate) to overcome this limitation is to run another MXD map service for the MSD map service. The MSD map service makes the maps and the MXD map service is only used to obtain and use the features of the  custom server object extension. I do really hate the idea of running a another expensive map service process on the AGS machine only for the purpose of using the features of the custom server object extensions. Although, I do have some ideas on a better ways of working around these limitations. The most frustrating part about this whole thing is that I think internally ESRI does have access to fine grained ArcObjects for the MSD map services and they just don’t let us know how to gain access to the fine grained ArcObjects. Why/How do I say that? The reason for my suspicions that the above is the case is because, the WMS and KML server object extensions in-fact DO work with MSD map services also. Which means that those server object extensions are able to access fine-grained ArcObjects to do what they need to do.

Update : Another change that relates to SOEs in AGS 9.3.1 is regarding it registration process. As many might have noticed, their SOEs shows (or doesn’t showup :)) up without any label in the ArcCatalog->Service Properties->Capabilities tab. It took me a while to realize what was happening. I had initially assumed that the SOE wasn’t showing up in the list. But later, luckily I discovered that the SOE was in-fact displaying but there was just no label for it in the Capabilities list box. In 9.3, the IServerObjectExtensionType interface has been superseded by the IServerObjectExtensionType2 interface ArcObjects style. So, when you register the SOE with ArcGIS Server by calling the CreateExtensionType, the returned object should be casted to the type IServerExtensionType2 and the ‘DisplayName‘ property should be set on it before calling AddExtensionType. This ‘DisplayName‘ property is what shows up in the capabilities tab and if it is not set, the entry for the SOE in the capabilities tab will be empty. Earlier, the extension’s Name property used to show up in the capabilities tab. But no longer, it is the DisplayName property that is used to label the entry in the capabilities tab list box. I hope ESRI finds some time to update their documentation about this over here.

GeoJSON & GPX support added to the ESRI Silverlight API Contrib

I’ve added support for the GeoJSON and GPX layer types to the ESRI Silverlight API Contrib project on CodePlex. Adding GeoJSON or GPX layer to the ESRI Silverlight API map is real easy and you would add it just like adding a FeatureLayer to the map.

Adding a GeoJSON layer to the map

<esri:Map x:Name="MyMap" Grid.Row="1">
            <esri:Map.Layers>
                <esri:ArcGISTiledMapServiceLayer ID="StreetMapLayer"
Url="http://server.arcgisonline.com/ArcGIS/rest/services/ESRI_Imagery_World_2D/MapServer"/>
                <slContrib:GeoJSONLayer ID="geoJSONLayer"
                     URL="http://localhost/SLMaps.Web/Data/JSON.txt" />
            </esri:Map.Layers>
</esri:Map>

Adding a GPX layer to the map

<esri:Map x:Name="MyMap" Grid.Row="1">
            <esri:Map.Layers>
                <esri:ArcGISTiledMapServiceLayer ID="StreetMapLayer"
Url="http://server.arcgisonline.com/ArcGIS/rest/services/ESRI_Imagery_World_2D/MapServer"/>
                <slContrib:GPXLayer ID="gpxLayer"
                 URL="http://localhost/SLMaps.Web/Data/fells_loop.txt" />
            </esri:Map.Layers>
</esri:Map>

This adds to the GeoRSS layer support already present in the library.

Other than just adding these data sources as layers to the map, The GeoJSONReader & GPXReader classes can also be used to just parse the data source and obtain the geometry and attribute values of the features in the data source if/as required. This should allow for the users to upload GeoJSON & GPX data from their GPS units directly from their computers to the Silverlight map control without the need to upload files to the server. Allowing the workflow to be smoother, faster and simple both on the server-side & client-side.