Bulk loading data into ESRI FeatureClasses

Yesterday, I was talking to a developer who was writing import scripts to load data from his legacy databases into ArcSDE FeatureClasses. Half way into our discussion, I realized that he wasn’t taking advantage of the IFeatureClassLoad interface in ArcObjects which speeds up the importing of data into the ArcSDE by magnitudes. It achieves it performance boost by suspending updates to the spatial index after every feature is added to FeatureClass and by updating the spatial index only after the FeatureClass is taken out of the load-only mode.

To quote ESRI’s documentation

IFeatureClassLoad is an optional interface supported by feature classes in ArcSDE and feature classes and tables in File Geodatabase. It improves performance of data loading.

With ArcSDE, putting a feature class in load-only mode disables updating of the spatial index while data is loading. In a File Geodatabase, putting a feature class or table in load-only mode disables updating of spatial and attribute indexes while data is loading. Taking the feature class or table out of load-only mode rebuilds the indexes.

While a feature class or table is in load-only mode, other applications cannot work with the data. A feature class or table should be placed in load-only mode only after acquiring an exclusive schema lock on the feature class via the ISchemaLock interface.

The dataset extent reported by the IGeoDataset::Extent property is recalculated when a feature class is removed from LoadOnlyMode to reflect the data present in the feature class.

Dig into this code sample here for more details on using IFeatureClassLoad. And if you end up locking up your FeatureClass while using IFeatureClassLoad, use the ‘sdetable‘ with the ‘-o normal_io‘ option to get yourself out of trouble.

Quick notes on Reflection performance in .NET

Here are some quick notes to keep in mind if you are using Reflection in your projects. You probably know that using Reflection takes a toll on the performance of your application. Bearing the following in mind when you are using Reflection in your projects can help you design better performing applications.

Fast and Light Functions	Costly Functions
typeof Object.GetType typeof == Object.GetType Type equivalence APIs (including typehandle operator overloads) get_Module get_MemberType Some of the IsXX predicate APIs New token/handle resolution APIs in the .NET Framework 2.0	GetXX APIs (MethodInfo, PropertyInfo, FieldInfo, and so on) GetCustomAttributes Type.InvokeMember Invoke APIs (MethodInfo.Invoke, FieldInfo.GetValue, and so on) get_Name (Name property) Activator.CreateInstance

 

 This figure illustrates the invocation mechanisms and their performance overheads.

Type.InvokeMember is the slowest of the late-bound invocation mechanisms because there are two functions that InvokeMember needs to perform to properly invoke a member. First, it must figure out the exact member it’s supposed to invoke, as determined by the user’s string input, and then perform checks to be sure the invocation is safe.

MethodBase.Invoke, on the other hand, doesn’t need to figure out which method it must call because the MethodBase already holds the identity of the method.

REST web services in .NET 2.0

I was looking into creating REST web services in .NET 2.0 and did not have too much success finding much information on the subject. I was able to find information on creating REST web services using .NET 3.5 but not using .NET 2.0. Recently, I stumbled across the Dream framework that enables you to create REST web services in .NET 2.0. So, if you are looking into creating REST web services but are still using .NET 2.0, you might want to take a look at the Dream framework.

Deciphering the Arc mumbo jumbo

There has been quite a bit of new Arc terms coming out of ESRI lately. New Arc terms means new product offering from ESRI. I spent some time investigating some of these new products from ESRI and put together the following for others who might also be interested in these new ESRI technologies.

ArcGIS Explorer – This one everyone knows about. This is Google Earth like GIS viewer application from ESRI with 2D and 3D capabilities. It can use data from ArcGIS Server, ArcIMS, ArcWeb services, WMS along with local data and imagery. More customizable than Google Earth and integrates well with ESRI products. Can access geoprocessing and analytical services provided by ESRI and also allows for creating custom tasks with the freely available SDK.

ArcWeb Services – This is a set of ESRI hosted data and GIS services. This set of services is mainly focused towards developers. Provides access to a variety of 2D basemaps. Also provides access to basic and advanced GIS services. This set of services allows for developers to build their own custom spatial solutions using the GIS data and services provided by ESRI. This is a paid-for service and is not available freely even for developers. There is a public services feature that is available for non-commercial use with severely restricted features.

Developers can access ArcWeb services from a variety of APIs listed below

• SOAP (Simple Object Access Protocol) – offers the most functionality and customization of the ArcWeb Services APIs
  • Easily communicate with ArcWeb in Java, .NET, ColdFusion, or any other programming language.
  • Perform advanced procedures such as creating thematic maps using ESRI or custom data, merging two map images, and generating reports.
  • Employ wireless location services and advanced geocoding functionality (batch, reverse, incomplete address, etc.).
•  JavaScript
  • Quickly add maps to your application with minimal development effort.
  • Take advantage of extensive mapping and geocoding functionality available in ArcWeb Explorer
• REST – offers developers a lightweight, intuitive environment
  • Customize map types and navigation.
  • Send map requests to ArcWeb via URL.
  • Take advantage of the automated data source option, wherein ArcWeb automatically selects the most appropriate data source for your requests.
• Flex
  • Quickly add maps to your application with minimal development effort.
  • Take advantage of extensive mapping and geocoding functionality available in ArcWeb Explorers
• Java MicroEdition (for embedded devices)
  • Deploy map requests to Java-enabled devices such as cell phones and PDAs.
  • Build mapping, routing, geocoding, and point-search applications.
• OpenLS (OpenGIS Location Services) – provides an Open Geospatial Consortium, Inc. (OGC)-compliant platform for wireless services
  • Support geocoding, mapping, locating wireless devices, routing, and finding points of interest.

ArcGIS Online – This is still in Beta and this is also where things start to get blurry. This a set of services that is a whole lot similar to ArcWeb Services but is geared more towards to ArcGIS users rather than developers unlike ArcWeb Services. ArcGIS Online provides free access to a set of 2D and 3D basemaps for use in ESRI clients. The data is available in the form of layer files and MXD files and is more familiar to ArcGIS users. These layers files and MXD available from ArcGIS Online can be consumed by ArcGIS clients like ArcGIS, ArcReader, ArcGIS Explorer. ArcGIS Explorer accesses its base data from ArcGIS Online. This also provides access to basic GIS services that can be easily consumed by GIS users.

ESRI understands that its customers are going to have a hard time getting their head wrapped around the difference between ArcWeb Services and ArcGIS Online. They have posted this movie which helps to get difference across to the customers.

ArcWeb Services	ArcGIS Online
For Web developers	For ArcGIS users
No software required (Browser access)	Requires an ArcGIS application
Includes 2D basemaps and basic and advanced GIS services (e.g., real-time traffic)	Includes 2D and 3D basemaps and basic GIS services (e.g., geocoding)
Build a complete spatial solution with ESRI data and services	Enhance your existing datasets with additional basemaps and tasks

 

ArcWeb Explorer 2.0 – This is still in Beta. This allows you to create web applications with browser based map rendering for improved speed and performance. In ArcGIS Server web applications, the maps are rendered on the ArcGIS Server unlike with ArcWeb Explorer where the map are rendered on the browser. This allows for greater control over map appearance and a desktop like functionality. It also exposes the ArcWeb Services through easy-to-use widgets. The ArcWeb Explorer 2.0 application can be customized and embedded using either the ArcWeb Explorer 2.0 JavaScript or ArcWeb Explorer 2.0 Flex API. Click here for ArcWeb Explorer demo.

ArcWeb Explorer 2.0 JavaScript API

Offers the use of a powerful subset of ArcWeb Services functionality Enables easy integration of mapping functionality and content into HTML Web applications Supports projected maps Provides access to street, street tile, aerial imagery, hybrid (integrated street and aerial), shaded relief, and topographic maps Provides access to Spatial Query point data sources

 

ArcWeb Explorer 2.0 Flex API

Provides access to ArcWeb Services Enables developers to build and/or customize applications Allows for easy and powerful ArcWeb integration inside Adobe Flex Builder Permits JavaScript developers to access Flex functionality via the JavaScript to Flex Bridge Provides access to ArcGIS, ArcIMS, and GeoRSS via API extensions as well as the ability to create your own API extension Allows you to build your own widgets Supports projected maps Allows you to request a thematic map from a Map Image data source and get back a map as a URL Provides access to street, street tile, aerial imagery, hybrid (integrated street and aerial), shaded relief, and topographic maps Provides access to Spatial Query point, line, and polygon data sources