ERROR ! Expected Class, delegate, enum, interface, or struc…

After following the step-by-step instructions in the ESRI tutorial, I tried compiling my code, and got this error:

Expected Class, delegate, enum, interface, or struc...

Most likely, it’s because I wasn’t sure where in the code to place the code snippets from the Snippet Finder. I will have to investigate. If you would like to help me out, here is the entire code example:

using System;
using System.Drawing;
using System.Runtime.InteropServices;
using ESRI.ArcGIS.ADF.BaseClasses;
using ESRI.ArcGIS.ADF.CATIDs;
using ESRI.ArcGIS.Framework;
using ESRI.ArcGIS.ArcMapUI;

Starting VC# ( and Visual Office 2008) for ArcGIS

So I have begun sinking my teeth into Visual C# to use with ArcGIS. Reading the first few chapters of the book mentioned in my last post, I seemed to notice certain similarities between C# and Python (I know, this will make some people’s hair stand on end) when it comes to lists and dictionaries, while I noted the use of 2-D or 3-D arrays (such as I remember from BASIC) in C#, which I don’t think exists in Python. Anyway, it will be interesting to explore C# and .NET further.

Having gotten my feet wet, I wanted to start a little ArcGIS project in C#, and found this page here: Getting Started: Walkthrough 1 C#, Creating a Simple Command for ArcMap, which looked like a great introduction, although it wasn’t written for Visual Office 2008. But I ran into a problem at the bottom of page one – locating ESRI.ArcObjects.Core.  The reason I couldn’t find it was explained here. Apparently I was using the old name for said component, and in fact, looking at the link above, I realized that it was for ArcGIS 8.3.  The updated tutorial for 9.3 can be found here. 

I must say this is just one example of many of how the ESRI website is cluttered with debris from yesteryear. Maybe I should avoid googling down dead-end one-ways for ESRI solutions, and instead always start searching at esri.com. But I find the website unnecessarily difficult to navigate.

PS: Somehow, I initially forgot to install the ArcGIS Desktop SDK for .NET (that comes with ArcGIS Desktop) to begin with, and when I tried creating a new C# project, all I could see were the templates for ArcGIS Engine SDK for .NET (that came with my EDN package).

Red Hat Enterprise Linux & ArcGIS Server

I thought trying out ArcGIS Server on Linux would be an experiment worth undertaking. I saw this post about installing on Ubuntu, which I already have running on an older PC. But I was curious how Ubuntu compares to Red Hat, the server edition of which (Red Hat Enterprise Linux) is listed as a supported platform for AGS on the ESRI website. Fortunately, rather than paying for Red Hat Linux, there is CentOS. The free version of unfree RHL. I thought I’d simply download the latest version (5.4) and install it on my #2 PC. That, however, didn’t work because apparently the Intel Celeron PGA478 does not support Physical Address Extension (PAE ), whereas Centos 5.4 does. I couldn’t quite figure out starting with what version a PAE-kernel is included, and I didn’t want to get into the nitty gritty of switching out the kernel or compiling one myself. Ergo no luck. So I will try installing version 4.8 instead. Interestingly, I’ve never had any trouble like that with Ubuntu, which has installed out of the box on the Celereron, an even older Celeron, and my Lenovo laptop. So, if version 4.8 fails, I will install AGS on Ubuntu.

Bedtime Reading – Visual C# Unleashed

Yesterday, I got my copy of Visual C# Unleashed in the mail. – It’s funny, I was following a GIS related discussion thread on linkedin.com last week, and a number of posters were pointing out the benefits of Safari book subscriptions, i.e. paying a monthly fee for accessing a whole online library of books. It sounds like a great if you’ve moved into the digital book age. But I’m afraid I haven’t. I get a lot of my information online and many answers to specific questions on the internet. But when it comes to good primers on a subject, there is nothing like an old-fashioned 2-lb book. To those who say that books kill trees I say: not if I buy a used copy through amazon.com. That cost me $4.95 and doesn’t kill trees. On the contrary it’s carbon sequestration !

My experience with the “Unleashed” books has been positive, and I’m looking forward to using this one to teach me some Visual Office programming skills using C#.

Loops/Iteration in Python – Geoprocessing

Having tried to learn Python by reading Python books rather than ESRI help on Python, I always thought that the looping/iteration with .Next() that I kept seeing in scripts online looked funny, or redundant, or just annoying.

fcs = gp.ListFeatureClasses() 
fc = fcs.Next()  
while fc: 
   # Do something
   fc = fcs.Next()

More Pythonic, I would’ve written it like this:

fcs = gp.ListFeatureClasses() 
for fc in fcs: 
   # Do something

And I understand from Michalis Avraam’s page here (where I saw the above code samples) that as of version 9.3, the way to go is the Pythonic one. Great !

Uninstalling ArcGIS 9.3, Installing ArcGIS 9.4 (or 10 or whatever)…

I finally have installed ArcGIS9.4 Beta 2. The install required uninstalling my functioning 9.3, and that wasn’t as easy as other deinstallations. At first, I came across the same issue Jithen Singh describes, i.e. ArcGIS uninstaller stumbles over itself. Ignoring that, still left me with what seemed like a complete installation. So I simply deleted all the files and directories, and wiped the registry clean manually. Kind of strange that this wouldn’t be more straight forward.

For licensing of 9.4, check the ESRI site, here.

Anyway, it’s running, and I like the new look:

Creating SHP file from X,Y ASCII file…

Below is the script I came up with yesterday by merging my previous script with part of a script I found here (thanks, Morgan, credit you !) and adding more functionality, e.g. selecting workspace and output file names using rawinput.

import arcgisscripting
gp = arcgisscripting.create()

# Set OverWriteOutput to “True” so script can over write files with the same name
gp.OverWriteOutput = True
try:
    outFolder = raw_input (“Enter Workspace: “)
except:
    print “Does this workspace exist ?”
    print gp.getmessage(2)
   
gp.Workspace = outFolder
gp.toolbox = “management”
newfile=[]
# Open a source TXT file with coordinate data
try:
    file_name = raw_input(“Enter file name: “)
    text_file = open(file_name, “r”)
except:
    print “Cannot access this file. Does it exist ?“
    print gp.getmessage(2)

# Read coordinates into a list, line by line
lines = text_file.readlines()
for line in lines: 
    alist=line.split()
    point=alist[0]
    latitude=alist[1]
    longitude=alist[2]
    # Collect the lists with coordinate data in another list
    newfile.append(alist)
text_file.close()

sr = gp.CreateSpatialReference (“C:\Program Files\ArcGIS\Coordinate Systems\Geographic Coordinate Systems\World\WGS 1984.prj”)
outFile = raw_input(“Enter Shapefile Name : “)

try:
    gp.CreateFeatureClass (outFolder, outFile, “Point”, “#”, “#”, “#”, sr)
except:
    print “There was a problem creating the shapefile.“
    print gp.getmessage(2)
   
for borehole in newfile:
    cur = gp.InsertCursor(outFile)
    row = cur.NewRow()
    pnt = gp.CreateObject(“Point”)
    pnt.x = borehole[2]
    pnt.y = borehole[1]

    row.shape = pnt
    cur.InsertRow(row)
   
del cur, row

Now, I don’t know how Pythonic this is, or whether I could’ve been more terse or accurate. I haven’t used try/except much, and maybe there is a better way of doing this. Please let me know. I did run into some errors initially, e.g.

Traceback (most recent call last):
  File “C:\GeoProc\ShP_from_XY.py”, line 43, in <module>
    cur = gp.InsertCursor(outFile)
RuntimeError: ERROR 999999: Error executing function.
A locator with this name does not exist.

I initially thought there might be something wrong with the gp.OverWriteOutput statement, and found that there are indeed differences between ArcGIS geoprocessing 9.2 and 9.3 with this statement, (see also here). But specifying 9.3 (gp = arcgisscripting.create(9.3) didn’t change anything. In fact, after that, I was told I had to use Python2.5 when I happened to be in 2.3 (too many versions on my computer). Finally, I figured out that entering an output name without the .SHP extension was causing the error. I will probably have to add a reminder/check that this is entered or added, if not entered by user.

GIS on a Shoestring

After my extremely simple script last week, I was embarassed. Simply too simple, I thought. I had to dive in again and cobble together something more involved.

Here is the back story: One of the first things we did with ArcGIS at my office was to digitize borehole locations for approximately 25 years of “paper only” engineering files. Too many times had we started drilling a geotechnical project only to realize too late that, 10 years prior, another engineer had conducted a similar project nearby that could’ve provided useful information. Creating a GIS containing all boreholes ever drilled, referenced by project number, and maybe linked to borehole logs, made a lot of sense, and the effort seemed to be more about man hours (pulling files, looking up reports, etc.) than technological.

I quickly found that

1. Recent files contained borehole coordinates and borehole logs while few older files did.
2. Many older (>10 years) files, however contained reliable drawings/maps showing borehole locations, and borehole logs.
3. Some older files contained only vague project descriptions, such as Hwy X @ X Creek, and borehole logs
4. Few files contained only borehole logs with unclear/questionable site descriptions.

Moreover, I found that my colleagues in engineering had been using Google Earth (GE) to locate proposed boring locations for which clients provided only coordinates and no maps. That added up to several 100s of X,Y points of surveyed borehole data in GE. They had entered Lat/Long data into GE and were accumluating such data there.

Going forward, using Google Earth for entering the mountain of historical borehole data was, therefore, an obvious choice, while keeping track of the quality (’survey data’, ‘taken from project map’, ‘eyeballed’, ‘estimated guess’) in a spreadsheet. Most queries of our GIS would be for site geology/soils; therefore, surveyed coordinate data wasn’t really a necessity. Plus, by using Google Earth, locating and “place-marking” of historic boreholes based on project maps and descriptions could be delegated to staff w/o access to or experience with ArcGIS .

Digitizing, i.e. “placing placemarks” labelled PROJECT#-BOREHOLE# in GE, took a few weeks. After that, I had a KML file with approximately 10,000 boreholes covering most of North Central Texas. To get this into ArcMAP, I used arc2earth (Cost at the time $199), which – with a few mouse clicks – created a geodatabase (MDB) that I could open/add in ArcMap. Linking to borehole logs, for which we converted all paper logs to PDF, remains an unfinished task to date. But we have, so far, gotten by with pulling the paper files of interest, once the projects have been identified using GIS.

As new projects were initiated and new borehole survey data came in, I was getting tired of having to enter this manually in GE, and didn’t really like the gobble-de-gook results when importing CSV files from Excel into GE, either. Luckily, I had begun learning Python and wrote this little script to convert simple coordinate data in 3-column, tabbed ASCII text files (Borehole#,Latitude, Longitude) to clean, uncluttered KML files that I could open in GE. This was plain Python, without using arcgisscripting.

Well, that wasn’t good enough anymore. So, given some downtime, I wrote a new script yesterday that reads the same kind of TXT file with X,Y borehole data and creates a shapefile, skipping the detour through arc2earth. I will put the code in a separate post, and discuss writing it there.

“Cropping” Shapefiles – Clipping Features

I have gotten a little carried away teaching myself Python, wrapping my brain around OOP and designing classes. But I’ve had relatively little opportunity using all that Python for what I intended it to help me with – Geoprocessing in ArcGIS. So I decided on a little review of what I learned a while back and apply that to a very common task in my “day job”.

What I spend about 80% of my GIS time doing is making pretty maps for engineering reports. Typically, these are based on SHP data for infrastructure (street, highways, interstates), hydrology (streams, rivers, lakes), administrative boundaries (city, county), geology, and project related information (e.g. boreholes, property boundaries, building foot print).

For North Texas projects, I was using GIS data from the North Central Texas Council of Governments (NCTCOG). But for projects in other regions of the state, I have been using ESRI data, USGS-NHD data, TNRIS data (especially for Texas geology), and data from other local sources, e.g. CAPCOG, for the Austin, Texas, region.

The “problem” with some of the ESRI data, apart from its datedness (When you’re working on construction related projects, it doesn’t help you that entire subdivisions or brandnew toll roads are missing from your data sets.), has been that it comes in SDC (Smart Data Compression) format. Which is great for cramming nationwide data onto DVDs but a pain when you end up adding Alaska roads to a Tarrant County, Texas, project MXD. Hey, wait, hear me out - of course, I know that this is not a “problem”. I’ve just been too lazy (busy, both) to fix it since half the time I can manage with only SHP files instead of SDC.

So how to “crop” those large files down to the region of interest ? Python ! While you can perform geoprocessing on feature classes in a geodatabase, you’re limited when it comes to geoprocessing with feature classes in SDC files. So one way out is to extract the feature class you need in ArcCatalog. I was amazed at the compression of SDC when I saw that extraction of one feature class from a 300MB SDC file created a 1.34 GB SHP file and took 34 minutes. That begs the question whether working with only the SHP you need is even more efficient (faster) than using the SDC file.

But I needed only a small portion of the SHP file – enough to cover a 15-county area. So I wrote this little micro Python script:

import arcgisscripting
gp = arcgisscripting.create()

try:
    # Set the workspace
    gp.Workspace = “c:/GeoProc“

    # Process: Clip the det_riv shapefile to the extent of the metro_county shapefile and name it metro_riv
    gp.Clip_analysis(“det_riv.shp”, “metro_county.shp”, “metro_riv.shp”)

except:
    # In the event of an error, print messages
    print gp.GetMessages()

This created a 5MB shapefile with only rivers in the 15-county metro area I do most of my work in. Okay, this wasn’t magic, and a dozen lines of code are barely programming. But it worked, and I am glad I finally got around to using some Python on something I’ve been meaning to try in a long time.

ESRI = Future of GIS ?

When I first started working with ArcGIS, I didn’t give much thought to the “makers” of it. ESRI made software just like other companies, and having worked with Mapinfo years ago, I figured they weren’t lonely players in the field of GIS.

Later I started researching “Open Source GIS” and found what seemed like a parallel universe of Open Source GIS bundles and applications. There is a lot of development going on with GIS software that is totally free.  So why choose ArcGIS ?

Obviously, ESRI software has become something of a standard, and for most jobs it’s a minimum requirement. If you want to work in GIS, you need to know how to work with ESRI products. When I thought about returning to my Mapinfo “roots”, I noticed it had retreated to something of a niche market (please correct me if I’m wrong!).

I’ve always liked the Open Source idea. It took me a long time to warm up to the idea of using Linux. But when I tried it, I quickly became convinced that the Era of Windows would end, and Google has certainly given a huge boost to the Open Source Universe.

There are a lot of things you do with ArcGIS that you can probably do just as well and easily with Open Source GIS. When it comes to other functionality, e.g. I have heard people mention Spatial Analyst, maybe you can’t. So when I decided to pay for an EDN license, it was because I figured I needed the ESRI specific skills to work in GIS.

I’ve come across two interesting items today that make me wonder if ESRI will continue dominating the GIS market, and which encourage me to continue following the happenings in the Open Source world. In my last post, I linked to James Fee’s blog, where I read a few interesting things, e.g. :

The legacy they [ESRI] may be fighting is the COM framework on which most of ArcObjects is based. At past Developers Conferences I’ve asked if there are plans to migrate their libraires to pure .Net. (Dennis Geasan)

Quad core Win7 workstation for ESRI? Remember, no multithreading, no multicore support…and you’re stuck in 32 so don’t bother with anything more than 4gigs of RAM, my friend. (Archie Belaney)

That after I just bought a new PC, quad core and 6GM RAM to run ArcGIS Server.

And then this page from last year, where ESRI bailed from the WMS Server Shootout against Mapserver and Geoserver.

Now, none of this makes me enjoy working with ArcGIS less than before. But it brought up the question where the mainstream in GIS will be headed in the future.