Monthly Archives: August 2010

Pro Python (Apress)

Here is another one of my shamelessly incomplete book reviews…. Let me start with an analogy. I am a rock climber. While I love to climb long routes outdoors, I am forced to spend a great deal more time climbing indoors for training. To climb at a comfortable level outdoors, I try to climb at a much harder level indoors. I learn a lot about easy climbing by climbing hard.

Same with Python. My Python programming level is: EASY. But I find that I learn a lot about how to think like a programmer by reading about advanced Python topics, and this book here is a great example. I had some time recently, sitting in 110 Fahrenheit weather watching a drill rig slowly grind its way into the ground, to read about half of this book, and while I am not about to immediately use much of what I learned, it will still be very handy sooner or later. I knew nothing, for example, about decorators. Now, I know a little.

The organizatino of the book is also interesting. While many book give a chapter a topic and then cram everything about that topic in the chapter, this book’s different. Chapter 1 will cover topics A, B, and C. Chapter 2 will add topic D, and then expound on topics B.1 thru B.3 and C.1 thru C.3, followed by chapter 3 which talks about topic 4 and special cases such as in chapter B.1.1 thru B.1.3 and C.2.1 thru C.s.5…. Okay, if I have thoroughly confused you now, I’m sorry. The structure seems odd at first, but it gradually introduces you to some topic and only gives you more detail when you’re ready for it. If you’ve read a bunch of introductory texts on Python and are ready for more, Marty Alchin’s Pro Python is the best bang for you buck.

Advertisements

Leave a comment

Filed under Uncategorized

XML & Python (Cont’d)

I have been fighting with XML. Here is the code I’ve been trying to run in Python.

 

(My apologies for including this as <img>. As soon as I have my blog running under my own domain, it will hopefully be easier to format code using the Google Syntax Highlighter for WordPress.)

The file I was using this time looks like this:

<?xml version=”1.0″?>
<dataroot>
<POINT>
<Name>B-1</Name>
<North>10342486</North>
<East>3259140</East>
</POINT>

<POINT>
<Name>B-2</Name>
<North>10342671</North>
<East>3259434</East>
</POINT>

<POINT>
<Name>B-3</Name>
<North>10342542</North>
<East>3259375</East>
</POINT>

</dataroot>

and the successful output looks like this:

Found Name
Found Northing
Found Easting
Found Name
Found Northing
Found Easting
Found Name
Found Northing
Found Easting
Borehole  B-1 has a Northing of 10342486 and Easting of 3259140
Borehole  B-2 has a Northing of 10342671 and Easting of 3259434
Borehole  B-3 has a Northing of 10342542 and Easting of 3259375

So what I have I learned ? I can now read the gINT created XML files into Python and turn the data into objects created from my existing classes. (Granted the POINT class used here is VERY basic.) I will probably have to re-write some of the def’s for the classes in my Project module so that I can use the same code for creating an instance, whether the data came from key board entry or was read from XML, and I need to reverse the process and let my program write XML. In fact, somewhere I read about the xml_pickle module. Will have to look at that, too.

Here is a short list of links to some helpful pages :

A Technical Introduction to XML
http://www.xml.com/pub/a/98/10/guide0.html

More XML Basics
http://www.xmlnews.org/docs/xml-basics.html#xmldecl

Columns by Uche Ogbuji
http://www.xml.com/pub/au/845.4

Understanding XML (Python & XML)
http://etutorials.org/Programming/Python.+Text+processing/Chapter+5.+Internet+Tools+and+Techniques/5.4+Understanding+XML/

A Brief History of SAX
http://ftp.accu.org/index.php/journals/515

Charming Python #2 (XML.DOM Module)
http://gnosis.cx/publish/programming/charming_python_2.html

On the ‘Pythonic’ treatment of XML documents as objects
http://www.ibm.com/developerworks/xml/library/xml-matters1/index.html

A Summary fo XML.SAX Exceptions
http://pydoc.org/2.5.1/xml.sax._exceptions.html#SAXParseException

Leave a comment

Filed under Uncategorized

XML Parsing with Python (Turning Objects into XML)

What I said at the end of my last post got me thinking about XML, and I have been reading about reading/writing XML with Python. There is a tremendous amount of information out there. But when you’re starting from square 0, it can be confusing because of the number of different XML modules to choose from. I’ll post a few links later. For now, here is a very short piece of code that opens and reads through an XML file, and looks for elements named “POINT”:

from xml.sax.handler import ContentHandler
from xml.sax import make_parser

class gintHandler(ContentHandler):
   
    def startElement(self, name, attrs):
         if name == “POINT”:
              print “Found a Point”
              for (k,v) in attrs.items():
                  print k + ” ” + v
  
parser = make_parser()
parser.setContentHandler(gintHandler())
parser.parse(open(“point.xml”,”r”))

The test XML file I used was a simplified version of my data exported from gINT (see previous posts), and looked like this:

<?xml version=”1.0″?>
<dataroot>
<POINT>
<North>10342486</North>
<East>3259140</East>
</POINT>

<POINT>
<North>10342671</North>
<East>3259434</East>
</POINT>

</dataroot>

OUTPUT:

Found a Point
Found a Point

The code I ended up using has been adapted from a number of sources. Most helpful (easiest) was this Devshed page. But as you might’ve noticed the reference to attrs (attributes) and the two lines iterating through attrs.items() aren’t all that helpful. I learned that XML tags can have attributes, e.g. <POINT Date=”08/22/2010″>, and those 2 lines refer to those attributes. But my XML file doesn’t have any attributes. The Python POINT object’s attributes are in tags that are enclosed in the POINT tags, a child-parent relationship.

<PARENT>
<CHILD></CHILD>
</PARENT>

The best example for this type of structure is that familiar from HTML tables.

<TABLE>
  <TR>
    <TD></TD>
    <TD></TD>
  </TR>
  <TR>
    <TD></TD>
    <TD></TD>
 </TR>
</TABLE>

I found a good code example for I am trying to achieve in O’Reilly’ Python Cookbook (Recipe #12.5 – Converting an XML Document into a Tree of Python Objects) but I had some trouble adapting it to my needs. Recipe 12.7 for parsing Excel XML files was simpler… [to be continued]

Leave a comment

Filed under Uncategorized

Python – Populate Class from Data in TXT File

After 2 weeks of dirty field work – 22 geotech boreholes in the middle of nowhere, it’s back to GIS. Maybe you’ve been following my recent posts about my little Python project. I have successfully formatted and saved my Python generated data as TXT files that can be read by gINT. Now, I want to be able to read TXT files generated by exporting from gINT. For more background, check out these posts 1), 2) and 3).

A typical file (or portion thereof) would look something like this:

**PROJECT
Title,Title2,Name,Location,Location2,Number,Number2…
GEOTECHNICAL SERVICES,HIGHWAY 999, DALLAS, DALLAS COUNTY, TEXAS, GEO CORP., PROJECT NO.999

**POINT
PointID,HoleDepth,Elevation,Supervisor,Equipment,Date_Start,Date_End,Notes,Note…

B-1,30,,A.G.,CME-75,8/20/2010,8/20/2010,IMORTANT NOTE…
B-2,30,,A.G.,CME-75,8/20/2010,8/20/2010,IMORTANT NOTE…
B-3,30,,A.G.,CME-75,8/20/2010,8/20/2010,IMORTANT NOTE…
B-4,30,,A.G.,CME-75,8/20/2010,8/20/2010,IMORTANT NOTE…

A tag starting with “**” to indicate the class, followed by a line with the attribute names, followed by a line or lines with the attribute data. So how do I read data in a way that allows me to populate the classes I have set up for e.g. PROJECT or POINT data ? This is what I started out using:

 

Of course, this is just another unfinished project. What this gets me is a dictionary with the data to assign to my class instances. After reading a bit about classes and kwargs (keyword arguments) the other day, I am wondering if I need to re-write some of my classes to be more flexible and thus able to use the data from the dictionary directly…

The more I thought about finding the **TAG’s in my TXT I started wondering if it would be easier to work with XML files rather than text. And, in fact, gINT allows me export my data as XML. — I just exported a sample file, and it looks like that would be the smarter way to go because it creates one XML file for every class/object. This way I don’t have to search a TXT file for tags…  and basically the day’s work has been reduced to … well, a mere exercise in programming. 🙂

1 Comment

Filed under Uncategorized

Python Modules for GIS (on Michalis Avraam’s blog)

This post by Michalis Avraam features a neat selection of Python modules useful to the GIS practitioner. Having experimented with numpy by way of OpenEV last week, I was glad to read MA’s testimony that numpy is an essential GIS tool for rasters. As I currently have very little (nothing) to do with Networks, I will skip NetworkX. But the Excel reader/writer I should look at. I knew there was such a thing, just as there is for CSV. But for importing simple data, e.g. coordinates, I have been able to make to with TXT. Anyway, I’d highly recommend MA’s summary to anyone.

Leave a comment

Filed under Uncategorized