Blogging Journey 2.0

Several years ago, I was working as a Professional Geologist, sitting by drill rigs, logging borehole cuttings, preparing paper logs, and arguing with the drillers about ROP and TCR… these days I don’t leave my desk, I’m logging automated data loads, manage digital well logs and wrestle with RDP access or TCP packets. Yet, I’m still in the geosciences. It’s been an interesting journey.

Initially, I used this blog to log my baby steps of becoming tech savvy. After making the official transition from a technological geologist to IT professional, there must have been to much to learn to find the time and blog about it. That’s when my editorial output waned. So, I’m making another attempt at reviving my page.  I have no particular audience in mind. Instead I like to think of it as merely a collection of useful tips and tricks for geo-technologists like myself who never went through any rigorous IT schooling but have learned to leverage technology to solve problems.

By now, I have 6 years of Python coding under my belt. I have moved from GIS to supporting a range of geoscience applications in the petroleum industry. I spent 3 years working with SQL Server and have spent the last year learning about Oracle. I once dabbled in Linux but will soon have to brush up on that. Also, after successfully completing a 4-course Python certificate, I’m now working towards completing one for C#/.NET.

Along the way, I’ve collected my share of notes and made my share of rookie mistakes. That’s not to say I don’t still feel like a rookie at least once every day. But I have gotten better about finding solutions quickly, asking the right questions, and knowing when it’s time to call in the Marines. So welcome to my blog, and leave me a message if you found something useful.

Leave a comment

Filed under Uncategorized

Good Intro to using SQL Server efficiently (Joes 2 Pros)

If you’re like me and you’re tired of being a SQL hack, producing SQL code that is the product of trial-and-error mixed in with google search results, then you might like to spend $100 on these four (4) volumes. I ordered the whole package on amazon and have it decorating my office shelf. Just about to finish the first book, and while little in it was new, it’s been a good refresher. The tone is light. The chapters are bite size in length. Yet the table of contents shows this series is quite comprehensive. So if you’re just getting started with SQL or feel like it’s time to get that first SQL book you’ve been avoiding, give these a chance.

Leave a comment

Filed under Uncategorized

Webservices in Python (or FME?)

Finally – to round off my triple of blog posts here (I’m getting caught up) – I have looked into consuming web services in Python with suds (https://fedorahosted.org/suds/). It looks straight forward. At this point, the only limitation is my knowledge of web services. Something as simple as:

import suds
from suds.client import Client
url = 'http://www.webservicex.net/usaddressverification.asmx?WSDL'
client = Client(url) 
print client
 

gets you a description of the service and its methods for this zip code verification service. This just happens to be a free service I could use for testing. One fine day, I hope to figure out how to use this for downloading data from a vendor who provides a wsdl web service portal. In fact, FME may offer a web services reader, too, and I need to take a look at that as well.

Leave a comment

Filed under Uncategorized

Writing to ArcSDE from FME Desktop

I have been doing a lot of simple ETL using FME Desktop, but ran into some problems with writing directly to ArcSDE. So I ended up just writing back to SHP and exporting to SDE from ArcCatalog.

Turns out I was using the wrong writer. Insetad of using the ESRI ArcSDE writer, which writes directly to (in my case) SQL Server but doesn’t utilize ArcObjects, you need to use the ESRI Geodatabase (ArsSDE) writer. Other wise you’re creating tables in your database but they’re not registered with the geodatabase. I figured this out the hard way when my newly created feature classes didn’t show up in ArcCatalog but I could see them in SQL Server Management Studio.

What I like about FME is the simplicity. I’ve read that FME speed outperforms Python scripting. I suppose that’s true. I haven’t run any tests. But when it comes to ESRI geoprocessing, I much prefer FME over the ESRI tools even for simple tasks. Using the a few wildcard characters and merging feature types is a lot easier than having to worry about matching schemas, gp.workspaces, or glob.glob() searches with ESRI Python.

Leave a comment

Filed under Uncategorized

Accessing SQL Server from Python

So, I am working on a Python Programming Certificate and had to spend a bit of time working with MySQL. But I was wishing I could learn how to acess SQL Server from Python. Turns out that’s pretty straight forward:

import pyodbc

server = "myserver"
driver = "SQL Server"
database = "mydatabase"

connStr = "DRIVER={%s};SERVER=%s;DATABASE=%s;Trusted_Connection=yes"%(driver,server,database)
conn = pyodbc.connect(connStr)
dbCursor = conn.cursor
query = "SELECT * FROM sys.tables"
dbCursor.execute(query)
for table in dbCursor:
     print table[0]

That brings up a list of all the table names in mydatabase and prints it to the screen. Time to work more with SQL Server in Python.

You can download pyodbc from http://code.google.com/p/pyodbc/.

Leave a comment

Filed under Uncategorized

Weird Maps in the Newspaper…

You can learn a lot about how little thought goes into a lot of maps by studying maps in the newspaper. What’s wrong with this one (as seen in the Dallas Morning News, a few days ago ?

Leave a comment

Filed under Uncategorized

From GIS Technologist to Geo Technologist

I feel blessed to be so busy at work that I haven’t found the time to blog. But it’s a shame I’ve been so blogless.

For the past 6 months or so, I have been working on all kinds of geospatial challenges. I’m slowly moving from GIS into data management. Working for an oil-gas company, I’m a lot more exposed to non-GIS software application used in the E/P industry. So I haven’t only learned how to operate Trimble GPS hardware and software, and better understand ArcSDE, I am also getting better with SQL Server and SQL databases. I’ve been involved in our company’s enterprise wide Master Data Management solutionefforts, developing schemata for organizing unstructured E/P data in RDBMS and investigating use and applications of PPDM (Public Petroleum Data Model) and pipeline models… the list goes on.

Sometimes, I wonder if I’m still a GIS guy since much of this isn’t really geospatial. But that’s okay. Branching out makes life and work more interesting. Also, I’ve had the chance to get better with Python. I signed up for a 4-course Python Certificate online, and am about 50% done. Most of what was covered in the 1st course I already knew. But the 2nd course was a good bang for my buck, and I’m excited about the rest.

What I’d like to check out is the extension ofPtyhon’s DB API for SQL Server. I’ve spent a little time working with the FME SQl Server edition to grab data from various SQL Server databases and plug them into something else (e.g. ArcSDE), and FME isnt just easy to use, it’s also nice and fast. But I would love to be able to just write in Python and do the same thing. So while Python/MySQL was covered in my 2nd course, I need to look into SQL Server. Well,  that’s wraps it up for the day. I hope to be a regular on my blog again soon. Headed to the ESRI PUG 2012 in Houston next. So hopefully that will help re-connect with GIS and provide enough inspiration for new GIS challenges and blog posts.

 

Leave a comment

Filed under Uncategorized