24. The strength of Coding is
NESTED LOOPS
col = 0
for row in xrange(dam_sheet.nrows):
if row > 0:
print dam_sheet.cell_value(row, col)
name = dam_sheet.cell_value(row, col)
rows = arcpy.SearchCursor(inputxs)
def round_figures(x, n):
return round(x, int(n
- math.ceil(math.
log10(abs(x)))))
for row in rows:
station = str(round_figures
(row.ProfileM, 7))
flow = str(row.Breach_Flow)
28. Make liberal use of
Windows Scheduler to run scripts
I may be playing,
but my computer
is working
29. Summary:
1. Efficiency | to end drudgery
2. Automate | learn the tools
3. ModelBuilder | for iteration
4. FME | for conversion
5. Code| for nested loops
30.
31. Get in touch
Bond Harper is an Analyst for
Arup (a global engineering firm)
in Los Angeles, CA
Email: bond.harper@arup.com
Twitter: @bondaharper
Linkedin: Bond Harper
Editor's Notes
Work conference story
This is where efficiency comes in. If I start out loving GIS, but am overrun every day by endless minutiae, I won’t be very happy. However, if I can take a portion of my time and develop processes that do the same tasks in less time, I can continually pursue the more creative aspects of life.
What things do you enjoy? I love being outdoors and interacting with the natural world. If I’m not careful, GIS which is very computer oriented can tie me to my desk
Being more efficient helps free my time
Some people are hesistant to be more efficient. They feel like if they do their work faster, their boss will just pile more work on them. Then it becomes an endless cycle of ever increasing workload.
If your work doesn’t offer you creative freedom in return for efficient results, be honest with yourself that it might be time to find a job that does.
There are many ways to be efficient, but today I’m going to talk about using automation methods related to GIS data.
I found this graph awhile back. I think it is a good example of different ways of thinking about repetitive tasks. The blue line representing status quo is reacting to all tasks the same whether they need to be done over and over or only once. The red line is realizing there is a point at which the extra effort to automate will pay increasing dividends in the end.
Personally, I tend to overestimate how long it will take to automate something, and underestimate the benefits or need of automating it.
Today I’ll address 3 automation tools. The first is ModelBuilder which is a built in ArcGIS tool. The second is FME which is a separate software that can be worth its cost many times over. And thirdly is coding or scripting which is free and software independent.
Modelbuilder is a great place to start because it is easy to use yet quite powerful. ESRI has a lot of short tutorials that will get you going.
If you are a QGIS user, never fear, there is a similar tool called Sextante that enables you to string tools together
For me the true power of ModelBuilder is iteration. There are several different kinds of iterators available to you. You can only use one per model, but that gets you quite a lot.
You can run analytics on an entire geodatabase. This is very handy if you have an inventory of lots of similar items that need to be processed in the same way. I’ve used this to run through an inventory of dams and run a breach analysis.
You can also run through a folder. This allows you to access some of the more open geospatial data types like shapefiles.
Whether you need the iterator or not, you can chain any combination of default analytic or data manipulation tools together to create a complete process that can be run at the click of a button.
Next up is FME.
FME’s biggest benefit is the ability to transform data between an impressive variety of formats. It helps you to never have to turn data away because it isn’t in your GIS software’s default or accepted format.
For me working in an engineering firm, a lot of data is only useful to a handful of people in standard GIS formats. With FME, we can semlessly accept and deliver data in whatever format is preferred so no one need work in a software that they aren’t comfortable with.
FME is in my experience more adroit at cleaning up data than default GIS software tools. If you frequently acquire data that has not been well standardized, you can flag and fix problems.
You can also have it detect changes between different datasets and output the changes to a file for inspection. This is handy if you need quality control or if you outsource data creation for cost savings.
If you have FME Server, you can really impress people and free up your time by allowing them to convert their data using an email. They simply email a file and then get a converted file back. You never have to see the data or the email unless you go check the logs.
Lastly is writing your own code or scripts. Don’t be intimidated by writing code. If you can search the internet you can borrow from lots of freely shared code samples to cobble together your own.
The best part of coding for me is the ability to do nested loops. You can be as complex as you want in code, with minimal pain. Take an evening to learn the basics of a language like Python and you will be well on your way. This code sample takes both an input table and geodatabase and runs through each record to gather dam breach flows to be input into a hydraulic model.
Writing your own code also makes it easier to use decision trees to go in any direction based on either the data or a user input during the script.
The best part of coding is you can run things in many different programs through you code so you aren’t so limited.
If you are working on a large project, I highly recommend using version control which allows you to collaborate either with your coworkers using software like Gitlab and Tortoise Hg or with anyone on the internet using Github. Even if your scripts are relatively small, version control saves a lot of headaches if you need to backtrack or take it in a different direction.
Code is particularly well suited to being added to Windows Scheduler to run when you aren’t around. Your computer can be criunching away at data while you are at lunch or home sleeping.
In summary, we want to increase efficiency to enjoy our work more. We can choose a tool that best suits our needs to get the most out of what we put in.
The founder of our company was a creative engineer named Ove Arup. I’ll read you a quote from what is called the key sppech which is a guidance document for how our company (which is employee owned) operates.
