Lesson 1: Introduction to GIS modeling and Python
Lesson 1: Introduction to GIS modeling and Python jed124The links below provide an outline of the material for this lesson. Be sure to carefully read through the entire lesson before returning to Canvas to submit your assignments.
Lesson 1 Overview
Lesson 1 Overview jed124Welcome to Geography 485. Over the next ten weeks, you'll work through four lessons and a final project dealing with ArcGIS automation in Python. Each lesson will contain readings, examples, and projects. Since the lessons are two weeks long, you should plan between 20 - 30 hours of work to complete them, although this number may vary depending on your prior programming experience. See the Course Schedule section of this syllabus, below, for a schedule of the lessons and course projects.
As with GEOG 483 and GEOG 484, the lessons in this course are project-based with key concepts embedded within. However, because of the nature of computer programming, there is no way this course can follow the step-by-step instructional design of the previous courses. You will probably find the course to be more challenging than our courses on GIS fundamentals. For that reason, it is more important than ever that you stay on schedule and take advantage of the course message boards and private email. It's quite likely that you will get stuck somewhere during the course, so before getting hopelessly frustrated, please seek help from me or your classmates!
I hope that by now that you have reviewed our Orientation and Syllabus for an important course site overview. Before we begin our first project, let me share some important information about the textbook and a related Esri course.
Textbook and Readings
The textbook for this course is Python Scripting for ArcGIS Pro by Paul A. Zandbergen. As you read through Zandbergen's book, you'll see material that closely parallels what is in the Geog 485 lessons. This isn't necessarily a bad thing; when you are learning a subject like programming, it can be helpful to have the same concept explained from two angles.
My advice about the readings is this: Read the material on the Geog 485 lesson pages first. If you feel like you have a good understanding from the lesson pages, you can skim through some of the more lengthy Zandbergen readings. If you struggled with understanding the lesson pages, you should pay close attention to the Zandbergen readings and try some of the related code snippets and exercises. I suggest you plan about 1 - 2 hours per week of reading if you are going to study the chapters in detail.
In all cases, you should get a copy of the textbook because it is a relevant and helpful reference.
Note:
The Zandbergen textbook is up to the 3rd Edition as of Summer 2024. The free copy of the book available through the PSU library is the 2nd Edition. Differences between the two editions are relatively minor and you may assume that the section numbers referenced here in the lessons are applicable to both editions unless otherwise noted.
You may see that in Esri's documentation, shapefiles are also referred to as "feature classes." When you see the term "feature class," consider it to mean a vector dataset that can be used in ArcGIS.
Esri Virtual Campus Course Python for Everyone
There is a free Esri Virtual Campus course, Python for Everyone, that introduces a lot of the same things you'll learn this term in Geog 485. Python for Everyone consists of a series of short videos and exercises, some of which might help toward the projects. If you want to get a head start, or you want some reinforcement of what we're learning from a different point of view, it would be worth your time to complete that Virtual Campus course.
All you need in order to access the course is an Esri Global Account, which you can create for free. You do not need to obtain an access code from Penn State.
The course moves through ideas very quickly and covers a range of concepts that we'll spend 10 weeks studying in depth, so don't worry if you don't understand it all immediately or if it seems overwhelming. You might find it helpful to quickly review the course again near the end of Geog 485 to review what you've learned.
Questions?
If you have any questions now or at any point during this week, please feel free to post them to the Lesson 1 Discussion Forum. (To access the forums, return to Canvas via the Canvas link. Once in Canvas, you can navigate to the Modules tab, and then scroll to the Lesson 1 Discussion Forum.) While you are there, feel free to post your own responses if you are able to help a classmate.
Now, let's begin Lesson 1.
Lesson 1 Checklist
Lesson 1 Checklist jed124This lesson is two weeks in length. (See the Calendar in Canvas for specific due dates.) To finish this lesson, you must complete the activities listed below. You may find it useful to print this page so that you can follow along with the directions.
- Download the Lesson 1 data and extract it to C:\PSU\Geog485 or your own desired path.
- Work through the online sections of Lesson 1.
- Read the assigned pages from the Zandbergen textbook. In the online lesson pages, I have inserted instructions on the sections to read from Zandbergen when it is most appropriate to read them.
- Complete Project 1, Part I and submit the deliverables to the course drop box in Canvas.
- Complete Project 1, Part II and submit the deliverables to the course drop box in Canvas.
- Complete the Lesson 1 Quiz in Canvas.
Do items 1 - 3 (including any of the practice exercises you want to attempt) during the first week of the lesson. You will need the second week to concentrate on the project and quiz.
Lesson Objectives
By the end of this lesson, you should:
- be able to create automated workflows in ArcGIS Pro ModelBuilder;
- be familiar with the PyScripter development environment;
- know some of Python’s most important basic data types and how to use variables in Python scripts;
- be familiar with the basic concepts of object oriented programming (e.g. classes & inheritance);
- be able to use arcpy tool functions to achieve basic spatial analysis tasks; and
- know how to create a simple script tool and pass a parameter to a script.
1.1.1 The need for GIS automation
1.1.1 The need for GIS automation jed124A geographic information system (GIS) can manipulate and analyze spatial datasets with the purpose of solving geographic problems. GIS analysts perform all kinds of operations on data to make it useful for solving a focused problem. This includes clipping, reprojecting, buffering, merging, mosaicking, extracting subsets of the data, and hundreds of other operations. In the ArcGIS software used in this course, these operations are known as geoprocessing and they are performed using tools.
Successful GIS analysis requires selecting the most appropriate tools to operate on your data. ArcGIS uses a toolbox metaphor to organize its suite of tools. You pick the tools you need and run them in the proper order to make your finished product.
Suppose you’re responsible for selecting sites for a chain restaurant. You might use one tool to select land parcels along a major thoroughfare, another tool to select parcels no smaller than 0.25 acres, and other tools for other selection criteria. If this selection process were limited to a small area, it would probably make sense to perform the work manually.
However, let’s suppose you’re responsible for carrying out the same analysis for several areas around the country. Because this scenario involves running the same sequence of tools for several areas, it is one that lends itself well to automation. There are several major benefits to automating tasks like this:
- Automation makes work easier. Once you automate a process, you don't have to put in as much effort remembering which tools to use or the proper sequence in which they should be run.
- Automation makes work faster. A computer can open and execute tools in sequence much faster than you can accomplish the same task by pointing and clicking.
- Automation makes work more accurate. Any time you perform a manual task on a computer, there is a chance for error. The chance multiplies with the number and complexity of the steps in your analysis, as well as the fatigue incurred by repeating the task many times. In contrast, once an automated task is configured, a computer can be trusted to perform the same sequence of steps every time for a potentially endless number of cycles.
The ArcGIS platform provides several ways for users to automate their geoprocessing tasks. These options differ in the amount of skill required to produce the automated solution and in the range of scenarios that each can address. The text below touches briefly on these automation options, in order from requiring the least coding skill to the most.
The first option is to construct a model using ModelBuilder. ModelBuilder is an interactive program that allows the user to “chain” tools together, using the output of one tool as input in another. Perhaps the most attractive feature of ModelBuilder is that users can automate rather complex GIS workflows without the need for programming. You will learn how to use ModelBuilder early in this course.
Some automation tasks require greater flexibility than is offered by ModelBuilder, and for these scenarios it's recommended that you write short computer programs, or scripts. The bulk of this course is concerned with script writing.
A script typically executes some sequential procedure of steps. Within a script, you can run GIS tools individually or chain them together. You can insert conditional logic in your script to handle cases where different tools should be run depending on the output of the previous operation. You can also include iteration, or loops, in a script to repeat a single action as many times as needed to accomplish a task.
Although ArcGIS supports various scripting languages for working with its tools, Esri emphasizes Python in its documentation and includes Python with the ArcGIS Pro and ArcGIS Server installations. In this course, we’ll be working strictly with Python for this reason, as well as the fact that Python can be used for many other file and data manipulation tasks outside of ArcGIS. You’ll learn the basics of the Python language, how to write a script, and how to manipulate and analyze GIS data using scripts. Finally, you’ll apply your new Python knowledge to a final project, where you write a script of your choosing that you may be able to apply directly to your work.
A more recently developed automation option available on the ArcGIS platform is the ArcGIS for Python API (Application Programming Interface). The ArcGIS API is written to work with the web based architecture of the ArcGIS Enterprise and other web based data. The API provides programmatic administration to ArcGIS Enterprise, as well as methods to consume and perform analyses from web-based GIS data, integrating popular data science packages such as pandas, numpy and Jupyter Notebook environments. The use of the Python API in a Jupyter Notebook environment is a topic in our Advanced Python class, GEOG 489
For geoprocessing tasks that require support for user interaction with the map or other UI elements, the ArcGIS Pro SDK (Software Development Kit) offers the ability to add custom tools to the Pro interface. The Pro SDK requires programming in the .NET framework using a .NET language such as Visual Basic .NET or C#. Working with this SDK's object model provides greater flexibility in terms of what can be built, as compared to writing Python scripts around their geoprocessing framework. The tradeoff is a higher level of complexity involved in the coding.
Finally, developers who want to create their own custom GIS applications, typically focused on delivering much narrower functionality than the one-size-fits-all ArcGIS Pro, can develop apps using the ArcGIS Maps SDKs (previously called Runtime SDKs). The Maps SDKs make it possible to author apps for Windows, Mac, or Linux desktop machines, as well as for iOS and Android mobile devices, again involving a greater level of effort than your typical Python geoprocessing script. In the past, there was a native version for MacOS but that has been retired. Instead, programmers can use the Maps SDK for Java version to develop for MacOS.
This first lesson will introduce you to concepts in both model building and script writing. We’ll start by just getting familiar with how tools run in ArcGIS and how you can use those tools in the ModelBuilder interface. Then, we’ll cover some of the basics of Python and see how the tools can be run within scripts.
1.2.1 Exploring the toolbox
1.2.1 Exploring the toolbox jed124The ArcGIS software that you use in this course contains hundreds of tools that you can use to manipulate and analyze GIS data. Back before ArcGIS had a graphical user interface (GUI), people would access these tools by typing commands. Nowadays, you can point and click your way through a whole hierarchy of toolboxes using the Catalog window in ArcGIS Pro.
Although you may have seen them before, let’s take a quick look at the toolboxes:
- Open a new or existing project in ArcGIS Pro.
- If the Geoprocessing pane isn't visible, click the Analysis tab, then click Tools.
- In the Geoprocessing pane, click the Toolboxes tab heading. Notice that the tools are organized into toolboxes and toolsets. For example, the IDW and Spline tools can be found in the Interpolation toolset within the Spatial Analyst toolbox. Sometimes it’s faster to use the Find Tools box at the top of the Geoprocessing tab to find the tool you need instead of browsing this tree.
-
Let’s examine a tool. Expand Analysis Tools > Proximity > Buffer, and double-click the Buffer tool to open it.
You've probably seen this tool in past courses, but this time, really pay attention to the components that make up the user interface. Specifically, you’re looking at a dialog with many fields. Each geoprocessing tool has required inputs and outputs. Those are indicated by the red asterisks. They represent the minimum amount of information you need to supply in order to run a tool. For the Buffer tool, you’re required to supply an input features location (the features that will be buffered) and a buffer distance. You’re also required to indicate an output feature class location (for the new buffered features).
Many tools also have optional parameters. You can modify these if you want, but if you don’t supply them, the tool will still run using default values. For the Buffer tool, optional parameters are the Side Type, End Type, Method, and Dissolve Type. Optional parameters are typically specified after required parameters.
-
Hover your mouse over any of the tool parameters. You should see a blue "info" icon to the left of the parameter. Moving your mouse over that icon will show a brief description of the parameter in a pop-out window.
If you’re not sure what a parameter means, this is a good way to learn. For example, viewing the pop-out documentation for the End Type parameter will show you an explanation of what this parameter means and list the two options: Round and Flat.
If you need even more help, each tool is more expansively documented in the ArcGIS Pro web-based help system. You can access a tool's documentation in this system by clicking on the blue ? icon in the upper-right of the tool dialog, which will open the help page in your default web browser.
- Open the web-based help page for the Buffer tool. As mentioned, the help page will provide extensive documentation of the tool's usage. In the upper right of the help page, you should see a list of links that are shortcuts to various sections of the tool's documentation.
- Click the Parameters link to jump to that section. There you should find that the information is divided between two tabs (Dialog and Python), with the Dialog tab displayed by default. The Dialog tab focuses on providing help to those who are executing the tool through the ArcGIS Pro GUI, while the Python tab focuses on providing help to those who are invoking it through a Python script.
- Click on the Python tab, since that's our focus in this class. You should see the name, a description, and the data type associated with each of the tool's parameters. That information is certainly helpful, but often even more helpful is the Code sample section that appears below the parameter list. Every tool's help page has these programming examples showing ways to run the tool from within a script, and they will be extremely valuable to you as you complete the assignments in this course.
1.2.2 Environments for accessing tools
1.2.2 Environments for accessing tools jed124You can access ArcGIS geoprocessing tools in several different ways:
- Sometimes you just need to run a tool once, or you want to experiment with a tool and its parameters. In this case, you can open the tool directly from the Geoprocessing pane and use the tool’s graphical user interface (GUI, pronounced gooey) to fill in the parameters.
- ModelBuilder is also a GUI application where you can set up tools to run in a given sequence, using the output of one tool as input to another tool.
- If you’re familiar with the tool and want to use it quickly in Pro, you may prefer the Python window approach. You type the tool name and required parameters into a command window. You can use this window to run several tools in a row and declare variables, effectively doing simple scripting.
- If you want to run the tool automatically, repeatedly, or as part of a greater logical sequence of tools, you can run it from a script. Running a tool from a script is the most powerful and flexible option.
We’ll start with the simplest of these cases, running a tool from its GUI, and work our way up to scripting.
1.2.3 Running a tool from its GUI
1.2.3 Running a tool from its GUI jed124Let’s start by opening a tool from the Catalog pane and running it using its graphical user interface (GUI).
- If, by chance, you still have the Buffer tool open from the previous section, close it for now so you can add some data.
- Download the Lesson 1 data and extract Lesson1.gdb.zip into your C:\PSU\Geog485 (or other desired) folder. The zip file should extract to a file geodatabase called Lesson1.gdb, which contains all of the data used in Lesson 1.
- Open a new project in Pro with a new empty map if you don't have one open already.
- Add the us_boundaries and us_cities feature classes from the USA feature dataset to your map.
- Open the Geoprocessing pane if necessary and browse to the Buffer tool as you did in the previous section.
- Double-click the Buffer tool to open it.
-
Examine the first required parameter: Input Features. Click the Browse button
and browse to the path of your cities dataset C:\PSU\Geog485\Lesson1.gdb\USA\us_cities. Notice that once you do this, a name is automatically supplied for the Output Feature Class (and the output path is your project's default workspace). The software does this for your convenience only, and you can change the name/path if you want.A more convenient way to supply the Input Features is to just select the cities map layer from the dropdown menu. This dropdown automatically contains all the layers in your map. However, in this example, we browsed to the path of the data because it’s conceptually similar to how we’ll provide the paths in the command line and scripting environments.
- Now you need to supply the Distance parameter for the buffer. For this run of the tool, set a Linear unit of 5 Statute Miles. When we run the tool from the other environments, we’ll make the buffer distance slightly larger, so we know that we got distinct outputs.
- The rest of the parameters are optional. The Side Type parameter applies only to lines and polygons, so it is not even available for setting in the GUI environment when working with city points. However, change the Dissolve Type to Dissolve all output features into a single feature. This combines overlapping buffers into a single polygon.
- Leave the Method set to Planar. We're not going to take the earth's curvature into account for this simple example.
- Click Run to execute the tool.
- The tool should take just a few seconds to complete. Examine the output that appears on the map, and do a “sanity check” to make sure that buffers appear around the cities, and they appear to be about 5 miles in radius. You may need to zoom in to a single state in order to see the buffers.
- Click on Pro's Analysis tab, then on History (in the Geoprocessing button group). This will open the History tab in Pro's Catalog pane, which lists messages about successes or failures of all recent tools that you've run.
-
Hover over the Buffer tool entry in this list to see a pop-out window. This window lists the tool parameters, the time of completion, and any problems that occurred when running the tool (see Figure 1.1). These messages can be a big help later when you troubleshoot your Python scripts. The text of these messages is available whether you run the tool from the GUI, from the Python window in Pro, or from scripts.
1.2.4 Modeling with tools
1.2.4 Modeling with tools jed124When you work with geoprocessing, you’ll frequently want to use the output of one tool as the input into another tool. For example, suppose you want to find all fire hydrants within 200 meters of a building. You would first buffer the building, then use the output buffer as a spatial constraint for selecting fire hydrants. The output from the Buffer tool would be used as an input to the Select by Location tool.
A set of tools chained together in this way is called a model. Models can be simple, consisting of just a few tools, or complex, consisting of many tools and parameters and occasionally some iterative logic. Whether big or small, the benefit of a model is that it solves a unique geographic problem that cannot be addressed by one of the “out-of-the-box” tools.
In ArcGIS, modeling can be done either through the ModelBuilder graphical user interface (GUI) or through code, using Python. To keep our terms clear, we’ll refer to anything built in ModelBuilder as a “model” and anything built through Python as a “script.” However, it’s important to remember that both things are doing modeling.
1.3.1 Why learn ModelBuilder?
1.3.1 Why learn ModelBuilder? jed124ModelBuilder is Esri’s graphical interface for making models. You can drag and drop tools from the Catalog pane into the model and “connect” them, specifying the order in which they should run.
Although this is primarily a programming course, we’ll spend some time in ModelBuilder during the first lesson for two reasons:
- ModelBuilder is a nice environment for exploring the ArcGIS tools, learning how tool inputs and outputs are used, and visually understanding how GIS modeling works. When you begin using Python, you will not have the same visual assistance to see how the tools you’re using are connected, but you may still want to draw your model on a whiteboard in a similar fashion to what you saw in ModelBuilder.
- ModelBuilder can frequently reduce the amount of Python coding that you need to do. If your GIS problem does not require advanced conditional and iterative logic, you may be able to get your work done in ModelBuilder without writing a script. ModelBuilder also allows you to export any model to Python code, so if you get stuck implementing some tools within a script, it may be helpful to make a simple working model in ModelBuilder, then export it to Python to see how ArcGIS would construct the code. (Exporting a complex model is not recommended for beginners due to the verbose amount of code that ModelBuilder tends to create when exporting Python).
1.3.2 Opening and exploring ModelBuilder
1.3.2 Opening and exploring ModelBuilder jed124Let’s get some practice with ModelBuilder to solve a real scenario. Suppose you are working on a site selection problem where you need to select all areas that fall within 10 miles of a major highway and 10 miles of a major city. The selected area cannot lie in the ocean or outside the United States. Solving the problem requires that you make buffers around both the roads and the cities, intersect the buffers, then clip to the US outline. Instead of manually opening the Buffer tool twice, followed by the Intersect tool, then the Clip tool, you can set this up in ModelBuilder to run as one process.
- Create a new Pro project called Lesson1Practice in the C:\PSU\Geog485 folder.
- Add the us_cities, us_roads, and us_boundaries feature classes from the Lesson1 file geodatabase to a new map.
- In the Catalog pane, right-click your Lesson1Practice toolbox and click New > Model. You’ll see ModelBuilder appear in the middle of the Pro window.
- Under the ModelBuilder tab, click the Properties button.
- For the Name, type SuitableLand and for the Label, type Find Suitable Land. The label is what everyone will see when they open your tool from the Catalog. That’s why it can contain spaces. The name is what people will use if they ever run your model from Python. That’s why it cannot contain spaces.
-
Click OK to dismiss the model Properties dialog.
You now have a blank canvas on which you can drag and drop the tools. When creating a model (and when writing Python scripts), it’s best to break your problem into manageable pieces. The simple site selection problem here can be thought of as four steps:
- Buffer the cities
- Buffer the roads
- Intersect the buffers
- Clip to the US boundary
Let’s tackle these items one at a time, starting with buffering the cities.
- With ModelBuilder still open, go to the Catalog pane > Geoprocessing tab and browse to Analysis Tools > Proximity.
-
Click the Buffer tool and drag it onto the ModelBuilder canvas. You’ll see a gray rectangular box representing the buffer tool and a gray oval representing the output buffers. These are connected with a line, showing that the Buffer tool will always produce an output data set.
In ModelBuilder, tools are represented with boxes and variables are represented with ovals. Right now, the Buffer tool, at center, is gray because you have not yet supplied the required parameters. Once you do this, the tool and the variable will fill in with color.
- In your ModelBuilder window, double-click the Buffer box. The tool dialog here is the same as if you had opened the Buffer directly from the Catalog pane. This is where you can supply parameters for the tool.
- For Input Features, browse to the path of your us_cities feature class on disk. The Output Feature Class will populate automatically.
- For Distance [value or field], enter 10 Statute Miles.
- For Dissolve Type, select Dissolve all output features..., then click OK to close the Buffer dialog. The model elements (tools and variables) should be filled in with color, and you should see a new element to the left of the tool representing the input us_cities feature class.
-
An important part of working with ModelBuilder is supplying clear labels for all the elements. This way, if you share your model, others can easily understand what will happen when it runs. Supplying clear labels also helps you remember what the model does, especially if you haven’t worked with the model for a while.
In ModelBuilder, right-click the us_cities element (blue oval, at far left) and click Rename. Name this element "US Cities."
- Right-click the Buffer tool (yellow-orange box, at center) and click Rename. Name this “Buffer the cities.”
-
Right-click the buffer output element (green oval, at far right) and click Rename. Name this “Buffered cities.” Your model should look like this.
Figure 1.2 The model's appearance following step 15, above. - Save your model (ModelBuilder tab > Save). This is the kind of activity where you want to save often.
-
Practice what you just learned by adding another Buffer tool to your model. This time, configure the tool so that it buffers the us_roads feature class by 10 miles. Remember to set the Dissolve type to Dissolve all output features... and to add meaningful labels. Your model should now look like this.
Figure 1.3 The model's appearance following step 17, above. - The next task is to intersect the buffers. In the Catalog pane's list of toolboxes, browse to Analysis Tools > Overlay and drag the Intersect tool onto your model. Position it to the right of your existing Buffer tools.
- Here’s the pivotal moment when you chain the tools together, setting the outputs of your Buffer tools as the inputs of the Intersect tool. Click the Buffered cities element and drag over to the Intersect element. If you see a small menu appear, click Input Features to denote that the buffered cities will act as inputs to the Intersect tool. An arrow will now point from the Buffered cities element to the Intersect element.
- Use the same process to connect the Buffered roads to the Intersect element. Again, if prompted, click Input Features.
-
Rename the output of the Intersect operation "Intersected buffers." If the text runs onto multiple lines, you can click and drag the edges of the element to resize it. You can also rearrange the elements on the page however you like. Because models can get large, ModelBuilder contains several navigation buttons for zooming in and zooming to the full extent of the model in the View button group on the ribbon. Your model should now look like this:
Figure 1.4 The model's appearance following step 21, above. - The final step is to clip the intersected buffers to the outline of the United States. This prevents any of the selected area from falling outside the country or in the ocean. In the Catalog pane, browse to Analysis Tools > Extract and drag the Clip tool into ModelBuilder. Position this tool to the right of your existing tools.
- As you did above, set the Intersected buffers as an input to the Clip tool, choosing Input Features when prompted. Notice that even when you do this, the Clip tool is not ready to run (it’s still shown as a gray rectangle, located at right). You need to supply the clip features, which is the shape to which the buffers will be clipped.
- In ModelBuilder (not in the Catalog pane), double-click the Clip tool. Set the Clip Features by browsing to the path of us_boundaries, then click OK to dismiss the dialog. You’ll notice that a blue oval appeared, representing the Clip Features (US Boundaries).
-
Set meaningful labels for the remaining tools as shown below. Below is an example of how you can label and arrange the model elements.
Figure 1.5 The completed model with the clip tool included. - Double-click the final output element (named "Suitable Land" in the image above) and set the path to C:\PSU\Geog485\Lesson1.gdb\USA\suitable_land. This is where you can expect your model output feature class to be written to disk.
- Right-click Suitable Land and click Add to display.
- Save your model again.
- Test the model by clicking the Run button.
You’ll see the by-now-familiar geoprocessing message window that will report any errors that may occur. ModelBuilder also gives you a visual cue of which tool is running by turning the tool red. (If the model crashes, try closing ModelBuilder and running the model by double-clicking it from the Catalog pane. You'll get a message that the model has no parameters. This is okay [and true, as you'll learn below]. Go ahead and run the model anyway.) -
When the model has finished running (it may take a while), examine the output on the map. Zoom into Washington state to verify that the has Clip worked on the coastal areas. The output should look similar to this.
Figure 1.6 The model's output in ArcGIS Pro.
That’s it! You’ve just used ModelBuilder to chain together several tools and solve a GIS problem.
You can double-click this model anytime in the Catalog pane and run it just as you would a tool. If you do this, you’ll notice that the model has no parameters; you can’t change the buffer distance or input features. The truth is, our model is useful for solving this particular site-selection problem with these particular datasets, but it’s not very flexible. In the next section of the lesson, we’ll make this model more versatile by configuring some of the variables as input and output parameters.
1.3.3 Model parameters
1.3.3 Model parameters jed124Most tools, models, and scripts that you create with ArcGIS have parameters. Input parameters are values with which the tool (or model or script) starts its work, and output parameters represent what the tool gives you after its work is finished.
A tool, model, or script without parameters is only good in one scenario. Consider the model you just built that used the Buffer, Intersect, and Clip tools. This model was hard-coded to use the us_cities, us_roads, and us_boundaries feature classes and output a feature class called suitable_land. In other words, if you wanted to run the model with other datasets, you would have to open ModelBuilder, double-click each element (US Cities, US Roads, US Boundaries, and Suitable Land), and change the paths that were written directly into the model. You would have to follow a similar process if you wanted to change the buffer distances, too, since those were hard-coded to 10 miles.
Let’s modify that model to use some parameters, so that you can easily run it with different datasets and buffer distances.
- If it's not open already, open the project C:\PSU\Geog485\Lesson1Practice.aprx in ArcGIS Pro.
- In the Catalog window, find the model you created earlier in the lesson, which should be under Toolboxes > Lesson1Practice.atbx > Find Suitable Land.
- Right-click the model Find Suitable Land and click Copy. Now, right-click the Lesson1Practice toolbox and click Paste. This creates a new copy of your model that you can work with to create model parameters. Using a copy of the model like this allows you to easily start over if you make a mistake.
- Rename the copy of your model Find Suitable Land With Parameters or something similar.
- In your Lesson 1 toolbox, right-click Find Suitable Land With Parameters and click Edit. You'll see the model appear in ModelBuilder.
- Right-click the element US Cities (should be a blue oval) and click Parameter. This means that whoever runs the model must specify the cities dataset to use before the model can run.
- You need a more general name for this parameter now, so right-click the US Cities element and click Rename. Change the name to just "Cities."
-
Even though you "parameterized" the cities, your model still defaults to using the C:\PSU\Geog485\Lesson1.gdb\USA\us_cities dataset. This isn't going to make much sense if you share your model or toolbox with other people because they may not have the same us_cities feature class, and even if they do, it probably won't be sitting at the same path on their machines.
To remove the default dataset, double-click the Cities element and delete the path, then click OK. Some of the elements in your model may turn gray. This signifies that a value has to be provided before the model can successfully run.
- Now you need to create a parameter for the distance of the buffer to be created around the cities. Right-click the element that you named "Buffer the cities" and click Create Variable > From Parameter > Distance [value or field].
- The previous step created a new element Distance [value or field]. Rename this element to "Cities buffer distance" and make it a model parameter. (Review the steps above if you're unsure about how to rename an element or make it a model parameter.) For this element, you can leave the default at 10 miles. Your model should look similar to this, although the title bar of your window may vary:
Figure 1.7 The "Find Suitable Land With Parameters" model following Step 10, above, and showing two parameters. Click link to expand for a text description of Figure 1.7.A gray oval labeled Cities and a light blue oval labeled Cities Buffer Distance lead to a gray box labeled Buffer the cities, which leads to a gray oval labeled buffered cities.
A blue oval labeled US roads leads to a yellow box labeled Buffer the roads, which leads to a green oval labeled Buffered roads.
The gray oval labeled Buffered cities and the green oval labeled Buffered roads both lead to a gray box labeled Intersect. This gray box leads to a gray oval labeled Intersected buffers.
The gray oval labeled Intersected buffers and a blue oval labeled US Boundaries lead to a gray box labeled Clip.
The gray box labeled Clip leads to a gray oval labeled Suitable land.
- Repeating what you learned above, rename the US Roads element to "Roads," make it a model parameter, and remove the default value.
- Repeating what you learned above, make a parameter for the Roads buffer distance. Leave the default at 10 miles.
- Repeating what you learned above, rename the US Boundaries element to Boundaries, make it a model parameter, and remove the default value. Your model should look like this (notice the five parameters indicated by "P"s):
Figure 1.8 The "Find Suitable Land With Parameters" model following Step 13, above, and showing five parameters. Click link to expand for a text description of Figure 1.8.A gray oval labeled Cities and a light blue oval labeled Cities buffer distance lead to a gray box labeled Buffer the cities which leads to a gray oval labeled Buffered cities.
A gray oval labeled Roads and a light blue oval labeled Roads buffer distance both lead to a gray box labeled Buffer the roads, which leads to a gray oval labeled Buffered roads.
The gray oval labeled Buffered cities and the gray oval labeled Buffered roads both lead to a gray box labeled Intersect. This gray box leads to a gray oval labeled Intersected buffers.
The gray oval labeled Intersected buffers and a white oval labeled Boundaries lead to a white box labeled Clip.
The gray box labeled Clip leads to a gray oval labeled Suitable land.
- Save and close your model.
-
Double-click your model Lesson 1 > Find Suitable Land With Parameters and examine the tool dialog. It should look similar to this:
Figure 1.9 The model interface, or tool dialog, for the model "Find Suitable Land With Parameters."People who run this model will be able to browse to any cities, roads, and boundaries datasets, and will be able to control the buffer distance. The red asterisks indicate parameters that must be supplied with valid values before the model can run.
- Test your model by supplying the us_cities, us_roads, and us_boundaries feature classes for the model parameters. If you like, you can try changing the buffer distance.
The above exercise demonstrated how you can expose values as parameters using ModelBuilder. You need to decide which values you want the user to be able to change and designate those as parameters. When you write Python scripts, you'll also need to identify and expose parameters in a similar way.
1.3.4 Advanced geoprocessing and ModelBuilder concepts
1.3.4 Advanced geoprocessing and ModelBuilder concepts jed124By now, you've had some practice with ModelBuilder, and you're about ready to get started with Python. This page of the lesson contains some optional advanced material that you can read about ModelBuilder. This is particularly helpful if you anticipate using ModelBuilder frequently in your employment. Some of the items are common to the ArcGIS geoprocessing framework, meaning that they also apply when writing Python scripts with ArcGIS.
Managing intermediate data
GIS analysis sometimes gets messy. Most of the tools that you run produce an output dataset, and when you chain many tools together, those datasets start piling up on disk. Esri has programmed ModelBuilder's default behavior such that when a model is run from a GUI interface, all datasets besides the final output -- referred to as intermediate data -- are automatically deleted. If, on the other hand, the model is run from ModelBuilder, intermediate datasets are left in their specified locations.
When running a model on another file system, specifying paths as we did above can be problematic, since the folder structure is not likely to be the same. This is where the concept of the scratch geodatabase (or scratch folder for file-based data like shapefiles) environment variable can come in handy. A scratch geodatabase is one that is guaranteed to exist on all ArcGIS installations. Unless the user has changed it, the scratch geodatabase will be found at C:\Users\<user>\Documents\ArcGIS\scratch.gdb on Windows 7/8. You can specify that a tool write to the scratch geodatabase by using the %scratchgdb% variable in the path. For example, %scratchgdb%\myOutput.
The following topics from Esri go into more detail on intermediate data and are important to understand as you work with the geoprocessing framework. I suggest reading them once now and returning to them occasionally throughout the course. Some of the concepts in them are easier to understand once you've worked with geoprocessing for a while.
Looping in ModelBuilder
Looping, or iteration, is the act of repeating a process. A main benefit of computers is their ability to quickly repeat tasks that would otherwise be mundane, cumbersome, or error-prone for a human to repeat and record. Looping is a key concept in computer programming, and you will use it often as you write Python scripts for this course.
ModelBuilder contains a number of elements called Iterators that can do looping in various ways. The names of these iterators, such as For and While actually mimic the types of looping that you can program in Python and other languages. In this course, we'll focus on learning iteration in Python, which may actually be just as easy as learning how to use a ModelBuilder iterator.
To take a peek at how iteration works in ModelBuilder, you can visit the ArcGIS Pro ModeBuilder help book for model iteration. If you're having trouble understanding looping in later lessons, ModelBuilder might be a good environment to visualize what a loop does. You can come back and visit this book as needed.
Readings
Read Zandbergen Chapter 3.1 - 3.6, and 3.8 to reinforce what you learned about geoprocessing and ModelBuilder.
1.4.1 Introducing Python Using the Python Window in ArcGIS
1.4.1 Introducing Python Using the Python Window in ArcGIS jed124The best way to introduce Python may be to look at a little bit of code. Let’s take the Buffer tool which you recently ran from the Geoprocessing pane and run it in the ArcGIS Python window. This window allows you to type a simple series of Python commands without writing full permanent scripts. The Python Window is a great way to get a taste of Python.
This time, we’ll make buffers of 15 miles around the cities.
- Open a new empty map in ArcGIS Pro.
- Add the us_cities dataset from the Lesson 1 data.
- Under the View tab, click the Python window button
(you can also access this from the Analysis tab but there you have to make sure to use the drop-down arrow to open the Python window rather than a Python Notebook). Type the following in the Python window (Don't type the >>>. These are just included to show you where the new lines begin in the Python window.)
>>> import arcpy >>> arcpy.Buffer_analysis("us_cities", "us_cities_buffered", "15 miles", "", "", "ALL")- Zoom in and confirm that the buffers were created.
You’ve just run your first bit of Python. You don’t have to understand everything about the code you wrote in this window, but here are a few important things to note.
The first line of the script -- import arcpy -- tells the Python interpreter (which was installed when you installed ArcGIS) that you’re going to work with some special scripting functions and tools included with ArcGIS. Without this line of code, Python knows nothing about ArcGIS, so you'll put it at the top of all ArcGIS-related code that you write in this class. You technically don't need this line when you work with the Python window in ArcGIS Pro because arcpy is already imported, but I wanted to show you this pattern early; you'll use it in all the scripts you write outside the Python window.
The second line of the script actually runs the tool. You can type arcpy, plus a dot, plus any tool name to run a tool in Python. Notice here that you also put an underscore followed by the name of the toolbox that includes the Buffer tool. This is necessary because some tools in different toolboxes actually have the same name (like Clip, which is a tool for clipping vectors in the Analysis toolbox or tool for clipping rasters in the Data Management toolbox).
After you typed arcpy.Buffer_analysis, you typed all the parameters for the tool. Each parameter was separated by a comma, and the whole list of parameters was enclosed in parentheses. Get used to this pattern, since you'll follow it with every tool you run in this course.
In this code, we also supplied some optional parameters, leaving empty quotes where we wanted to take the default values, and truncating the parameter list at the final optional parameter we wanted to set.
How do you know the syntax, or structure, of the parameters to enter? For example, for the buffer distance, should you enter 15MILES, ‘15MILES’, 15 Miles, or ’15 Miles’? The best way to answer questions like these is to return to the Geoprocessing tool reference help topic for the Buffer tool. All of the topics in this reference section have Usage and Code Sample sections to help you understand how to structure the parameters. Optional parameters are enclosed in braces, while the required parameters are not. From the example in this topic, you can see that the buffer distance should be specified as ’15 miles’. Because there is a space in this text, or string, you need to surround it with single quotes.
You might have noticed that the Python window helps you by popping up different options you can type for each parameter. This is called autocompletion, and it can be very helpful if you're trying to run a tool for the first time, and you don't know exactly how to type the parameters. You might have also noticed that some code pops up like (Buffer() analysis and Buffer3D() 3d) when you were typing in the function name. You can use your up/down arrows to highlight the alternatives. If you selected Buffer() analysis, it will appear in your Python window.
Please note that if you do you use the code completion your code will sometimes look slightly different - esri reorganized how the functions are arranged within arcpy, they work the same they're just in a slightly different place. The "old" way still works though so you might see inconsistencies in this class, online forums, esri's documentation etc. So, in this example, arcpy.Buffer_analysis(...) has changed to arcpy.analysis.Buffer(....) reflecting that the Buffer tool is located within the Analysis toolbox in Pro.
There are a couple of differences between writing code in the Python window and writing code in some other program, such as Notepad or PyScripter. In the Python window, you can reference layers in the map document by their names only, instead of their file paths. Thus, we were able to type "us_cities" instead of something like "C:\\data.gdb\\us_cities". We were also able to make up the name of a new layer "us_cities_buffered" and get it added to the map by default after the code ran. If you're going to use your code outside the Python window, make sure you use the full paths.
When you write more complex scripts, it will be helpful to use an integrated development environment (IDE), meaning a program specifically designed to help you write and test Python code. Later in this course, we’ll explore the PyScripter IDE.
Earlier in this lesson, you saw how tools can be chained together to solve a problem using ModelBuilder. The same can be done in Python, but it’s going to take a little groundwork to get to that point. For this reason, we’ll spend the rest of Lesson 1 covering some of the basics of Python.
Readings
Zandbergen covers the Python window and some things you can do with it in Chapter 2.
2nd Edition: 2.7, and 2.9-2.13
3rd Edition: 2.8, and 2.10-2.14
1.4.2 What is Python?
1.4.2 What is Python? jed124Python is a language that is used to automate computing tasks through programs called scripts. In the introduction to this lesson, you learned that automation makes work easier, faster, and more accurate. This applies to GIS and many other areas of computer science. Learning Python will make you a more effective GIS analyst, but Python programming is a technical skill that can be beneficial to you even outside the field of GIS.
Python is a good language for beginning programming. Python is a high-level language, meaning you don’t have to understand the “nuts and bolts” of how computers work in order to use it. Python syntax (how the code statements are constructed) is relatively simple to read and understand. Finally, Python requires very little overhead to get a program up and running.
Python is an open-source language, and there is no fee to use it or deploy programs with it. Python can run on Windows, Linux, Unix, and Mac operating systems with very little change in syntax.
In ArcGIS, Python can be used for coarse-grained programming, meaning that you can use it to easily run geoprocessing tools such as the Buffer tool that we just worked with. You could code all the buffer logic yourself, using more detailed, fine-grained programming with the ArcGIS Pro SDK, but this would be time consuming and unnecessary in most scenarios; it’s easier just to call the Buffer tool from a Python script using one line of code.
In addition to the Esri help which describes all of the parameters of a function and how to access them from Python, you can also get Python syntax (the structure of the language) for a tool like this :
- Run the tool interactively (e.g., Buffer) from the Geoprocessing pane with your input data, output data and any other relevant parameters (e.g., distance to buffer).
- Beneath the tool parameters, you should see a notification of the completed running of the tool. Right-click on that notification. (Alternatively, you can click the History button under the Analysis tab to obtain a list of run tools.)
- Pick "Copy Python command."
- Paste the code into the Python code window or PyScripter (see next section) or to see how you would code the same operation you just ran in Pro in Python.
1.4.3 Installing PyScripter
1.4.3 Installing PyScripter jed124PyScripter is an easy IDE to install for ArcGIS Pro development. ArcGIS Pro 3.0 changed the way that Pro manages the environments and gives more control to the user. These steps below are written using Pro 3.1 as a reference. If you have an earlier version of Pro, these steps are similar in process but the output destination (step 4.1) will be set for you. To do this, follow these steps below and please let the instructor know if you run into any trouble.
- In the ArcGIS Pro Settings window (often the window that opens when you start Pro), click on Package Manager on the left side to open Pro's Python Package manager.
- Next to the Active Environment near the top right, click on the gear to open a window listing the Python environments.
- Find the environment named arcgispro-py3 (Default) and click on the symbol that looks like two pages or click on the ... to expand a menu to select clone.
- A window will pop up to allow you to name the new environment. The default destination path that Pro automatically sets is ok.
- The destination should be something like C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3-clone, or C:\Users\<username>\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone. Make a note of this path (you can highlight the path and ctrl+c to copy it to your clipboard) because you will need it for PyScripter.
- Wait while it clones the environment.
- If the cloning fails with an error message saying that a Python package couldn't be installed, you may need to run ArcPro as an Administrator (do a right-click -> Run as administrator on the ArcGIS Pro icon) and repeat the steps above (it's also helpful to mouseover that error box in Pro and see if it gives you any additional details).
- When the cloning is done, the new environment "arcgispro-py3-clone" (or whatever you choose to call it - but we'll be assuming it's called the default name) can be activated by clicking on the ... and selecting Activate.
- You may need to restart Pro to get this change to take effect, and you may see a message telling you to restart. If not, then click the ok button to get back to Pro.
Now perform the following steps to install PyScripter:
- Download the installer from PyScripter's soureceforge download page, or you can download the install exe from the course files (64bit version or 32bit version). If you have problems with one version, you can try the other version. Let the Instructor know if you have any problems during the installation.
- Follow the steps to install it, checking the additional shortcuts if you want those features. Create a desktop shortcut does what it says it does. A Quick Launch shortcut is placed in the Task Bar, and the Add 'Edit' with PyScripter' to File Explorer context menu adds the option to use PyScripter in the menu that appears when you right click on a file.
- Click the Install button and follow the prompts. Check the Launch PyScripter box and click Finish.
- Once PyScripter opens, we need to point it to the cloned environment we made. To do this, find the Python 3.x (64-bit) on the bottom bar. Your Python might be different, and that is ok for now. Click on the Python 3.x text and it will open a window listing all of the Python environments it found. If your clone is not listed, you need to click on the Add a new Python version button (gear with a plus) and navigate to where it was created. You can get this path by referring to the Package Manager in Pro. Once you get to the parent folder of the environment, select it and it will add it to the list of Unregistered Versions.
- Double click on the environment to activate it. If you get an Abort Error, click ok to close the prompt.
Verify that your environment is active. It will have a large arrow next to it. Close the Python Versions window. Note that the below image is pointing to the default ArcGIS Pro Python environment. You can use this environment, but adding packages to it has the potential to break Pro. It is recommended that you use a cloned environment.
Figure 1.21 PyScripter setting ArcGIS Pro conda Python environmentIt is a good idea to close the application and restart it to ensure that it saves your activated environment. If the settings revert back to the defaults, repeat step 4 through 7 again, and it should save.
Figure 1.22 PyScripter Installation
If you are familiar with another IDE, you're welcome to use it instead of PyScripter (just verify that it is using Python 3!), but we recommend that you still install PyScripter to be able to work through the following sections and the sections on debugging in Lesson 2.
1.4.4 Exploring PyScripter
1.4.4 Exploring PyScripter jed124Here’s a brief explanation of the main parts of PyScripter. Before you begin reading, be sure to have PyScripter open, so you can follow along.
When PyScripter opens, you’ll see a large text editor in the right side of the window. We'll come back to this part of the PyScripter interface in a moment. For now, focus on the pane in the bottom called the Python Interpreter. If this window is not open, and not listed as a tab along the bottom, you can open it by going to the top menu and selecting View> IDE Windows> then select Python Interpreter. This console is much like the Python interactive window we saw earlier in the lesson. You can type a line of Python at the In >>> prompt, and it will immediately execute and print the result if there is a printable result. This console can be a good place to practice with Python in this course, and whenever you see some Python code next to the In >>> prompt in the lesson materials, this means you can type it in the console to follow along.
We can experiment here by typing "import arcpy" to import arcpy or running a print statement.
>>> import arcpy
>>> print ("Hello World")
Hello world
You might have noticed while typing in that second example a useful function of the Python Interpreter - code completion. This is where PyScripter, like Pro's Python window, is smart enough to recognize that you're entering a function name, and it provides you with the information about the parameters that function takes. If you missed it the first time, enter print(in the IPython window and wait for a second (or less) and the print function's parameters will appear. This also works for arcpy functions (or those from any library that you import). Try it out with arcpy.Buffer_analysis.
Now let's return to the right side of the window, the Editor pane. It will contain a blank script file by default (module1.py). I say it's a blank script file, because while there is text in the file, that text is delimited by special characters that cause it to be ignored when the script is executed We'll discuss these special characters further later, but for now, it's sufficient to note that PyScripter automatically inserts the character encoding of the file, the time it was created, and the login name of the user running PyScripter. You can add the actual Python statements that you'd like to be executed beneath these bits of documentation. (You can also remove the function and if statement along with the documentation, if you like.)
Among the nice features of PyScripter's editor (and other Python IDEs) is its color coding of different Python language constructs. Spacing and indentation, which are important in Python, are also easy to keep track of in this interface. Lastly, note that the Editor pane is a tabbed environment; additional script files can be loaded using File > New or File > Open.
Above the Editor pane, a number of toolbars are visible by default. The File, Run, and Debug toolbars provide access to many commonly used operations through a set of buttons. The File toolbar contains tools for loading, running, and saving scripts. Finally, the Debug toolbar contains tools for carefully reviewing your code line-by-line to help you detect errors. The Debugging toolbar is extremely valuable to you as a programmer, and you’ll learn how to use it later in this course. This toolbar is one of the main reasons to use an Integrated Development Environment (IDE) instead of writing your code in a simple text editor like Notepad.
1.5.1 Working with variables
1.5.1 Working with variables jed124It’s time to get some practice with some beginning programming concepts that will help you write some simple scripts in Python by the end of Lesson 1. We’ll start by looking at variables.
Remember your first introductory algebra class where you learned that a letter could represent any number, like in the statement x + 3? This may have been your first exposure to variables. (Sorry if the memory is traumatic!) In computer science, variables represent values or objects you want the computer to store in its memory for use later in the program.
Variables are frequently used to represent not only numbers, but also text and “Boolean” values (True or False). A variable might be used to store input from the program’s user, to store values returned from another program, to represent constant values, and so on.
Variables make your code readable and flexible. If you hard-code your values, meaning that you always use the literal value, your code is useful only in one particular scenario. You could manually change the values in your code to fit a different scenario, but this is tedious and exposes you to a greater risk of making a mistake (suppose you forget to change a value). Variables, on the other hand, allow your code to be useful in many scenarios and are easy to parameterize, meaning you can let users change the values to whatever they need.
To see some variables in action, open PyScripter and type this in the Python Interpreter:
>>> x = 2
You’ve just created, or declared, a variable, x, and set its value to 2. In some strongly-typed programming languages, such as Java, you would be required to tell the program that you were creating a numerical variable, but Python assumes this when it sees the 2.
When you hit Enter, nothing happens, but the program now has this variable in memory. To prove this, type:
>>> x + 3
You see the answer of this mathematical expression, 5, appear immediately in the console, proving that your variable was remembered and used.
You can also use the print function to write the results of operations. We’ll use this a lot when practicing and testing code.
>>> print (x + 3) 5
Variables can also represent words, or strings, as they are referred to by programmers. Try typing this in the console:
>>> myTeam = "Nittany Lions" >>> print (myTeam) Nittany Lions
In this example, the quotation marks tell Python that you are declaring a string variable. Python is a powerful language for working with strings. A very simple example of string manipulation is to add, or concatenate, two strings, like this:
>>> string1 = "We are " >>> string2 = "Penn State!" >>> print (string1 + string2) We are Penn State!
You can include a number in a string variable by putting it in quotes, but you must thereafter treat it like a string; you cannot treat it like a number. For example, this results in an error:
>>> myValue = "3" >>> print (myValue + 2)
In these examples, you’ve seen the use of the = sign to assign the value of the variable. You can always reassign the variable. For example:
>>> x = 5 >>> x = x - 2 >>> print (x) 3
When naming your variables, the following tips will help you avoid errors.
- Variable names are case-sensitive. myVariable is a different variable than MyVariable.
- Variable names cannot contain spaces.
- Variable names cannot begin with a number.
- A recommended practice for Python variables is to name the variable beginning with a lower-case letter, then begin each subsequent word with a capital letter. This is sometimes known as camel casing. For example: myVariable, mySecondVariable, roadsTable, bufferField1, etc.
- Variables cannot be any of the special Python reserved words such as "import" or "print."
Make variable names meaningful so that others can easily read your code. This will also help you read your code and avoid making mistakes. A good rule of thumb for naming is to use nouns for variables and classes, and verbs for functions. For example, a variable named country_feature indicates the variable contains a country feature whereas a plural country_features may be a list of country features. The function name get_country_features() indicates a process to get country features.
You’ll get plenty of experience working with variables throughout this course and will learn more in future lessons.
Readings
Read Zandbergen section 4.5 on variables and naming.
- Chapter 4 covers the basics of Python syntax, loops, strings and other things which we will look at in more detail in Lesson 2, but feel free to read ahead a little now if you have time or come back to it at the end of Lesson 1 to prepare for Lesson 2.
- You'll note that the section on variable naming refers to a Python style guide, which recommends against the use of camel casing. As the text notes, camel casing is a popular naming scheme for a lot of developers, and you'll see it used throughout our course materials. If you're a novice developer working in an environment with other developers, we recommend that you find out how variable naming is done by your colleagues and get in the habit of following their lead.
1.5.2 Objects and object-oriented programming
1.5.2 Objects and object-oriented programming jed124The number and string variables that we worked with in the last section represent data types that are built into Python. Variables can also represent other things, such as GIS datasets, tables, rows, and the geoprocessor that we saw earlier that can run tools. All of these things are objects that you use when you work with ArcGIS in Python.
In Python, everything is an object. All objects have:
- a unique ID, or location in the computer’s memory;
- a set of properties that describe the object;
- a set of methods, or things, that the object can do.
One way to understand objects is to compare performing an operation in a procedural language (like C or FORTRAN) to performing the same operation in an object-oriented language. We'll pretend that we are writing a program to make a peanut butter and jelly sandwich. If we were to write the program in a procedural language, it would flow something like this:
- Go to the refrigerator and get the jelly and bread.
- Go to the cupboard and get the peanut butter.
- Take out two slices of bread.
- Open the jars.
- Get a knife.
- Put some peanut butter on the knife.
- etc.
If we were to write the program in an object-oriented language, it might look like this:
- mySandwich = Sandwich.Make
- mySandwich.Bread = Wheat
- mySandwich.Add(PeanutButter)
- mySandwich.Add(Jelly)
In the object-oriented example, the bulk of the steps have been eliminated. The sandwich object "knows how" to build itself, given just a few pieces of information. This is an important feature of object-oriented languages known as encapsulation.
Notice that you can define the properties of the sandwich (like the bread type) and perform methods (remember that these are actions) on the sandwich, such as adding the peanut butter and jelly.
1.5.3 Classes
1.5.3 Classes jed124The reason it’s so easy to "make a sandwich" in an object-oriented language is that some programmer, somewhere, already did the work to define what a sandwich is and what you can do with it. He or she did this using a class. A class defines how to create an object, the properties and methods available to that object, how the properties are set and used, and what each method does.
A class may be thought of as a blueprint for creating objects. The blueprint determines what properties and methods an object of that class will have. A common analogy is that of a car factory. A car factory produces thousands of cars of the same model that are all built on the same basic blueprint. In the same way, a class produces objects that have the same predefined properties and methods.
In Python, we can take advantage of pre-written modules which are a grouping of classes to help with your program. You import these modules into your code to tell your program what objects you’ll be working with and then use their pre-written methods to get a result. For example, the first line of most of the scripts you write in this course will be:
import arcpy
Here, you're using the import keyword to tell your script that you’ll be working with the arcpy module, which is provided as part of ArcGIS. After importing this module, you can create objects and processes that leverage ArcGIS functionality in your scripts.
Other modules that you may import in this course are os (allows you to work with the operating system), random (allows for generation of random numbers), csv (allows for reading and writing of spreadsheet files in comma-separated value format), and math (allows you to work with advanced math operations). These modules are included with Python, but they aren't imported by default. A best practice for keeping your scripts fast is to import only the modules that you need for that particular script. For example, although it might not cause any errors in your script, you wouldn't include import arcpy in a script not requiring any ArcGIS functions.
Readings
Read Zandbergen section 5.9 (Classes) for more information about classes.
1.5.4 Inheritance
1.5.4 Inheritance jed124Another important feature of object-oriented languages is inheritance. Classes are arranged in a hierarchical relationship, such that each class inherits its properties and methods from the class above it in the hierarchy (its parent class or superclass). A class also passes along its properties and methods to the class below it (its child class or subclass). A real-world analogy involves the classification of animal species. As a species, we have many characteristics that are unique to humans. However, we also inherit many characteristics from classes higher in the class hierarchy. We have some characteristics as a result of being vertebrates. We have other characteristics as a result of being mammals. To illustrate the point, think of the ability of humans to run. Our bodies respond to our command to run not because we belong to the "human" class, but because we inherit that trait from some class higher in the class hierarchy.
Back in the programming context, the lesson to be learned is that it pays to know where a class fits into the class hierarchy. Without that piece of information, you will be unaware of all of the operations available to you. This information about inheritance can often be found in documentation for the module or the applications API. For example, esri provides the classes for the arcpy Python module here: https://pro.arcgis.com/en/pro-app/latest/arcpy/classes/alphabetical-list-of-arcpy-classes.htm and ArcGIS Pro's SDK for C# here: https://pro.arcgis.com/en/pro-app/latest/sdk/api-reference.
1.5.5 Python syntax
1.5.5 Python syntax jed124Every programming language has rules about capitalization, white space, how to set apart lines of code and procedures, and so on. Here are some basic syntax rules to remember for Python:
- Python is case-sensitive, both in variable names and reserved words. That means it’s important whether you use upper or lower-case. The all lower-case "print" is a reserved word in Python that will print a value, while "Print" is unrecognized by Python and will return an error. Likewise, arcpy is very sensitive about case and will return an error if you try to run a tool without capitalizing the tool name.
- You end a Python statement by pressing Enter and literally beginning a new line. (In some other languages, a special character, such as a semicolon, denotes the end of a statement.) It’s okay to add empty lines to divide your code into logical sections.
- If you have a long statement that you want to display on multiple lines for readability, you need to use a line continuation character, which in Python is a backslash (\).
- Indentation is required in Python to logically group together certain lines, or blocks, of code. You should indent your code four spaces inside loops, if/then statements, and try/except statements. In most programming languages, developers are encouraged to use indentation to logically group together blocks of code; however, in Python, indentation of these language constructs is not only encouraged, but required. Though this requirement may seem burdensome, it does result in greater readability.
- You can add a comment to your code by beginning the line with a # character. Comments are lines that you include to explain what the code is doing. Comments are ignored by Python when it runs the script, so you can add them at any place in your code without worrying about their effect. Comments help others who may have to work with your code in the future, and they may even help you remember what the code does. The intended audience for comments is developers.
- You can add documentation to your code by surrounding a block of text with """ (3 consecutive double quotes) on either side). As with the # character, text surrounded by """ is ignored when Python runs the script. Unlike comments, which are sprinkled throughout scripts, documentation strings are generally found at the beginning of scripts. Their intended audience is end users of the script (non-developers).
1.6.1 Introductory Python examples
1.6.1 Introductory Python examples jed124Let’s look at a few example scripts to see how these rules are applied. The first example script is accompanied with a walkthrough video that explains what happens in each line of the code. You can also review the main points about each script after reading the code.
1.6.2 Example: Printing the spatial reference of a feature class
1.6.2 Example: Printing the spatial reference of a feature class jed124This first example script reports the spatial reference (coordinate system) of a feature class stored in a geodatabase. If you want to use the USA.gdb referenced in this example, you can run the code yourself.
# Opens a feature class from a geodatabase and prints the spatial reference
import arcpy
featureClass = "C:/Data/USA/USA.gdb/Boundaries"
# Describe the feature class and get its spatial reference
desc = arcpy.Describe(featureClass)
spatialRef = desc.spatialReference
# Print the spatial reference name
print (spatialRef.Name)This may look intimidating at first, so let’s go through what’s happening in this script, line by line. Watch this video (5:54) to get a visual walkthrough of the code.
Video: Walk-Through of First Python Script (5:33)
The purpose of this video is to walk through the first Python script in this course, which will be especially helpful if you're a beginner with Python. I have PyScripter open here, which is the development environment, or code editor, that we recommend you use throughout the class. The first couple lines of this script are a comment. And it says what we're going to do with this script, which is to look at a feature class inside of a geodatabase and then print the spatial reference of that feature class. In line four, we import the arcpy site package. Now if your script has anything at all to do with geoprocessing or looking at Esri datasets, this is what you're going to put at the top of your script. So you can get in the habit of using import arcpy at the top of your scripts in this class. In line six, we create a string variable. We call this variable featureClass, but we could really call it anything.
What to call the variables is up to us; what to assign the variable? Here is the path of the feature class that we want to look at, and we put that in quotes, which makes it a string in the eyes of Python. When you make a string, pyScripter is going to help you out by putting that text in orange so that it's easy to spot in your code. Notice in the path we use forward slashes. This is different from the common backslash that you would see typically when you define a path. The backslash in Python is a reserved character, so you have to use either two backslashes or a forward slash. You're going to see us use both in this course. In this case, the path is pointing at a file geodatabase. That's why you see the USA.gdb. Inside that file geodatabase is a feature class called boundaries. This is probably a polygon or a line feature class, who knows?
If you didn't know the exact path to get, you could open the Catalog view in ArcGIS Pro, navigate to that feature class and then you would see in the upper location bar the exact path to use. You could also navigate to it in your File Explorer in Windows. In line 9, we call a method in arcpy, the arcpy Describe method, which returns to us what's called a Describe object. We've mentioned in the course material that everything in Python is an object. Objects have properties that describe them, and methods that are things that they can do. If you want to learn about this particular object, the Describe object, you could look it up and you should look it up in the ArcGIS Pro Help. In this case, since we're describing a feature class, the Describe object that we're working with is going to have a spatialReference property.
Now, before we move off of line 9, I want to point out the parentheses there. When you call the method Describe, you need to put something inside that parentheses, which is the path of the feature class that you would like to Describe. Now, we don't have to type in that full path here because we made a variable earlier to store the path. This is where you would plug in the featureClass variable. Python is going to see that as the path stored in that variable.
After line 9, we've got a Describe object stored in our desc variable. That Describe object has a property which is the spatialReference, so in line 10, that's what we're getting. desc is the name of the Describe object, and spatialReference gets us, returns to us, a SpatialReference object. Now we can't print out an object, we can't do print(spatialRef). If you tried to do that. pyScripter is going to tell you that the thing you're trying to print is an object, which isn't going to be very helpful. We actually need to get a property from the SpatialReference object. The property we're going to get is the name. That's why in line 13 we do spatialRef.name. By this time we've drilled down far enough that we've made our way to a string. spatialRef.name gives you back a string, the name of the spatial reference. That's something that we are able to print to the Python Interpreter window.
Now I'm going to run this script. To run a script I just click on the Run button, which looks like a Play icon on the top. We'll see later in the course that scripts sometimes require arguments or pieces of information, like a path name or a number, in order to do their job. It's possible to supply such arguments, but this is a pretty simple script that doesn't require any kind of input, so we can just go ahead and click on the Run button. Now, anytime you run a script that includes a print statement, you should look to the bottom of the pyScripter application window to the area labeled as the Python Interpreter.
Before your print statement output, you'll see a message that indicates that you've re-initialized the Python interpreter. Then comes your output, which in this case is the word geographic, meaning that this particular feature class has a geographic coordinate system. The first time you run a script in any particular pyScripter session, it's going to take several seconds for the interpreter to load into memory. However, if you go on to run any other arcpy scripts, to re-run this script or another script, they should all run more quickly.
Again, notice that:
- a comment begins the script to explain what’s going to happen.
- case sensitivity is applied in the code. "import" is all lower-case. So is "print". The module name "arcpy" is always referred to as "arcpy," not "ARCPY" or "Arcpy". Similarly, "Describe" is capitalized in arcpy.Describe.
- The variable names featureClass, desc, and spatialRef that the programmer assigned are short, but intuitive. By looking at the variable name, you can quickly guess what it represents.
- The script creates objects and uses a combination of properties and methods on those objects to get the job done. That’s how object-oriented programming works.
Trying the example for yourself
The best way to get familiar with a new programming language is to look at example code and practice with it yourself. See if you can modify the script above to report the spatial reference of a feature class on your computer. In my example, the feature class is in a file geodatabase; you’ll need to modify the structure of the featureClass path if you are using a shapefile (for example, you'll put .shp at the end of the file name, and you won't have .gdb in your path).
Follow this pattern to try the example:
- Open PyScripter and click File > New.
- Paste the code above into the new script file and modify it to fit your data (change the path).
- Save your script as a .py file.
- Click the Run button to run the script. Look at the IPython console to see the output from the print keyword. The print keyword does not actually cause a hard copy to be printed! Note that the script will take several seconds to run the first time because of the need to import the arcpy module and its dependencies. Subsequent runs of arcpy-dependent scripts during your current PyScripter session will not suffer from this lag.
Readings
We'll take a short break and do some reading from another source. If you are new to Python scripting, it can be helpful to see the concepts from another point of view.
Read parts of Zandbergen chapters 4 & 5. This will be a valuable introduction to Python in ArcGIS, on how to work with tools and toolboxes (very useful for Project 1), and also on some concepts which we'll revisit later in Lesson 2 (don't worry if the bits we skip over seem daunting - we'll explain those in Lesson 2).
- Chapter 4 deals with the fundamentals of Python. We will need a few of these to get started, and we'll revisit this chapter in Lesson 2. For now, read sections 4.1-4.7, reviewing 4.5, which you read earlier.
- Chapter 5 talks about working with arcpy and functions - read sections 5.1-5.2 and 5.4-5.7.
1.6.3 Example: Performing map algebra on a raster
1.6.3 Example: Performing map algebra on a raster jed124Here’s another simple script that finds all cells over 3500 meters in an elevation raster and makes a new raster that codes all those cells as 1. Remaining values in the new raster are coded as 0. This type of “map algebra” operation is common in site selection and other GIS scenarios.
Something you may not recognize below is the expression Raster(inRaster). This function just tells ArcGIS that it needs to treat your inRaster variable as a raster dataset so that you can perform map algebra on it. If you didn't do this, the script would treat inRaster as just a literal string of characters (the path) instead of a raster dataset.
# This script uses map algebra to find values in an
# elevation raster greater than 3500 (meters).
import arcpy
from arcpy.sa import *
# Specify the input raster
inRaster = "C:/PSU/Geog485/Lesson1.gdb/foxlake"
cutoffElevation = 3500
# Check out the Spatial Analyst extension
arcpy.CheckOutExtension("Spatial")
# Make a map algebra expression and save the resulting raster
outRaster = Raster(inRaster) > cutoffElevation
outRaster.save("C:/PSU/Geog485/Lesson1.gdb/foxlake_hi_10")
# Check in the Spatial Analyst extension now that you're done
arcpy.CheckInExtension("Spatial")Begin by examining this script and trying to figure out as much as you can based on what you remember from the previous scripts you’ve seen.
The main points to remember on this script are:
- Unlike other tools/functions we've seen so far, map algebra functions and the Raster class aren't in the arcpy module, but in the arcpy.sa sub-module. Thus, this script imports both arcpy and arcpy.sa.
- The script uses an alternative syntax for importing modules: from <module> import <class> or *. When a module is imported with this syntax, any classes imported can be invoked without prefixing the parent module. In other words, using from arcpy.sa import * allows us to refer to the Raster class like this:
outRaster = Raster(inRaster) > cutoffElevation
whereas using import arcpy.sa would require this syntax when referring to the Raster class:
outRaster = arcpy.sa.Raster(inRaster) > cutoffElevation - You may notice that PyScripter may flag the from statement and later the statement that invokes the Raster class with a warning icon. Your script will still run, but the warning appears because it can be poor practice to import all classes/functions from a module. It is often better to import only those items that are needed for the task at hand. In this case, changing * to Raster would be the better practice and would eliminate the warning. We've used the * notation to be consistent with Esri's code samples, but also encourage you to limit what you import from modules when possible.
- Notice the lines of code that check out the Spatial Analyst extension before doing any map algebra and check it back in after finishing. In ArcGIS Pro, the checkout step is required only if you're operating in an environment where extension licenses are shared (i.e., using a Concurrent Use license). If you were to run this script on an educational copy of the software, the checkout and checkin statements could be omitted. We've left them in because they don't hurt, and they may be needed in your work environment.
- Because each line of code takes some time to run, avoid putting unnecessary code between checkout and checkin. This allows others in your organization to use the extension if licenses are limited. The extension automatically gets checked back in when your script ends, thus some of the Esri code examples you will see do not check it in. However, it is a good practice to explicitly check it in, just in case you have some long code that needs to execute afterward, or in case your script crashes and against your intentions "hangs onto" the license.
- inRaster begins as a string, but is then used to create a Raster object once you run Raster(inRaster). A Raster object is a special object used for working with raster datasets in ArcGIS. It's not available in just any Python script: you can use it only if you import the arcpy module at the top of your script.
- cutoffElevation is a number variable that you declare early in your script and then use later on when you build the map algebra expression for your outRaster.
- Your expression outRaster = Raster(inRaster) > cutoffElevation is saying, in plain terms, "Make a new raster and call it outRaster. Do this by taking all the cells of the raster dataset at the path of inRaster that are greater than the number I assigned to the variable cutoffElevation."
- outRaster is also a Raster object, but you have to call the method outRaster.save() in order to make it permanent on disk. The save() method takes one argument, which is the path to which you want to save.
Now try to run the script yourself using the FoxLake digital elevation model (DEM) in your Lesson 1 geodatabase. If it doesn’t work the first time, verify that:
- you have supplied the correct input and output paths;
- your path name contains forward slashes (/) or double backslashes (\\), not single backslashes (\);
- you have the Spatial Analyst extension installed and enabled. To check this, from ArcPro, click on the Project tab > Licensing and ensure Spatial Analyst is enabled;
- you do not have any of the datasets open in ArcGIS products;
- the output data does not exist yet. If you want to be able to overwrite the output, you need to add the line arcpy.env.overwriteOutput = True. This line can be placed immediately after import arcpy.
You can experiment with this script using different values in the map algebra expression (try 3000 for example).
Readings
ArcGIS Pro edition:
Read the sections of Chapter 5 that talk about environment variables and licenses (5.11 & 5.13) which we covered in this part of the lesson.
1.6.4 Example: Creating buffers
1.6.4 Example: Creating buffers jed124Think about the previous example where you ran some map algebra on an elevation raster. If you wanted to change the value of your cutoff elevation to 2500 instead of 3500, you had to open the script itself and change the value of the cutoffElevation variable in the code.
This third example is a little different. Instead of hard-coding the values needed for the tool (in other words, literally including the values in the script) we’ll use some user input variables, or parameters. This allows people to try different values in the script without altering the code itself. Just like in ModelBuilder, parameters make your script available to a wider audience.
The simple example below just runs the Buffer tool, but it allows the user to enter the path of the input and output datasets as well as the distance of the buffer. The user-supplied parameters make their way into the script with the arcpy.GetParameterAsText() function.
Examine the script below carefully, but don't try to run it yet. You'll do that in the next part of the lesson.
# This script runs the Buffer tool. The user supplies the input
# and output paths, and the buffer distance.
import arcpy
arcpy.env.overwriteOutput = True
try:
# Get the input parameters for the Buffer tool
inPath = arcpy.GetParameterAsText(0)
outPath = arcpy.GetParameterAsText(1)
bufferDistance = arcpy.GetParameterAsText(2)
# Run the Buffer tool
arcpy.Buffer_analysis(inPath, outPath, bufferDistance)
# Report a success message
arcpy.AddMessage("All done!")
except:
# Report an error messages
arcpy.AddError("Could not complete the buffer")
# Report any error messages that the Buffer tool might have generated
arcpy.AddMessage(arcpy.GetMessages())Again, examine the above code line by line and figure out as much as you can about what the code does. If necessary, print the code and write notes next to each line. Here are some of the main points to understand:
- GetParameterAsText() is a function in the arcpy module. Notice that it takes a zero-based integer (0, 1, 2, 3, etc.) as an argument. If you’re going to go ahead and make a tool out of this script, as we are going to do in the next page of this lesson, then it’s important you define the parameters in the same order you want them to appear in the tool’s dialog.
- When we called the Buffer tool in this script, we supplied only three parameters. By not supplying any more, we accepted the default values for the rest of the tool’s parameter (Side Type, End Type, etc.).
The try and except blocks of code are a way that you can prevent your script from crashing if there is an error. Your script attempts to run all of the code in the try block. If the script cannot continue for some reason, it jumps down and runs the code in the except block. Inserting try/except blocks like this is a good practice to follow once you think you've gotten all the errors out of your script, or when you want to make sure your code will run a certain line at the end, no matter what happens.
When you are first writing and debugging your script, sometimes it's more useful to leave out try/except and let the code crash because the error messages reported in the console sometimes give you better clues on how to diagnose the problem in your code. Suppose you put a print statement in your except block saying "There was an error. Please try again." For the end user of your script, this is nicer than seeing a nasty error message; however, as a programmer debugging the script, you want to see the (red) error message to get any insight you can about what went wrong.
Projects that you submit in this course require error handling using try/except in order to receive full credit.
The arcpy.AddMessage() and arcpy.AddError() functions are ways of adding additional messages to the user of the tool. Whenever you run a tool, the geoprocessor prints messages, which you have probably seen before (for example, “Executed (Buffer) successfully. End time: Sat Oct 05 07:37:31 2019”). You have the power to add more messages through these functions. The messages have differing levels of severity, hence different functions for AddMessage and AddError. Sometimes people choose to view only the errors instead of all the messages, or they do a quick visual scan of the messages for errors.
When you use arcpy.GetMessages(), you get all the messages generated by the tool itself. These will tell you things such as whether the user entered invalid parameters. Notice in this script the somewhat complex syntax you have to use to first get the messages, then add them: arcpy.AddMessage(arcpy.GetMessages()). If this line of code is confusing to understand, remember that the order of functions works like math operations: you start by working inside the parentheses first to get the messages, then you add them.
Readings
ArcGIS Pro edition:
Read the section of Chapter 5 that talks about working with tool messages (5.12) for another perspective on handling tool output.
1.7.1 Making a script tool
1.7.1 Making a script tool jed124User input variables that you retrieve through GetParameterAsText() make your script very easy to convert into a tool in ArcGIS. A few people know how to alter Python code, a few more can run a Python script and supply user input variables, but almost all ArcGIS users know how to run a tool. To finish off this lesson, we’ll take the previous script and make it into a tool that can easily be run in ArcGIS.
Before you begin this exercise, I strongly recommend that you scan the ArcGIS help topic Adding a script tool. You likely will not understand all the parts of this topic yet, but it will give you some familiarity with script tools that will be helpful during the exercise.
Follow these steps to make a script tool:
- Copy the code from Lesson 1.6.4 "Example: Creating Buffers" into a new PyScripter script and save it as buffer_user_input.py.
- Open your Lesson1Practice.aprx Pro project and display the Catalog window.
- Expand the nodes Toolboxes > Lesson1Practice.atbx.
- Right-click Lesson1Practice.atbx and click New > Script.
Fill in the Name and Label properties for your Script tool as shown below, depending on your version of ArcGIS Pro: The first image below shows the interface and the information you have to enter under General in this and the next step if you have a version earlier than version 2.9. The other two images show the interface and information that needs to be entered in this and the next step under General and under Execution if you have version 2.9 or higher.
Figure 1.10a Entering information for your script tool if you have a version < 2.9.Figure 1.10b Entering information for your script tool if you have a version ≥ 2.9: General tab.Figure 1.10c Entering information for your script tool if you have a version ≥ 2.9: Execution tab.- For version <2.9, click the folder icon for Script File on the General tab and browse to your buffer_user_input.py file. For version 2.9 or higher, do the same on the Execution tab; the code from the script file will be shown in the window below.
- All versions: On the left side of the dialog, click on Parameters. You will now enter the parameters needed for this tool to do its job, namely inPath, outPath, and bufferMiles. You will specify the parameters in the same order, except you can give the parameters names that are easier to understand.
- Working in the first row of the table, enter Input Feature Class in the Label column, then Tab out of that cell. Note that the Name column is automatically filled in with the same text, except that the spaces are replaced by spaces.
Next, click in the Data Type column and choose Feature Class from the subsequent dialog. Here is one of the huge advantages of making a script tool. Instead of accepting any string as input (which could contain an error), your tool will now enforce the requirement that a feature class be used as input. ArcGIS will help you by confirming that the value entered is a path to a valid feature class. It will even supply the users of your tool with a browse button, so they can browse to the feature class.
Figure 1.11 Choosing "Feature Class."- Just as you did in the previous steps, add a second parameter labeled Output Feature Class. The data type should again be Feature Class.
- For the Output Feature Class parameter, change the Direction property to Output.
- Add a third parameter named Buffer Distance. Choose Linear Unit as the data type. This data type will allow the user of the tool to select both the distance value and the units (for example, miles, kilometers, etc.).
Set the Buffer Distance parameter's Default property to 5 Miles. (You may need to scroll to the right to see the Default column.) Your dialog should look like what you see below:
Figure 1.12 Setting the Default property to "5 Miles."- Click OK to finish creation of the new script tool and open it by double-clicking it.
Try out your tool by buffering any feature class on your computer. Notice that once you supply the input feature class, an output feature class path is suggested for you. This is because you specifically set Output Feature Class as an output parameter. Also, when the tool is complete, examine the Details box (visible if you hover your mouse over the tool's "Completed successfully" message) for the custom message "All done!" that you added in your code.
Figure 1.13 The tool is complete.
This is a very simple example, and obviously, you could just run the out-of-the-box Buffer tool with similar results. Normally, when you create a script tool, it will be backed with a script that runs a combination of tools and applies some logic that makes those tools uniquely useful.
There’s another benefit to this example, though. Notice the simplicity of our script tool dialog compared to the main Buffer tool:

At some point, you may need to design a set of tools for beginning GIS users where only the most necessary parameters are exposed. You may also do this to enforce quality control if you know that some of the parameters must always be set to certain defaults, and you want to avoid the scenario where a beginning user (or a rogue user) might change the required values. A simple script tool is effective for simplifying the tool dialog in this way.
Readings: ArcGIS Pro edition
Read Zandbergen 3.9- 3.10 to reinforce what you learned during this lesson about scripts and script tools.
Lesson 1 Practice Exercises
Lesson 1 Practice Exercises jed124Each lesson in this course includes some simple practice exercises with Python. These are not submitted or graded, but they are highly recommended if you are new to programming or if the project initially looks challenging. Lessons 1 and 2 contain shorter exercises, while Lessons 3 and 4 contain longer, more holistic exercises. Each practice exercise has an accompanying solution that you should carefully study. If you want to use the USA.gdb referenced in some of the solutions you can find it here.
Remember to choose File > New in PyScripter to create a new script (or click the empty page icon). You can name the scripts something like Practice1, Practice2, etc. To execute a script in PyScripter, click the "play" icon.
Practice 1: Say hello
Create a string variable called x and assign it the value "Hello". Display the contents of the x variable in the Console.
- Practice 1 Solution
Solution:
x = "Hello" print (x)
Explanation:
The first line of the script tells the Python code interpreter to set aside space in the computer's memory to hold a value, to refer to that space in memory by the name x and to store the text string "Hello" in that space. The memory space is typically referred to as a variable. Unlike some languages that require declaring a variable and defining the type of data it will hold before assigning a value to it, Python allows variables to be defined without explicitly specifying their type. The data type (string, number, etc.) is inferred by the Python code interpreter based on the kind of value you're assigning to the variable.
The second line of the script tells the interpreter to print the contents of the x variable to the Python Interpreter.
Note that this already simple script could be simplified even further by doing away with the x variable and plugging "Hello" into the print statement directly as literal text:
print ("Hello")Note also that this script does not include lines that import arcpy since the script does not require any ArcGIS functionality.
Practice 2: Concatenate two strings
Create a string variable called first and assign to it your first name. Likewise, create a string variable called last and assign to it your last name. Concatenate (merge) the two strings together, making sure to also include a space between them.
- Practice 2 Solution
Solution:
first = "James" last = "Franklin" full = first + " " + last print (full)
Explanation:
In this script, two string variables are used to hold the two name pieces. The two pieces are merged together using the concatenation character (+ in Python). A space is inserted between them using a space within quotes.
If you wanted to display the name pieces last name first and with a comma in between, you would change the 'full' statement as follows:
full = last + ", " + first
Practice 3: Pass a value to a script as a parameter
Example 1.6.4 shows the use of the arcpy.GetParameterAsText() function. This function is typically used in conjunction with an ArcGIS script tool that has been designed to prompt the user to enter the required parameters. However, it is possible to supply arguments to the script from within PyScripter. To do so, you would select Run > Configuration per file, check the Command line options checkbox, then enter the arguments something like this:
C:\PSU\geog485\Lesson1.gdb\USA\us_cities C:\PSU\geog485\Lesson1.gdb\USA\us_cities_buffered "10 miles"
Note that the arguments to the two feature class parameters are unquoted, while the argument to the buffer distance string parameter is provided in double quotes.
For this exercise, write a script that accepts a single string value using the GetParameterAsText method. The value entered should be a name, and that name should be concatenated with the literal string "Hi, " and displayed in the console. Test the script from within PyScripter, entering a name (in double quotes) in the Command line options text box as outlined above prior to clicking the Run button.
- Practice 3 Solution
Solution:
import arcpy name = arcpy.GetParameterAsText(0) print ("Hi, " + name)Explanation:
There are other means of obtaining user input from the PyScripter IDE, but this course is focused on executing scripts from within ArcGIS, so the exercise asked you to use arcpy's GetParameterAsText function. That means that, unlike the first two exercises, this script requires importing arcpy. As mentioned, the script can be executed from within PyScripter by entering the name parameter as a command line option. GetParameterAsText is zero-based, so the script is set up to get parameter 0 (the first and only parameter).
Note that it would also be acceptable to do away with the name variable as follows:
print ("Hi, " + arcpy.GetParameterAsText(0))
Practice 4: Report the geometry type of a feature class
Example 1.6.2 demonstrates the use of the Describe function to report the spatial reference of a feature class. The Describe function returns an object that has a number of properties that can vary depending on what type of object you described. A feature class has a spatialReference property by virtue of the fact that it is a type of Dataset. (Rasters are another type of Dataset and also have a spatialReference property.)
The Describe function's page in the Help lists the types of objects that the function can be used on. Clicking the Dataset properties link pulls up a list of the properties available when you describe a dataset, spatialReference being just one of several.
For this exercise, use the Describe function again; this time, to determine the type of geometry (point, polyline or polygon) stored in a feature class. I won't tell you the name of the property that returns this information. But I will give you the hint that feature classes have this mystery property not because they're a type of Dataset as with the spatialReference property, but because they're objects of the type FeatureClass.
- Practice 4 Solution
Solution:
import arcpy fc = "C:/PSU/Geog485/Lesson1.gdb/USA/us-boundaries" desc = arcpy.Describe(fc) shapeType = desc.shapeType print ("The geometry type of " + fc + " is " + shapeType)Explanation:
This script begins like the one in example 1.6.2 since both require using the Describe function. As mentioned in the exercise instructions, you can see that the Describe function provides access to a spatialReference property by looking at the Describe function's help page for Datasets. In other words, if the object you're describing is a type of Dataset, it has a spatialReference property.
In this case, you were asked to determine the geometry type of a feature class and given the hint that the property you need belongs to objects of the type FeatureClass. This hint should have led you to click the FeatureClass properties link on the Describe function's Help page and scan through the list of properties until you found shapeType. The solution above reads the shapeType property, stores the return value in a variable, and then inserts that value into a user-friendly print message that also includes the name of the feature class.
Practice 5: Do some simple math and report your results
Create a variable score1 and assign it a value of 90. Then create a variable score2 and assign it a value of 80. Compute the sum and average of these values and output to the Console the following messages, by replacing the x with your scores:
The sum of these scores is x.
Their average is x.
In constructing your messages, you're probably going to want to concatenate text strings with numbers. We'll cover this more in Lesson 2, but this requires converting the numeric values into strings to avoid a syntax error. The str() function can be used to do this conversion. For example, I might output the score1 value like this:
print('Score 1 is ' + str(score1))
- Practice 5 Solution
Solution:
score1 = 90 score2 = 80 sum = score1 + score2 avg = sum / 2 print("The sum of these scores is " + str(sum) + ".") print("The average is " + str(avg) + ".") #Alternatively... #score1 = 90 #score2 = 80 # #print("The sum of these scores is " + str(score1 + score2) + ".") #print("The average is " + str((score1 + score2) / 2) + ".")Explanation:
This script begins with a couple of statements in which the score1 and score2 variables are created and assigned values. There are two approaches shown to displaying the required messages. The first shows creating variables called sum and avg to store the two simple statistics requested. The values held in those variables are then inserted into a pair of print() statements using string concatenation. Note the use of the str() function to convert the numbers into strings for the purpose of concatenating with the message text.
The second approach ("commented out" using the # character) shows how the script could be written without bothering to create the sum and avg variables. The required math can be embedded right within the print() statements. Note that in the statement that outputs the average the addition of the values held in score1 and score2 is enclosed in parentheses. If these parentheses were omitted, the script would still run, but would produce an incorrect average because algebra's order of operations would cause score2 to be divided by 2 first, then score1 would be added to that product.
A couple of other points can be made about this script:
You might be thinking that the second approach is pretty slick and that you should be striving to write your scripts in the shortest number of lines possible. However, in terms of performance, shorter scripts don't necessarily translate to higher performing ones. In this instance, there is practically no difference. And the second approach is a bit harder for another coder (or yourself at some later date) to interpret what's going on. That's a very important point to take into consideration when writing code.
You'll often see, including in the ArcGIS documentation, Python's format() function used to insert values into larger strings, eliminating the need for the concatenation operator. For example, we could display our sum message like so:
print('The sum is {}.'.format(sum))
With the format() function, curly braces are used within a string as placeholders for values to be inserted. The string (enclosed in single or double quotes) is then followed by .format(), with the desired value plugged into the parentheses. Note that the values supplied are automatically converted into a string format and any number of placeholders can be employed. For example, we could combine the two messages like so:
print('The sum is {}. The average is {}.'.format(sum, (score1 + score2) / 2))
Project 1, Part 1: Modeling precipitation zones in Nebraska
Project 1, Part 1: Modeling precipitation zones in Nebraska jed124Suppose you're working on a project for the Nebraska Department of Agriculture and you are tasked with making some maps of precipitation in the state. Members of the department want to see which parts of the state were relatively dry and wet in the past year, classified in zones. All you have is a series of weather station readings of cumulative rainfall for 2018 that you've obtained from within Nebraska and surrounding areas. This is a feature class of points called Precip2018Readings. It is in the Nebraska feature dataset in your Lesson1 geodatabase.
Precip2018Readings is a fictional dataset created for this project. The locations do not correspond to actual weather stations. However, the measurements are derived from real 2018 precipitation data created by the PRISM Climate Group at Oregon State University.
You need to do several tasks in order to get this data ready for mapping:
- Interpolate a precipitation surface from your points. This creates a raster dataset with estimated precipitation values for your entire area of interest. You've already planned for this, knowing that you are going to use inverse distance weighted (IDW) interpolation. Click the following link to learn how the IDW technique works. You've also selected your points to include some areas around Nebraska to avoid edge effects in the interpolation.
- Reclassify the interpolated surface into an ordinal classification of precipitation "zones" that delineate relatively dry, medium, and wet regions.
- Create vector polygons from the zones.
- Clip the zone polygons to the boundary of Nebraska.
Figure 1.15 Mapping the data.
It's very possible that you'll want to repeat the above process in order to test different IDW interpolation parameters or make similar maps with other datasets (such as next year's precipitation data). Therefore, the above series of tasks is well-suited to ModelBuilder. Your job is to create a model that can complete the above series of steps without you having to manually open four different tools.
Model parameters
Your model should have these (and only these) parameters:
- Input precipitation readings- This is the location of your precipitation readings point data. This is a model parameter so that the model can be easily re-run with other datasets.
- Power- An IDW setting specifying how quickly influence of surrounding points decreases as you move away from the point to be interpolated.
- Search radius- An IDW setting determining how many surrounding points are included in the interpolation of a point. The search radius can be fixed at a certain distance, including whatever number of points happen to fall within, or its distance can vary in order for it to always include a minimum number of points. When you use ModelBuilder, you don't have to set up any of these choices; ModelBuilder does it for you when you set the Search Radius as a model parameter.
- Zone boundaries- This is a table allowing the user of the model to specify the zone boundaries. By default, the table should be configured as shown in Figure 1.16 below: Precipitation values of 0 - 30000 will result in a reclassification of 1 (to correspond with Zone 1), 30000 - 60000 will result in a classification of 2 (to correspond with Zone 2), and so on. The way to get this table is to make a variable from the Reclassification parameter of the Reclassify tool and set it as a model parameter.
- Output precipitation zones- This is the location where you want the output dataset of clipped vector zones to be placed on disk.
As you build your model, you will need to configure some settings that will not be exposed as parameters. These include the clip feature, which is the state of Nebraska StateBoundary feature class. There are many other settings such as "Z Value field" and "Input barrier polyline features" (for IDW) or "Reclass field" (for Reclassify) that should not be exposed as parameters. You should just set these values once when you build your model. If you ever ask someone else to run this model, you don't want them to be overwhelmed with choices stemming from every tool in the model; you should just expose the essential things they might want to change.
For this particular model, you should assume that any input dataset will conform to the same schema as your Precip2018Readings feature class. For example, an analyst should be able to submit similar datasets Precip2019Readings, Precip2020Readings, etc. for more recent years with the same fields, field names, and data types. However, he or she should not expect to provide any feature class with a different set of fields and field names, etc. As you might discover, handling all types of feature class schemas would make your model more complex than we want for this assignment.
Important: Given the scenario of wishing to re-run the model for other years of data, it would be a good idea to set default values for the exposed model parameters. Therefore, we are asking you to set default values for all parameters that are exposed as model parameters including the Power value, Search radius value, and Zone boundaries classification table. When you double-click the model to run it, the interface should look like the following:
Running the model with the exact parameters listed above should result in the following (I have symbolized the zones in Pro with different colors to help distinguish them). This is one way you can check your work:
Once you are done, take a screenshot of the layout of your final model in ModelBuilder (similar to Figure 1.5 in Section 1.3.2) to include in your homework submission.
Tips
The following tips may help you as you build your model:
- Your model needs to include the following tools in this order: IDW (from the Spatial Analyst toolbox), Reclassify, Raster to Polygon, Clip (from the Analysis toolbox).
- An easy way to find the tools you need in Pro is to go to Analysis > Tools, then type the name of the tool you want in the search box. Be careful when multiple tools have the same name. You'll typically be using tools from the Spatial Analyst toolbox in this assignment.
- Once you drag and drop a tool onto the ModelBuilder canvas, double-click it and set all the parameters the way you want. These will be the default settings for your model.
- If there is a certain parameter for a tool that you want to expose as a model parameter, right-click the tool in the ModelBuilder canvas, then click Create Variable > From Parameter and choose the parameter. Once the oval appears for the variable, right-click it and click Parameter.
Project 1, Part II: Creating contours for the Fox Lake DEM
Project 1, Part II: Creating contours for the Fox Lake DEM jed124The second part of Project 1 will help you get some practice with Python. At the end of Lesson 1, you saw three simple scripting examples; now your task is to write your own script. This script will create vector contour lines from a raster elevation dataset. Don't forget that the ArcGIS Pro Help can indeed be helpful if you need to figure out the syntax for a particular command.
Earlier in the lesson, you were introduced to the Fox Lake DEM in your Lesson 1 geodatabase. It represents elevation in the Fox Lake Quadrangle, Utah. Write a script that uses the Contour tool in the Spatial Analyst toolbox to create contour lines for the quadrangle. The contour interval should be 25 meters, and the base contour should be 0. Remember that the native units of the DEM are meters, so no unit conversions are required.
Running the script should immediately create a feature class of contour lines on disk.
Follow these guidelines when writing the script:
- The purpose of this exercise is just to get you some practice writing Python code. Therefore, you are not required to use arcpy.GetParameterAsText() to get the input parameters. Go ahead and hard-code the values (such as the path name to the dataset).
- Consequently, you are not required to create a script tool for this exercise. This will be required in Project 2.
- Your code should run correctly from PyScripter. For full credit, it should also contain comments, attempt to handle errors, and use legal and intuitive variable names.
- You should assume that you're writing your script for an organization that has a limited number of concurrent Spatial Analyst licenses available.
Deliverables
The deliverables for Project 1 are:
- The .atbx file of the toolbox containing your Part I model. The easiest way to find it is to hover your mouse over the toolbox in the Catalog window and note its location in the box that appears.
- A screenshot showing the layout of your final model for Part 1 (the model in the ModelBuilder editor, not the tool interface and not the produced output of your model).
- The .py file containing your Part II script.
- A detailed written narrative that explains in plain English what the various lines of your script do. In this document, we will be looking for you to demonstrate that you understand how the code you're submitting works. You may find it useful to emulate the practice exercise solution explanations. Alternatively, you may record a video in which you walk the grader through your solution beginning to end.
- A short writeup (about 400 words) describing how you approached the two problems, any setbacks you faced, and how you dealt with those. Both writeups described here will be required on all projects.
Important: Successful delivery of the above requirements is sufficient to earn 95% on the project. The remaining 5% is reserved for efforts that go "over and above" the minimum requirements. For Part I, this could include (but is not limited to) analysis of how different input values affect the output, substitution of some other interpolation method instead of IDW (for example Kriging), documentation for your model parameters that guides the end user in what to input, or demonstration of how your model was successfully run on a different input dataset. For Part II, in addition to the Contour tool, you could run some other tool that also takes a DEM as an input.
As a general rule throughout the course, full credit in the "over and above" category requires the implementation of 2-4 different ideas, with more complex ideas earning more credit. Note that for future projects, we won't be listing off ideas as we've done here. Otherwise, it wouldn't really be an over and above requirement.
Finishing Lesson 1
To complete Lesson 1, please zip all your Project 1 deliverables (for parts I and II) into one file and submit them to the Project 1 Drop Box in Canvas. Then take the Lesson 1 Quiz if you haven't taken it already.