Lesson 7 Overview

Effective quality management has the potential to make a project shine. Ask an end user if the product was delivered on time, on budget, or included all of the products within its scope, and the best answer you can probably hope for is "yes." Ask the same end user about the quality of the product, and a happy customer is given the opportunity to extol the merits of your exemplary work. Quality Management is a PMI Project Knowledge Area. Quality cannot be ignored and should be embraced in project management. We will look at methods and metrics to make quality part of the planning and implementation processes of a project.

Learning Objectives

By the end of this lesson, you should be able to:

• understand the meaning of quality in the context of GIS projects
• understand the differences between quality planning, quality assurance, and quality control, and describe where each fits into the project life cycle
• appreciate the specific quality parameters associated with typical GIS projects and deliverables
• describe the history, techniques, and tools of quality control and management and cite organizations that development and promote quality standards
• understand the key processes and steps involved in quality management in GIS projects
• describe aspects of GIS projects that require special attention to quality

See the checklist page for readings, quiz, and assignment work in this Lesson.

Questions?

If you have any questions or would like to brainstorm ideas, please contact the instructor by phone or email. Also, feel free to communicate with your fellow students via the Discussion Forum or email.

Lesson 7 is one week in length. (See the Calendar for specific due dates.) To finish this lesson, you must complete the activities listed below. You may find it useful to print this page out first so that you can follow along with the directions.

Introduction

The Project Management Institute (PMI) defines quality, in a project context as, "the degree to which project deliverables meet requirements". This general definition places the focus on a need for well-defined requirements for project execution and deliverables. Quality planning should occur as part of the development of project deliverable specifications and work planning. Identification and characterization of quality as a basis for quality management procedures is helped by reference to accepted standards.

Standards Organizations

There are a number of government and independent organizations that develop and promote standards relating to different aspects of quality (of GIS products and project deliverables. Consider taking a look at resources of the following organizations to get a better understanding of accepted GIS-related standards:

• Open GIS Consortium (OGC): An independent standards organization with participation of industry and government organizations and coordination with professional associations and other standards organizations. The OGC has established standards for spatial data format, data classification, geospatial services and applications, and GIS operational practices. The OGC standards have been adopted by many GIS software and database companies and by user organizations.
• Federal Geographic Data Committee (FGDC): a U.S. Federal government organization which develops and approves standards for GIS data and metadata quality, format, content, and classification and related GIS data collection and maintenance practices.
• American Society of Photogrammetry and Remote Sensing (ASPRS): Professional association that prepares and approved formal standards for aerial data imagery and LiDAR acquisition and processing and for map and orthoimage positional accuracy.
• International Association for Standardization (ISO): An independent international standards organization with participation of industry and government organizations and coordination with professional associations and other standards organizations. The ISO approves and promotes standards for a wide range of quality topics (ISO 9000) and has organized Technical Committee 211 (Geographic Information and Geomatics) which deals with GIS-related software, data, and services.
• Urban and Regional Information Systems Association (URISA): An international professional and educational association that promotes standards and best practices for the management and use of GIS technology. URISA's GIS Management Institute (GMI) develops tools and best practices useful for GIS planning and management.

Other standards organizations, not specific to GIS or spatial data but which develop and promote standards some of which relate to information technology and quality parameters, include:

• American National Standards Institute (ANSI): An independent U.S. based standards organization addressing standards of all types (including many IT standards);
• National Institute of Standards and Technology (NIST): A U.S. government standards organization which adopts format standards for a wide range of products and services;
• Institute of Electrical and Electronics Engineers (IEEE): An independent standards organization with a focus on computer hardware, networks, and software;
• Internet Engineering Task Force (IETF): Global organization that sets standards for Internet communications and services; and
• Software Engineering Institute (SEI): An independent body based at Carnegie Mellon University that develops and promotes standards and best practices for software and application development.

Quality Control and Quality Assurance

There are two terms that are widely used in quality management (in GIS and other fields), quality control and quality assurance. These terms are used somewhat differently by different practitioners. In GIS projects (particularly for GIS database development), they are often used interchangeably. So, as you encounter these terms in project specifications, contractor service descriptions, and white papers, be aware that there is not a full consensus on their meaning. A practical usage of these terms in GIS projects is as follows:

For the purposes of this course we will refer to Quality control (QC) as the tools, processes, and range of automated and manual checks that are put in place to meet quality requirements as deliverables are being prepared (e.g., GIS database deliverables). The intent is to produce the deliverables that fully meet project specifications and quality requirements. The related term, quality assurance (QA), refers to tools and procedures used to assess adherence to specifications and quality requirements after initial deliverable completion and in a final step to check and approve the deliverables. QA checking should be performed as a separate step from deliverable production/preparation, and often by a separate group or people from those involved in deliverable production. Often that separate group is a client organization that has contracted GIS services (database development) to a private firm. QC and QA are related and may use similar tools and procedures, but their use in the entire workflow from deliverable preparation to final acceptance is different. In practice, if QA checking reveals problems with deliverables, the deliverable is subjected to additional steps to correct errors and re-submit it--at which point it is usually subjected to another round of QA checking.

Overview

Many of the tools and techniques in current use have their roots in post-World War II Japan. Dr. W. Edwards Deming used statistical methods to improve quality with a strong focus on the customer or end user, while in the process making organizations more productive and profitable. Although this user-needs focus is now widely accepted, it was an unconventional perspective at the time. Deming's approach was plan, do, check, and act; later he expanded these ideas to:

1. design the product
2. make it; test it in the production line and the laboratory
3. put it on the market
4. test it in service; find out what the user thinks of it, and why the nonuser has not bought it

Fitness of use is defined as both 1) freedom from defects and deficiencies, and 2) product features that meet the user's needs. These two ideas continue to evolve in more recent quality management ideas and practices. For example, the Six Sigma principle, which attempts to limit defective units per billion to two, is a disciplined example of the first definition of fitness of use above. Individuals working with human factors to understand how end users interact with various graphical user interfaces would be an example of the second definition. The concept of "fitness of use" relates directly to the PMI definition of "quality", the degree to which project deliverables meet requirements.

Tools/Techniques for Monitoring and Evaluating Quality

There are a number of tools and techniques to monitor, evaluate, and report on the quality of GIS services and project deliverable quality. A Pareto diagram is a histogram with columns or bars ordered from most common to least. Figure 7-1, below, shows an example of a Pareto diagram. It is a graphical way of summarizing where most problems occur with a product, or what most users would like to see included in a product. It is an important graphical display tool, as often a great majority of problems or needs fall into the same category. Often, the number of individuals providing input is somewhat limited, so that issues can be classified and enumerated from the entire population.

Text description of Figure 7-1: Pareto Diagram.

This figure is a Pareto diagram an explains the concept of quality analysis--a portrayal of the frequency of system problems. it is in the form of a bar graph with 5 vertical bars arranged horizontally. The five bars represent different types of system problems including (left to right): "Log-in problems", "System locks up", "system is too slow", "System is too hard to use", and "Reports are inaccurate". There is a vertical scale on the left side called "Number of complaints this week" with a scale that goes from 0 to 100, incremented in 10s. From left to right, the height of the bars are: 100 Complaints (Log-in problems), 55 Complaints (System locks up), 30 (System is too slow), 20 (System is too hard to use), and 5 (Reports are inaccurate). There is a vertical scale on the left side called "Number of complaints this week" with a scale that goes from 0 to 100 incremented in 10s. which corresponds to a solid line graph line which starts at 50% at the 1st bar (Log-in problems) and moves gradually upward and to the right, reaching 100% above the last bar (Reports are inaccurate).

Credit: Schwalbe (2016), Information Technology Project Management, 8th Edition. (Used with permission by Cengage Learning).

Statistical sampling may be appropriate for testing the quality of products produced, as looking at each individual product would be very time-consuming and cost prohibitive. The sample must be random, and large enough to represent the entire population of products with some degree of certainty. Statistical sampling as part of a quality control program is used most frequently in high-volume manufacturing processes. But it may be appropriate as a basis for automated and manual checks for large-scale GIS database development projects, especially in cases where data collection is more or less random. Whether or not statistical methods are used for determining a sample, the main point is that for some aspects of quality control and quality assurance checking, choosing a sample of the full project set is appropriate and effective. For instance, to check for positional accuracy of field data collection (e.g., positions of signs, hydrants, trees, etc.), a subset (e.g., 2% to 5%) of the full data set could be visually checked by comparing the captured position to the actual location on a high-resolution orthoimage or even with GNSS in the field.

Quality Parameters

With the general definition of quality, "the degree to which project deliverables meet requirements", the challenge for GIS projects is to properly define specifications and specific quality parameters that are appropriate for different types of GIS projects and deliverables. In GIS projects, quality can be defined (and ideally measured and assessed) for specific deliverables such as:

• reports and documents (needs assessment, design documents, plans, etc.)
• conceptual and physical database designs
• spatial data collected from aerial surveys and processed for delivery to users (orthoimagery, LiDAR elevation data)
• GIS data compiled from field data collection, map digitizing, or other means
• GIS applications (custom-designed user interface and functionality from GIS software)

Defining quality parameters for GIS project deliverables should be done as part of project planning--to provide a basis for project work and quality assurance activities. Documented specifications for project deliverables establishes the basis for quality management. Table 7-1 below identifies quality criteria associated with different types of GIS project deliverables.

NOTE: Review Table 7-1 closely. For Assignment #6, you will specifically need to address quality parameters for Metropolis deliverables that relate to the last two (right side) columns of this table.

*includes specified allowable "error of omission" level (required features captured) and "error of commission" (non-required features captured by error). An example of "error of commission" would be a GIS project involving the collection of manhole locations for a wastewater utility network. If the collection included, by error, some gas main manholes as well--there would be commission errors (the gas manholes should not be part of the database).

Quality planning establishes quality parameters and a level of expected quality for those parameters, and it describes the use of necessary tools and procedures to ensure that the level of quality is attained. Quality planning should occur as part of the development of project deliverable specifications and work planning.

As indicated in Table 7-1 above, quality planning for GIS projects may include several important aspects, some of which overlap with those identified for information technology projects, and some of which are unique to GIS. For example, functionality and usability are important aspects of many IT development projects. If you are customizing a user interface for a GIS application, you will need to address these issues of functionality and usability. If, however, you are creating paper copies of maps for botanists to use as they collect samples, functionality and readability relative to how the map will be used is important. The most rigorous and detailed quality planning and quality control procedures are associated with GIS data compilation and processing projects. For GIS data, there are fairly mature, documented standards, and data quality lends itself to a greater level of quantification than quality criteria that are more subjective. Several government and professional organizations have documented standards for spatial data content, format, and overall quality.

In GIS projects, there is considerable focus on quality management for GIS data and custom application development. Read Croswell, subsections 7.4 and 7.5, for an overview of GIS database and application development considerations. Also, take a look at the tutorial document on GIS database QC and QA.

Approaches for Ensuring Quality in GIS Projects

In major GIS database development projects, like the one being launched by the City of Metropolis, it is common practice to prepare an initial database design and set up database development procedures (including QC steps) and then initiate a pilot project--a task or subproject that is part of the overall database development project. The purpose of the pilot is to test the design, source material management, data capture and development procedures, QC approach, etc., and then to use the pilot project results to refine the design and procedures before initiating production work.

In GIS database work, the concept of incremental data improvement is important. Consider a major GIS database project for a water utility organization which has the objective of developing a complete GIS database of the water distribution network with such features as water mains, valves, hydrants, fittings, service lines, etc. The project also captures attribute information about the water facilities like pipe diameter, material, installation date, and others. Even a very well-planned project that uses available sources (as-built engineering drawings, work orders, water service line connection permits), may not result in a fully complete and accurate database (in relation to facilities actually in the field). In this case, a process of "incremental accuracy" may be put in place to improve data quality over time. The water utility organization has people in the field on a continual basis performing inspections and maintenance activity. These individuals observe facility status in the field and, with the right field-based tools deployed with mobile devices, can capture information that will improve GIS data quality over time. Such improvements may include: a) improved positional accuracy based on field-collected coordinates, b) populating attribute data missing from the initial data capture (e.g., correct pipe material), and c) capturing features located in the field (missed in the initial database development). This can be done by deploying field-based GIS applications with a location-aware devices (smartphone, tablet computer) that allow field personnel to capture new data that will drive GIS database improvements.

GIS software provides tools for customizing applications that meet the needs of users. The types of customization work that often come up in GIS application development projects include:

• automating access, integration with other systems or databases, or import/export of files with external databases and applications
• designing and developing “intelligent” interactive forms for attribute or graphic data entry (includes use of dropdown pick lists, automatic error checking, and other controls)
• developing application scripts that can be launched by a simple menu pick and combining a number of individual functions
• designing and creating templates for standard maps and textual reports
• creating data quality control and quality assurance applications using validation tools provided by the software package
• creating a library of standard queries that can be accessed through a menu
• programming complex analysis functions using basic GIS processing command
• building custom geospatial statistics or analysis models (e.g., network analysis)

The types of quality issues that are important in planning and execution for software and GIS application development projects are:

• proper functionality
• usability and adherence to user interface standards
• efficiency, performance, response time
• adherence to coding, programming standards
• flexibility and maintainability
• proper access to GIS data sources
• completeness and clarity of documentation

Software and application development work can follow a number of formal methodologies. The key point is that a formal, organized methodology should be put in place and followed during project execution. As a matter of quality management, key steps in the process (user application needs assessment, design, prototyping, operational deployment, documentation) should have defined steps for user review and comment which is used by the development team to incrementally refine the applications in a process that culminates in deployed applications that meet stated requirements.

For Assignment #6, you will work as a team to prepare a quality management plan for the City of Metropolis Geodatabase Development Project. The basis for this plan is the description of deliverables required by the City and, for GIS data deliverables, the specific quality criteria and levels defined in the RFP (Section 5).

Assignment 6 Overview

Assignments #6 (as well as Assignment #7) will be completed as team assignments. Assignment #6 is a Quality Management Plan for the City of Metropolis Geodatabase Project. You will be required to work with your assigned team (instructor will identify team composition) to complete this assignment. To carry out this assignment, assume that your team represents the Contractor providing project work to the City. The City's project manager (Lucille Geodata) has asked you to document your process for meeting the City's quality requirements and adhering to all required content, format, accuracy, and other quality criteria—as stated in the RFP and any additional quality criteria that you will put in place for the project. She wants to be comfortable that the deliverables provided by your firm will closely meet City requirements. In addition, she wants you to suggest steps that the City team should take to review deliverables as they are submitted, check for quality compliance, and to formally accept or reject deliverables. Only one jointly prepared submittal per team is required for Assignment 6 and will be submitted by the team leader.

Your Submittal for Assignment #6

With your teammates, create a Quality Management Plan for the Metropolis Geodatabase Development Project. A project must plan for quality of deliverables from the onset and put in place quality control and quality assurance checks to ensure that deliverables meet required level of quality. This plan will identify expected levels of quality for project deliverables and steps that your team will take to ensure that deliverables submitted to the City meet stated quality requirements. The main basis for your Quality Plan is Section 5 and Section 6 of the (City of Metropolis RFP). These parts of the RFP state expected quality criteria for the contractor’s deliverables—see Table 2 for a summary of the deliverables. Specifically, this Assignment should address quality management for the following deliverables (see RFP Table 2):

• MD2: GIS Data
• MD4: Design and development of custom GIS applications to support City update of data

The Quality Management Plan should cover the following topics:

• Cover page with prominent title and all necessary information identifying the course, assignment, author, and date. The main title of the document should be "Quality Management Plan". The Cover Page should also reference "City of Metropolis" and the full project name. At the bottom of the Cover Page (right side is best), include the course name and number, assignment number, Team number and team members, and date.
• Table of Contents
• Introduction about the purpose of this plan.
• Brief project background and scope overview—Mention the launch of this project and assembling of a Project Team with Publc Works in the lead. Explain that the City initiated the project with PW in lead. Mention the RFP and hiring of your firm. Make a brief statement about collaboration between the contractor and City team and summary of roles in the project.
• Summary of deliverables—description of the two deliverables (MD2 and MD4) which are the subject of this assignment. Don't include a lot of detail but describe them as a basis for getting into more details about quality management relating to them. You can use excerpts from Table 2 of the City's RFP. Provide a reasonable amount of detail becuase this is important to understand quality management workflows.
• Define the terms "quality" and "quality management" as it relates to this project.
• Explain the quality parameters for Deliverables MD2 and MD4 (NOTE: These come from the RFP and are also referenced in Canvas Table 7-2). Also make reference to applicaable standards from key professional and standards organizations.
• Present and explain the quality management workflow for MD2. A flow chart^† and accompanying description of the steps should be included. This should explain the workflow for contractor deliverable development and QC and submittal to the City AND the QA checking process and tools used by the City to accept (or reject) data deliverables. NOTE: Suggest making this a separate Section in the document
• Present and explain the quality management workflow for MD4. A flow chart^† and accompanying description of the steps should be included. The workflow should cover the design and development process (by the contractor) and the iterative review and comment process with the City team. The quality management workflow should show include final User Acceptance Testing and approval by the City, followed by preparation and review of documentation. Remember that there are two MD4 deliverables (the office-based and the field-based applications). You do NOT need to include a workflow diagram separately for each of these MD4 applications but the accompanying text should explain the quality management for these two applications. NOTE: Suggest making this a separate Section in the document.
• Include a summary of the automated and manual tools and processes that will be used to support the QC and QA work by the Contractor and the City Team. This should include some detail about the Web-based SD4 QA tools supporting City data QA checks. Include some discussion of testing work done by the contractor for the MD4 applications and the appraoch used by the City to review and comment on the applications at the different iterative review/comment parts of the workflow (e.g., a rating/comment form).

^†See the GIS Management Handbook for ideas on format and style for flow charts. Symbology uses "Unified Modeling Language" (UML) symbology. Mainly, you'll be using two symbol types with connecting arrowed lines: rectangles that are workflow steps and "decision diamonds" which represent a point where the workflow path may take more than one path based on a condition.

Make sure you describe the workflow explaining the complementary roles of the Contractor and the City in quality review and acceptance of the deliverables. For Deliverable MD2, this workflow includes the Contractor QC activities for data deliverables and then submits to the City for its QA review (culminating in acceptance or rejection).

As in all written assignments, you should include a cover page which includes the following information: a) course number and name, b) assignment number and name, c) your name, and d) submittal date. The cover page should also have the full project name, title ("Quality Management Plan"), and name of your contracted company. Your submitted assignment should be formatted as specified in the Format Quality of this assignment’s rubric below to earn maximum points. As you prepare this assignment, start with an outline, with sections and subsections that cover the topics above. We recommend that you use the Outline/Heading feature of your word processing software in document preparation. It is expected that you will organize the document into numbered and named sections. It is best practice today, for technical and management documents, to use a "decimal" outline numbering scheme (1., 1.1, etc.) as opposed to the older Roman Numeral numbering approach.

With the assignment of team members and a team leader, the groups should use the most appropriate means for communication and collaboration. The; Groups Space can be used for Group Discussion and file sharing.  Teams are also welcome to use other collaboration tools such as:

• email communication with attachments
• psu.zoom.us - (Penn State now supports zoom.us as a video conferencing tool)
• Google Docs allowing joint revision of documents
• the "Conferences" tool, and
• social media tools

The Quality Management Plan should be from about 3,500 to 5,500 words in length. As is the case for all written assignments, the word count is a target to give you an idea about the level of detail expected. As a general rule, it is best to keep it concise and as brief as possible while still covering the necessary topics. No points will be deducted for submittals if they exceed the maximum word count by a small amount.

Refer to the grading rubric below for guidelines on expected content and format.

Submitting the Assignment Submittal and Grading

View specific directions for Submitting Assignment #6. See Canvas Calendar for submittal date. The grading information and rubric is below.

This assignment is worth 70 points. The points awarded from the Instructor’s grading of this Assignment will be given to all members of the team.

The instructor may deduct points if the Assignment is turned in late, unless a late submittal has been approved by the Instructor prior to the Assignment submittal date.