In this lesson, we take a step further in the evaluation of technologies from the standpoint of environmental, economic, and social compatibility. Building upon the life cycle assessment concepts presented in Lesson 2, we will learn how to develop or select the metrics that would allow us to quantify the impacts and to decide on viability of technology projects. Metrics are important analytical tools when it comes to objective decisions, but they are not something predefined and ready to use. Metrics are meaningfully designed and tuned for a particular purpose, and it is the job of the evaluator to define that purpose prior to the analysis. This lesson overviews some of the methods that are used in environmental science and economics for technology evaluation. However, we are only scratching the surface here. Those areas of science are quite extensive and can fill whole books. So, while working through the basics and studying examples, be prepared to search further and specialize when you chose the metric set for your final course project down the road.

Learning Objectives

By the end of this lesson, you should be able to:

• understand different levels of metrics and their connection with assessment goals;
• perform calculations of some environmental metrics / indicators and interpret their meaning;
• review the fundamentals of the economic analysis of technologies. Perform calculations of some economic metrics using simple payback and discounted cash flow approaches.

Required Readings

Questions?

If you have any questions while working through this Lesson, please post them to our Message Board forum in Canvas. You can use that space any time to chat about course topics or to ask questions. While you are there, please feel free to post your own responses if you are able to help out a classmate.

One of the challenges in sustainability assessment of technologies or other elements of anthropogenic systems is designing meaningful and quantifiable metrics. Because sustainability frameworks are built on very diverse sets of environmental and economic values, there is difficulty in bringing them to common terms within a unified model.

When we ask the question "Is this process sustainable?", we often do not get a "yes/no" answer. Some parts of the system may benefit sustainable development, some parts may be in conflict with it, and some parts may be rather neutral or flexible. But how can we tell where we are on the scale of sustainability assessment? Further, if we do alter certain parts of the system, how much shift do we create towards sustainability goals? Can we measure the impact of those changes?

All these questions prompted attempts to develop quantifiable metrics or indicators, which would allow researchers and policy makers to make more accurate comparisons between different paths of system development and take better justified decisions.

The existing methods for evaluating products and technologies in terms of environmental impact and sustainable development are numerous, and the scope of their applications and purposes is extremely wide. Metrics have been and continue to be developed by different agencies, companies, and researchers to address a variety of issues, which were often placed in different frameworks. As a result, those methods of assessment may use widely diverging rationale, terminology, and approaches, and often come up with contrasting results. Consistency and compatibility are most difficult issues!

In this lesson, it would be hardly possible to learn all of the methods in detail, especially because most models and evaluation approaches are subject-specific and would rather be learned in the context of a particular technology. But we will take a tour over several evaluation platforms and will try to distill the most important questions to address as we proceed to characterization of particular technological areas in further lessons of this course.

Numerous technologies existing in the world differ in many aspects (e.g., profitability, social popularity, efficiency, scale, local need, resource consumption, etc.). However, our main goal will be to distinguish and characterize the technologies in terms of sustainable development. In the long run, the main question we aim to answer: Is the technology project sustainable or not sustainable? What are the criteria for "good" and "evil" here?

How are metrics developed?

What exactly is a metric? A metric is a system of measurement that includes:

• the item being measured (what we measure),
• the method of measurement (how we measure), and
• the inherent value associated with the metric (why we measure, or what we intend to achieve by this measurement).

According to Werner and Souder [Werner and Souder, Research-Technology Management, 40(2), 1997, 34-42], the choice of an appropriate measurement metric depends on the user’s needs and purpose, the area of study, and the available data. Setting the purpose of evaluation is the key. Without it, the metrics are simply data. There should be a decision focus.

Metrics are categorized into quantitative-subjective, qualitative, and integrated metrics. The type is often determined by the availability and accuracy of raw data. Data must be accessible and affordable; otherwise, assumptions and surrogate information would inevitably undermine the adequacy and validity of assessment.

Standardization and coherence in rules of construction of technology evaluation metrics are yet to be achieved. In the broad area of science and technology, the practice of creating and using metrics is in the form of a "menu." Evaluators select combination of measures, data sources, and instruments that will address their specific objectives and needs.

The following issues make metrics not a straightforward matter:

• A metric may be composed of a single quantity or by a more complex set of measures (for example, indexes and "macro" metrics).
• Metrics in science and technology evaluation are designed to measure a variety of activities, events, and phenomena—some simple and short-lived, others highly complex and durable along an extended time frame and therefore using different units and scales.
• The absence of a unique and single building-block increases the role of subjective reasons for the construction and selection of metrics, which sometimes requires better formalization.

Designing comprehensive universal metrics, which would work as "magic crystal" for decision makers, is difficult, if at all possible, because different stakeholders care about different impacts. Most analytical approaches separate contexts and rather develop multi-metric frameworks for assessment. The table below lists some examples of metrics used for technology evaluation in various contexts. This list is not exhaustive, by any means, and in each assessment project metrics must be justified and modified for specific research purpose.

[Source: National Renewable Energy Laboratory]

While working on this course, you should feel free to modify existing metrics or create new ones for the specific needs of your assessment. There are no "mandatory" criteria for evaluation - it all depends on the purpose and the message you try to deliver.

Lagging versus leading metrics

• Lagging metrics are those that indicate what has already happened (past). For example:
  • amount of soil eroded
  • electricity cost per kWh
  • average annual temperature
  • battery efficiency measured
• Leading metrics are those that indicate what may happen (future). For example:
  • deforestation rate
  • input / output ratio
  • wind direction and speed
• Lagging metrics are mainly actual measures; leading metrics are usually indicators marking rates or trends

Complexity levels of metrics

(In parentheses, some examples of metrics are given for the case of a wastewater treatment facility.)

• Level 1 – Measure a technology’s level of compliance with regulations or its conformance with industrial performance standards (e.g., output concentration of hazardous chemicals versus EPA tolerance standards).
• Level 2 – Measure the inputs, outputs, and performance of a technology during system operation (e.g., cost of purified water per gallon, amount of solid waste generated per year, treatment efficiency).
• Level 3 – Evaluate the potential impact of the technology and associated operation on facility personnel, the surrounding environment, and communities (e.g., eutrophication, local air quality trends, carbon emission cost of facility operation).
• Level 4 – Evaluate the lifecycle effects of the technology. This will involve Level 1-3 metrics as necessary to assess the impact of technology’s manufacturing, operation, and disassembly stages (e.g., amounts of raw materials consumed for building the facility, % carbon emissions related to transportation and infrastructure maintenance, water filter life and disposal).
• Level 5 – Assess how the technology-related activities affect the sustainability balance at the scale of society, region, or planet (if the technology is scaled-up) (e.g., % renewable materials or energy used at different stages of operation, community quality-of-life indicators).

In the above hierarchy, Levels 1 and 2 may be sufficient for understanding the promise of technical performance. These levels would be primary guides in relationships between research and development sector and industry, which look for reliable and efficient systems. Ecological perspective would involve Level 3 in order to understand and track environmental impacts. Furthermore, sustainability analysis would have to involve Level 4 at the community scale and Level 5 at the economy scale, since only via thorough lifecycle assessment and system analysis is it possible to identify the correct targets for metric design.

One of the ways to understand if the choice of metric that is adequate to the purpose is sensitivity analysis. By varying impact factors, see how metrics respond. Simulating a series of "what-if" and "what-if-not" scenarios will lead you to designing a proper metric model and defining the boundaries.

Sustainability Indicators

The framework of sustainability indicators deals with much wider context than just technology assessment. It was developed for assessing socio-ecologic development of communities and associated resources and services, and technology can be certainly part of that context. Applying this framework to technology, we can see how technical performance metrics (such as efficiency and useful output) are connected to the economic, social, and environmental specifics of the locale, thus allowing us to estimate the promise of this technology in a local setting.

Systems of sustainability indicators are typically customized to a particular case study. One of the rationales is to take a more detailed approach to assessing sustainability and move beyond the traditional three-pillar approach, which conventionally classifies factors within social, economic, and environmental domains. Indicators can go through those boundaries and address specific needs of assessment.

Good indicators must be:

1. relevant to the problem or system considered in assessment;
2. understandable and easily interpretable by stakeholders and societies that may use the assessment;
3. reliable; i.e., based on trustworthy information and also sensitive to data variation;
4. built on accessible information (not something that is hidden or will become available in the future).

Collection of data and information for calculating sustainability indicators may be a big task. While much of the data can be available in local, state, and federal reports, and many of those can be available online these days, in some cases, you would need to contact respective agencies and offices to request missing information depending on your specific interest.

One of the criticisms of sustainability metrics and indicators is that they attempt to encapsulate an array of diverse processes and interactions in a few simple measures. Is that simplification fair? And what is the risk of using those simplified measures for taking rational decisions?

In fact, designing metrics can be an obvious approach to deal with the complex world in manageable bits. It is common for scientists to deal with a complex system by breaking it down and studying its components separately before studying how they work together. For example, in natural sciences, scientists have been dealing with complex ecosystems for years, and specific indicators have been long used as tools for gauging ecosystem health and development (Bell and Morse, 2008).

According to Slobodkin (1994):

"Any simplification limits our capacity to draw conclusions, but this is by no means unique to ecology. Essentially, all science is the study of either very small bits of reality or simplified surrogates for complex whole systems. How we simplify can be critical. Careless simplification leads to misleading simplistic conclusions."

To learn more about sustainability indicators, please turn to the following readings:

Environmental metrics are designed to assess the environmental impact of technology or activity. Such impacts are primarily related to using natural resources (lifecycle INPUTS) and generating waste and emissions (lifecycle OUTPUTS). The ultimate sustainability goal is to minimize the environmental impacts due to using non-renewable resources and minimizing waste and pollution. Since the complete elimination of these impacts is hardly possible (any technology has its environmental costs!), it is also important to evaluate the rate at which environment can absorb the impacts and become remediated.

There are a number of common metrics designed to characterize the lifecycle inputs and outputs. Some examples are given below:

* The units in the metrics are typically normalized by unit mass, unit volume, or unit area of a material or product, depending on application. For example, for lifecycle energy, it is common to see the units such as J/kg, which indicate how many jules of energy were spent for manfacturing, use, and disposal of 1 kg of the material or product over its lifecycle.

It should be noted that input metrics primarily characterize the sources of impacts (not impacts themselves), while output metrics aim to quantify the consequences - how the extraction and manufacturing processes and technologies may affect natural ecosystems, human health, and environmental values at large. 

Next we are going to take a closer look at some of the metrics and show how those can be estimated. 

Emergy / Transformity

The concepts of emergy and transformity are introduced as universal measures in the environmental accounting theory. The basic approach in that theory is to use energy units for assessing inputs and outputs of various natural and industrial systems. All types of energy and real wealth products are related to the primordial source - solar energy - through transformity. It is stated that going through multiple transformations via both technological and natural converters, available energy acquires new quality, while the load on the environment due to those transformations increases.

For example, on a hot day, we use an air conditioner. The energy to power the air conditioner comes from the electricity grid. Prior to that, the electric energy is generated at a power plant via conversion of thermal energy of steam and kinetic energy of the turbine into electricity. The thermal energy is, in turn, produced by combustion of fossil fuels. Energy stored in the fossil fuels (which were originally biomass) was actually the solar energy transformed by plants to organic matter via photosynthesis. So, solar energy is indeed the original component of the final energy used by the air conditioner (Figure 3.1).

Figure 3.1. Energy transformations resulting in a new quality of usable energy.

Credit: Mark Fedkin

Transformations of the primary types of energy through an array of energy conversion processes and technologies described in the example above demonstrate the idea of transformity applied to a particular type of usable energy, such as electricity. In other words, transformity indicates how many transformations are necessary to obtain a particular sort of energy in a usable form and also show how “costly” those transformations are for the environment. Quantification of the energy inputs and outputs in terms of primary solar energy may be a tricky task; however, a number of studies provided such data and enabled energy flow analysis for environmental systems.

Emergy is defined as the available solar energy used up directly or indirectly to make a service, product, fuel, or another form of usable resource. This term essentially means the solar energy equivalent. Transformity, in this case, is the equivalence factor:

Emergy [seJ] = Energy stored or available [J] x Transformity [seJ/J]

Emergy is usually measured in solar energy joules [seJ], and transformity is therefore expressed as a ratio of solar energy Joules to regular Joules.

Embodied Energy

Embodied energy is another popular representation of the same concept. By definition: embodied energy is the sum of all the energy required to produce any material or product considered as if that energy was incorporated or 'embodied' in the product itself (Wikipedia, Embodied Energy).

The main difference between the embodied energy and emergy is that the former does not include the energy content in the raw resource (e.g. energy content in growing trees), but rather just accounts for the subsequent energy expenditures associated with the extraction, processing, and manufacturing stages.

Embodied energy is often expressed in the units of energy per making a unit mass [J/kg], unit volume [J/liter], or unit area [J/m²] of material depending how the its amount is accounted. Some data are given below for illustration.

For more complex products consisting of multiple raw materials, the embodied energy increases significantly. For instance, for a common monocrystalline silicon solar PV panel, the embodied energy was estimated at ~4750 MJ/m².

Global Warming Potential

Global Warming Potential (GWP) is a very common way to account for greenhouse gas emissions of a project. It can be used within the Lifecycle Assessment and outside of it as a criterion for choosing the most climate-friendly solution among the alternatives. GWP scale is also commonly used to compare different atmospheric gases and pollutants with respect to their ability to cause greenhouse effect. In that sense GWP is a relative metric - it is always related to carbon dioxide CO₂ as the universal benchmark.

GWP of a gas depends on:

• Heat (IR radiation) absorption properties
• Spectral location of the absorbed wavelength (specific for Earth's radiation range)
• Atmospheric lifetime (longer = more impact)
• Time span of assessment (commonly 100 years)

For example, GWP of methane (CH₄) = 25. That means that a ton of methane causes 25 as much warming in the atmosphere as a ton of CO₂ due to greenhouse effect over a 100-years period.

Here is the list a few common atmospheric pollutants graded on the GWP scale.

It is easy to notice the correlation between the longer lifetime and GWP. Some gases are not necessarily potent IR absorbers, but due to very long persistence in the atmosphere, the overall impact is compounded in the long term.

Let us see how these GWP factors for these chemicals can be used in project assessment.

Example

Assume greenhouse gas (GHG) emissions from an agricultural project are estimated as follows:

• Carbon dioxide: m(CO₂) = 5 ton/year (gasoline and diesel-based machinery, irrigation pump, electric grid emissions)
• Methane: m(CH₄) = 0.1 ton/year (organic decay, on-site waste management)
• Nitrous oxide (N₂O) = 0.01 ton/year (transportation emissions)

All these emissions will contribute to the greenhouse effect and global warming, but not equally. Total contribution can be calculated based on GWP metrics:

Figure 3.2. Contribution of gases to cumulative climate impact according to their GWP factors. 

Credit: Mark Fedkin

As another example of using GWP, you can look at the LCA study by Stoessel et al., 2012, that assessed various crops with respect to a number of environmental metrics. Figure 2 of the paper makes an interesting illustration of environmental costs of production and sales of different types of agricultural produce.

Kaya Equation

Kaya Equation (introduced by the economist Yochi Kaya) is another example of environmental metric, which helps to estimate the total CO₂ emissions of a country based on some common social and economic information, such as population, gross domestic product, energy intensity, and carbon intensity:

$$C{O}_2(emitted)=(P)\times (\frac{GDP }{P})\times (\frac{E}{GDP })\times (\frac{C{O}_2}{E})$$

Where P = population, GDP = gross domestic product, (GDP/P) = GDP per capita, (E/GDP) = energy intensity per unit of GDP, and (CO₂/E) = carbon intensity, i.e., emissions per unit energy consumed.

Obviously, the population is an important factor here since more people means more energy use, so it is included as the first term in this equation. GDP is commonly determined the market value of all officially recognized final goods and services produced within a country in a given period of time. GDP per capita is often considered an indicator of a country's economic well-being and standard of living. GDP per capita appears as the next term in the formula, since bigger economy means higher energy use. The next two terms - energy intensity and carbon intensity - are technology related. As we develop more efficient ways to convert energy or produce goods through technological innovation, we expect that it will take less energy to increase our GDP by another dollar. As efficiency grows, the E/GDP term should go down. Finally, the carbon intensity is primarily affected by the ways we generate energy. As we develop and gradually switch to renewable energy sources and minimize the use of fossil fuels, we should see CO2/E factor to decrease. As a result, less carbon dioxide will be emitted per kW of power produced.

The last metric in the Kaya equation is also useful to understand the real "carbon cost" of the energy converting technologies. The more fossil fuel burning is involved in the production of consumable energy (energy conversion), the higher the “carbon cost” of each bit of that energy. Renewable energy technologies, such as solar, wind, and others are characterized by lower (CO₂/E), or even approach zero carbon in an ideal case. However, from the systems perspective, zero emission is not always achievable, since manufacturing, maintenance, and support system operation of such energy conversion systems may still require a certain amount of energy from fossil fuels.

The Kaya model allows estimating how changing technological solutions for energy conversion can help the economy in terms of emission reduction. Determination of the CO₂/E factor provides a quantitative scale for measuring environmental impact in terms of “carbon cost”. The CO₂/E metric is common in many assessment studies discussing alternative energy sources. We need to keep in mind that reported values usually reflect the lifecycle, "cradle to grave" emissions, i.e.,those related to raw material extraction, manufacturing, delivery, operation, maintenance, and decommissioning altogether (not just operational emissions).

Data Reading Exercise:

Take a look at this example of a National Renewable Energy Laboratory (NREL) study that had a goal to compare the lifecycle greenhouse gas emissions of various energy technologies. The study took into account the total estimated emissions from more than 2100 LCA publications and related those to the total amount of energy generated by those systems during their lifetime operation. The carbon intensity results are summarized on the bar graph (p.2 of the fact sheet). Interpreting the graph, answer the following questions for yourself and write answers in your notes:

• What units are used to express the CO₂/E metric?
• Energy from which technology (out of those studied) has the highest "carbon cost"? Which one has the lowest "carbon cost"?
• Which stage of the technology lifecycle does result in the most CO₂ emissions: in case of renewable energy systems and in case of fossil fuel energy systems?

In summary, a review of the Kaya equation indicates that the development of technologies can lower the global and country's carbon emissions in two ways: (1) increasing conversion efficiency and (2) decreasing carbon content in the lifecycle.

When a systems approach is used for technology evaluation, the financial dimension of the system life cycle cannot be omitted. While it is not the purpose of this course to teach the entire theory of economic assessment, reviewing some fundamentals and practical tools for economic evaluation should be useful here.

The purpose of economic metrics is to provide the quantitative information needed to make a judgment or a decision on deployment of a new technology or to select alternative options. The most complete analysis of an investment in a technology or a project requires the analysis of each year of the life of the investment, taking into account relevant direct costs, indirect and overhead costs, taxes, and returns on investment, plus any externalities, such as environmental impacts that are relevant to the decision to be made.

Cash Flow Analysis

Cash flow is a tool used to show how the project expenses and revenues vary over the term of the project - it is a financial timeline. For the basic cash flow, the following terms need to be defined:

• Term of the project – for how many years the process, technology, or facility will be deployed.
• Initial cost (capital investment) – one-time expense at the beginning (e.g., purchase of major assets - land, equipment, buildings, labor).
• Annuity – annual increment of cash related to the operation of the technology:
  • positive, if it brings revenue;
  • negative, if it brings expense;
  • Note that net (expenditure vs. revenue) balance should be positive in order to repay the investment cost.
• Salvage value – one-time positive cash flow at the end of the planning period (if everything is sold in its actual condition). Usually, salvage value is low compared to initial cost.

General cash flow scheme can be visualized as follows (Figure 3.3):

Figure 3.3. Generic cash flow diagram showing the phases of a technology project.

Click for a text description.

A horizontal line is labeled years and is numbered zero through seven. Above the line is revenue and below the line is maintenance/operation costs. At year zero, a large arrow points down, labeled initial cost investment. From year one to six, double-headed arrows show both revenues and maintenance. Revenues remain constant while maintenance fluctuates. At year seven, a single headed arrow points up and is taller than year one through six revenues and is labeled salvage. Years zero through seven are labeled annuity.

Credit: Mark Fedkin

Modeling the cash flow helps assess the financial viability of a project and answer some of the important questions before the decision is made to start the project.

In this section, we are going to consider two basic approaches to cash flow analysis for a project: (1) Simple Payback approach and (2) Discounted Cash Flow analysis.

The first method is attractive for its simplicity and can be used as a quick-check calculation before any further, more sophisticated analysis is performed. It is best suited to short-term projects, in which the money value is not significantly impacted by inflation. The second method is preferred for long-term projects, when the money value is expected to significantly change over time or if interest is applied to investment over an extended period of time.

Referenced below are two reading sources that provide background on the economic evaluation, which will introduce several key economic metrics. The first source is more important, and its content is linked to one of the homework assignments given in this Lesson. Additional explanations and examples to the concepts discussed in the Vanek and Albright's book are provided further in this section.

Simple payback approach

This approach is suitable for short-term projects with quick return on investment. In this case, discounting (for money value declining over time) may be unnecessary.

In simple payback evaluation, all cash flows into and out of the project are added up to find Net Present Value (NPV) metric. That includes initial cost, annuities, and salvage value.

NPV = – Initial Cost + S(Annuities) + Salvage Value

The break-even point, i.e., the year when the sum of annuities surpasses the initial cost and the initial expenditures have been paid back, is characterized by the Simple Payback Period (SPB):

SPB (years) = Initial Cost / Net annuity

For the case described above:

Capital Recovery Factor (CRF) evaluates the relationship between the cash flow and investment cost. This evaluation is applicable to short-term investments (within N=10 years).

CRF = ACC / NPV

where ACC = Annual Capital Cost

ACC = Annuity – NPV/N

Here, the NPV/N term is the average share of the net present value per each year of the project. So, ACC is the part of the annuity that goes each year to cover the investment; it does not go towards profit.

Discounted Cash Flow Analysis

This approach is better applied to long-term projects with slow payback. Money value declines over time, so it must be taken into account.

For example, many renewable energy projects generate low positive annuity at the beginning, while having high initial costs, so it takes more years to pay back investments. In this case, the discounted evaluation should be used.

In the case of discounted cash flow, we need to evaluate how much any cash flow element would value in the future. That would depend on the interest rate (i) imposed on initial investment and the number of years (N) the project is underway. The following conversion factors are used:

$$\frac{\mathrm{present value} }{\mathrm{future value} }=\frac{P}{F}=\frac{1}{{(1+i) }^N}$$

$$\frac{\mathrm{present value} }{\mathrm{annuity} }=\frac{P}{A}=\frac{{(1+i) }^N-1 }{i{(1+i) }^N}$$

Then the NPV can be calculated using the following equation: 

NPV = – Initial Cost + (P/A) × Annuity + (P/F) × Salvage Value

Another useful metric associated with the discounted cash flow analysis is Internal Rate of Return (IRR), which corresponds to the marginal interest rate that would allow the project to break even in the end of the term.

By calculating NPV future value for the end of the project term at different interest rates, one can find the rate at which NPV is equal to zero. The rate corresponding to that condition is IRR (Figure 3.4).

Figure 3.4. Determining of the Internal Rate of Return (IRR) as the limiting interest rate for a project.

Credit: Mark Fedkin

Another illustration of the comparison of the simple payback and discounted cash flow methods is given by the Example on p.67 of the book [Vanek and Albright, 2008].

Listed below are some other economic measures that can be used in different analyses as metrics to evaluate technological systems:

• TLCC = Total life cycle cost
• LCOE = Levelized cost of energy
• RR = Revenue requirements
• B/C = Benefit to cost ratio
• SIR = Savings to investment ratio

You can refer to supplemental reading source [Short et al., 1995] mentioned above for more details on how these metrics are useful and how they can be estimated.

According to the Western Australia Council of Social Services (WACOSS):

"Social sustainability occurs when the formal and informal processes; systems; structures; and relationships actively support the capacity of current and future generations to create healthy and livable communities. Socially sustainable communities are equitable, diverse, connected and democratic and provide a good quality of life."

When we talk about environmental metrics, we focus on well-being of environment; when we talk about economic metrics, we focus on well-being of economy. Hence, the social metrics should be measures of well-being of society or particular groups of people involved as stakeholders. While understanding the importance of sustainable development, people do not still want to give up wealth, capabilities, convenience of life. Although changes in lifestyles and consumption habits can be considered a necessary sacrifice, social analysis seeks to reveal the ways of social transformation that would be less stressful, yet more efficient in reaching sustainability goals. Comparison of different avenues for development would require establishing social metrics.

The following dimensions can be identified in the social context:

Quality of life - basic needs are met and a good quality of life for all members is fostered at the individual, group, and community level (e.g., health, housing, education, employment, safety).

When evaluating a technology project, one can use the following questions as a checklist to see how the development affects or improves:

• affordable and appropriate housing opportunities for the target group;
• physical health outcomes for the target group;
• mental health outcomes for the target group;
• education, training, and skill development opportunities for the target group;
• employment opportunities for the target group;
• access to transport for the target group;
• ability of the target group to meet their basic needs;
• safety and security for the target group;
• access to community amenities and facilities for the target group.

Equity - equitable opportunities and outcomes for all its members, particularly the poorest and most vulnerable members of the community.

Check how the technology project will:

• reduce disadvantage for the target group;
• assist the target group to have more control over their lives, socially and economically;
• identify the causes of disadvantage and inequality and look for ways to reduce them;
• identify and aim to meet the needs of any particularly disadvantaged and marginalized people within the target group;
• be delivered without bias and promote fairness.

Diversity – co-existence of different viewpoints, practices, ethnic, cultural, racial groups in the community.

Check how the technology project will:

• identify diverse groups within the target group and look at ways to meet their particular needs;
• recognize diversity within cultural, ethnic, and racial groups;
• allow for diverse viewpoints, beliefs, and values to be taken into consideration;
• promote understanding and acceptance within the broader community of diverse backgrounds, cultures, and life circumstances.

Interconnected/Social cohesions – establishment of processes, systems, and structures that promote connectedness within and outside the community at the formal, informal, and institutional level.

Check how the technology project will:

• help the target group to develop a sense of belonging in the broader community;
• increase participation in social activities by individuals in the target group;
• improve the target groups’ understanding of and access to public and civic institutions;
• build links between the target group and other groups in the broader community;
• result in the provision of increased support to the target group by the broader community;
• encourage the target group to contribute towards the community or provide support for others.

Democracy and governance – ensuring democratic processes and open and accountable governance structures.

Check how the technology project will:

• allow for a diverse range of people (especially the target group) to participate and be represented in decision-making processes;
• facilitate a clear decision-making process understandable by staff and stakeholders;
• have a budget sufficient to ensure adequate delivery by qualified, trained staff;
• ensure that the use of volunteers is appropriate and properly governed;
• have duration sufficient to achieve the desired outcomes;
• have Plan B - what will happen when the project ceases.

Maturity - an individual accepts the responsibility of consistent growth and improvement through broader social attributes (e.g., communication styles, behavioral patterns, indirect education, and philosophical explorations).

Check how the technology project will:

• be dependent on the responsible decisions of individuals in the target group;
• require additional knowledge and education of stakeholders.

Most of the social metrics are hard to quantify. In assessments, we have to develop a rubric that explains the low and high values on metric scale and choose a reference system for consistent comparison.

Several quantitative metrics have been constructed by Brown and Ulgiati (1997), based on the emergy theory (see system diagram in Figure 3.5). The treatment below provides a good example of how environmental metrics can be blended with economic and social aspects and link them to the system sustainability in a broader sense.

Figure 3.5. Emergy-based indices, accounting for local renewable emergy inputs (R), local nonrenewable inputs (N), and purchased inputs from outside the system (F).

Credit: Brown, M.T. and Ulgiati, S., Ecological Engineering 9 (1997) 51–69

Figure 3.5 is a system diagram showing the energy flows and transformations within a generic locale (surrounded by the system boundary). The Economic Use box can be seen as a "transformer" of the available energy and resources into some Yield (Y), i.e., some product directly related to the function of this system. The inputs to the system are classified as renewable resources, non-renewable resources, local resources, and non-local (purchased) resources. In this model, it is presumed that system sustainability is favored by using renewable energy resources and local energy resources. The resources that are both renewable and local are denoted by R on this diagram. On the contrary, non-renewable local (N) and any non-local, i.e., purchased (F) resources are assumed to lower overall sustainability of the system. These assumptions set the basis for devising a few sustainability metrics in this study.

One of such metrics, which characterizes the environmental impact of an energy flow, is Environmental Loading Ratio (ELR):

ELR = (F + N) / R

From this relationship, we can see that the more non-renewable and outside resources are involved in the process, the higher the ERL index. An increase in renewable energy use in the denominator translates into a lower ELR value. As you can guess, lower ELR is beneficial for the environment.

Another index introduced here is Energy Yield Ratio (EYR):

EYR = Y / F

This metric characterizes the system's capability to exploit local resources (renewable or not). The more the system depends on imported resources or services (increasing F), the lower the EYR, and the higher the system's vulnerability.

Finally, the Sustainability Index (SI) combines both ELR and EYR as follows:

SI = EYR / ELR

Obviously, for higher sustainability “score”, we are interested in having the highest EYR versus the lowest ELR. Within this approach, SI can be used as an aggregate measure to characterize the sustainability function of a given process, technology, or economy.

Please see further explanation of this method and example calculations of metrics in the reading material referenced below.

Note that the above-described approach to assessing a system sustainability is just a single illustration of how sustainability metrics can be devised. The parameters chosen by the authors were specific to their objectives. Calculations they provide answer some of the questions, but may not answer other questions that different stakeholders may have. In that respect, setting the objectives for your assessment and stating clear definitions and assumptions is a very important step in any assessment study in order to make the results meaningful.

Ideally, we would like to see the environmental, economic and social dimensions, and benefits of new technologies balanced. However, most real-life situations would gravitate differently towards those three dimensions. Results of the metric analysis need to be presented in a way that provides a clear and informative message to stakeholders and investors. Presented below are a couple of examples from sustainability assessments performed by government organizations.

The radial diagram in Figure 3.6 was presented by the National Renewable Energy Laboratory (NREL) to describe the sustainability profiles of several energy technologies. Six selected criteria plotted in 6 different directions in the form of a propeller provide an illustration of balance or lack of balance in system analysis. Note that each of the metrics is not directly comparable to others (like we saw in the case of the energy analysis, when all impacts are normalized to the same unit and scale). In this case, the scale for each metric needs to be defined independently versus boundary conditions (minimum and maximum values) so that it covers the appropriate range of evaluation.

Figure 3.6. A visual representation of different categories of sustainability metrics. Click for a text description

This diagram is basically a six-sided star with one category of sustainability metrics shown at each point. From the top right, and continuing clockwise, are the following categories:

• Climate Friendliness - GHG Emission (g-CO2 eq/kWh) - min=1000, Max=0
• Water Conservation - Annually Available Freshwater Usage (%) Min=35, Max=0
• Safety - Fatalities from Severe Incidents (fatalities/GW-year) Min=0.12, Max=0.000248
• Local Employment Impacts - Contribution to County Employment (%) - Min=0, Max=1
• Energy Affordability - LCOE (USD/kWh) - Min=0.39, Max=0
• Energy Diversity - Change in Diversity Indicator (%) - Min=-5, Max=5

Credit: National Renewable Energy Laboratory

The diagram below (Figure 3.7.) presents another example of how different categories of metrics are balanced to characterize the sustainability profile of a city. From this representation, we can immediately recognize that the most problematic areas the city may want to address first are Emission and Waste, which create a critically bad impact, and Materials and Energy flows (the lowest: red and orange scores in the pie). At the same time, Cultural Engagement and Identity is the most attractive feature of the city (the highest: bright green score). We can also conclude just from a quick glance that the ecological part of this sustainability system is most suppressed, while the cultural part is probably most developed and sound. On the political and economic fronts, some of the impacts are in the favorable range, while others are down to satisfactory. This snapshot of the disbalance provides a tool for comparison when other systems (cities) are evaluated against the same metrics.

Figure 3.7. Radial diagram showing the sustainability of metropolis of Melbourne (Australia) as developed by UN Global Compact Cities Programme.

Credit: Wikimedia Commons

When considering specific technologies in the context of sustainability, the ideal expectation is that they score equally well on all three evaluation domains – environmental, economic, and social. The scoring metrics should be chosen wisely, with primary consideration of the main stakeholders’ interests. When comparing different technologies as alternatives for a project, go with the same set of metrics. Collection of location-specific and accurate data is critical for accurate assessment. Multi-metric assessment may not give you a simple answer, but may provide a more realistic ground for decision making. Resources listed in this lesson will be especially useful in choosing the path for the assessment study in your final course project.

References and Resources:

Bell, S. and Morse, S., Sustainability Indicators: Measuring Immeasurable?, 2nd Ed., London - Sterling VA, 2008.

Brown, M.T. and Ulgiati, S, Emergy-based indices and ratios to evaluate sustainability: monitoring economies and technology toward environmentally sound innovation, Ecological Engineering 9 51–69 (1997).

Hammond, G.P. and C.I.Jones, Embodied Energy and Carbon Footprint Database, Department of Mechanical Engineering, University of Bath, United Kingdom (2006).

Odum, H.T., Environmental Accounting, John Wiley & Sons, 1996.

Short, W., Packey, D.J., and Holt, T., A Manual for the Economic Evaluation of Energy Efficiency and Renewable Energy Technologies, National Renewable Energy Laboratory, Golden, CO, 1995.

Vanek, F.M., and L.D. Albright, Energy Systems Engineering. Evaluation and Implementation, McGraw Hill, 2008.