DIS-16-01, How the CNSC Considers Information on Costs and Benefits: Opportunities to Improve Guidance and Clarity

Preface

Discussion papers play an important role in the selection and development of the regulatory framework and regulatory program of the Canadian Nuclear Safety Commission (CNSC). They are used to solicit early public feedback on CNSC policies or approaches.

The use of discussion papers early in the regulatory process underlines the CNSC’s commitment to a transparent consultation process. The CNSC analyzes and considers preliminary feedback when determining the type and nature of requirements and guidance to issue.

Discussion papers are made available for public comment for a specified period of time. At the end of the first comment period, CNSC staff review all public input, which is then posted for feedback on the CNSC website for a second round of consultation.

The CNSC considers all feedback received from this consultation process in determining its regulatory approach.

Executive Summary

The CNSC’s mandate includes regulating the use of nuclear energy and materials to protect health, safety, security and the environment. The CNSC policy document P-242, Considering Cost-benefit Information, states that, when making a decision under the Nuclear Safety and Control Act (NSCA), the Commission or its designated officers will consider relevant cost-benefit information submitted by a participant in the process. The CNSC is proposing to update this policy and incorporate the material into its regulatory framework.

Section 2 of this discussion paper provides an overview of the CNSC’s current policy on the consideration of cost-benefit information that is submitted to the Commission or its designated officers for decisions under the NSCA. This overview includes information on who submits information on costs or benefits to the CNSC, types of methodologies used to produce cost-benefit information, whether the submission of cost-benefit information is a requirement, the types of cost-benefit information the CNSC considers, and how the CNSC considers this information in its decision making. This section also describes several examples of cost-benefit information that has been submitted to the CNSC. These examples are intended to help illustrate how the CNSC may consider such information, and to provide background context for the subsequent discussion questions. 

Section 3 provides an overview of the CNSC’s current policy on how it considers cost-benefit information when making changes to its regulatory framework (i.e., through regulations or regulatory documents).

Section 4 provides proposed guidance for stakeholders when preparing information on costs or benefits for submission to the CNSC. This guidance is proposed for inclusion in a regulatory document along with the content currently in P-242. Once published, P-242 would be replaced by a new regulatory document within series 3.5 of the CNSC’s regulatory framework.

Section 5 asks questions that solicit stakeholder feedback and explore how to update P-242, as well as whether the CNSC should provide additional clarity or guidance to help ensure that cost-benefit information submitted to it is fit for purpose and of high quality.

1. Introduction

As Canada’s nuclear regulator, the Canadian Nuclear Safety Commission (CNSC) regulates the use of nuclear energy and materials to protect health, safety, security and the environment, to implement Canada's international commitments on the peaceful use of nuclear energy; and to disseminate objective scientific, technical and regulatory information to the public. The CNSC’s top priority is safety.

In 2000 the CNSC published regulatory policy P-242, Considering Cost-benefit Information. This document confirms that, when making a decision under the Nuclear Safety and Control Act (NSCA), the Commission or its designated officers will consider relevant cost-benefit information submitted by any participant in the process.

The CNSC makes independent, fair and transparent decisions every day on the licensing of nuclear-related activities in Canada. Good regulatory decisions – decisions that achieve desirable results in an efficient manner – require decision-making processes that balance a number of important factors and considerations, including potential costs and benefits. The CNSC will not compromise safety when balancing these factors.

The CNSC strives to ensure that its decisions are commensurate with the risks being managed, and that unreasonable risks to the environment or to the health and safety of persons are prevented. The approach the CNSC uses to achieve this objective is beyond the scope of this paper. However, it does discuss the role that information on costs and benefits may play in the CNSC’s decision-making processes.

The CNSC makes a wide variety of decisions, so the role of cost-benefit information in any specific decision also varies and depends on many factors. However, in all cases, costs and benefits are only one consideration that the CNSC may take into account when making a decision, and this is always done in a manner that puts safety first.

With 15 years’ experience since the publication of P-242, the CNSC is reviewing the need to update this policy and seeks feedback from stakeholders to determine if there is a need for more guidance on its expectations for the submission of information on costs and benefits. A discussion document on this subject is timely for several reasons.

It may be useful to first recognize that there are many different types of decisions made by the CNSC that can vary widely in their complexity and potential impacts. The following examples provide an idea of the depth and breadth of the types of decisions that the CNSC makes when regulating nuclear-related activities:

• whether to license a new or existing facility, such as nuclear power plant, uranium mine or a fuel processing plant
• whether to accept a proposal from a licensee on how it intends to meet the CNSC’s regulatory requirements (see section 2 of this discussion paper for examples)
• whether to set a new requirement in its regulatory framework (or modify or remove an old requirement); for example, the CNSC may consider changes to a regulation or a regulatory document (see section 3 of this discussion paper for examples)

Firstly, during the past two years, the CNSC has adopted a new framework to organize its suite of regulatory documents. Policy documents such as P-242 are scheduled to be incorporated into this new structure by 2018. The CNSC plans to incorporate the existing content of P-242 into a new regulatory document with a new title, to be located in series 3.5 of the framework, focused on CNSC processes and practices. This exercise provides an opportunity to determine if updates to the document would be beneficial.

Secondly, in response to recent stakeholder feedback, the CNSC has created a new opportunity for stakeholders to share information on the potential impacts (financial and other) of new or amended regulatory documents. CNSC impact statements now explicitly request feedback from stakeholders on the alternatives, costs and other potential impacts associated with new or recently amended draft regulatory documents. This discussion paper may help facilitate a useful exchange of views on the type and quality of information that could be provided to the CNSC in response to this new initiative.

Draft guidance that the CNSC is considering for inclusion in a regulatory document is provided in this paper for stakeholder review. The objective of this guidance is to help stakeholders understand the CNSC’s expectations for the development of cost-benefit information that is of high quality and fit for purpose to meet CNSC regulatory requirements without compromising safety.

The CNSC welcomes feedback on this discussion paper from all stakeholders.

2. Overview of the CNSC’s Current Policy on Consideration of Costs and Benefits for Decisions Under the NSCA

The CNSC’s current policy, as outlined in regulatory policy P-242, indicates:

“When conducing a proceeding for purposes of a decision under the Nuclear Safety and Control Act that involves a licence or an order, the Commission or its designated officers will consider relevant information on costs or benefits that is submitted by a person who is participating in the process.”

The CNSC receives information on costs and benefits when a licence applicant or licensee wishes to propose alternative approaches to meeting regulatory requirements, taking costs and benefits into account. The following subsections provide additional context.

2.1 Who can submit cost-benefit information to the CNSC?

Consistent with regulatory policy P-242, when making a decision under the NSCA, the Commission or its designated officer will consider relevant cost-benefit information submitted by any participant in the process. In practice, information on costs and benefits are usually submitted to the CNSC by licence applicants or licensees to support regulatory decision making, if they choose to do so.

2.2 What types of methodology are used to produce cost-benefit information?

Regulatory policy P-242 indicates that cost-benefit information that is submitted to the CNSC may be quantitative or qualitative in nature. 

Many types of methodologies and analyses have been used to produce quantitative cost-benefit information. These have included formal, comprehensive approaches, such as the production of cost-benefit analyses (CBA), cost-effectiveness analyses (CEA), and multi-criteria decision analyses (MCDA).^{Footnote 1} Within each of these, methods and approaches are described in scholarly articles and are in use across many sectors and industries.

Less formal types of qualitative information on costs and benefits, such as, general considerations of one approach over another, have also been provided to the CNSC. Therefore, the scope of cost-benefit information touched on by this paper may vary with the complexity and potential risk of the situation under consideration.

It is also worth noting that assigning monetary weight to certain types of benefits can be a challenge. Some intangible benefits, such as maintaining public confidence in the Canadian nuclear regulatory regime, would be difficult to objectively and fairly weight according to a dollar value.

2.3 Does the CNSC require cost-benefit information to be submitted?

The NSCA and its regulations do not explicitly require submission of cost-benefit information to the CNSC. This discussion paper does not propose any new legal requirements for licensees and applicants. However, there are occasions when consideration of cost-benefit information can be useful to the CNSC when making a regulatory decision and its submission would be recommended.

2.4 What types of costs-benefit information does the CNSC consider?

Regulatory Policy P-242 confirms that, when making a decision under the NSCA, the Commission or its designated officer will consider relevant cost-benefit information. What does the CNSC consider “relevant”? The CNSC would not consider cost-benefit information that pertains to issues outside of its mandate to regulate nuclear activity in Canada. For example, the CNSC is not responsible for making decisions on the use of a nuclear technology relative to non-nuclear alternatives. Cost-benefit information on, for example, nuclear power versus natural gas would not be relevant to the CNSC’s mandate. Proponents considering submitting information on costs and benefits are encouraged to consult with the CNSC if they wish to confirm the scope of information considered relevant to the decision at hand.

2.5 How does the CNSC consider information on costs and benefits?

The CNSC’s current policy, as outlined in Regulatory Policy P-242, indicates:

“When receiving or considering any relevant information on costs or benefits that is submitted in relation to a decision involving a licence or order, the Commission or its designated officers will be governed by the following principles:

• Information on costs and benefits is only one factor that may be considered in making “regulatory decisions” or taking “regulatory actions” under the Act, and does not displace legal requirements and other valid regulatory considerations.
• The information on costs or benefits may be quantitative or qualitative in nature.
• Consideration of the information on costs or benefits may be quantitative or qualitative in nature.”

Many complex decisions made by the CNSC are guided by a formal process for risk-informed decision-making (RIDM), which takes into account costs and benefits after a risk assessment has occurred. Where there is no immediate risk to the public or the environment, and when there is more than one acceptable approach to meet a safety objective, the CNSC will generally accept the approach proposed by an applicant or licensee.

During routine operations, when the CNSC reviews proposals from applicants and licensees or considers changes to its requirements, there may be more than one acceptable approach to address the identified safety objective. It is in this context that the CNSC may use information on costs or benefits to support effective regulatory decision making.

2.6 Examples of how the CNSC has considered information on costs or benefits for decisions under the NSCA

The following subsections provide examples of how the CNSC has considered information on costs and benefits that have been submitted for purposes of decisions under the NSCA.

2.6.1 Darlington new nuclear project: condenser cooling water option assessment

In 2006, Ontario Power Generation (OPG) initiated the process to obtain federal approvals for the construction and operation of up to four new power reactors at the existing Darlington Nuclear Generating Station site. This process included an environmental assessment conducted in accordance with the Canadian Environmental Assessment Act^{Footnote 2}. Accordingly, the Minister of the Environment and the CNSC President mandated a joint review panel to assess the environmental effects of the proposed project, including a review of OPG’s proposed condenser cooling water technology. In its 2011 report, the panel noted that OPG’s proposed solution was selected from among several options based on a qualitative rationale. It recommended that a formal, quantitative cost-benefit analysis be completed to compare cooling tower and once-through condenser cooling water systems, applying the principle of best available technology economically achievable (BATEA).

OPG undertook an MCDA. This methodology allowed factors that are difficult to quantify in economic terms, such as the visual impact of cooling towers, to be taken into account. The attributes considered in the analysis and used to compare the options included health and safety, environmental, technological, public perception and financial aspects.

OPG’s analysis concluded that both options would protect public and worker safety, and comply with environmental protection regulatory requirements. However, the once-through cooling option would cost significantly less to implement and was strongly favoured by local stakeholders.

The CNSC reviewed the analysis and concluded that OPG’s methodology satisfied the intent of the joint review panel’s recommendation and provided an adequate basis for making a decision on the application. No fundamental barriers were found to licensing the once-through cooling water system with the incorporation of the latest in mitigative technology and techniques.

2.6.2 Pickering Nuclear Generating Station mitigation options for fish impingement

Electric power plants take in cooling water from nearby lakes and rivers to condense steam after its use in turbine generators. Incoming cooling water is typically screened to prevent fish and debris from entering the plant. However, fish and fish egg losses occur when they are caught on these screens or drawn into pipes during cooling water intake. Depending on the magnitude and nature of these losses, there may be impacts on the environment.

In 2008, the CNSC set a target to reduce fish impingement by 80 percent at the OPG-operated Pickering Nuclear Generating Station. Several options to reduce fish losses were possible, such as the use of barrier nets, sound deterrents, variable speed pumps and restoration of fish stocks. OPG undertook an analysis to determine which alternative would satisfy the target in the most cost-effective manner.

The analysis provided to the CNSC considered a variety of benefits and costs for each mitigation alternative, such as the value of commercial and recreational fish, impacts on threatened or endangered species, and costs to construct, operate and maintain the mitigation option.

OPG’s proposed alternative combined the installation of a barrier net with restocking certain species of fish, an approach that was subsequently shown to effectively achieve the objective set by the CNSC. The CNSC accepted OPG’s proposal to employ the most cost-effective option while meeting the established goal of reducing fish mortality by 80 percent.

2.6.3 Uranium mill tailings management options

Uranium mill tailings are the waste material that remains after uranium ore has been processed into uranium concentrate. They are a mixture of sandy rock particles, water and processing reagents, and need to be stored in containment in a manner that minimizes contamination of groundwater and surface water. In the past, tailings have been stored in surface impoundments, which use geographical features and man-made barriers such as dams to contain the tailings. In recent years, tailings have been stored in mined-out open pits, to avoid reliance on man-made structures that require active perpetual maintenance.

The CNSC’s RD/GD-370, Management of Uranium Mine Waste Rock and Mill Tailings, recommends that applicants select the most suitable mine waste disposal alternative from an environmental, technical, economic and socio-economic perspective, and strive to achieve consensus on the decision from a broad stakeholder group. In order to do so, the CNSC encourages applicants to complete a multi-criteria decision analysis to evaluate alternatives, pursuant to Environment Canada’s Guidelines for the Assessment of Alternatives for Mine Waste Disposal. MCDA is similar to traditional cost-benefit analysis; however it does not use dollar value as an equalizer to compare different aspects of the alternatives under consideration. This can be useful in situations where it may be challenging to evaluate considerations in monetary terms; e.g., ecological impacts or quality of life.

Although considering alternatives is not mandatory for mine waste management, except in specific scenarios,^{Footnote 3} the CNSC has found value in the use of this methodology for comparing options and for helping it to understand the applicant’s proposal. Information produced using this or other similar methodologies has been submitted to the CNSC and used to support regulatory decision making in a number of cases including:

• 2014: AREVA’s McClean Lake tailings management site, northern Saskatchewan
• 2013: Gunnar mine site, northern Saskatchewan
• 2013: Lorado tailings site, Beaverlodge, Saskatchewan
• 1995: Quirke and Panel Mines and Mills, Elliot Lake, Ontario

2.6.4 Periodic safety reviews of nuclear power plants

Nuclear power plant licensees conduct regular reviews of their performance to ensure safety is maintained. Licensees perform periodic safety reviews (PSRs) of their operations, facilities and equipment as a licence requirement, but also to support refurbishment of existing facilities. A PSR takes into account operating experience in Canada and around the world, new knowledge from research and development activities, and advances in technology. It results in a prioritized plan for making improvements to structures, systems, components and programs to assure the continued safe operation of a nuclear power plant, in accordance with modern standards and operating practices.

The CNSC’s REGDOC-2.3.3, Periodic Safety Reviews, indicates that all gaps identified in the course of a PSR should be categorized and prioritized according to their safety significance. Licensees describe the process and methodology they used to identify and prioritize these gaps, and to explain how they evaluated and decided among available alternatives for addressing the gaps. Cost-benefit methodology has been employed by licensees (e.g., at Point Lepreau, units 5-8 at Pickering and the National Research Universal reactor) to help evaluate alternative approaches to address identified issues.

Typically, cost-benefit analysis is a component of the licensee’s process for determining an approach to address gaps identified in the review of the facility against modern codes and standards. After that, the licensee may make submissions on design changes (or why design changes are not being implemented) supported by cost-benefit analysis without compromising safety. In many cases, licensees have used cost-benefit analysis guidance developed by the CANDU Owners Group in the context of a PSR.

2.6.5 Applying the ALARA principle

The Radiation Protection Regulations require CNSC licensees to implement measures to keep radiation doses received by workers and members of the public as low as reasonably achievable (ALARA). It is insufficient for a licensee to simply respect the appropriate dose limits; efforts must be made to further reduce doses. The ALARA principle considers social and economic factors (such as the costs of measures to reduce doses) balanced against the benefit obtained. Taking actions to further reduce dose levels without any added safety benefits may not be economically justifiable in every case.

The CNSC’s regulatory guide G-129, Keeping Radiation Exposures and Doses As Low As Reasonably Achievable, notes that cost-benefit analysis is one method that can be used to help judge reasonableness. Safety literature as well as IAEA Safety Reports Series No. 21, Optimization of Radiation Protection in the Control of Occupational Exposure, provides additional information on a number of decision-aiding techniques including CBA and CEA.

OPG’s experience with the installation of sub-micron filters at the Darlington Nuclear Generating Station illustrates a cost-effective application of the ALARA principle. Workers at nuclear power plants in Canada are exposed to radiation from various sources, but a significant portion results from nuclear substances dispersed in the plant’s primary heat transport system. Purification systems remove radioactive particles from these systems.

In 2002, OPG installed a new type of filter that reduced radiation doses to workers. The cost of installing the filters was approximately $32,000. In accordance with methods outlined in international guidance such as IAEA Safety Reports Series No. 21, OPG calculated the net benefit of installing the filters at $28,000 within six months, a benefit that would continue to accrue over their lifetime.

Installing sub-micron filters was reasonably achievable and an economically sound means to reduce dose levels. Sub-micron filters were later installed in all of the purification systems at Darlington Nuclear Generating Station.

2.6.6 Applying the BATEA principle

Best available technology economically achievable (BATEA) is a principle used to help drive continuous improvement through the adoption of new and innovative technologies and techniques, as they become economically feasible. BATEA is similar in some respects to ALARA, but where ALARA applies to radiation doses, BATEA has been applied to environmental performance in various industrial sectors. Just as there is a need to determine what is reasonable in ALARA, a cost-benefit analysis, or other types of analyses and methods can be used to establish what is economically achievable when applying the BATEA principle. There is no one best method to do this. In each sector or situation where the BATEA principle is used, the parties involved must determine an appropriate method to determine what will be considered economically achievable, to achieve a level of safety that has been set by the CNSC.

3. Overview of the CNSC’s Current Policy on Consideration of Cost-Benefit Information in Its Regulatory Framework

The CNSC’s current policy, as outlined in regulatory policy P-242, indicates:

“When conducting consultations on a draft regulatory standard or a draft regulatory policy [now called regulatory documents], the Commission will take into account, when fixing the deadline for submission of comments, the time that may be required for the preparation of submissions on the costs and benefits related to the proposed standard or policy.”

The following subsections provide additional context on how the CNSC takes information on costs or benefits into account while considering changes to regulations or regulatory documents.

3.1 Consideration of costs and benefits in regulations

When the CNSC amends any of its existing 13 regulations or develops a new regulation, the process for doing so includes analyzing and estimating the associated costs and impacts on the regulated community and all Canadians. The analysis is conducted by the CNSC and summarized in a Regulatory Impact Analysis Statement (RIAS). The RIAS, along with the proposed regulation, is included in consultation material posted in the Canada Gazette, Part I. Once consultation is complete, the CNSC considers feedback received and makes adjustments. The RIAS and regulation are published in the Canada Gazette, Part II after they have been approved by the Commission and the Governor in Council.

The Government of Canada and the Treasury Board of Canada maintain the process for amending or creating new regulations. The process for amending or creating regulations, including information on the RIAS, is outlined in the Cabinet Directive on Regulatory Management.

3.2 Example of consideration of costs and benefits for regulations

In 2015, the Packaging and Transport of Nuclear Substances Regulations were repealed and replaced by the Packaging and Transport of Nuclear Substances Regulations, 2015. The process to replace the regulation included an analysis of the estimated associated costs and impacts on the regulated community and all Canadians. This analysis was conducted by the CNSC and summarized in a RIAS, and pre-consultation was conducted through a discussion paper (DIS‑12‑06, Proposal to Amend the Packaging and Transport of Nuclear Substances Regulations). The proposed regulation and the RIAS were published for consultation in the Canada Gazette, Part I, and the final regulations and RIAS were published in the Canada Gazette, Part II.

3.3 Consideration of costs and benefits for regulatory documents

The CNSC develops regulatory documents to help clarify the CNSC’s expectations for compliance with regulations and licence conditions. The CNSC consults stakeholders and welcomes feedback on the costs, benefits or other impacts of these documents whenever a regulatory document is created or amended.

In July 2015, the CNSC began including impact statements along with draft regulatory documents being issued for consultation. CNSC impact statements now explicitly welcome additional information from stakeholders on the potential impacts associated with new or recently amended draft regulatory document or any proposed alternative approaches that could be considered that meet the document’s safety objectives.

3.4 Example of consideration of costs and benefits for changes to a regulatory document

In 2015, the CNSC released an impact statement for draft REGDOC 2.2.4, Fitness for Duty. The impact statement provides additional information for stakeholders on the proposed regulatory document, including background information, objectives, regulatory approach and forecasted implementation plan. The CNSC also welcomes additional information from stakeholders on the the regulatory document’s potential impacts or any proposed alternative approaches that meet the document’s safety objectives. Comments received will be considered by the CNSC and will become part of the public record.

4. Draft Guidance on the Submission of Cost-Benefit Information to the CNSC

This section contains proposed guidance that licensees or other stakeholders should consider when preparing to submit information on costs and benefits to the CNSC. This will help ensure that cost-benefit information provided to the CNSC is fit for purpose and of high quality.

The CNSC is considering including this guidance in a regulatory document, which would also include the content currently in P-242. Once complete, P-242 would be retired and replaced by a new regulatory document within series 3.5 of the CNSC’s regulatory framework.

4.1 Purpose of guidance

Regardless of the methodology used to produce cost-benefit information (e.g., cost-benefit analysis, multi-criteria decision analysis, cost-effectiveness analysis or other), the CNSC expects the information received to be of high quality and to be developed in accordance with best practices.

4.2 Scope of guidance

This guidance would inform licensees when preparing a cost-benefit analysis to submit to the Commission, or its designated officers, to consider for the purposes of a decision under the NSCA. The guidance would also be recommended to stakeholders who submit information on costs and benefits to the CNSC during consultation on its regulatory framework (i.e., regulations or regulatory documents).

4.3 New guidance material

Stakeholders producing cost-benefit information should consider the following recommendations, where applicable. 

4.3.1 Level of analysis

In general, minor routine decisions with minor potential consequences should not demand the same level of analysis as exceptional decisions having major potential consequences. If investing in more data, research or analysis is not expected to change the ranking of alternatives, further investment is not justified. Stakeholders should show that the level of analysis is commensurate with the nature of the decision that needs to be made.

4.3.2 Rationale

Various terms are used to refer to the rationale for a project (e.g., goal, objective, purpose, need). Having a clear rationale statement provides a good foundation for identifying reasonable alternative courses of action and for determining if each alternative is capable of satisfying the purpose of a proposed action or project. The rationale should clearly set out the problem or opportunity that is being addressed and the desired outcome.

4.3.3 Boundaries of the analysis

Boundaries that have been decided on for the evaluation (e.g., time period or geographical area considered) should be described precisely and a compelling rationale provided to explain why they were selected. A key measure of the reasonableness of decisions on boundaries is evidence that expanding or contracting the boundaries of an evaluation would not likely change the preference ordering of the alternatives.

4.3.4 Factors to consider

Depending on the nature of the decision being taken, in addition to an analysis focused on the proponent, it may be appropriate to consider other factors, such as human health and environmental. In all cases, information provided should be relevant to the CNSC’s mandate.

4.3.5 Alternatives

An essential requirement for good decision-making is a full consideration of all reasonable options. Including a comprehensive range of reasonable alternatives is also essential to demonstrating that an analysis is unbiased. The following should be considered:

• Have all possible alternatives for addressing the problem or opportunity been identified and listed?
• Have all reasonable alternatives been analyzed?
• Has each alternative been developed to a reasonable level of detail for a sufficiently accurate evaluation of costs and benefits to reliably distinguish among the alternatives?
• Has each alternative been designed using an unbiased and consistent approach?
• Has the analysis of each alternative been comprehensive and based on a common set of data, relationships and assumptions?
• Should the status quo (i.e., do nothing) alternative be considered? This option may not be applicable in cases where the CNSC has established an objective that must be met.

4.3.6 Forecasting

Forecasting the expected impact of an alternative course of action may involve many disciplines (e.g., engineering, environmental sciences, human health, economics) and many different types of data (e.g., biophysical, engineering, economic and social). A high-quality analysis makes the best use of the best available information.

4.3.7 Valuation or weighting

Valuation^{Footnote 4} or weighting involves estimating the relative importance of a risk, cost or benefit; in economics, market prices and imputed market prices are often used for valuation. In multi-criteria decision analysis (MCDA), valuation is undertaken through participatory processes. An extensive literature dealing with appropriate valuation and accounting methods exists. High-quality cost-benefit information should explain and rationalize the valuation method(s) that have been used for risks, costs or benefits.

4.3.8 Uncertainty

All forecasts involve some level of uncertainty. Uncertainty can result in the actual outcome being above or below the expected outcome. A high-quality analysis should include a systematic and comprehensive uncertainty analysis.

4.3.9 Sensitivity analysis

An analysis may be based on many individual data points, relationships and assumptions; however, a small subset of these may have a disproportionate influence on the overall evaluation of alternatives. Knowing which data, relationships and assumptions have the greatest influence on the overall result is important from a decision-making perspective.

Sensitivity analysis is valuable in deciding whether more investment in data collection and research is warranted before a decision on a project is made. High-quality cost-benefit information should include a sensitivity analysis that ranks key data points, relationships and assumptions relative to their impact on the results of the evaluation of alternatives. Of particular importance is to identify those data points, relationships and assumptions that are likely to alter the final ranking of the alternatives.

Sensitivity analysis, however, can be essentially open-ended with larger projects involving many data points, relationships and assumptions. Prudence and judgment are required to balance demands for more sensitivity analysis with the likelihood of new insights being provided. A high-quality analysis should provide a compelling rationale for the limits to the sensitivity analysis conducted.

4.3.10 Replicability

Thorough, clear and accessible documentation is critical to producing a high-quality analysis. All data, sources, forecasting methods, assumptions and calculations should be fully documented and understandable. An analysis is considered replicable if a qualified third party would be able to duplicate the evaluation of alternatives and reach the same conclusion, using the same data and methods. 

4.3.11 Discount rate

Discounting allows for the calculation of costs and benefits that occur over several years, taking inflation and other factors into account. Choosing an appropriate discount rate is an important consideration because it will affect the calculation of net costs and benefits and potentially have an impact on the conclusion. In all cases, the discount rate used in the analysis should be clear.

4.4 Resources

The following resources are available to help stakeholders develop cost-benefit information:

• Treasury Board Canada Secretariat has produced the Canadian Cost-Benefit Analysis Guide: Regulatory Proposals for the development of cost-benefit analysis which applies to new regulations or regulatory amendments. This information may be useful guidance in other regulatory decision making.
• Environment Canada’s Guidelines for the Assessment of Alternatives for Mine Waste Disposal is recommended in the CNSC’s RD/GD-370, Management of Uranium Mine Waste Rock and Mill Tailings, to aid applicants in selecting the most suitable mine waste disposal alternative from an environmental, technical, economic and socio-economic perspective.
• The Canadian Environmental Assessment Agency published an operational policy statement with guidance to help proponents of major projects determine the technical and economic feasibility of alternative means of carrying out a project.

5. Questions for Stakeholders

Stakeholder feedback is sought on the following questions to help the CNSC determine the need to update P-242 and include new guidance in a regulatory document:

5.1 Question 1: Should any elements be added or removed from the draft guidance?

Section 4 contains a sample of guidance material that the CNSC could publish in a regulatory document. This guidance would be intended to help stakeholders develop high-quality cost-benefit information. Data is considered high-quality if it is fit for its intended purpose, provides enough information to allow an evaluation to be completed, and if it accurately reflects the real-world situation to which it refers. If the CNSC receives data on costs or benefits that is not of high quality, the CNSC may ask the person who provided the information to make clarifications or changes. Data quality may also impact the degree to which the CNSC takes it into account in decision-making. If data on costs and benefits is unclear or its accuracy cannot be substantiated, the CNSC may not have sufficient confidence to consider it in a meaningful way. Thus clarifying the CNSC’s expectations may be helpful to stakeholders when developing information on costs and benefits for submission.

5.2 Question 2: Are there other resources the CNSC should include in the draft guidance?

Section 4.4 contains references to other methods and tools that may help stakeholders develop cost-benefit information.  Are there other resources which could be added to the draft guidance material?

5.3 Question 3: Is there a need for further discussion on methodologies or certain aspects of estimating costs in use by the Canadian nuclear sector?

The CNSC intends to continue to provide as much flexibility as possible to applicants, licensees and other stakeholders to determine the type of information on costs and benefits and the level of detail they wish to provide to the CNSC. The general nature of the proposed guidance in section 4 aligns with this philosophy. However, there could be areas where more specific guidance or a more standardized approach to estimating costs may be beneficial. The CNSC is interested in stakeholder views on the need for further discussion. For example, could there be a benefit from:

• a discussion of approaches and considerations when selecting an appropriate discount rate?
• a discussion on the “reference monetary value of a person-Sievert?”
• others?

5.4 Question 4: Are there alternative ways of obtaining information on costs and benefits?

P-242 permits stakeholders to produce and submit relevant cost-benefit information to the CNSC for consideration. Some stakeholders may not have the capacity to develop detailed cost-benefit analyses themselves. For example, many of the CNSC’s licensees are small businesses or individuals for whom performing detailed analyses may not be feasible. Are there areas of CNSC’s regulatory framework or areas of regulatory policy in which the CNSC should consider proactively undertaking cost-benefit analysis, to better inform the delivery of our mandate?

5.5 Question 5: Should the CNSC identify specific program areas in which the submission of a formal cost-benefit analysis by the applicant should be considered?

The CNSC’s regulatory approach is to allow licensees considerable flexibility, wherever possible, in how they comply with requirements. In general, the CNSC reviews what an applicant or licensee proposes to do, and assesses whether the proposed approach will meet requirements and safety objectives. Weighing the costs and benefits of alternative approaches is the domain of the proponent. However, in some instances licensees are encouraged to share a detailed assessment of different alternatives with the CNSC (e.g. when considering mine waste disposal options as specified in RD/GD-370, or assessing alternative means of completing a major project under the Canadian Environmental Assessment Act, 2012).

There may be other areas where submission of formal, structured, cost-benefit information (e.g. CBA, CEA or MCDA) would be beneficial. Should the CNSC specify additional areas in a regulatory document? For example, these areas might include:

• when applicants are selecting a decommissioning strategy
• when applicants are choosing between options for the design of a facility for which there is high community interest

5.6 Question 6: Did we miss anything?

The CNSC welcomes any other feedback that stakeholders would like it to consider while it reviews the need to update P-242 and replace it with a new regulatory document within series 3.5 of the CNSC’s regulatory framework.

Glossary

cost-benefit analysis (CBA)

Cost-benefit analysis is a form of structured decision analysis based on the theory of micro-economics. Well-developed accounting practices for deriving reliable and accurate costs and benefits have been developed and standardised. Social cost-benefit analysis has emerged to broaden conventional measures of costs and benefits and to encompass many non-market goods and services that are valued by people (e.g., public health, ecological goods and services). Specialized methods have been developed for deriving proxy prices for these non-market values, facilitating the aggregation of all costs and benefits to arrive at a comprehensive net benefit bottom line. The alternative that yields the greatest net benefit is generally determined to be the best alternative.

cost effectiveness analysis (CEA)

Cost effectiveness analysis is a variant of cost-benefit analysis. With CEA, the range of alternatives is confined to those that will achieve a predetermined outcome (e.g., a regulated level of pollution emission). The goal of CEA is to find the alternative that will satisfy the predetermined outcome at the least cost. The alternative that satisfies the predetermined outcome at the least cost is generally determined to be the preferred alternative.

multi-criteria decision analysis (MCDA)

Multi-criteria decision analysis is a form of structured decision analysis derived from the discipline of operations research. A number of different MCDA methodologies have been developed and a large literature on the various methods and their application is available. Most MCDA methods rely on participatory processes to derive relative importance assessments (i.e., weights) for individual decision criteria. The alternative that yields the best balance of advantages and disadvantages is generally determined to be the preferred alternative.

valuation

With CBA, costs and benefits are summed to derive a net benefit. With CEA, just costs are summed. Valuation is the process of deriving prices (i.e., relative significance measures) for each cost and benefit. With conventional marketed goods and services, market prices are generally used to estimate their value. With non-market goods and services (e.g., many human health and environmental costs and benefits), their value is derived using stated or revealed preference methods.

weighting

With MCDA, advantages and disadvantages are compared and combined to arrive at a preferred alternative. Weighting is the process of deriving weights (i.e., relative significance measures) for each decision criterion. Weights are typically derived through a participatory process.