Audit of the Directorate of Nuclear Substance Regulation Inspection Processes
Table of contents
Executive summary
The Canadian Nuclear Safety Commission (CNSC) has a mandate, under the Nuclear Safety and Control Act (NSCA), to regulate all nuclear facilities and nuclear-related activities in Canada. Licensing and certification activities are in place to issue licences or certify persons and prescribed equipment involved in nuclear-related activities. The Directorate of Nuclear Substance Regulation (DNSR) is responsible for verifying that nuclear substances and prescribed equipment are used in accordance with regulatory requirements. The Operations Inspection Division (OID) and the Accelerators and Class II Facilities Division (ACFD) within DNSR assess licensee compliance through desktop reviews and Type I and Type II inspections.
The Office of the Auditor General of Canada (OAG) issued the 2016 Fall Reports of the Commissioner of the Environment and Sustainable Development Report 1—Inspection of Nuclear Power Plants—Canadian Nuclear Safety Commission (the “OAG Audit Report”). The OAG Audit Report included five recommendations. The CNSC agreed with each of the OAG Audit Report recommendations and provided a response that detailed actions the CNSC had taken or intended to take to address the findings.
The following are the five recommendations outlined in the OAG audit report.
- The CNSC should develop a systematic and risk-informed planning process that is well documented and includes a five-year baseline inspection plan and the minimum required frequency and type of inspections (Recommendation 1, paragraph 1.33 of the OAG report)
- The CNSC should develop detailed criteria to identify when to conduct Type I inspections (Recommendation 2, paragraph 1.35 of the OAG report)
- The CNSC should ensure that its inspections follow its own procedures (Recommendation 3, paragraph 1.48 of the OAG report)
- The CNSC should document its lessons learned in carrying out inspections (Recommendation 4, paragraph 1.50 of the OAG report)
- The CNSC should issue timely final inspection reports (Recommendation 5, paragraph 1.61 of the OAG report)
In the summer of 2016, the CNSC President and Executive Committee instructed all non-nuclear power plant directorates (i.e., those not within the scope of the OAG audit) that conduct licensee inspections to address the OAG Audit Report recommendations as they relate to their inspection programs.
The objective of the audit was to determine the adequacy of the implementation of the actions taken by DNSR to respond to each of the five recommendations in the OAG Audit Report. There was limited opportunity to assess the implementation and effectiveness of some recently-completed actions by DNSR. An assessment of implementation and effectiveness of recently-completed actions will require a focused follow-up audit or specified audit procedures in a year or two, subject to Risk-Based Audit Plan priorities.
DNSR completed a DNSR Compliance Program Self-Assessment (the “self-assessment”) of the Directorate’s licensee compliance program using the same criteria used in the 2016 OAG audit report. DNSR has been making continuous improvement to their inspection operations since 2017 as part of their self-assessment and in order to address the OAG Audit Report recommendations. The self-assessment noted that the OAG Audit Report criteria were used with minor changes to reflect the context of the compliance activities performed in DNSR. The self-assessment included an analysis for each in-scope DNSR division, breaking down each OAG Audit Report recommendation into subtopics (appendix C), and included improvements by OID and ACFD.
The following is a summary of audit findings by Division.
OID
The audit found that OID took actions to address self-assessed improvements in conducting inspections (OAG Recommendation 3) and completing inspection reports in a timely manner (OAG Recommendation 5), and recently implemented improvements to the lessons-learned process (OAG Recommendation 4). The audit identified opportunities for further improvement in the planning for inspections (OAG Recommendation 1) and developing detailed criteria for Type I inspections (OAG Recommendation 2).
The following are the audit findings related to OID’s response to each of the OAG Audit Report recommendations:
1) Develop a systematic and risk-informed planning process that is well documented and includes a five-year baseline inspection plan and the minimum required frequency and type of inspections (Recommendation 1, paragraph 1.33 of the OAG Audit Report)
OID determined that a systematic, multi-year inspection plan was not applicable to the Division, but it did identify opportunities for improvement in planning. OID developed instructions for annual inspection planning, guidance on conducting inspections and a methodology to identify inspection type. OID identified a gap in inspection performance and analyzed resources. It was not possible to assess the risk-informed program review of performance, as it was a work in progress.
2) Develop detailed criteria to identify when to conduct type I inspections (Recommendation 2, paragraph 1.35 of the OAG Audit Report)
The audit found that OID had developed criteria for identifying Type I inspections and incorporated them in the process document; however, the criteria did not appear to be implemented. OID only partially used the criteria in its process for selecting Type I inspection candidates. Further, the audit found that the selected Type I candidates could not be tracked to the ranked list of candidates.
3) Ensure that inspections follow approved procedures (Recommendation 3, paragraph 1.48 of the OAG Audit Report)
The audit found that OID’s worksheet guidance and inspection procedures were developed and implemented at an appropriate level within the Division, providing an adequate response to the OAG Audit Report recommendations.
4) Document lessons learned in carrying out inspections (Recommendation 4, paragraph 1.50 of the OAG Audit Report)
OID addressed lessons learned through formal internal meetings, providing an adequate response to the OAG Audit Report recommendations. As OID rolled out a survey in September 2018, it was not possible to determine whether the lessons-learned process for licensees and other stakeholders was fully implemented.
5) Issue timely final inspection reports (Recommendation 5, paragraph 1.61 of the OAG Audit Report)
OID determined that the existing processes at a division level sufficiently met both the Type I and Type II performance business standards for issuing final inspection reports and did not propose any recommendations for improvements associated with the timeliness of issuing final inspection reports.
ACFD
The audit found that ACFD took actions to address self-assessed improvements to the lessons-learned process (OAG Recommendation 4) and recently implemented improvements in planning for inspections (OAG Recommendation 1), conducting inspections (OAG Recommendation 3) and completing inspection reports in a timely manner (OAG Recommendation 5). The audit identified opportunities for further improvement in developing detailed criteria for Type I inspections (OAG Recommendation 2).
The following are the audit findings related to ACFD’s response to each of the OAG Audit recommendations:
1) Develop a systematic and risk-informed planning process that is well documented and includes a five-year baseline inspection plan and the minimum required frequency and type of inspections (Recommendation 1, paragraph 1.33 of the OAG Audit Report)
ACFD addressed the OAG Audit Report recommendation in its regulatory program oversight document. As the update to ACFD’s regulatory program oversight document was published in July 2018, it was not possible to determine whether the procedures were fully implemented.
2) Develop detailed criteria to identify when to conduct type I inspections (Recommendation 2, paragraph 1.35 of the OAG Audit Report)
The audit found that ACFD did not consistently use their criteria and processes for determining the inspection type and the process. Further, the audit found that the justification for selecting the inspection type was not consistently used and adequately documented.
3) Ensure that inspections follow approved procedures (Recommendation 3, paragraph 1.48 of the OAG Audit Report)
ACFD’s regulatory program oversight was made available to Division staff in January 2018. As the update to ACFD inspection procedures were published in July 2018, it was not possible to determine whether the procedures were fully implemented.
4) Document lessons learned in carrying out inspections (Recommendation 4, paragraph 1.50 of the OAG Audit Report)
ACFD determined that the existing processes at a division level were sufficient and did not propose any recommendations for improvement associated with the lessons-learned process for inspections.
5) Issue timely final inspection reports (Recommendation 5, paragraph 1.61 of the OAG Audit Report)
ACFD addressed the Audit Report recommendation in the Division’s inspection procedures. As the update to ACFD’s inspection procedures were published in July 2018, it was not possible to determine whether the procedures were fully implemented.
The OAE proposes that a focused audit or specified audit procedures be conducted to assess the implementation and effectiveness of DNSR inspection processes in a year or two.
Introduction
Background
The Canadian Nuclear Safety Commission (CNSC) regulates the use of nuclear energy and materials to protect health, safety, security and the environment; implement Canada’s international commitments on the peaceful use of nuclear energy; and disseminate objective scientific, technical and regulatory information to the public. The Directorate of Nuclear Substance Regulation (DNSR) is responsible for verifying that accelerators, Class II nuclear facilities and Class II prescribed equipment are safely operated. The Operations Inspection Division (OID) and the Accelerators and Class II Facilities Division (ACFD) within DNSR determine licensee compliance through desktop reviews and Type I and Type II inspections.
The CNSC regulates the development, production and use of nuclear energy and materials under the Nuclear Safety and Control Act (NSCA) so that the environment and the health, safety, and security of Canadians are protected, and Canada’s international commitments on the peaceful use of nuclear energy are implemented.
Inspections serve as one of the key tools the CNSC uses for the purpose of determining whether a licensee is in compliance with the requirements of the regulatory framework. DNSR conducts regulatory inspections of licensees which culminate in inspection reports to the licensees.
DNSR had 31 active certified inspectors and 6 inspectors in training in 2018 19. The CNSC 2017 18 Annual Report indicated that DNSR conducted 820 regulatory inspections and compliance reviews in 2017 18 of licensees under the Directorate`s Nuclear Substances and Prescribed Equipment Program.
The CNSC site inspection process is as follows:
- Plan inspection, which includes confirming scope, compiling inspection criteria and reviewing prior reports for outstanding issues
- Notify licensee
- Start field inspection and collect facts
- Analyze inspection facts and develop preliminary findings
- Communicate preliminary findings to licensee
- Conduct final analysis
- Prepare and issue final inspection report to licensee
- Follow up on non-compliance
The Office of the Auditor General of Canada (OAG) issued the 2016 Fall Reports of the Commissioner of the Environment and Sustainable Development Report 1—Inspection of Nuclear Power Plants—Canadian Nuclear Safety Commission (the “OAG Audit Report”). The OAG Audit Report focused on whether the CNSC had adequately managed its site inspections of Canadian nuclear power plants to verify that the environment and the health, safety, and security of Canadians were protected. More specifically, the OAG audit examined whether the CNSC adequately planned for and carried out site inspections of nuclear power plants. The OAG audit also examined whether the CNSC applied enforcement measures to ensure that the deficiencies it identified were corrected to comply with regulatory and licence requirements. The OAG audit focused on the CNSC’s management of its site inspections, and not on the overall safety of nuclear power plants in Canada.
The OAG Audit Report included five recommendations in relation to three areas of audit findings, namely planning inspections, conducting inspections and enforcing compliance with regulatory and licence requirements (appendix A). The CNSC agreed with each of the OAG Audit Report recommendations and provided a response that detailed actions the CNSC had taken or intended to take to address the findings.
In the summer of 2016, the CNSC President and Executive Committee instructed all non-nuclear power plant directorates that conduct licensee inspections (i.e., directorates that did not fall within the scope of the OAG audit) to address the OAG Audit Report recommendations as they relate to their inspection processes.Authority
The audit was part of the CNSC’s approved Risk-Based Audit Plan for 2018 19 to 2020 21 as the audit of the management of inspection processes (non-nuclear power plants Phase II) DNSR.
Audit objective, scope and approach
The objective of the audit was to determine the adequacy of the implementation of actions taken by DNSR to respond to each of the five recommendations in the OAG Audit Report. There was limited opportunity to assess the implementation and effectiveness of some actions DNSR recently completed. An assessment of the implementation and effectiveness of recently completed actions will require a focused follow-up audit or specified audit procedures in a year or two, subject to Risk-Based Audit Plan priorities. The audit focused on management’s assessment, response and implementation of actions taken to address the OAG Audit Report recommendations, and not on the effectiveness of the inspection processes.
The audit considered information from DNSR’s management action plans and deliverables from the time of the Directorate’s 2018 self-assessment, completed in March 2018, to the end of September 2018.
The audit approach included but was not limited to the following:
- Conduct interviews with key stakeholders, including senior management and staff
- Review and analyze documentation related to DNSR’s 2018 documented self-assessment related to the 2016 OAG Audit Report recommendations
Statement of conformance
The audit was conducted in conformance with the Internal Auditing Standards for the Government of Canada and was supported by the OAE’s quality assurance and improvement program.
Acknowledgement
The OAE would like to acknowledge and thank management and staff for their support throughout the conduct of this audit.
Audit observations and recommendations
DNSR performed a DNSR Compliance Program Self-Assessment(the “self-assessment”) of the Directorate’s licensee compliance program using the same criteria used in the OAG audit report. The self-assessment noted that the OAG Audit Report criteria were used with minor changes to reflect the context of the compliance activities performed in DNSR.
The self-assessment completed in March 2018 included an analysis for each in-scope DNSR division breaking down each OAG Audit Report recommendation into subtopics (appendix C) and including improvements for OID and ACFD.
The assessment was performed at a service line level by senior Division staff members and designated inspectors in OID and ACFD. The assessment verified the adequacy of compliance processes based on a desktop review of inspection reports and the approved management system documents.
The audit included an assessment of DNSR’s response and deliverables by OID and ACFD for each of the OAG Audit Report recommendations.1) Develop a systematic and risk-informed planning process that is well documented and includes a five-year baseline inspection plan and the minimum required frequency and type of inspections (Recommendation 1, paragraph 1.33 of the OAG Audit Report)
OID
OID determined that a five-year safety and control area (SCA) planning cycle was not applicable like it is with nuclear power plants, as OID included all applicable SCAs when verifying compliance. Further, OID determined that DNSR conducted a periodic review of its risk assessment by use type that was an equivalent process that better suits DNSR licensees. The audit found that OID licensees were such that all SCAs were included in the scope of each licensee inspection. OID assessment noted that the last review of the Risk Informed Regulatory Program was in 2014, and it resulted in adjustments to some risk levels for certain use types and a recommended change to the baseline inspection frequency.
OID identified an opportunity to improve its inspection planning processes. OID determined that it needed to adequately analyze and assess the reason some planned inspections were not conducted during the year. OID also found that a staffing assessment was needed to determine whether the Division had the appropriate number and levels of staff to carry out the planned number of inspections.
The audit found that the Director, OID was responsible for creating and monitoring the annual inspection plan through performance meetings. A tracking tool was used to report on inspection dates and status of completion in the annual plan and documentation on inspections that were not completed.
OID assessment noted that the Division could not conduct the recommended minimum number of inspections for each medium-risk use type. OID identified challenges with ensuring adequate staff to conduct inspections, affected by the number of licensees, the number of planned inspections and the logistics and travel to the number of disparate locations across Canada. As a result, OID completed a resourcing analysis for inspections for 2018 23 that reported the Division’s capacity to conduct the baseline number of inspections over the next several years. OID also deemed necessary and planned a longer-term, strategic program review of performance and professional development, including an assessment of the baseline of inspections, and staffing and recruiting needs. The audit found that the strategic program review was a work in progress.
Conclusion
OID determined that a systematic, multi-year inspection plan was not applicable to the Division and identified opportunities for improvement in planning. OID developed plans to address the gaps, including the development of instructions for annual inspection planning, guidance on conducting inspections and a methodology for determining inspection type. OID identified a gap in inspection performance, analyzed resources and initiated development of a risk-informed program review of performance and professional development. The OAE proposes that a focused audit be conducted or specified audit procedures be followed to assess the implementation and effectiveness of planning processes in a year or two.
Recommendation 1
It is recommended that the Director General of DNSR take appropriate action to ensure the review of its risk-informed regulatory program is completed in order to develop a multi-year resource management plan that includes inspections.
Management response and action plan
Management agrees.
DNSR has completed a resource needs analysis and is in the process of completing the quinquennial review of its risk-informed regulatory program.
These will be integrated to develop a multi-year resource management plan for the division that takes into account compliance needs for the industry.
Target completion date: June 2019
(See appendix D: Audit recommendations and management action plans)
ACFD
ACFD assessment identified a need for a well-documented planning procedure and found that the planning process lacked clear criteria when determining the type of inspection to be conducted.
ACFD planned to develop inspection process guidance in the Division’s regulatory program oversight document, including procedures to determine the type and frequency of inspections.
The audit found that the regulatory program oversight document provided guidance on the frequency of inspections, which alternated between Type I and Type II inspections and was based on the relative risk ranking of SCAs by use type. The audit also found that ACFD published Type I and Type II inspection procedures in July 2018. ACFD developed a five-year baseline plan that documented a rolling five-year compliance plan, detailing mandatory inspections and the frequency of inspections, based on priority level.
In terms of resource management, ACFD noted that a resource analysis was conducted at a division level to assess capacity for inspections. The audit found that ACFD resources were assigned and allocated using the workload management tool.
Conclusion
ACFD assessed a gap in the Directorate’s inspection planning and tracking process and addressed it in its regulatory program oversight document. As the update to the ACFD regulatory program oversight document was published in July 2018, it was not possible to determine whether the procedures were fully implemented. The OAE proposes that a focused audit be conducted to assess the implementation and effectiveness of the inspection planning process.
2) Develop detailed criteria to identify when to conduct Type I inspections (Recommendation 2, paragraph 1.35 of the OAG Audit Report)
OID
In its assessment of the OAG Audit Report recommendations, OID identified an opportunity to review and update Type I inspection triggers and provide guidance on when a Type I inspection should be recommended. OID also identified a need to facilitate the annual planning process for the resources required for inspections.
The OID self-assessment identified various triggers for a Type I inspection. The OID self-assessment revealed that Type I inspections were not included in the Division’s baseline inspection plan; rather, Type I inspections were conducted when triggered by an event, as defined by the criteria. The self-assessment found that the triggers identified as part of the criteria were not detailed and did not provide sufficient guidance to consistently identify the need for a Type I inspection.
It its self-assessment, OID planned to review and update Type I inspection triggers in the Division’s guidance document for selecting an inspection type. OID also planned to maintain a list of potential Type I inspection candidates as input into the annual planning process.
The audit found that criteria for identifying Type I inspections had been developed in a memorandum that was referenced in the Division’s guidance document for selecting an inspection type. The memorandum provided clear guidance for recommending Type I inspections to management. The revised OID guidance document for selecting an inspection type was communicated to Division staff in January 2018.
Guidance for selecting potential licensees for Type I inspections used a three-phase process where licensees are analyzed, evaluated and ranked to identify the highest priority licensees. The three-phase process is as follows:
-
Phase I creates an initial list of licensees using the complexity criterion
- Step 1 uses criteria based on the number of locations (three or more)
- Step 2 uses the number of licenses held by a licensee
- Step 3 uses a use type risk identifier associated with the licensee (all use-type licenses associated with the licensee are High, High or Medium or Low, and Medium or Low
- Phase 2 analyzes the licensees’ risk profile by using standardized elements to ensure that the analysis is consistent and reliable (the more elements that apply, the higher the risk profile)
- Phase 3 uses the complexity criterion to rank licensees using the elements scored from highest to lowest
The guidance indicates that professional judgement is to be used to select the final planned Type I inspections from the list of candidates.
The audit found that OID developed a tool that identified Type I inspections planned for 2018 19; however, the criteria in the guidance for selecting Type I inspections was not fully captured in the tool, so OID was unable to provide a ranked list of Type I candidates for 2018 19. OID explained that the candidates on the list of 2018 19 were selected based on the inspectors’ professional judgment and knowledge, and did not make full use of the criteria in OID guidance.
Conclusion
OID developed selection criteria and a process for selecting inspection types. The process for determining the Type I inspection candidates for 2018 19 was not fully implemented, as the selection criteria and prescribed process were not used for providing a ranked list of Type I candidates and selecting Type I inspections.
ACFD
ACFD determined that there was no formal criteria for selecting the inspection type, and that the Division-level criteria for conducting a Type I inspection were of minor significance. ACFD developed a management action plan to formally document the criteria for selecting an inspection.
The audit found that ACFD published a regulatory program oversight document January 2018 that provided guidance on the frequency of inspections. The oversight document included guidance on developing a rolling five-year baseline plan, detailing mandatory inspections and determining the frequency of inspections, based on priority level. The guidance in the regulatory program oversight document on selecting an inspection type included the following for certain licensees:
- the pattern typically alternates between Type I and Type II inspections
- specific Type II inspections are planned coincident with the general inspection
- where the scope of operations and inherent risks are minimal, the inspection frequency may be reduced and/or the type of inspection may be limited to Type II only, as determined on a case-by-case basis
Examples of criteria for reducing inspection frequency or scope included:
- an acceptable or exceptional compliance performance history
- limited level of operations related to the prescribed equipment or conduct of the licensed activity
- limited radiological risks
ACFD indicated that a Type I inspection should be conducted at least once every five years for each licensee. ACFD also indicated that Type I inspections were conducted for any new licensees. The ACFD assessment indicated that a Type I inspection may be scheduled if a Type II inspection indicates possible systemic issues. These criteria for conducting a Type I inspection were not included in the program oversight document guidance.
The purpose of these combined descriptions was to provide a common understanding for selecting a Type I or Type II inspection. The audit noted that the use of the above documented and undocumented criteria makes it difficult to determine the inspection type that should be selected. The audit noted that there are opportunities to add more comprehensive definitions for the criteria. The criteria above contain phrases such as “scope of operations”, “inherent risk” and “potential systemic issues” that should be defined in detail.
Conclusion
The audit found that the criteria and process for determining the inspection type were not consistently documented. Further, the audit found that the process and justification for selecting the inspection type were not consistently used.
Recommendation 2
It is recommended that the Director General of DNSR take appropriate action to ensure that the process for selecting Type I inspections includes clearly defined criteria for selecting an inspection type and the process results in a clearly documented justification for the selection.
Management response and action plan
Management agrees.
The Director General of DNSR agrees with this recommendation.
Criteria exist for the selection of Type I inspections in OID, and planning procedures will be updated to indicate the need to document their application. ACFD regulatory oversight document will be revised to specify criteria for off-baseline selection of Type I inspections.
Target completion date: December 2018
(See appendix D: Audit recommendations and management action plans)
3) Ensure that inspections follow approved procedures (Recommendation 3, paragraph 1.48 of the OAG Audit Report)
OID
In its assessment of the OAG Audit Report recommendations, OID identified an opportunity to conduct a regular review of the Division’s internal guidelines and procedures to bring them up to date with changes in the regulatory environment.
OID planned to review and update the Division’s inspection worksheet guidance documents to make them consistent with changes in compliance expectations.
OID also planned to review and update the Division’s inspection procedure document to ensure the Division’s inspection procedures were aligned with the CNSC corporate-level inspection guidance, consistent with changes to certain OID inspection practices and including expectations on document retention.
The audit found that OID updated its series of inspection-related worksheet guides and the OID inspection procedure. Each worksheet guide was updated to align the expectations from the Nuclear Substances and Radiation Devices Licensing Division (NSRDLD) and OID working group. The audit found that the OID inspection procedure document was updated to include recent changes to OID inspection practices. The retention expectations for the OID inspection procedure document were included in OID’s inspection procedures.
Conclusion
The audit found that OID’s updated worksheet guidance and inspection procedure document were implemented. The audit also found that the worksheet guidance and OID inspection procedures were developed and implemented at an appropriate level within the Division, providing an adequate response to the OAG Audit Report recommendations.
ACFD
In its assessment of the OAG Audit Report recommendations, ACFD identified a need to conduct a regular review and update of the Division’s internal procedures and guidelines.
ACFD planned to review and update the Division’s inspection procedures to make them consistent with changes in compliance expectations, specifically in relation to inspection document retention and Type I and Type II ACFD inspection procedures. ACFD also planned to include inspection planning guidance in the Division’s regulatory program oversight document, including procedures for changing the baseline inspection frequency.
The audit found that ACFD updated its inspection procedures in relation to Type I and Type II inspections and included expectations regarding inspection document retention. The audit found that ACFD published its regulatory program oversight document in July 2018 to communicate how management would oversee the regulatory program in ACFD. The audit also found that the ACFD program oversight document provided guidance on inspection planning, including procedures for changing inspection frequency.
Conclusion
The regulatory program oversight was made available to Division staff in January 2018. As the update to ACFD inspection procedures were published in July 2018, it was not possible to determine whether the procedures were fully implemented. The OAE proposes that a focused audit be conducted to assess the implementation and effectiveness of the inspection process.
4) Document lessons learned in carrying out inspections (Recommendation 4, paragraph 1.50 of the OAG Audit Report)
OID
In its assessment of the OAG Audit Report recommendations, OID identified an opportunity to incorporate feedback received on its inspection program from licensees and other stakeholders and an opportunity to improve its internal lessons-learned process.
OID developed a plan to conduct a formal review of the feedback received on its inspection program from licensees and other stakeholders, looking for improvement opportunities. OID also planned to develop terms of reference for monthly inspector meetings that included specific reference to lessons learned in conducting inspections.
The audit found that OID rolled out a licensee feedback survey focused on licensee inspections. The survey addressed a broad range of topics associated with the inspection process and provided the reader an opportunity for comments throughout the survey. OID also published terms of reference for the OID Inspector Meeting Group.
Conclusion
The terms of reference for the OID Inspector Meeting Group were published in June 2017 and the corresponding lessons learned from those meetings were reviewed. The survey was rolled out in September 2018 which did not allow sufficient time to determine whether the lessons-learned process for licensees and other stakeholders was fully implemented. The OAE proposes that a focused audit be conducted to assess the implementation and effectiveness of the inspection lessons-learned process.
ACFD
In its assessment of the OAG Audit Report recommendations, ACFD noted that lessons learned were brought forward during monthly technical review meetings at which a presentation on each and every inspection conducted was made by the inspection teams. ACFD noted that presentations were retained as part of the inspection record, and further, that action was being taken to resolve issues related to improving the inspection process during weekly ACFD operations meetings.
ACFD determined that the existing processes at a division level were sufficient to mitigate risks associated with lessons learned to an acceptably low level; therefore, no actions were taken as a direct result of the OAG Audit Report recommendations. ACFD did not propose any recommendations for improvement associated with the lessons-learned process for inspections.
Conclusion
The audit found that ACFD adequately assessed the OAG Audit Report recommendation related to the lessons-learned process.
5) Issue timely final inspection reports (Recommendation 5, paragraph 1.61 of the OAG Audit Report)
OID
In its assessment of the OAG Audit Report recommendations, OID found that the Division was meeting both the Type I and Type II performance business standards for issuing inspection reports. OID noted that 41 of 44 (93%) Type I final inspection reports were issued within 60 business days (target of 80%) and 4,643 of 4,947 (94%) Type II final inspection reports were issued within 40 business days (target of 80%) between 2013-14 and the second quarter of 2016 17.
OID determined that the existing processes at a division level sufficiently met both the Type I and Type II performance business standards for issuing inspection reports; therefore, no actions were taken as a direct result of the OAG Audit Report recommendation. OID did not propose any recommendations for improvement associated with the timeliness of issuing final inspection reports.
Conclusion
The audit found that OID adequately assessed the OAG Audit Report recommendation related to the timeliness of final inspection reports.
ACFD
In its assessment of the OAG Audit Report recommendations, ACFD identified an area for improvement in facilitating performance metrics. ACFD developed a plan to clarify the compliance program procedures.
The audit found that the gap was addressed using a management tool that was embedded within ACFD inspection procedures, published in July 2018.
Conclusion
Because the update to ACFD inspection procedures was published in July 2018, it was not possible to determine whether the procedures were fully implemented. The OAE proposes that a focused audit be conducted to assess the implementation and effectiveness of the process to issue final inspection reports.
Overall conclusion
The audit found that DNSR assessed each of the five recommendations in the OAG Audit Report. DNSR developed management action plans and deliverables in response to gaps, and the full implementation of deliverables will ensure an adequate response.
The audit identified opportunities for further improvement in the process for selecting inspection types. The audit also identified an opportunity for OID to develop and implement multi-year plans to sustain its program for compliance activities, including the baseline number of inspections and staffing and recruiting needs.
1) Develop a systematic and risk-informed planning process that is well documented and includes a five-year baseline inspection plan and the minimum required frequency and type of inspections (Recommendation 1, paragraph 1.33 of the OAG Audit Report)
Although OID determined that a systematic, multi-year inspection plan was not applicable to the Division, it did identify opportunities for improvement in planning. OID developed instructions for annual inspection planning, guidance on conducting inspections and a methodology to determine inspection type. OID identified a gap in inspection performance, analyzed resources and initiated the development of a risk-informed program review of performance.
ACFD deliverables addressed the Audit Report recommendation to ensure a systematic and risk-informed planning process that is well documented; however, given that the update to the ACFD regulatory program oversight document was published in July 2018, it was not possible to determine whether the procedures were fully implemented.
2) Develop detailed criteria to identify when to conduct type I inspections (Recommendation 2, paragraph 1.35 of the OAG Audit Report)
OID and ACFD did not adequately address the OAG Audit Recommendation by developing, consistently using and documenting the justification for the inspection type selected.
3) Ensure that inspections follow approved procedures (Recommendation 3, paragraph 1.48 of the OAG Audit Report)
OID adequately addressed the OAG Audit Report recommendation to ensure that inspections follow approved procedures.
ACFD deliverables addressed the OAG Audit Report recommendation to ensure that inspections follow approved procedures; however, as the update to ACFD inspection procedures was published in July 2018, it was not possible to determine whether the procedures were fully implemented.
4) Document lessons learned in carrying out inspections (Recommendation 4, paragraph 1.50 of the OAG Audit Report)
The OID deliverables addressed the Audit Report recommendation related to lessons learned; however, as OID rolled out a survey in September 2018, it was not possible to determine whether the lessons-learned process for licensees and other stakeholders was fully implemented.
ACFD adequately addressed the OAG Audit Report recommendation related to the lessons-learned process for inspections.
5) Issue timely final inspection reports (Recommendation 5, paragraph 1.61 of the OAG Audit Report)
OID adequately addressed the OAG Audit Report recommendation to issue timely final inspection reports.
ACFD deliverables addressed the Audit Report recommendation to issue timely final inspection reports; however, given that the updates to ACFD inspection procedures were published in July 2018, it was not possible to determine whether the procedures were fully implemented.
Acronyms
The following table presents acronyms used in this document.
| ACFD | Accelerators and Class II Facilities Division |
|---|---|
| CESD | Commissioner of the Environment and Sustainable Development |
| CNSC | Canadian Nuclear Safety Commission |
| DAC | Departmental Audit Committee |
| DNSR | Directorate of Nuclear Substance Regulation |
| NSCA | Nuclear Safety and Control Act |
| NSRDLD | Nuclear Substances and Radiation Devices Licensing Division |
| OAG | Office of the Auditor General of Canada |
| OAE | Office of Audit and Ethics |
| OID | Operations Inspection Division |
| SCA | Safety and control area |
Appendix A: Lines of enquiry and audit criteria
The OAE used both OAG recommendations as the basis for the criteria in its audit.
|
1. Line of Enquiry 1 Assess the adequacy of DNSR deliverables stemming from the OAG Audit Report recommendations that have been implemented. |
|
Audit criteria |
|
|
|
2. Line of Enquiry 2 Where the deliverables related to the OAG Audit Report recommendations have not been implemented, assess the adequacy, completeness and timeliness of DNSR plans for responding to the OAG Audit Report recommendations. |
|
Audit criteria |
|
|
Appendix B: OAG Audit Report recommendations
The following table is an excerpt from the findings and recommendations in the OAG Audit Report. The number(s) in the table indicate the paragraph where each recommendation appears in the OAG Audit Report.
| Findings | Recommendations |
|---|---|
|
Planning inspections The Canadian Nuclear Safety Commission could not show that it had an adequate, systematic, risk-informed process for planning site inspections at nuclear power plants. |
1.33 The Canadian Nuclear Safety Commission should develop and implement a well-documented planning process for site inspections of nuclear power plants that can demonstrate that the process is systematic and risk-informed. This should include determining the minimum required frequency and type of inspections needed to verify compliance, updating the five-year baseline inspection plan and assessing whether it is assigning the appropriate number and levels of staff to carry out the number of inspections required to verify compliance. |
|
1.35 The Canadian Nuclear Safety Commission should develop detailed criteria to help it identify when to conduct Type I inspections. |
|
|
Conducting inspections The Canadian Nuclear Safety Commission did not always follow its own inspection procedures. |
1.48 The Canadian Nuclear Safety Commission should ensure that its inspections follow its own procedures. This requires that it develop approved inspection guides with appropriate criteria before conducting inspections to assess that nuclear power plants are complying with applicable regulatory and licence requirements. The Canadian Nuclear Safety Commission should also clearly explain to its staff how to decide which documents should be considered transitory and which documents should be retained after they issue inspection reports. |
|
1.50 The Canadian Nuclear Safety Commission should ensure that it documents lessons learned in carrying out its inspections to help it make continuous improvements to its inspection practices. |
|
|
Enforcing compliance with regulatory and licence requirements The Canadian Nuclear Safety Commission followed up to confirm that nuclear power plants corrected compliance violations it identified, but did not always issue final reports on time. |
1.61 The Canadian Nuclear Safety Commission should determine why it does not issue timely final inspection reports and decide whether it needs to make any changes to its processes or standards. |
Appendix C: DNSR self-assessment
DNSR performed its self-assessment, defining subtopics related to the OAG Audit Report recommendations as follows:
| OAG Audit Report recommendation | Subtopics defined by DNSR self-assessment |
|---|---|
| Planning Inspections | |
|
The Canadian Nuclear Safety Commission should develop and implement a well-documented planning process for the site inspections of nuclear power plants that can demonstrate that the process is systematic and risk-informed. This should include determining the minimum required frequency and type of inspections needed to verify compliance, updating the five-year baseline inspection plan and assessing whether it is assigning the appropriate number and levels of staff to carry out the number of inspections required to verify compliance. |
a) Planning process is systematic and risk-informed |
b) Five-year baseline inspection plan is updated |
|
c) Minimum required frequency and type of inspections needed have been determined |
|
|
d) Assessment is conducted to determine whether appropriate number and levels of staff have been assigned to carry out the inspections |
|
|
The Canadian Nuclear Safety Commission should develop detailed criteria to help it identify when to conduct Type I inspections. |
e) Detailed criteria have been developed to identify when Type I inspections should be conducted |
| Conducting Inspections | |
|
The Canadian Nuclear Safety Commission should ensure that its inspections follow its own procedures. This requires that it develop approved inspection guides with appropriate criteria before conducting inspections to assess that nuclear power plants are complying with applicable regulatory and licence requirements. The Canadian Nuclear Safety Commission should also clearly explain to its staff how to decide which documents should be considered transitory and which documents should be retained after they issue inspection reports. |
a) Inspections follow internal procedures |
b) Approved inspection guides are developed with appropriate criteria |
|
|
c) Explanation is given to staff on how to decide which documents should be considered transitory and which documents should be retained after they issue inspection reports |
|
|
The Canadian Nuclear Safety Commission should ensure that it documents lessons learned in carrying out its inspections to help it make continuous improvements to its inspection practices. |
d) Lessons learned in carrying out inspections are documented |
| Enforcement Compliance | |
|
The Canadian Nuclear Safety Commission should determine why it does not issue timely final inspection reports and decide whether it needs to make any changes to its processes or standards. |
a) Timely final inspection reports are issued |
b) A decision is made as to whether any changes are needed to its processes or standards |
Appendix D: Audit recommendations and management action plans
The following table presents recommendations made in audit observations and recommendations and the associated management action plans provided by DNSR.
| Recommendations | Management response and action plan | Target completion date |
|---|---|---|
|
1. It is recommended that the Director General of DNSR take appropriate action to ensure the review of its risk-informed regulatory program is completed in order to develop a multi-year resource management plan that includes inspections. |
Management agrees. DNSR has completed a resource needs analysis and is in the process of completing the five-year review of its risk-informed regulatory program. These will be integrated to develop a multi-year resource management plan for the Division that takes into account compliance needs for the industry. |
June 2019 |
|
2. It is recommended that the Director General of DNSR take appropriate action to ensure the process for selecting Type I inspections includes clearly defined criteria for selecting an inspection type and the process results in clearly documented justification for the selection. |
Management agrees. Criteria exist for the selection of Type I inspections in OID, and planning procedures will be updated to indicate the need to document their application. ACFD regulatory oversight document will be revised to specify criteria for off-baseline selection of Type I inspections. |
December 2018 |
Page details
- Date modified: