REGDOC-2.2.3: Personnel Certification: Initial Certification Examinations

1. Purpose

This regulatory document provides requirements and guidance regarding initial certification examinations that persons seeking initial certification by the Canadian Nuclear Safety Commission (CNSC) must complete, in accordance with paragraph 9(2)(b) of the Class I Nuclear Facilities Regulations.

REGDOC-2.2.3 specifies the programs and processes that nuclear power plant (NPP) licensees must implement in the design, development, conduct, marking and grading of knowledge-based and performance-based certification examinations. These examinations are aimed at demonstrating that persons seeking initial certification by the CNSC have acquired the knowledge and skills required to work competently and safely and in accordance with the regulatory requirements. In addition, it ensures that NPP licensees administer the certification examinations equitably and consistently.

This regulatory document also provides guidance within appropriate sections to further clarify the regulatory requirements.

2. Scope

This regulatory document sets out the licensee's obligations with respect to the administration of initial certification examinations for persons seeking initial certification by the CNSC for the operating positions specified in the power reactor operating licence. The CNSC generic position titles and the licensee's equivalent position titles are listed in table 1. The plant shift supervisor position cited in this regulatory document refers exclusively to the shift supervisor position at single-unit NPPs. At the time of this document's printing, additional certification examinations were not required to advance from control room shift supervisor to shift manager at a multi-unit NPP.

These certification examinations aim to assure the CNSC that - at the time of initial certification - candidates have acquired the level of knowledge and skills required to work competently and safely in their assigned positions, and are capable of responding to abnormal operating conditions at the specific NPP for which initial certification is sought.

3. Relevant Legislation

The CNSC is the federal agency that regulates the use of nuclear energy and materials in Canada, in order to prevent unreasonable risk to the health and safety or persons, to the environment and to national security while implementing Canada's international commitments on the peaceful use of nuclear energy. The NSCA requires persons or organizations to be licensed by the CNSC for carrying out the activities referred to in Section 26 of the NSCA, unless otherwise exempted. The associated regulations stipulate prerequisites for CNSC licensing and the obligations of licensees.

The following provisions of the NSCA and the regulations made under the NSCA are relevant to this document:

1. Paragraph 21(1)(i) of the NSCA empowers the Commission to "certify and decertify persons referred to in paragraph 44(1)(k) as qualified to carry out their duties under this Act or the duties of their employment."
2. Paragraph 44(1)(k) of the NSCA empowers the Commission to make regulations "respecting the qualifications for, and the training and examination of,...nuclear energy workers and other persons employed in a nuclear facility..."
3. Paragraph 37(2)(b) of the NSCA states that the Commission may authorize a Designated Officer to "certify and decertify persons referred to in paragraph 44(1)(k)."
4. Subsection 24(5) of the NSCA empowers the Commission to impose any licence condition that the Commission considers necessary for the purposes of this Act, including conditions respecting qualifications, training and examination of nuclear energy workers.
5. Subsection 24(4) of the NSCA prohibits the Commission from issuing, renewing, amending or replacing a licence, "unless, in the opinion of the Commission, the applicant
   1. is qualified to carry on the activity that the licence will authorize the licensee to carry on
   2. will, in carrying on that activity, make adequate provision for the protection of the environment, the health and safety of persons and the maintenance of national security and measures required to implement international obligations to which Canada has agreed."
6. Subsection 12(1)(a) of the General Nuclear Safety and Control Regulations obliges every licensee to "ensure the presence of a sufficient number of qualified workers to carry on the licensed activities safely and in accordance with the Act and the licence."
7. Subsection 12(1)(b) of the General Nuclear Safety and Control Regulations obliges every licensee to "train the workers to carry on the licensed activity in accordance with the Act, the regulations made under the Act and the licence."
8. Subsection 9(2) of the Class I Nuclear Facilities Regulations sets out the provisions whereby
   "The Commission or a designated officer authorized under paragraph 37(2)(b) of the Act may certify a person referred to in paragraph 44(1)(k) of the Act for a position referred to in a licence after receiving from the licensee an application stating that the person
   1. meets the applicable qualification requirements referred to in the licence
   2. has successfully completed the applicable training program and examination referred to in the licence
   3. is capable, in the opinion of the licensee, of performing the duties of the position."
9. Subsection 9(4) of the Class I Nuclear Facilities Regulations sets out the provisions whereby
   "A certification expires five years after the date of its issuance or renewal."

Part I: General Requirements To Support Initial Certification Examinations

4. Policies and Procedures

The licensee shall have documented policies, processes and procedures in place to ensure that all the requirements set forth in this regulatory document are met and implemented.

These policies, processes and procedures shall include:

1. the assurance that all persons involved in the administration of initial certification examinations follow the requirements set out in this regulatory document
2. the assurance that all persons involved in the administration of the initial certification examinations are trained, qualified and maintain their qualifications
3. the assurance that the key positions involved in the certification examination process include the:
   1. lead examiner
   2. examiners
   3. training manager
   4. support team members for performance-based examinations
4. the availability of a sufficient number of qualified personnel, resources and time to design, develop, conduct, mark and grade all certification examinations in accordance with this regulatory document
5. the assurance of independence between training and examination personnel assigned to each certification examination

6. the assurance that any person who has participated in the training of candidates scheduled to take a certification examination does not participate in the selection of the examination topics and in the preparation of the examination questions in the areas covered by the training given by this person
7. the assurance that once any person has started working on the development of an examination or once a person has any information on the content of an examination, that the individual no longer trains or gives training feedback to the candidates scheduled to take that examination until all of the candidates have completed the examination
8. the assurance that each examination is validated prior to conduct and the validation process includes the requirement of a person acting as the candidate, who is currently or previously certified at the NPP for which the test is designed and who has not been involved in any aspect of the examination design and development, to validate the examination
9. rules regarding the control of frozen documentation for all phases of all examinations
10. the invigilation requirements for knowledge-based certification examinations and an invigilation form specifying the examination title, date, time limit, starting time, a list of candidate names, completion time of the examination, a set of rules or instructions the invigilators must follow and their declaration that they have complied with the these set of rules or instructions
11. a code of conduct and instructions for briefing the examination team
12. a code of conduct and instructions for briefing the candidates
13. confirmation that the plant full-scope simulator meets the minimum acceptable capability requirements for simulator-based examinations specified in appendix A
14. a documented program that assures simulator fidelity and effectiveness

15. the assurance of a high-quality audiovisual recording system capable of clearly recording the performance of the candidates during a certification examination, and what the examiners and support team members are performing and verbalizing including the asking of questions by the examiners and the answers of the candidates

16. documentation that categorizes the level of risk associated with serious errors for the critical and significant error determination
17. the assurance that any significant knowledge or performance deficiencies revealed by a candidate or by the examination program are addressed in accordance with the principles of a systematic approach to training (SAT)
18. the assurance that any significant deficiencies in the NPP documentation revealed during the design, development, conduct, marking or grading of a certification examination is addressed accordingly
19. the assurance that a copy of the authorized examination and the corresponding authorized marking guide or corresponding authorized examiner's guide, the examination checklist, the test scenario checklists and all applicable forms for each examination are sent to the CNSC and that the CNSC is formally notified of the results of each candidate
20. the handling of appeals by candidates regarding their examination results
21. the assurance that the CNSC is informed of the tentative schedule of all certification examinations in any given calendar year, at least three months before the beginning of the year, and as soon as changes are made to the schedule
22. the assurance that the CNSC is provided with the names of the qualified lead examiners for each certification examination in any given calendar year, at least three months before the beginning of the year, and as soon as changes are made to the list of names.
23. the assurance that persons participating in the administration of certification examinations will not benefit from any performance incentive related to the success or failure of the candidates taking the certification examinations

5. Responsibilities and Qualifications

The titles of the positions used in this regulatory document are generic. The responsibilities of those positions shall be assigned by the licensee to persons holding the equivalent positions with the required qualifications at the specific NPP.

5.1 Lead examiner

5.1.1 Responsibilities

The lead examiner shall be responsible for the design, development, conduct, marking, grading, recording and reporting of knowledge-based and performance-based certification examinations.

A lead examiner assigned to a given examination shall remain as the lead examiner for the entire examination process.

Knowledge-based certification examinations

The lead examiner assigned to a given written or oral certification examination shall:

1. coordinate the design, the development, the conduct and the marking of the examination, and ensure that the requirements and criteria contained in this regulatory document are followed during all stages of the examination process
2. inform all persons participating in the design, development, conduct and marking of an examination of the requirements regarding examination security and ensure that each person has signed the applicable security agreement
3. ensure that the members of the examination team are fully aware of their roles and responsibilities and the rules they must abide by
4. ensure no changes are made to any approved examinations and approved marking guides without their prior approval
5. select the candidates who require a second marking of their examination
6. document and report the candidate's final results
7. ensure that a final marking package is compiled for each candidate
8. record and report any significant knowledge deficiencies revealed by a candidate or by an examination
9. record and report significant deficiencies in the NPP documentation found during the design, development, conduct and marking of the examination

Performance-based certification examinations

The lead examiner assigned to a given simulator-based certification examination shall:

1. coordinate the design, the development, the conduct and the grading of the examination, and ensure that the requirements and criteria contained in this regulatory document are followed during all stages of the examination process
2. inform all persons participating in the design, development, conduct and grading of an examination of the requirements regarding examination security and ensure that each person has signed the applicable security agreement
3. ensure that the persons assigned to the support team have the required knowledge and skills to play their respective roles effectively
4. ensure the support team is qualified and is fully familiar with their respective roles and responsibilities and the relevant parts of this regulatory document
5. ensure no changes are made to any approved examinations and approved examiner's guides without prior approval
6. only make minor adjustments to the approved examiner's guides, where required, at the time of the examination
7. ensure compliance with section 6 of this document regarding observers
8. ensure the simulator and the data collection devices are set properly for the conduct of the examination
9. ensure the simulator fidelity response meets the requirements of the examination

10. determine when to start, abort or end a test scenario during the conduct of an examination
11. select the candidates who require a second grading of their examination
12. document and report the candidate's final results
13. ensure that a final assessment package is compiled for each candidate
14. record and report any significant performance deficiencies revealed by a candidate or by an examination
15. record and report significant deficiencies in the NPP documentation found during the design, development, conduct and grading of the examination

5.1.2 Qualifications

Knowledge-based certification examinations

The lead examiner for a given written or oral certification examination shall:

1. be fully familiar with the relevant parts of this regulatory document applicable to the specific NPP
2. be fully qualified as an examiner

3. be fully qualified as the lead examiner for the examination to which they have been assigned

4. have previously participated as an examiner in the entire examination process for at least one written or one oral certification examination for the position for which the examination is intended at the specific NPP
5. have for the general examinations, the knowledge in science fundamentals and equipment principles required by currently certified persons working at the specific NPP in the position for which the examination is intended
6. have for the given NPP-specific examinations, one of the qualifications (a) or (b), as follows, that is applicable to the position for which the examination is intended:
   1. for the reactor operator, control room shift supervisor or plant shift supervisor position, the lead examiner must be currently or have been previously certified by the CNSC:
      1. in the position for which the examination is intended at the specific NPP and if previously certified, be fully familiar with the knowledge requirements of the position
      2. in an operating position other than unit 0 operator at any NPP, with at least one year of experience in the position for which the examination is intended, and be fully familiar with the knowledge requirements of this position at the specific NPP
   2. for the unit 0 operator position, the lead examiner must be currently or have been previously certified by the CNSC:
      1. as a unit 0 operator at the specific NPP and if previously certified, be fully familiar with the knowledge requirements of the position
      2. in an operating position, other than reactor operator, at any NPP with at least one year of experience in the position and be fully familiar with the knowledge requirements of this position at the specific NPP

Performance-based certification examinations

The lead examiner for a given simulator-based certification examination shall:

1. be fully familiar with the relevant parts of this regulatory document applicable to the specific NPP
2. be fully qualified as an examiner

3. be fully qualified as the lead examiner for the examination to which they have been assigned

4. have previously participated as an examiner in the entire examination process in at least two simulator-based initial certification examinations of the same type at the specific NPP for the position for which the examination is intended or for control room shift supervisor or plant shift supervisor candidates
5. be fully familiar with the assessment techniques used during testing on a full-scope simulator
6. be fully familiar with the documented performance expectations for certified shift personnel at that NPP
7. have one of the qualifications (a) or (b), as follows, that is applicable to the position for which the examination is intended:
   1. for the reactor operator, control room shift supervisor or plant shift supervisor position, the lead examiner must:
      1. be currently or have been previously certified by the CNSC in the position for which the examination is intended at the specific NPP and if previously certified, be fully familiar with the current knowledge and performance requirements of the position
      2. be currently or have been previously certified by the CNSC in an operating position other than unit 0 at any NPP with at least one year of experience in the position for which the examination is intended, and be fully familiar with all the knowledge and performance requirements of this position at the specific NPP
   2. for the unit 0 operator position, the lead examiner must:
      1. be currently or have been previously certified by the CNSC as a unit 0 operator at the specific NPP and if previously certified, be fully familiar with the current knowledge and performance requirements of the position
      2. be currently or have been previously certified by the CNSC in an operating position other than reactor operator at any NPP, with at least one year of experience in the position, and be fully familiar with all the knowledge and performance requirements of this position at the specific NPP

5.2 Examiners

5.2.1 Responsibilities

The examiners involved in the design, development, conduct, marking and grading of knowledge-based and performance-based certification examinations shall be under the direction of the lead examiner.

Knowledge-based certification examinations

The examiners assigned to a given written or oral certification examination team shall:

1. participate in the design, development and marking of the examination
2. participate in the conduct of the examination when the examination is conducted orally
3. obtain the data necessary from the NPP full-scope simulator to develop examination questions and answers
4. record and report significant deficiencies in the NPP documentation found during the design and development, conduct and marking of the examination
5. record and report significant deficiencies in the NPP training documentation and training program found during the entire examination process

Performance-based certification examinations

The examiners assigned to a given simulator-based certification examination team shall:

1. participate in the design, development, conduct and grading of the examination as directed by the lead examiner

2. bring to the attention of the lead examiner any simulator shortcoming that may affect the simulation of the test scenarios
3. record the performance of each candidate in the examiner's guides during the conduct of an examination
4. record and report significant deficiencies in the NPP documentation found during the design, development, conduct and grading of the examination
5. record and report significant deficiencies in the NPP training documentation and training program found during the entire examination process
6. recommend to the lead examiner to abort a test scenario, when warranted

5.2.2 Qualification requirements

Knowledge-based certification examinations

Each examiner on the examination team for a given written or oral certification examination shall:

1. be fully familiar with the relevant parts of this regulatory document applicable to the specific NPP
2. be fully qualified as an examiner

3. possess the applicable qualifications, knowledge and skills required to design, develop, conduct and mark the certification examination to which they have been assigned
4. have, for the general examinations, the knowledge in science fundamentals and equipment principles required by currently certified persons working at the specific NPP in the position for which the examination is intended
5. have, for the NPP-specific examinations, one of the qualifications (a) or (b), as follows, that is applicable to the position for which the examination is intended:
   1. for the reactor operator, control room shift supervisor or plant shift supervisor positions, have qualification (i) and, if applicable, qualification (ii)
      1. be currently or have been previously certified by the CNSC in an operating position other than unit 0 operator at any NPP, with at least one year of experience in the position
      2. if never certified by the CNSC at the specific NPP in the position for which the examination is intended, be fully familiar with the knowledge requirements of this position at that NPP
   2. for the unit 0 operator position, have qualification (i) and, if applicable, qualification (ii):
      1. be currently or have been previously certified by the CNSC in an operating position other than reactor operator at any NPP, with at least one year of experience in the position
      2. if never certified by the CNSC at the specific NPP as a unit 0 operator, be fully familiar with the knowledge requirements of this position at that NPP

6. in addition to the requirements above, at least one examiner on an examination team for a given NPP-specific examination shall:
   1. be sufficiently familiar with the operations of CANDU NPPs to ensure the examination questions and answers are operationally focused
   2. be either:
      1. currently certified at the specific NPP, or at a similar NPP on the same site, in the position for which the examination is intended or as a control room shift supervisor or a plant shift supervisor, or
      2. previously certified at the specific NPP, or at a similar NPP on the same site, in the position for which the examination is intended or as a control room shift supervisor or a plant shift supervisor and have the knowledge required by currently certified persons working in the position for which the examination is intended at the specific NPP

Performance-based certification examinations

Each examiner on the examination team for a given simulator-based certification examination shall:

1. be fully familiar with the relevant parts of this regulatory document applicable to the specific NPP
2. be fully qualified as an examiner

3. be fully familiar with the assessment techniques used during testing on a full-scope simulator
4. have one of the qualifications (a), (b) or (c) listed below that is relevant, taking into account the position for which the examination is intended:
   1. for the reactor operator, control room shift supervisor and plant shift supervisor positions, have qualification (i) and, if applicable, qualification (ii):
      1. be currently or have been previously certified by the CNSC in an operating position other than unit 0 operator at any NPP, with at least one year of experience in the position
      2. if never certified by the CNSC at the specific NPP in the position for which the examination is intended, be fully familiar with the knowledge and skill requirements of this position at that NPP
   2. for the unit 0 operator position, have qualification (i) and, if applicable, qualification (ii):
      1. be currently or have been previously certified by the CNSC in an operating position other than reactor operator at any NPP, with at least one year of experience in the position
      2. if never certified by the CNSC at the specific NPP as a unit 0 operator, be fully familiar with the knowledge and skill requirements of this position at that NPP
   3. if never certified by the CNSC in a relevant operating position at any NPP, be fully familiar with the knowledge and skill requirements of the position for which the examination is intended at the specific NPP; the licensee shall have a documented process and procedure in place that ensures the examiners have the knowledge required by currently certified persons working in that position at that NPP

5. in addition, at least one examiner on the examination team for a given simulator-based examination shall:
   1. have up-to-date knowledge of the simulator operational capabilities, its modelling limitations and the equipment and system malfunctions that it can simulate
   2. be either:
      1. currently certified at the specific NPP, or at a similar NPP on the same site, in the position for which the examination is intended, or as a control room shift supervisor or a plant shift supervisor
      2. have been previously certified at the specific NPP, or at a similar NPP on the same site, in the position for which the examination is intended, or as a control room shift supervisor or a plant shift supervisor and have the knowledge required by currently certified persons working in the position for which the examination is intended at the specific NPP

5.3 Training manager

5.3.1 Responsibilities

The training manager shall be responsible for the implementation of the knowledge-based and performance-based certification examination requirements set forth in this regulatory document.

Knowledge-based and performance-based certification examinations

The training manager shall:

1. ensure that all of the requirements contained in this regulatory document are complied with during all stages of the certification examination process
2. ensure all security requirements are met by all personnel throughout the certification examination process
3. for each certification examination, assign at least two qualified examiners to the examination team and ensure one is qualified to be the lead examiner

4. determine the need to assign subject matter experts to assist the qualified examiners on the examination team
5. ensure the rules governing frozen documentation are followed at all times
6. determine and approve, in accordance with section 10 the need to conduct a given supplementary NPP-specific examination orally
7. review and approve the knowledge-based certification examinations and corresponding marking guides prior to the conduct of the examinations
8. review and authorize the marking guide and the examination results of each candidate for a given knowledge-based examination
9. review and approve the performance-based certification examinations and corresponding examiner's guides prior to the conduct of the examinations
10. review and authorize the examiner's guides prior to grading a given performance-based certification examination
11. review and authorize the examination results of each candidate for a given performance-based certification examination
12. ensure simulator fidelity response meets the given examination design requirements
13. ensure that the NPP full-scope simulator meets the requirements of appendix A.1 regarding simulation capabilities for simulator-based certification examinations
14. approve the presence of observers during the conduct of the performance-based certification examinations
15. ensure that a copy of all authorized examinations and corresponding authorized marking guides or corresponding authorized examiner's guides, examination checklists, test scenario checklists and all applicable forms are made available upon request to the CNSC following each examination
16. ensure that the CNSC is formally notified of the pass and fail results of each candidate
17. formally notify the CNSC of the tentative schedule of all certification examinations in any given calendar year, at least three months before the beginning of the year, and as soon as changes are made to the schedule

5.3.2 Qualification requirements

The licensee shall ensure there are documented processes and procedures in place specifying the qualification requirements, including the maintaining of qualification requirements, of training managers at the specific NPP involved in the certification examination process.

Knowledge-based and performance-based certification examinations

The training manager shall:

1. be currently or have been previously certified by the CNSC in any operating position at any NPP or have the equivalent knowledge and experience
2. have expert knowledge of all parts of this regulatory document related to certification examinations applicable to the specific NPP

5.4 Support team members

5.4.1 Responsibilities

The support team members, under the direction of the lead examiner, shall be responsible for supporting the performance-based certification examinations during rehearsal and validation of the test scenarios and during the conduct of a given examination.

The licensee shall ensure that restrictions are placed on the number of support team members in the control room and on their individual performance during the conduct of a given examination. These restrictions will allow the candidates the opportunity to demonstrate their ability to independently monitor the evolution of plant conditions, recognize abnormalities and diagnose malfunctions.

Performance-based certification examinations

During the certification examination process, the support team members shall:

1. perform the role of the NPP control room operating crew, as prescribed in the examiner's guides

2. direct the activities or perform the role of the field operators, as prescribed in the examiner's guides

3. act in such a way as not to interfere with or adversely influence the conduct of the examination or change the intent and scope of the test scenario

4. operate, or have a qualified person operate, the NPP full-scope simulator as prescribed in the examiner's guides

5. bring to the immediate attention of the lead examiner, during the rehearsals, validation or the conduct of a test scenario any simulator deficiencies, deficiencies in NPP documentation or other circumstances that may affect the validity of the test scenarios

5.4.2 Qualification requirements

Performance-based certification examinations

Each support team member shall:

1. be fully familiar with the relevant parts of this regulatory document applicable to the specific NPP
2. have the required knowledge and skills to perform the assigned roles correctly and effectively, as prescribed in the examiner's guides

At least one support team member, who is assigned to operate the simulator during a certification examination, shall have up-to-date knowledge of the simulator operational capabilities, any known modelling limitations and the equipment and system malfunction capabilities.

6. Observers

The licensee shall have a documented policy, process and procedures in place regarding the participation of observers who, due to their duties, need to observe the conduct of a simulator-based certification examination.

The licensee's policy, process and procedures shall include the requirements for security agreements and code of conduct of observers, and ensure the following conditions:

1. observers shall in no way participate in the administration of any part of a certification examination nor interfere with the conduct of an examination
2. observers shall not comment under any circumstances on the performance of a candidate nor participate in the evaluation of the candidate's performance
3. observers shall obtain approval from the training manager prior to the conduct of a certification examination
4. observers shall sign a security agreement specified in section 7 prior to attending the conduct of the certification examination

7. Security Requirements

7.1 Security policy and process

The licensee shall have documented policies, processes and procedures in place regarding the security requirements for all phases of the knowledge-based and performance-based certification examinations and shall include detailed security agreements.

The documented policies, processes and procedures shall include the requirement to:

1. document the physical, electronic and administrative measures and requirements , including those specifically designed for the simulator, which must be in place to minimize the risk of compromising the security of certification examinations
2. control all certification examinations and all material associated with those examinations at all times
3. ensure no information on the content of a certification examination is revealed in any way to the candidates scheduled to take the examination
4. ensure that any required discussions on a certification examination take place with specified examination personnel in a secure location to prevent compromising the security of the examination
5. limit the access to the certification examinations and to the associated examination material to persons with a need to know
6. ensure there is at least one barrier in place at all times to prevent unauthorized access to examinations and associated examination material

7. ensure all draft material, notes and other documents generated or consulted during any phase of the examination process be handled in a way that prevents compromising the security of the examinations
8. ensure all persons participating in any part of the certification examination process for any certification examination are informed of:
   1. the licensee's documented physical, electronic and administrative measures and requirements, applicable to the person's role in the examination or part there of, to ensure the security of the certification examinations
   2. the terms of the licensee's security agreement process for certification examinations including the consequences of violating its terms
   3. the requirement to sign the licensee's security agreement
9. ensure that any person involved in any phase of the examination process who suspects unauthorized access to information related to the content of a given certification examination immediately informs the training manager or the lead examiner who must promptly undertake an investigation; the examination must not be used as soon as the investigation shows that its security has been compromised
10. ensure all candidates taking a certification examination are informed of the terms of the licensee's security agreement for certification examinations including the consequences of violating its terms and sign the licensee's specific security agreement
11. ensure that in the event of a security breach, including a violation of the terms of a signed agreement, a process is in place to determine if the examination is to be immediately terminated and no longer used

7.2 Security agreements

The security agreements shall stipulate the requirements outlined in section 7.1 specific to the role of the person who is required to sign the agreement.

Prior to participating in any part of the certification examination process for any certification examination, and prior to the conduct of any certification examination, the following persons shall have signed a security agreement attesting to its terms and conditions:

1. examiners and examination support personnel who are assigned or appointed to work on a regular basis in the design, development and conduct of a given certification examination
2. persons, other than those defined in point 1 above, who have approval to participate in the design, development and conduct of a given certification examination such as subject matter experts and observers
3. candidates who are scheduled to take a certification examination

8. Retention of Initial Certification Examination Records

8.1 Knowledge-based certification examination records

1. The licensee shall retain, for a minimum of five years from the date of approval or signature, the following records for each written or oral examination:
   1. the approved examination:
   2. the authorized marking guide
   3. the examination checklist
   4. the invigilation form
   5. the names of the persons who developed the examination
   6. the name and signature of the person who approved the examination, with the date of signature
   7. the name and signature of the person who approved and who authorized the marking guide, with the date of signature
   8. the candidates' answer booklets for a written examination and if applicable, the recordings of the candidates' oral examinations
   9. the signed security agreements
   10. any approved report documenting the analysis of the examination results
2. the licensee shall retain, for the period prescribed in paragraph 14(5) of the Class I Nuclear Facilities Regulations, a record of the written and if applicable, oral examinations taken by each candidate, containing:
   1. the position sought by the candidate
   2. the title and date of each examination taken
   3. the completed authorized marking guide for each examination taken by the candidate
   4. all documentation pertaining to the outcome of any appeal regarding the candidate's examination results

8.2 Performance-based certification examination records

1. the licensee shall retain, for a minimum of five years from the end date of conducting the examination, the following records for each simulator-based examination:
   1. an approved examination package that includes:
      1. the approved examiner's guides
      2. the test scenario checklists and the examination checklist
      3. the names of the persons who developed the examiner's guides
      4. the name and signature of the person who approved the examiner's guides and the examination package, with the date of signature
   2. the authorized examiner's guides with the name and signature of the person who authorized them, with the date of signature
   3. the examiner's guides marked up by the examiners and all the data collected during the conduct of the examination including the audiovisual recordings
   4. the signed security agreements
   5. any approved report documenting the analysis of the examination results
2. the licensee shall retain, for the period prescribed in paragraph 14(5) of the Class I Nuclear Facilities Regulations, a record of the simulator-based examinations taken by each candidate, containing:
   1. the position sought by the candidate
   2. the type and date of each examination taken
   3. the final assessment package of the candidate for each examination, consisting of the documents specified in section 18.5 of this regulatory document
   4. all documentation pertaining to the outcome of any appeal regarding the candidate's examination results

Part II: Requirements for Knowledge-Based Certification Examinations

This part establishes the required written initial certification examinations, (or oral certification examinations where prior approval has been given) and the requirements regarding the design, development, conduct and marking of these examinations that must be successfully completed by reactor operator (RO), unit 0 operator (U0O), plant shift supervisor (PSS) and control room shift supervisor (CRSS) candidates.

The knowledge-based examinations must be successfully completed in sequence by each candidate, and pass results must be obtained before the performance-based initial certification examinations are taken.

9. Written and Oral Certification Examinations

The required knowledge-based initial certification examinations shall consist of:

1. A general examination: This examination shall examine the basic knowledge requirements of the position at the NPP where certification is sought covering the science fundamentals relevant to the operation of that NPP and the principles of operation of the equipment at that NPP.
2. An NPP-specific examination: This examination shall examine the knowledge requirements of the position relevant to the operation of the NPP where certification is sought covering the principles of nuclear safety and their application at the NPP, as well as the integrated operation of the NPP systems including, where applicable, interactions between the systems of a reactor unit and those of other reactor units and of unit 0, including the relevant operating documentation.
3. A supplementary NPP-specific examination: This examination shall examine specific knowledge requirements of the PSS and of the CRSS positions that are in addition to those of the RO position and Unit 0 as applicable. This shall include accident assessment and emergency operations.

The design and development of each of the required examinations specific to each position is described in sections 9.3-10.6.

9.1 Total marks and time limits per certification examination

In preparation for the design and development of the examinations (sections 9.3-10.6), table 2 lists the required certification examinations together with the candidates to whom the examination applies. Table 2 also provides the required total marks, the estimated duration each examination must be designed towards and the maximum allowable time limits for the candidate to complete the examination.

9.2 Examination bounding envelope

Written and oral certification examinations shall be designed to examine the knowledge requirements of a position at the specific NPP for which initial certification is sought. Each examination shall ensure that the knowledge objectives for each position, developed in accordance with a SAT, are adequately tested using the applicable topics within each of the required topic groups outlined in Appendices B.1-B.11. The design requirements are specified in section 9.3 under Examination Design and Development.

Licensees shall ensure that:

1. knowledge objectives, specific to their NPP, are being developed, documented and updated on a regular basis in accordance with a SAT. The licensees shall inform the CNSC of any major changes to their CNSC approved knowledge objectives template which may have an impact on the training and examination envelopes

   The knowledge objectives or learning objectives are specific to each NPP and are developed in accordance with a SAT. The objective template used by multi-unit NPPs was approved by the CNSC in December 2003. Single-unit NPPs use the Generic Station System Knowledge Objectives for Control Room Operators (GSSKO) document in the development of their NPP-specific learning objectives.
   
   The details and the process by which the licensees are to inform the CNSC of any changes to this template was further confirmed in a letter dated June 14, 2010. These documents are listed in the Reference section.

2. the knowledge objectives selected for a given position meet the following criteria:
   1. each selected objective supports one or more tasks that are rated as difficult and important in the NPP's approved, up-to-date SAT-based job and task analysis for the position

      As part of the job and task analysis phase of SAT for each position, a DIF rating system should be used (D = level of difficulty, I = level of importance, F=frequency of use).

   2. the selected objectives cover all the knowledge required for the person to perform safely and competently in the position
   3. the selected objectives are suitable to formulate questions for certification examinations for the position
3. the topic groups adequately cover the different knowledge areas addressed by the knowledge objectives template for each of the positions requiring certification

The source of the applicable knowledge objectives that are suitable to formulate questions for an examination for a given position are identified in column two of table 3. Table 3 also identifies the required certification examinations for each position.

9.3 Examination design

The knowledge-based certification examinations shall be designed and developed to be conducted in the form of written examinations. For NPPs where prior concurrence from the CNSC has been sought, the identical examination may be conducted orally as per section 10.

The examination team, under the direction of the lead examiner, shall design the examination questions, examination marking guide, examination checklist and final examination for each position in accordance with the requirements set out below.

9.3.1 General examinations for RO, Unit 0, PSS and CRSS candidates

The examination team shall design the examination questions as follows:

1. Select one or more specific examination topics for each of the topic groups listed in either appendix B.1 for RO, PSS and CRSS candidates or appendix B.2 for Unit 0 candidates.
2. Prepare, for each specific topic selected, questions whose answers are worth the number of marks allocated to the topic, according to the following criteria:
   1. at least 80% of the questions on the certification examination shall be newly designed and not based upon questions previously used in any examination, test, candidate evaluation or training
   2. no more than 20% of the questions on the certification examination shall be based on previously used questions, and all reused questions must be significantly modified and never before seen by the candidates
   3. to the maximum possible extent, prepare questions that measure integrated understanding or higher-level knowledge required by ROs or by U0Os to perform their job competently
   4. select the individual parts of the relevant knowledge objectives in science fundamentals and equipment principles approved by the CNSC that will be used to formulate the questions on the corresponding topic, according to the following:

      The knowledge objectives or learning objectives are specific to each NPP and are developed in accordance with a SAT. The objective template used by the multi-unit NPPs was approved by the CNSC in December 2003. Single-unit NPPs should use the GSSKO document in the development of their NPP-specific learning objectives.

      1. to the maximum possible extent, select objectives that require integrated understanding or higher-level knowledge

         In order to examine higher-level knowledge, the examination questions should test at the cognitive level of comprehension, application or analysis as defined by Bloom's taxonomy. For example, questions requiring the candidate to "describe" or to "explain" would be categorized as higher-order learning objectives, while requesting the candidate to "list" or to "state" would be classified as lower-order objectives.

      2. objectives shall not be used verbatim to formulate questions

         Ideas for formulating questions can be found in the following documentation:

         • NPP and industry wide significant event reports
         • operating manuals and operating memos
         • design manuals and other technical basis documents
         • NPP safety report
         • licensee and NPP administrative procedures and associated documentation related to plant operation and maintenance
3. verify that each question is operationally focused
4. develop the examination and corresponding marking guide as specified in section 9.4

9.3.2 NPP-specific examinations for RO candidates

The examination team shall design the examination questions as follows:

1. for each group of topics listed in appendix B.3, select one or more specific topics for use in the examination
2. allocate a number of marks to each selected topic such that the total number of marks for the group is within the range indicated beside the group in appendix B.3
3. for each topic selected in 1 above, select from appendix B.4 one or more knowledge groups to be used for designing the corresponding examination questions and, using an examination design matrix (EDM) that outlines the topic groups vertically and the knowledge groups horizontally as shown in appendix B.8, distribute in the matrix the number of marks for the topic among the knowledge area(s) selected. Ensure that the total marks allocated to the various knowledge groups are within the range indicated beside the group in appendix B.4
4. for each cell of the EDM where marks are indicated, prepare questions whose answers are worth the marks indicated in the cell, according to the following criteria:
   1. at least 80% of the questions on the certification examination shall be newly designed and not based upon questions previously used in any examination, test, candidate evaluation or training
   2. no more than 20% of the questions on the certification examination shall be based on previously used questions, and all reused questions must be significantly modified and never before seen by the candidates
   3. to the maximum possible extent, questions shall measure integrated understanding or higher-level knowledge required by ROs to perform their job competently

      In order to examine higher-level knowledge, the examination questions should test at the cognitive level of comprehension, application or analysis as defined by the Bloom's taxonomy.For example, questions requiring the candidate to "describe" or to "explain" would be categorized as higher-order learning objectives, while requesting the candidate to "list" or to "state" would be classified as the lower-order objectives. Integrated understanding or higher-level knowledge is tested by questions that require the candidate to state, analyze and describe.

   4. when a cell is associated with the generic station system knowledge objectives or with radiation protection knowledge objectives, select the individual parts of the relevant objectives that will be used to formulate the questions on the corresponding topic, according to the following:
      1. to the maximum possible extent, select GSSKOs according to appendix D.2
      2. avoid repeated use of the same objective in any knowledge group for different topics
      3. objectives shall not be used verbatim to formulate questions

         Appendix B.4 and appendix D.2 provide the GSSKO requirements. The GSSKO document serves as a reference manual and accompanying user guide.

         These documents define the knowledge of station systems and procedures that ROs of NPPs require to perform their job competently. Altogether, the GSSKOs cover the design of systems, their instrumentation and control, their operation, overall unit operation and emergency operation.

         The GSSKOs provide NPPs with a template for developing NPP-specific training objectives for RO candidates. The user guide gives detailed guidance for preparing the NPP-specific system training objectives and for developing the supporting training material.

         Ideas for formulating questions can be found in the following documentation:

         • NPP and industry wide significant event reports
         • operating manuals and operating memos
         • design manuals and other technical basis documents
         • NPP safety reports
         • licensee's and NPP administrative procedures and associated documentation related to plant operation and maintenance
         • actual simulator data
5. verify that each question on NPP systems and procedures is within the scope of the applicable GSSKOs, or of the NPP-specific knowledge objectives derived from them

   For any question for which specific data from the NPP are needed:

   • obtain the plant or simulation data required to allow completion of the formulation of the question
   • identify the aspects of the data obtained that may be interesting to explore in the question and complete the formulation of the question
6. develop the examination and corresponding marking guide as specified in section 9.4

9.3.3 NPP-specific examinations for unit 0 operator candidates

The examination team shall design the examination questions as follows:

1. select, for each group of topics listed in appendix B.5 one or more specific topics for use in the examination
2. allocate a number of marks to each selected topic such that the total number of marks for the group is within the range indicated beside the group in appendix B.5
3. for each topic selected in 1 above, select from appendix B.6one or more knowledge groups to be used for designing the corresponding examination questions and, using an examination design matrix (EDM) that outlines the topic groups vertically and the knowledge groups horizontally shown in appendix B.8, distribute in the matrix the tentative number of marks for the topic among the knowledge area(s) selected; ensure that the total marks allocated to the various knowledge groups are within the range indicated beside the group in appendix B.6
4. prepare, for each cell of the EDM where marks are indicated, questions whose answers are worth the marks indicated in the cell, according to the following:
   1. at least 80% of the questions on the certification examination shall be newly designed and not based upon questions previously used in any examination, test, candidate evaluation or training
   2. no more than 20% of the questions on the certification examination are based on previously used questions, and all reused questions must be significantly modified and never before seen by the candidates
   3. to the maximum possible extent, prepare questions that measure integrated understanding or higher-level knowledge required by U00s to perform their job competently

      In order to examine higher-level knowledge, the examination questions should test at the cognitive level of comprehension, application or analysis as defined by Bloom's taxonomy. For example, questions requiring the candidate to "describe" or to "explain" would be categorized as higher-order learning objectives while requesting the candidate to "list" or to "state" would be classified as the lower-order objectives. Integrated understanding or higher-level knowledge are questions that require the candidate to state, analyze and describe.

   4. when a cell is associated with GSSKOs or with radiation protection knowledge objectives, select the individual parts of the relevant objectives that will be used to formulate the questions on the corresponding topic, according to the following:
      1. to the maximum possible extent, select GSSKOs according to appendix D.4
      2. avoid repeated use of the same objective in any knowledge group for different topics
      3. objectives shall not be used verbatim to formulate questions
5. verify that each question on NPP systems and procedures is within the scope of the applicable GSSKOs, or of the NPP-specific knowledge objectives derived from them

   For any question for which specific data from the NPP are needed:

   • obtain the plant or simulation data required to allow completion of the formulation of the question
   • identify the aspects of the data obtained that may be interesting to explore in the question and complete the formulation of the question
6. develop the examination and corresponding marking guide as specified in section 9.4

9.3.4 Supplementary NPP-specific examinations for PSS and CRSS candidates

The examination team shall design the examination questions as follows:

1. select, for each of the topic groups listed in appendix B.7, one or more specific topics for use in the examination
2. allocate a number of marks to each topic selected such that the total number of marks for the group is within the range indicated beside the group in appendix B.7
3. prepare, for each specific topic selected, questions whose answers are worth the number of marks allocated to the topic, according to the following:
   1. at least 80% of the questions on the certification examination shall be newly designed and not based upon questions previously used in any examination, test, candidate evaluation or training
   2. no more than 20% of the questions on the certification examination are based on previously used questions, and all reused questions must be significantly modified and never before seen by the candidates
   3. to the maximum possible extent, prepare questions that measure integrated understanding or higher-level knowledge required by PSSs or by CRSSs to perform their job competently
   4. select the individual parts of the relevant knowledge objectives for PSS or for CRSS approved by the CNSC that will be used to formulate the questions on the corresponding topic, according to the following:
      1. to the maximum possible extent, select objectives that require integrated understanding or higher-level knowledge

         In order to examine higher-level knowledge, the examination questions should test at the cognitive level of comprehension, application or analysis as defined by Bloom's taxonomy.For example, questions requiring the candidate to “describe” or to “explain” would be categorized as higher-order learning objectives, while requesting the candidate to “list” or to “state” would be classified as the lower-order objectives. Integrated understanding or higher-level knowledge are questions that require the candidate to state, analyze and describe.

      2. objectives shall not be used verbatim to formulate questions

         Ideas for formulating questions can be found in the following documentation:

         • NPP and industry wide significant event reports
         • operating manuals and operating memos
         • design manuals and other technical basis documents
         • NPP safety reports
         • licensee's and NPP administrative procedures and associated documentation related to plant operation and maintenance
         • actual simulator data
4. check that each question is within the scope of the knowledge objectives for PSSs or for CRSSs approved by the CNSC

   For any question for which specific data from the NPP is needed:

   • obtain the plant or simulation data required to allow completion of the formulation of the question
   • identify the aspects of the data obtained that may be interesting to explore in the question and complete the formulation of the question
5. develop the examination and corresponding marking guide as specified in section 9.4

9.4 Examination and marking guide development

The examination team, under the direction of the lead examiner, shall develop a given examination and the answers to that examination in a marking guide in accordance with the following requirements:

1. ensure the cover page of the examination is of similar format and contains the information shown in appendix B.9
2. ensure the formatting of the questions is similar to that shown in appendix B.10

   Group together the questions that are logically related, integrating them into a single multi-part question, according to the following guidelines:

   • the parts and subparts of the question are arranged in a logical order
   • the answer to a part of the question is not completely dependent on the answer to a previous part
3. prepare the answers in a marking guide for each given examination
4. ensure that the answers for all questions:
   1. are in line with the intent of the questions
   2. are complete and technically accurate
   3. contain only the information specifically asked by the question
   4. reflect the level of knowledge that candidates are expected to have on the topics examined
   5. do not conflict with the information contained in the frozen documentation
   6. are broken into elements used to determine the number of marks to be assigned to the question
5. identify, for each question, the reference documents required to prepare the answer and check that the revision number and date of issue of each document correspond to those of the frozen documentation
6. ensure that the level of difficulty of each question is appropriate to discriminate between those candidates who have sufficient understanding of the subject covered by the question and those who do not.
7. determine the number of marks to be allocated to each question using the relevant table in appendix B.11

   The total number of marks to be allocated to each question and the estimated time limits are determined by using the applicable table of elements in appendix B.11.

8. ensure the examination has the total number of marks allocated for each question and has the breakdown of marks with the corresponding number of elements for each part of the question as shown in appendix B.11

   To assist the candidate, the estimated time to answer each part of a question can also be shown in addition to the number of elements and marks which typically appear to the left of the letter designating the question part.

9. ensure the cover page of the marking guide is of similar format and contains the information shown in appendix B.12
10. ensure the formatting of the answers in the marking guide is similar to that shown in appendix B.13
11. finalize the examination and corresponding marking guide as specified in section 9.5

9.5 Final examination and marking guide

The examination team shall finalize a given examination and corresponding marking guidein accordance with the following requirements:

1. ensure each question and its answer is clear and technically accurate

   This includes the verification that the identification codes of equipment and documents used in the questions are correct.

2. ensure the examination has been verified that the number of marks is based on the total number of elements in the answer to all parts of the questions

   This includes the verification that the breakdown of each answer into elements is appropriate.

3. ensure the questions clearly identify any reference material given to the candidates and give appropriate instructions for its use
4. ensure that any instruction or other information in a question quoted from licensee's or NPP documentation appears verbatim
5. ensure for each question, the reference material given to the candidates is clearly identified
6. ensure no reference material is given to a candidate when a question covers an area that candidates have to know from memory
7. ensure the content of approved procedures and flowsheets given as reference material is not altered
8. ensure that whenever an acronym is used in a question, it is defined the first time it appears in that question.

   In each question, the key words that indicate the degree of development required in the answer should be typed in bold and underlined to help candidates to give a complete answer.

   The examination should also have only one question per page and the stems used in a question should be placed immediately before the parts or subparts of the question to which they apply.

9. compile a package containing the questions with their answers, a copy of the reference material used to prepare each answer and the knowledge objectives covered by each question
10. complete an examination checklist similar to that shown in appendix B.14 and confirm that the examination meets the criteria specified in this checklist
11. ensure the final version of the EDM, the examination, its marking guide and the examination checklist are verified, approved and signed by the lead examiner and the training manager
12. ensure no changes are made to an approved examination and corresponding marking guide without the prior concurrence of the lead examiner and training manager

9.6 Examination validation

The examination team shall, under the direction of the lead examiner, ensure that each certification examination for a given position is validated and meets or exceeds the requirements set out in this regulatory document prior to the conduct of the examination.

This validation shall be performed prior to obtaining the required approvals of the marking guide for the conduct of a given examination.

At a minimum, this examination validation shall be done:

1. in real time, by a person currently or previously certified at the NPP for the position for which the examination is designed and who has not been involved in any aspect of the examination design, acting as the candidate
2. to confirm that the applicable requirements in sections 9.4 and 9.5 are met

10. Conduct of Knowledge-Based Certification Examinations

The licensee shall have documented policies, processes and procedures regarding the conduct of knowledge-based examinations.

The licensee shall ensure:

1. that knowledge-based certification examinations are conducted in the form of written examinations and invigilated at all times
2. that under exceptional circumstances, when an identical written examination may need to be conducted orally with a candidate, prior notification has been provided to the CNSC; the training manager shall formally notify the CNSC of the need to conduct a specific certification examination orally; the request must be submitted to the CNSC with written justification within a reasonable time before the anticipated scheduling of the examination

   An example of an exceptional circumstance would be if a candidate was unable to write a given examination as a result of an unanticipated incident (e.g., physical injury).

3. that candidates who voluntarily withdraw from an examination or cheat be automatically assigned a fail grade
4. a secure location for conducting an examination with restricted access preventing contact between candidates and other personnel
5. acceptable seating of candidates for a written examination to allow for sufficient spacing between them to ensure confidentiality of the examination and prevent cheating
6. examination confidentiality and security
7. that all applicable security agreements have been signed prior to the conduct of an examination
8. the control of all examination material
9. the control of any material brought into and out of the examination room including handheld electronic devices
10. that only reference material that is part of the examination package shall be permitted

    Candidates should not have access to any reference material other than the material given with the examination. The candidates should not bring anything in or take anything out of the examination room. Only the invigilator or lead examiner may provide the candidates with:

    • the examination questions and applicable reference material attached
    • booklets for the candidates to write in their answers
    • blank writing paper
    • articles such as pens, pencils, highlighters, erasers, ruler, a magnifying glass and a non-programmable calculator
11. documented invigilation requirements for written examinations including a form with rules for the invigilator and the requirement for individual attestations
12. that for washroom breaks during the conduct of an examination, only one candidate at a time is permitted to leave the examination room under escorted supervision, and that communication with any person shall be prohibited and can result in a fail grade
13. the presence of an invigilator in the examination room at all times during a written examination

    The invigilation form and the instructions should be similar to that found in appendix D.6.

14. documented rules of conduct and instructions for the candidate and the signing of a security agreement
15. documented rules of conduct and instructions for the examination team including the operator of the audiovisual recording system for an oral examination

    The security agreements and individual instructions should be similar to those found in appendix D.1.

16. that the examiners do not provide any additional information or clarification of questions to the candidates
17. that the examinations are conducted in accordance with the applicable maximum time limits set in table 2 of this document, with no time extensions permitted
18. that oral examinations:
    1. are conducted by the lead examiner and at least one other examiner who has participated in the examination design and development, and only the operator of the recording system shall be permitted in the examination room with the examiners
    2. are recorded; the audiovisual system must be capable of recording clearly the questions of the examiners and the answers of a candidate during the examination

       The audiovisual recordings need to be of high quality to clearly distinguish between what is being asked by the examiners from what is being answered by the candidate during an oral examination. An oral examination will not be recognized by the CNSC unless it has been properly recorded. In addition, these recordings are used by the CNSC when performing compliance activities.

19. that during an oral examination:
    1. a copy of the questions are given to the candidate one at a time and read out by an examiner on the examination team
    2. that all answers given by the candidate are documented by each examiner on the examination team in their copy of the marking guide
    3. if a candidate appears to have missed or misunderstood a question, the examination team rephrases the question without giving hints on the answer

11. Marking of Examinations

11.1 General requirements for written and oral examinations

The licensee shall have documented policies, processes and procedures in place regarding the marking of knowledge-based examinations. This shall include:

1. examinations shall be marked as soon as practicable

   The marking of examinations is an integral part of the administration of the certification examination process. Marking should be completed immediately following the conduct of a certification examination. This regulatory document does not stipulate an absolute deadline for marking examinations to allow licensees to deal with unforeseen circumstances. However, as part of administering the certification examinations, licensees should have the necessary resources at the onset of planning a given examination.

2. the marking of examinations shall only begin when all candidates have been examined
3. only the lead examiner and the examiners involved in the design of the examination shall perform the first and second marking of the examinations
4. the first and second marking of the examinations shall be done in accordance with sections 11.2-11.4
5. for an oral examination, the first and second marking shall only be performed by the examiners who conducted the examination
6. the first marking shall be completed on all candidates prior to performing the second marking
7. the second marking shall be performed on selected candidates by an examiner who did not perform the first marking on those candidates
8. to ensure independence between the first and second marking of a written examination, the second marker shall not have access to the results of the first marking until the second marking has been completed
9. an examination result form that contains the information in format similar to that shown in appendix B.15 shall be completed for each candidate

11.2 First marking of written and oral examinations

The lead examiner shall ensure that the first marking is performed in accordance with the following:

1. use a new copy of the approved marking guide for each candidate
2. mark the answer of all candidates to a given question before proceeding to mark the next question
3. mark the candidate's answers against the approved marking guide by checking-off the answer elements directly in the candidate's marking guide
4. for each question, record in the candidate's marking guide:
   1. any misconceptions, errors and deficiencies, and their significance
   2. any deficiencies in the answer which are not, or not sufficiently, accounted for by the marking guide
   3. any relevant information in the answer that is not included in the marking guide
5. once marking of a question is completed, record in an examination comments file:
   1. any perceived difficulty encountered by a number of candidates with the question, as indicated by answers that are consistently incomplete or different from the answer in the approved marking guide
   2. any recommendation for changes to the answer in the approved marking guide that the lead examiner considers necessary before the marking guide is authorized
6. once the first marking of all candidates is completed:
   1. review the answers of each candidate to determine whether credit may be given for an answer element not mentioned explicitly in a candidate's answer to a question, because it is concluded from an answer to a related question that the candidate has this knowledge

      When marking a written examination that was conducted orally, the examiners should reference the recordings as required to verify the answers made by the candidate.

   2. document the reasons for giving such credits in the candidate's marking guide
   3. determine the significance of misconceptions, errors and deficiencies recorded in each candidate's marking guide and document any conclusions in the marking guide
7. calculate the individual question scores and examination scores obtained by each candidate
8. assign a pass or fail result in accordance with section 11.4
9. on the cover page of each candidate's marking guide document the marks obtained by the candidate and the recommended examination results
10. complete an examination result form for each candidate

11.3 Second marking of written and oral examinations and reconciliation of results

Upon completion of the first marking of all candidates in accordance with sections 11.2 and 11.4, the second marking of candidates shall be performed in accordance with the following:

1. the training manager shall:
   1. approve the candidates selected by the examiners to undergo a second marking
   2. select a minimum of one candidate marked by each examiner who performed the first marking

      The minimum second marking requirement is to perform a calibration check on the accuracy of the examiner's guide and assessment of the marking process.

   3. select candidates for whom the first marker recommends a second marking
   4. ensure the second marking of a candidate is performed by the member of the examination team who did not perform the first marking
   5. ensure the second marking is performed one candidate at a time using the authorized marking guide in accordance with the instructions for the first marking given in paragraphs 1 through 10 in section 11.2
2. the lead examiner shall select candidates for a second marking according to the following criteria:
   1. select candidates with an examination score in the range from 57 percent to 63 percent
   2. other candidates at the discretion of the training manager or lead examiner
3. the examiners shall:
   1. meet to resolve any discrepancies between the two markings and document how these discrepancies were resolved
   2. review the complete examination of one candidate at time
   3. document the reconciled examination results and the justifications in the original knowledge-based examination result form for each candidate.
   4. ensure that any final changes made to answers in the approved marking guide and the examination result form are reviewed and authorized by the training manager

11.4 Assignment of examination results

The examination team shall assign a pass or fail result to a given examination in accordance with the following criteria:

1. a pass result shall be assigned to an examination if a candidate has scored 70 percent or higher in the examination
2. a fail result shall be assigned to a given examination if a candidate has either:
   1. not met the pass criteria above
   2. met the pass criteria above but has shown knowledge deficiencies the lead examiner considers to be serious

The examination team shall

1. document the final examination results and the justifications in the original knowledge-based examination result form for each candidate
2. ensure the results form is reviewed and signed by the training manager

12. Examination Follow-Up

1. The examination team shall document and report to the training manager any knowledge deficiencies revealed by a candidate or by an examination.

   The deficiencies revealed by an examination should include those found in the examination design, marking, in NPP procedures or training program.

2. The examiners shall ensure that a final marking package is compiled for each candidate consisting of:
   1. the completed knowledge-based examination result formsigned by both examiners and authorized by the training manager
   2. the final scores obtained in the examination
   3. the consolidated marking guide signed by both examiners
3. The training manager shall:
   1. after authorizing the results of any given knowledge-based certification examination, ensure that a copy of the approved examination, authorized marking guides, examination checklists, EDMs and examination result forms are made available to the CNSC upon request and that the CNSC is formally notified of the results of each candidate
   2. ensure that each candidate who was assigned a pass result receives and completes remedial training and formal evaluations in every area identified as deficient by the marking process to confirm and document that the candidate has successfully attained the required level of knowledge required to competently perform in the certified position
   3. ensure that any deficiencies revealed in paragraph 1 above are addressed

Part III: Performance-Based Certification Examination Requirements

This part establishes the required simulator-based initial certification examinations and the requirements regarding the design, development, conduct and grading of these examinations that must be successfully completed by reactor operator (RO), unit 0 operator (U0O), plant shift supervisor (PSS) and control room shift supervisor (CRSS) candidates who have successfully completed all the prerequisite requirements including the required knowledge-based initial certification examinations.

All performance-based examinations shall be conducted in a full-scope simulator.

13. Simulator-Based Certification Examinations

The required simulator-based examinations shall consist of a number of test scenarios that examine the performance requirements of each position relevant to the operation of the NPP where initial certification is sought. The examination specific to the position shall cover a series of NPP malfunctions that creates abnormal NPP conditions, failures and transients requiring the candidates to demonstrate their skills in all applicable competency areas. The required categories of malfunctions are described in subsection 13.4.

The test scenarios shall be designed to measure the candidate's level of competency in five generic competency areas at the specific NPP:

1. monitoring
2. actions taken without reference to procedures
3. diagnosis and decision making
4. procedure compliance
5. communication and crew interaction skills

Over the entire examination, the candidates shall be tested at least 25 times in each of the competency areas, with the exception of competency area 2 (actions taken without reference to procedures) where the candidates shall be tested at least 15 times. The required competency areas are described in subsection 13.5.

The required performance-based initial certification examinations shall be as follows:

1. Reactor operator examination: This examination shall consist of three comprehensive test scenarios (CTS). These test scenarios examine the performance requirements of the RO position relevant to the operation of the single-unit or multi-unit NPP where initial certification is sought
2. Unit 0 operator examination: This examination shall consist of two CTSs at NPPs that have a Unit 0. These test scenarios examine the performance requirements of the U0O position relevant to the operation of the applicable multi-unit NPP where initial certification is sought
3. Plant shift supervisor examination: This examination shall consist of two CTSs and two abridged test scenarios (ATSs), including a number of specific standard questions. These test scenarios examine the performance requirements of the PSS position relevant to the operation of the single-unit NPP where initial certification is sought specifically covering the necessary skills for when the RO is temporarily absent from the control room
4. Control room shift supervisor examination: This examination shall consist of two CTSs, three diagnostic test scenarios (DTSs) and one panel check test scenario(PCTSs), including several specific standard questions. These test scenarios examine the performance requirements of the CRSS position relevant to the operation of the multi-unit NPP where initial certification is sought

13.1 Test scenario descriptions

The required test scenarios referred to above that shall form the examinations relevant to the positions at an NPP are as follows:

1. CTS: This shall be a dynamic test scenario consisting of an integrated sequence of primary and secondary malfunctions that simulates a succession of abnormal NPP conditions, failures or transients requiring the candidates to respond to in accordance with the NPP-documented performance expectations.

   For the PSS and CRSS positions, these tests also include a number of specific standard questions at the end of the dynamic portion of each test. The questions are designed to determine the significant concerns the candidates have at the end of each test scenario including the levels of impairment, to outline the required course of actions to address these concerns and determine the priority for these actions.

2. ATS: This shall be a dynamic test scenario consisting of a number of primary and secondary malfunctions that simulates a number of abnormal NPP conditions, failures or transients, requiring PSS candidates to demonstrate their ability to respond to these abnormal situations when the RO is temporarily absent from the control room.

   For the PSS position, these tests also include several specific standard questions at the end of the dynamic portion of each test. The questions are designed to determine the accuracy of the candidates' diagnosis, any other observed problems, to determine the significant concerns candidates have at the end of each test scenario and to outline the required course of actions and the priority for these actions.

3. DTS: This shall be a dynamic test scenario consisting of a single primary malfunction and a number of secondary malfunctions requiring the CRSS candidates to demonstrate their ability to:

   1. independently monitor the evolution of multi-unit NPP conditions
   2. recognize abnormalities
   3. determine the significance
   4. diagnose malfunctions
   5. select the applicable procedures
   6. determine the required course of actions when procedures do not exist or are deficient

   For the CRSS position, these tests also include a number of specific standard questions at the end of the dynamic portion of each test. The questions are designed to determine the accuracy of the candidates' diagnosis, any other observed problems, to determine the significant concerns candidates have at the end of each test scenario including the levels of impairment, and to outline the required course of actions and the priority for these actions.

4. PCTS: This shall be a non-dynamic test scenario during which NPP conditions are stable requiring the candidates to demonstrate their ability to perform a verification task of control room panels relevant to the CRSS position by recognizing panel anomalies or abnormal conditions and by determining the significance.

   For the CRSS position, this test also includes a number of specific standard questions at the end of the test scenario. The questions are designed to demonstrate the candidates' ability to determine the significance of the anomalies and abnormal conditions and the required course of actions to address them.

The criteria for the test scenarios and the specific standard questions are covered in section 14 under "examination design and development".

13.2 Test scenario and examination time durations

Table IV summarizes the required number and type of test scenarios in an examination for each position requiring certification, and the estimated test scenario dynamic duration for each scenario. The table also provides examination duration estimates for each test scenario and examination.

13.3 Examination bounding envelope

Simulator-based certification examinations shall be designed and developed to examine the performance requirements of a position at the NPP for which initial certification is sought. Each simulator-based certification examination shall consist of a specific number of test scenarios with a defined number and type of NPP malfunctions to test all the applicable competency areas for the position.

No previously conducted test scenario shall be reused in its entirety to design and conduct another test scenario or certification examination.

The following subsections outline the categories of NPP malfunctions and the competency areas required for the design and development of each certification examination detailed in section 14.

13.4 Primary and secondary malfunctions

13.4.1 Primary malfunctions

A primary malfunction shall consist of a failure of equipment, of a control device, of a component of a system, or a combination of such failures, that creates at the time of its occurrence during a simulator-based test scenario the main abnormal condition, failure or transient that must be addressed by the candidate.

Primary malfunctions shall be divided into the following four categories:

Category 1: Malfunctions requiring the execution of a routine operation

Any deterioration or failure of one or more components or pieces of equipment affecting a single system that requires the candidate to perform a routine equipment changeover, within a limited time frame, to prevent equipment damage or a system transient due to an impending automatic action to protect the equipment involved. If properly addressed, such a primary malfunction will not cause any significant system or unit transient.

Note: Category 1 primary malfunctions shall not to be used in examinations for PSS or CRSS candidates.

Category 2: Malfunctions requiring the execution of a non-routine operation with no immediate significant reactor unit transient

Any deterioration or failure of one or more components or pieces of equipment affecting one or more systems that requires the candidates to perform or direct non-routine actions, within a limited time frame, for one of the following reasons:

• to prevent equipment or system damage while minimizing the deterioration of system, unit or NPP conditions resulting from these actions
• to minimize the deterioration of system, unit or NPP conditions following any automatic action to protect the equipment involved
• to prevent or correct an impairment of a safety or safety support system

If properly addressed, such a primary malfunction occurring on a reactor unit will not by itself cause a change in reactor power or in turbine-generator load of greater than 10%. However, a change in reactor power or in turbine-generator load of greater than 10% may subsequently take place due to the required operator actions.

Category 3: Malfunctions causing a significant reactor unit transient

Any failure of one or more components or pieces of equipment that causes a change in reactor power or in turbine-generator load of greater than 10%.

This category also includes any malfunction other than those causing an emergency condition, as defined under Category 4 below, that results in any one or any sequence of the following:

1. reactor setback
2. reactor stepback
3. reactor trip
4. turbine trip, runback or load rejection

Category 4: Malfunctions causing an emergency condition

Any failure of one or more components or pieces of equipment that results in one of the following emergency conditions affecting one or more reactor units and the application of one or more of the applicable emergency operating procedures:

1. loss of heat transport system inventory on a reactor unit: defined as a sustained loss of inventory outside the heat transport system envelope that is greater than the make-up capability of the available feed pumps or that requires the manual or automatic initiation of the emergency core cooling system; primary malfunctions that may cause this emergency condition include boiler tube failures
2. loss of steam pressure on a reactor unit: defined as any sustained loss of steam pressure that causes a shutdown system trip on low heat transport system pressure or any steam line break that creates a widespread hazard to NPP personnel and equipment
3. total loss of Class IV, Class III or Class II electrical power on one or more units, as appropriate
4. total loss of Class IV and Class III electrical power on one or more units, as appropriate
5. loss of instrument air on a reactor unit: defined as a non-recoverable loss of instrument air pressure
6. loss of boiler feedwater on a reactor unit: defined as a sustained loss of feedwater to the boilers that requires the use of a back-up heat sink; auxiliary boiler feed pumps or feedwater ties are not considered as back-up heat sinks in this context
7. total loss of high- or low-pressure service water on a reactor unit: includes a loss of powerhouse upper level service water at Darlington NGS, if the loss is due to a break, and a loss of recirculated service water system at a 600 MW plant
8. complete loss of both control computers of a reactor unit

Note: Only those Category 4 primary malfunctions that require a complex intervention by U0Os shall be credited as a Category 4 primary malfunction in test scenarios for U0O candidates.

13.4.2 Secondary malfunctions

Secondary malfunctions shall be supplementary malfunctions consisting of either additional malfunctions or monitoring malfunctions as follows:

1. Additional malfunctions consist of a failure of a piece of equipment, of a control device or of a component of a system to respond correctly when called upon to act, either automatically or by operator intervention, or a failure of an operating piece of equipment, subsequent to a primary malfunction.
   Each additional malfunction shall meet all of the following criteria:
   1. it has an observable effect on the indications on the simulator control room panels or it is reported from the field
   2. it is an equipment or component failure that could occur during NPP operation, including failure of poised equipment to respond
   3. if not properly addressed by the candidate, it will cause deviation to an unacceptable value of one or more system parameters, or it will create another undesirable system condition
   4. it only requires a limited number of corrective actions from the candidates

   Additional malfunctions shall be included in the test scenarios to aid in assessing the ability of the candidates to monitor changes in system and unit conditions and to respond to these changes.

2. Monitoring malfunctions consist of a failure of any indicating device on the control room panels.
   Each monitoring malfunction shall meet all of the following criteria:
   1. the indicating device should fail as it would during operation at the NPP
   2. the indicating device is expected to be used by the candidates during the test scenario
   3. there are at least two other means to obtain or derive the same information from simulator control room or field indications
   4. the failure of the indicating device does not influence the applicable system process

Monitoring malfunctions shall be included in the test scenarios to aid in assessing the ability of the candidates to monitor system and unit conditions.

13.5 Competency areas and generic performance expectations

The five competency areas referred to in section 13 each have associated generic performance expectations (GPEs) that shall be used for assessing the performance of candidates in the position at the NPP where initial certification is sought. Each associated list of GPEs per competency area specifically addresses performance requirements for a given position.

The standard of acceptable performance used with each generic performance expectation is dictated by the NPP's corresponding documented performance expectations applicable for each position. The required competency areas and the corresponding GPEs for each position are specified in the applicable appendices C.1 to C.3.

The integration of the GPEs is specified in section 14 under the examination design and development requirements.

14. Examination Design and Development

In accordance with the minimum requirements specified in subsections 14.1 to 14.4 and under the direction of the lead examiner, the examination team shall, design and develop the required certification examinations, referred to in section 13, for each position at the specific NPP where initial certification is sought.

In addition, the examination team shall:

1. develop a design checklist for each test scenario CTS, ATS, DTS and PCTS by using, at a minimum, all the design requirements specified in sections 14.1 to 14.4 and Appendices C.4 to C.10; the design checklist must also specify the NPP, the test scenario title, the position for which the candidates are seeking certification, the examination date, exactly what criteria and where in section 3 of the examiner's guides each criteria has been met, and the printed names and signatures of the lead examiner and training manager

   The design checklist for each test scenario should be similar to the one shown in appendix D.7.

2. develop an examination design checklist for each examination using, at a minimum, the design requirements specified in sections 14.1 to 14.4 and include a check to indicate that the examination has not been used for the conduct of a previous examination; the examination design checklist must also specify the examination title indicating the position for which the candidates are seeking certification, the examination date, the test scenario that has met each of the criteria, and the printed names and signatures of the lead examiner and training manager

   The examination design checklist for each examination specific to a position should be similar to the one shown in appendix D.8.

14.1 Reactor operator examination

The simulator-based examination for RO candidates shall be designed to meet the following criteria:

1. the entire examination shall consist of three CTSs
2. each CTS shall be designed in accordance with the criteria in appendix C.4, and in accordance with section 14.5 for the applicable examiner's guide
3. the CTSs shall cover a broad range of system operations, equipment malfunctions and unit transients
4. duplication in the required operator actions shall be minimized among CTSs
5. initial NPP conditions, including reactor power, shall be varied among CTSs
6. at least one CTS shall:
   1. start with initial NPP conditions different from normal full power conditions that have a significant impact on the actions to be taken by the candidates
   2. include a category 4 primary malfunction as defined in paragraph 13.4.1
   3. include a failure of a major automatic action during a transient that must be detected by the candidates and that requires them to take the appropriate corrective actions
   4. include a situation where a requirement in the operating policies and principles (OP&Ps) is not met that must be recognized and addressed by the candidates
   5. include the occurrence of an impairment of a special safety system or standby safety support system that must be recognized and addressed by the candidates
   6. at a multi-unit NPP, include conditions on one or more reactor units, other than the simulated reactor unit, that prevent or significantly delay the arrival at this reactor unit of one or both assisting ROs from other units, or that require one or both assisting ROs to subsequently leave the simulated reactor unit during the CTS
7. at least two CTSs shall:
   1. include one or more situations that require the candidates to determine or recommend appropriate course of actions because procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective
   2. include challenging concurrent malfunctions or unit conditions that require the candidates to assign priorities to their actions or to the actions of the support team
8. over the entire examination, candidates shall be tested in all competency areas listed in appendix C.1 as follows:
   1. at least 25 times in competency areas 1, 3, 4 and 5
   2. at least 15 times in competency area 2
9. the test scenarios and the entire examination shall be designed to be conducted within the estimated time durations shown in table 4
10. following the completion of designing a test scenario, a design checklist shall be completed to confirm that the specific test scenario meets the criteria in the checklist
11. each test scenario design checklist shall be reviewed and approved, with the date, name and signature of the lead examiner and training manager prior to developing the corresponding examiner's guide
12. following the completion of designing the three CTSs, an examination design checklist shall be completed to confirm that the examination meets the criteria in the checklist
13. the examination design checklist shall be reviewed and approved, with the date, name and signature of the lead examiner and training manager prior to developing the corresponding examiner's guide

14.2 Unit 0 operator examination

The simulator-based examinations for U0O candidates shall be designed to meet the following criteria:

1. the entire examination shall consist of two CTSs
2. each CTS shall be designed in accordance with the criteria in appendix C.5, and in accordance with section 14.5 for the applicable examiner's guide
3. the CTSs shall cover a broad range of system operations, equipment malfunctions and transients relevant to the U0O position
4. duplication in the required operator actions shall be minimized among CTSs
5. initial NPP conditions, including conditions of the reactor units, shall be varied among CTSs
6. at least one CTS shall:
   1. start with initial conditions of systems operated by U0Os different from normal operating conditions specified in operating procedures that have a significant impact on the actions to be taken by the candidates
   2. include a Category 4 primary malfunction, as defined in paragraph 13.4.1, affecting one or more reactor units that requires a complex intervention by the candidates
   3. include a failure of a major automatic action during complex operations of unit 0 systems that must be detected by the candidates and that requires them to take the appropriate corrective actions
   4. include a situation where a requirement in the OP&Ps is not met, that must be recognized and addressed by the candidates
   5. include the occurrence of an impairment of a special safety system or standby safety support system that must be recognized and addressed by the candidates
7. one CTS shall test the candidates in the role of the lead U0O and the other shall test the candidates in the role of the assisting U0O
8. there shall be at least two situations that require the candidates to determine or recommend appropriate course of actions because procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective
9. there shall be at least two occurrences of challenging concurrent malfunctions or conditions of systems operated by U0Os that require the candidates to assign priorities to their actions or to the actions of the support team
10. in both CTSs, the arrival of the second U0O in the control room shall be delayed for approximately 15 minutes
11. over the entire examination, candidates shall be tested in all competency areas listed in appendix C.1 as follows:
    1. at least 25 times in competency areas 1, 3, 4 and 5
    2. at least 15 times in competency area 2
12. the test scenarios and the entire examination shall be designed to be conducted within the estimated time durations shown in table 4
13. following the completion of designing each test scenario, a design checklist shall be completed to confirm that the specific test scenario meets the criteria in the checklist
14. each test scenario design checklist shall be reviewed and approved, with the date, name and signature of the lead examiner and training manager, prior to developing the corresponding examiner's guide
15. following the completion of designing the two CTSs, an examination design checklist shall be completed to confirm that the examination meets the criteria in the checklist
16. the examination design checklist shall be reviewed and approved, with the date, name and signature of the lead examiner and training manager, prior to developing the corresponding examiner's guide

14.3 Plant shift supervisor examination

The simulator-based examinations for PSS candidates at single-unit NPPs shall be designed to meet the following criteria:

1. the entire examination shall consist of two CTSs and two ATSs
2. each CTS shall be designed in accordance with the criteria in appendix C.6, and in accordance with section 14.5 for the applicable examiner's guide
3. each ATS shall be designed in accordance with the criteria in appendix C.7 , and in accordance with section 14.5 for the applicable examiner's guide
4. the test scenarios shall cover a broad range of system operations, equipment malfunctions and NPP transients
5. duplication in the required actions shall be minimized among test scenarios
6. initial NPP conditions, including reactor power, shall be varied among test scenarios
7. at least one test scenario shall:
   1. start with initial NPP conditions different from normal full-power conditions that have a significant impact on the response expected from the candidates
   2. include a category 4 primary malfunction as defined in paragraph 13.4.1
   3. include a failure of a major automatic action during a transient that must be detected by the candidates and that requires them to take the appropriate corrective actions
   4. include the occurrence of an impairment of a special safety system or standby safety support system that must be recognized and addressed by the candidates
8. at least one CTS shall include concurrent malfunctions or NPP conditions that require the candidates to give directions to the support team on the execution of two complex procedures or courses of actions that need to be implemented promptly
9. at least one ATS shall include concurrent malfunctions or NPP conditions that require the candidates to assign priorities to their actions or to the actions of the support team
10. at least two test scenarios shall:
    1. include one or more situations that require the candidates to determine appropriate course of actions because procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective
    2. include a situation where a requirement in the OP&Ps is not met that must be recognized and addressed by the candidates
11. over the entire examination, candidates shall be tested in all competency areas listed in appendix C.2 as follows:
    1. at least 25 times in competency areas 1, 3, 4 and 5
    2. at least 15 times in competency area 2
12. the test scenarios and the entire examination shall be designed to be conducted within the estimated time durations shown in table 4
13. following the completion of designing each test scenario, a design checklist shall be completed to confirm that the specific test scenario meets the criteria in the checklist
14. each test scenario design checklist shall be reviewed and approved, with the date, name and signature of the lead examiner and training manager, prior to developing the corresponding examiner's guide
15. following the completion of designing the two CTSs and the two ATSs, an examination design checklist shall be completed to confirm that the examination meets the criteria in the checklist
16. the examination design checklist shall be reviewed and approved, with date, name and signature of the lead examiner and training manager, prior to developing the corresponding examiner's guide

14.4 Control room shift supervisor examination

The simulator-based examinations for CRSS candidates at multi-unit NPPs shall be designed to meet the following criteria:

1. the entire examination shall consist of two CTSs, three DTSs and the one PCTS
2. each CTS shall be designed in accordance with the criteria in appendix C.8 and in accordance with section 14.5 for the applicable examiner's guide
3. each DTS shall be designed in accordance with the criteria in appendix C.9 , and in accordance with section 14.5 for the applicable examiner's guide
4. the PCTS shall be designed in accordance with the criteria in appendix C.10 , and in accordance with section 14.5 for the applicable examiner's guide
5. the test scenarios shall cover a broad range of system operations, equipment malfunctions and NPP transients
6. duplication in the required actions shall be minimized among test scenarios
7. initial NPP conditions, including reactor power, shall be varied among test scenarios
8. at least one CTS shall:
   1. start with initial NPP conditions different from normal full power conditions that have a significant impact on the response expected from the candidates
   2. include concurrent malfunctions or plant conditions that require the candidates to give directions to the support team on the execution of two complex procedures or courses of actions that need to be implemented promptly
   3. include challenging concurrent malfunctions on the reactor unit and on unit 0, or on the reactor unit and on simulated systems common to more than one reactor unit, that require the candidates to assign priorities to their actions or to the actions of the support team
9. during one CTS, the candidates shall be required to give detailed instructions to the support team on the course of actions to be taken in a situation that necessitates a prompt execution of a complex sequence of operator actions because the RO or the U0O needs assistance from the CRSS
10. at least one DTS shall include challenging concurrent abnormal NPP conditions that require the candidates to assign priorities to the actions required to address those conditions
11. at least one CTS or DTS shall:
    1. include a Category 4 primary malfunction as defined in paragraph 13.4.1
    2. include a failure of a major automatic action during a transient that must be detected by the candidates and that requires them to take the appropriate corrective actions
12. at least one test scenario shall include the occurrence of an impairment of a special safety system or standby safety support system that must be recognized and addressed by the candidates
13. at least two test scenarios shall:
    1. include one or more situations that require the candidates to determine an appropriate course of actions because procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective
    2. include a situation where a requirement in the OP&Ps is not met that must be recognized and addressed by the candidates
14. over the entire examination, candidates shall be tested in all competency areas listed in appendix C.3 as follows:
    1. at least 25 times in competency areas 1, 3, 4 and 5
    2. at least 15 times in competency area 2
15. the test scenarios and the entire examination shall be designed to be conducted within the estimated time durations shown in table 4
16. following the completion of designing each test scenario, a design checklist shall be completed to confirm that the specific test scenario meets the criteria in the checklist.
17. each test scenario design checklist shall be reviewed and approved, with the date and signature of the lead examiner and training manager, prior to developing the corresponding examiner's guide
18. following the completion of designing the two CTSs, the three DTSs and the PCTS, an examination design checklist shall be completed to confirm that the examination meets the criteria in the checklist
19. the examination design checklist shall be reviewed and approved, with date of signature, by the lead examiner and training manager prior to developing the corresponding examiner's guide

14.5 Examiner's guides

The examination team shall, further to the test scenario design criteria specified in appendices C.4 to C.10, develop a corresponding examiner's guide for each test scenario in accordance with the following criteria:

1. each examiner's guide shall have a cover page with the information shown in appendix C.11
2. for test scenarios CTS, ATS and DTS, the examiner's guide shall contain a number of sections and parts within the applicable sections with the information specified in 14.5.1
3. for test scenario PCTS, the examiner's guide shall contain a number of sections and parts within the applicable sections with the information specified in 14.5.2

14.5.1 Examiner's guide for test scenarios CTS, ATS and DTS

Section 1 - Test scenario summary

Section 1 of the examiner's guide, shown in appendix C.12, shall consist of at least three parts and shall be completed during the design of each test scenario and modified to reflect any significant change made to the scenario during its development.

Part A of this section of the examiner's guide shall contain:

1. a detailed description of the scenario:
   1. the initial unit conditions and the initiating malfunctions that start the test
   2. the evolution of the reactor unit, or unit 0 as applicable, and key systems in response to the initiating malfunctions and subsequent malfunctions or candidate actions
   3. a description of the expected candidate's responses to the reactor unit, or unit 0 as applicable, and key systems conditions or malfunctions
   4. the key procedures the candidate is expected to access
   5. any expected prioritization of the candidates' responses
   6. any required notification to be made by the candidates
   7. when each role player is expected to arrive, what their role is, and when they should leave
   8. any other detail on how the scenario is designed to unfold when conducted
   9. a clearly defined end point of the test
2. a list of all the malfunctions used, the corresponding procedures to be used with the revision number and impact on the candidate or NPP evolution
3. a list of the key alarms and annunciations
4. a list of all the equipment that is out of service with the corresponding impact on the candidate or NPP evolution

Part B of this section of the examiner's guide shall contain the system parameters to be recorded during the conduct of the examination.

Part C of this section of the examiner's guide shall contain the minor adjustments made during conduct and reasons for the adjustments.

Section 2 - Initial NPP conditions

This section of the examiner's guide shall contain details of the initial NPP conditions defined during the design of the test scenario:

1. reactor power of each reactor unit
2. generator load of each reactor unit
3. unit control mode of the simulated reactor unit, if applicable
4. average liquid zone level of the simulated reactor unit
5. state of major systems of the various units
6. equipment out of service
7. fuelling machine activities in progress
8. routine tests and maintenance activities in progress
9. other NPP conditions

This section of the examiner's guide shall also detail where each member of the control room support team will be at the start of the test scenario.

This information shall be given to the candidates during the turnover at the start of the test scenario.

Section 3 - Candidate action checklist

This section of the examiner's guide shall:

1. detail the sequence of actions and checks expected from the candidates in response to the malfunctions and conditions of the test scenario in a checklist format similar to that shown in appendix C.13. Each check in the checklist shall include the corresponding GPE identified from appendix C.1, C.2 or C.3, as applicable to the position
2. contain detailed instructions for the individual members of the control room support team and for the person directing the activities of the field operators or playing their role. These instructions shall be written according to the criteria in appendix C.14
3. contain, for PSS and CRSS candidates, the standard questions to be asked at the end of the test scenario, specified in appendix C.15, with complete and accurate answers and with the corresponding GPE identified from appendix C.2 or C.3 as applicable to the position.

The candidate action checklist shall be developed to allow the examiners to record accurately the response of the candidates to the conditions of the scenario, independent of the audiovisual recordings and of the operator action monitor. It shall reflect as closely as possible the performance of a typically qualified incumbent of the position sought by the candidates and shall contain the following information arranged in the anticipated sequence of occurrence, with references to the appropriate GPEs for each step:

For CTS and ATS test scenarios:

The following are candidate action checklist items that shall be included for a CTS and an ATS:

1. the primary malfunctions and when each is initiated
2. the sequence of key alarms and annunciations received on the control room panels and cathode-ray tubes (CRTs) or computer monitors after the initiation of each primary malfunction
3. the secondary malfunctions that follow the occurrence of a primary malfunction and when each is observable
4. any key alarm or annunciation on the control room panels and CRTs (or computer monitors) associated with each secondary malfunction
5. when each member of the control room support team summoned will be instructed to arrive at the relevant unit in the control room
6. the alarms, annunciations, equipment malfunctions and any other information that the members of the control room support team must communicate to the candidates and when they must be communicated
7. the field conditions and any other information that the person directing the activities or playing the role of the field operator must communicate to the candidates and when they must be communicated
8. the checks of automatic actions that the candidates must perform, specifying the indicators to be used (see note below). For checks of automatic actions of complex systems where a number of identical devices are actuated (e.g., emergency core cooling system), generic statements may be used (e.g., check D2O isolation valves open)
9. the checks of system parameters and indicators that the candidates must perform, including those to determine the nature of a primary malfunction and to identify the applicable operating procedures, specifying the indicators to be used
10. the actions and checks that the candidates must perform or must instruct the members of the control room support team to perform, based on diagnosis, prior to accessing the applicable operating procedures, specifying the controls or devices to be operated and the indicators to be used to perform the checks
11. for RO and U0O candidates and for PSS candidates during an ATS, the actions and checks that the candidates must perform or must instruct the members of the control room support team to perform to implement each applicable step of the relevant procedures, specifying the controls or devices to be operated and the indicators to be used to perform the checks; these actions and checks include any standard operating practice that is not mentioned explicitly in a step of the procedure, but is associated with its execution, based on training and on the expectations of NPP management
12. for PSS and CRSS candidates during a CTS, the specific instructions that the candidates must give to the members of the control room support team to direct them to implement the applicable operating procedures
13. for PSS and CRSS candidates during a CTS, the actions and checks that the candidates must instruct the members of the control room support team to perform whenever they request assistance from the shift manager while implementing a procedure, specifying the devices to be operated and the indicators to be used to perform the checks
14. for PSS and CRSS candidates, the actions and checks that the candidates must perform to implement any step of an applicable procedure specifically assigned to the PSS or CRSS
15. when operating procedures do not specifically address a malfunction or NPP condition, the actions and checks that the candidates must perform or must instruct the members of the control room support team to perform to place the unit, the systems or the equipment in the required state, specifying the controls or devices to be operated and the indicators to be used to perform the checks
16. the checks that the candidates must perform to determine the nature of any secondary malfunction and to identify the applicable operating procedure, if any, specifying the indicators to be used
17. the actions and checks that the candidates must perform or must instruct the members of the control room support team to perform upon detection of each secondary malfunction, specifying the controls or devices to be operated and the indicators to be used to perform the checks
18. for RO and U0O operator candidates and for PSS candidates during an ATS, the specific instructions with the appropriate demonstration, if required, that the candidates should give to the members of control room support team when delegating an activity and, for each, the specific information to be reported back once it is completed
19. all requests for activities in the control equipment room expected from the candidates and for each activity, the time that would be required for completing it at the NPP and the information to be reported back to the candidates by the member of the control room support team involved once the activity is completed
20. all requests for field activities expected from the candidates and for each activity, the number of field operators and the time that would be required for completing it at the NPP and the information to be reported back to the candidates once the activity is completed
21. the specific information that the candidates should give to the support team members
22. the notifications and any request for approval that the candidates must make
23. for PSS and CRSS candidates, the checks of system, unit and NPP conditions, constraints and limits that the candidates must perform before approving request by the support team
24. at multi-unit NPPs, when any certified person on the support team will be instructed to leave the simulated reactor unit or unit 0 during the test scenario and, if applicable, when the person will be instructed to return to the relevant unit
25. for PSS and CRSS candidates, the standard questions in part A of appendix C.15, with complete and accurate answers

The checklist shall specify the indicator or the set of indicators to be used for checking or monitoring a parameter or condition when it is specified in the relevant operating procedure, or when only one indicator or set of indicators is reliable under the existing circumstances.

For a DTS test scenario:

The following are candidate action checklist items that shall be included for a DTS:

1. the primary malfunction
2. the sequence of key alarms and annunciations received on the control room panels and CRTs (or computer monitors) after the initiation of the primary malfunction
3. any secondary malfunction that follows the occurrence of the primary malfunction and when it is observable
4. any key alarm or annunciation on the control room panels and CRTs (or computer monitors) associated with a secondary malfunction
5. the field conditions and any other information that the person playing the role of the field operators must communicate to the candidates and when they must be communicated
6. the checks of major automatic actions and major system parameters that the candidates must perform, specifying the indicators to be used (see note above). For checks of automatic actions of complex systems where a number of identical devices are actuated (e.g., emergency core cooling system), generic statements may be used (e.g., check D2O isolation valves open)
7. the checks or combinations of checks of system parameters and indicators that the candidates must perform to determine unambiguously the nature of the primary malfunction and to identify the applicable operating procedures, specifying the indicators to be used (see note above); when there is more than one approach to reach the correct diagnosis, each acceptable combination of checks that supports a unique conclusion must be documented
8. the checks that the candidates must perform to determine the nature of any secondary malfunction and to identify the applicable operating procedures, if any, specifying the indicators to be used (see note above)
9. the actions that the candidates must instruct the members of the control room support team to perform when a major automatic action fails to occur
10. all requests for verifications in the control equipment room expected from the candidates and for each verification, the time that would be required for completing it at the NPP and the information to be reported back to the candidates
11. all requests for verifications in the field expected from the candidates and, for each verification, the minimum time that would be required for completing it at the NPP and the information to be reported back to the candidates; when it is acceptable to report back, before the minimum time, field information requested that is essential to make the correct diagnosis, the control room checks that a candidate must have performed before the field information is reported back
12. the DTS dynamic duration allowed for making the correct diagnosis
13. the standard questions in part B of appendix C.15, must reflect the actual conditions of the scenario, with complete and accurate answers

Section 4 - Simulator operator instructions

This section of the examiner's guide shall detail the activities of the simulator operator during the test scenario. It shall contain the following instructions arranged in the planned sequence of execution:

For CTS and ATS test scenarios:

1. instructions to set up the simulator
2. the checks required before the start of the scenario to ensure that:
   1. the simulator is in the proper configuration to simulate the initial unit conditions and the equipment out of service at the beginning of the scenario
   2. all equipment out of service is correctly identified on the unit panels
   3. the required malfunctions of poised equipment are programmed
   4. all panel lights are functional and the panel horn is on
   5. all data collection devices are operational and synchronized
   6. the telephone system, the radiation emergency warning siren, the fire emergency warning siren and the public address system are functional
3. the sequence of primary and secondary malfunctions to be entered during the test scenario, with the conditions for their activation; these conditions may be any of the following:
   1. a specific system or unit condition
   2. the completion of a specified step in an operating procedure
   3. the completion of a specified action in the control room by a candidate or by a member of the support team
   4. a signal from the lead examiner
   5. a specified time after a given occurrence during the scenario

      Caution should be taken by the lead examiner using a specified time as the condition for activation of a malfunction unless it can be accurately determined based on an existing condition.

4. the actions required to reproduce in the control room the outcome of each operation performed in the field or in the control equipment room and the time after which the outcome will be seen in the control room, based on the time that would be required for completing the operation at the NPP
5. the specific data to be collected after completion of the test scenario, based on the list in appendix C.16

For a DTS test scenario:

1. instructions to set up the simulator
2. the checks required before the start of the scenario to ensure that:
   1. the simulator is in the proper configuration to simulate the initial unit conditions and the equipment out of service at the beginning of the scenario
   2. all equipment out of service is correctly identified on the unit panels
   3. any secondary malfunction associated with poised equipment is programmed
   4. all panel lights are functional and the panel horn is on
   5. all data collection devices are operational and synchronized
   6. the telephone system, the radiation emergency warning siren, the fire emergency warning siren and the public address system are functional
3. the primary malfunction to be entered at the start of the DTS
4. any additional malfunction associated with an operating piece of equipment to be entered during the DTS, with the conditions for its activation
5. the specific data to be collected after completion of the DTS, based on the list in appendix C.16

Section 5 - Instructions for support team staff in the control room

This section of the examiner's guide shall contain separate instructions for each member of the control room support team that specify the actions they must perform during the test scenario. It is completed after the rehearsal of the scenario with the support team. The sequence of instructions for each member is compiled from the candidate action checklist section of the examiner's guide.

Section 6 - Field operator instructions

This section of the examiner's guide shall contain instructions that specify the actions that the person directing the activities or playing the role of the field operators must perform during the test scenario. It is completed after the rehearsal of the scenario with the support team. The sequence of instructions for each field operation expected is compiled from the candidate action checklist section of the examiner's guide.

14.5.2 Examiner's guide for test scenario PCTS

The examiner's guide for the test scenario PCTS shall contain a number of sections and parts within the applicable sections similar to that shown in appendix C.12 with the following required information:

Section 1 - Test scenario summary

Section 1 of the examiner's guide, shown in appendix C.12, shall consist of at least two parts and shall be completed during the design of the PCTS and modified to reflect any significant change made to the scenario during its development.

Part A of this section of the examiner's guide shall contain:

1. a detailed description of the verification task to be performed by the candidates
2. the list of panel anomalies to be identified and their impact on the candidate or NPP

Part B of this section of the examiner's guide shall contain the minor adjustments made during conduct and reasons for the adjustments.

Section 2 - Initial NPP conditions

This section of the examiner's guide shall contain the same information specified above under initial NPP conditions for the CTS, ATS and DTS examiner's guides.

Section 3 - Candidate action checklist

This section of the examiner's guide shall detail the answers expected of a typically qualified CRSS to the standard questions asked at the end of the PCTS.

This candidate action checklist shall be developed to allow the examiners to accurately record the answers given by the candidates independently of the audiovisual recordings. It shall contain the following information:

1. the control room indications associated with each panel anomaly
2. any expected request from the candidates for field information required to assess the abnormal system condition associated with a panel anomaly and for each, the specific information that the lead examiner must give to the candidates
3. the maximum time allowed for completing the verification task
4. the standard questions shown in part C of appendix C.15, with complete and accurate answers

Section 4 - Simulator operator instructions

This section of the examiner's guide shall contain details of the activities of the simulator operator before the start of the PCTS. It shall contain the following instructions:

1. instructions to set up the simulator
2. the checks required before the start of the scenario to ensure that:
   1. the simulator is in the proper configuration to simulate the initial unit conditions and the equipment out of service at the beginning of the scenario
   2. all equipment out of service is correctly identified on the unit panels
   3. all panel anomalies are set up properly
   4. all panel lights are functional and the panel horn is on
   5. the audiovisual system is operational

Section 5 - Instructions for the candidates

This section of the examiner's guide shall contain specific instructions regarding the verification task to be performed by the candidates during the PCTS.

14.6 Examination validation

The examination team shall, under the direction of the lead examiner ensure that each CTS, ATS, DTS and PCTS examiner's guideis validated on the NPP simulator for which initial certification is sought.

This validation shall be performed prior to obtaining the required approvals of the examiner's guide for the conduct of a given examination.

At a minimum, this examination validation shall be done in accordance with the following requirements:

1. ensure a person who is currently or previously certified at the NPP for which the test is designed, and who has not been involved in any aspect of the test scenario design, acts as the candidate
2. perform the validation in real time on the NPP simulator with the person acting as the candidate to confirm that:
   1. each test scenario unfolds as planned in the corresponding examiner's guide

      Closely verify the fidelity of the simulator response to the malfunctions and other scenario conditions inserted and to the expected operator actions.

   2. the alarms received in the simulator and the parameters listed in the checklist of appendix C.17 are verified to ensure that:
      1. the key alarms are received when expected and in the correct sequence
      2. the values of each simulated system parameter have approximately the correct magnitude and their excursions, if any, have approximately the correct magnitude and duration, based on the applicable laws of physics and the characteristics of the equipment and systems of the reference unit
      3. the relevant system logic control circuits operate correctly
      4. the response of the simulated unit and systems to the malfunctions and the expected candidate responses are realistic and that there are no misleading differences between the simulated response and that of the NPP reference unit
   3. the required candidate actions and checks and the applicable operating documentation used in response to the sequence of malfunctions are consistent with the NPP-documented performance expectations
   4. the sequence of the required candidate actions is clear and unique and that a typically qualified incumbent of the position sought by the candidate would be expected to respond to each malfunction
   5. for test scenarios that include situations for which the operating procedures give no specific instructions, that the course of expected candidate actions is clear and unique and consistent with the NPP-documented performance expectations
   6. the timing of the malfunctions is such that a typically qualified incumbent of the position sought by the candidate would have sufficient time to respond as expected to each malfunction
   7. any secondary malfunction only affects those indicators, equipment, components or control devices that the candidates are expected to check or operate during the dynamic duration of the test scenario and can be clearly detected by the candidate
   8. for the DTS, there is sufficient information and time for the candidate to respond as expected to the secondary malfunctions and to make the correct diagnosis, and the unit conditions have not deteriorated to a level that would prevent, or interfere with, making the required diagnosis
   9. for the PCTS, there is sufficient time for the candidate to complete the verification task as would a typically qualified CRSS
   10. for the DTS, any appropriate course of actions to respond to a failure of a major automatic action prior to diagnosing the primary malfunction is clear and does not have an impact on unit conditions that would prevent, or interfere with, making the correct diagnosis
   11. the dynamic duration of each test scenario does not significantly exceed the prescribed time limit in the examiner's guide and table 4 above
3. confirm that the list of system parameters to be recorded during conduct of the test scenarios, which is contained in section 1 (test scenario summary) of the examiner's guides, is appropriate and complete
4. for the CTS and ATS, confirm that the steps in section 3 (candidate action checklist) of the examiner's guides do not contain situations where the candidates are most likely to make errors or create deviations that could force the early termination of the test scenario or change the planned evolution of the test scenario to where the examiner's guide could no longer be used to reliably record the performance of the candidate
5. confirm that for each CTS and DTS, there is no duplication in the candidate action checklist (section 3) of the examiner's guides between the performance items measured during the dynamic portion of the test scenario and the answer items for the questions asked at the end of each scenario
6. confirm that each item of the candidate action checklist (section 3) of each examiner's guide is assigned the appropriate generic performance expectations from appendix C.1 to C.3, as applicable
7. for the PCTS, confirm that the panel anomalies can be clearly detected from the control room panels by the candidate during the execution of the verification task
8. for the PCTS, confirm that the control room indications associated with each panel anomaly are the same as they would be at the NPP reference unit under the defined initial unit conditions
9. confirm, at the completion of the dynamic portion of each test scenario, that the answers to the standard questions are complete and technically accurate by asking the person acting as the candidate to respond to the test standard questions from the applicable part of appendix C.15 which have been included in the examiner's guide
10. ensure the instructions for the support team are compiled in sections 5 (instructions for support team staff in the control room) and 6 (field operation instructions) of the examiner's guides
11. confirm that any anticipated requests for information, corrective actions or notifications from the candidate to support team staff are identified in section 5 (instructions for support team staff in the control room) of the examiner's guide
12. confirm that any actions and communications performed by support team staff are technically accurate and consistent with the NPP-documented performance expectations
13. confirm that, over the entire examination, each competency area 1, 3, 4 and 5 is tested at least 25 times and competency area 2 is tested at least 15 times
14. finalize the version of the examiner's guides in the light of the outcome of the validation of each test scenario with the support team
15. modify the test scenario and corresponding examiner's guide to reflect the response expected from a typically qualified incumbent of the position sought by the candidate if at a minimum:
    1. any of steps 2 through to 12 performed can not be confirmed or ensured
    2. any malfunction and its consequence cannot be simulated realistically
    3. conditions exist for which no unique sequence of actions can be predicted with confidence
    4. to correct any technical error and have it reflect the response expected from a typically qualified operating team

       If during validation it is found that a malfunction is not simulated correctly, it is acceptable to reproduce the desired effect by simulator operator intervention provided this will not be noticeable to the candidates. Otherwise, another malfunction should be used. Similarly, if the test scenario as a whole cannot be simulated correctly, another test scenario should be designed.

16. ensure that any modifications made to the test scenario or corresponding examiner's guide that:
    1. the test design criteria is still met
    2. another validation is performed by repeating steps 2 to 12 above as determined by the lead examiner
17. complete the applicable test scenario validation checklist with, at a minimum, the information shown in appendix C.17
18. ensure that the simulator is restored to a normal configuration so as not to reveal the content of the test scenarios developed or to be conducted, before releasing the simulator for other uses

14.7 Final examiner's guides and examination design

The lead examiner, at the completion of each required examination specified in sections 14.1 to 14.4 with the corresponding examiner's guides specified in section 14.5. and after completing all the validation steps in section 14.6, shall ensure that the final examiner's guides, the test scenario design checklists, the examination design checklists and the test scenario validation checklists are reviewed and approved, with the date and signatures of the lead examiner and training manager before the rehearsal of each test scenario. The rehearsal of each approved test scenario shall be performed immediately prior to the conduct of the test scenario in accordance with subsection 15.1.

15. Conduct of Performance-Based Certification Examinations

The licensee shall have documented policies, processes and procedures in place regarding the conduct of performance-based examinations.

The licensee shall ensure that:

1. the examinations are conducted by the lead examiner and at least one other examiner who was responsible for the examination design and development
2. the lead examiner coordinates all the activities of the examination team
3. the examination team members do not prompt the candidates or offer any suggestion or solution regarding expected diagnoses of malfunctions, decisions to be made or actions to be performed

   Any prompting or directing of the candidate towards the correct answer or path can result in rendering a test scenario and/or a certification examination invalid for the purposes of certification.

4. the examination team members minimize verbal communications and the use of body language during the conduct of an examination
5. the code of conduct and instructions for briefing the examination team, the support team and for briefing the candidates have been explained
6. the standard questions conducted after the dynamic portion of a test scenario are performed within a reasonable time limit and any supplementary questions asked by the lead examiner do not lead the candidates to the correct answer
7. the certification examinations are conducted in accordance with the applicable maximum time limits set in table 4 of this document
8. observers are briefed in accordance with section 6 of this regulatory document
9. all the test scenarios are recorded using an audiovisual system

   A test scenario will not be recognized by the CNSC unless it has been properly recorded.

10. the simulator is restored to a normal configuration after any certification examination-related activity to avoid revealing the content of the test scenarios developed or to be conducted, before releasing the simulator for other uses

The lead examiner shall schedule the candidates for a given examination and ensure that:

1. in advance of the scheduled examinations, each candidate is briefed on the roles and responsibilities of the various persons involved in the conduct of the examination, on the performance expected from the candidate during the conduct of the examination including the need to verbalize their actions for the recordings and on the rules they must abide by; the minimum requirements for the lead examiner in this briefing are specified in appendix C.18
2. all candidates are examined, one at a time, on a given test scenario before conducting the next test scenario
3. the order in which the candidates are examined must be varied from one test scenario to another
4. the total duration of the test scenarios conducted each day does not place excessive demands on the candidates or on the examination team

   Each candidate test needs to be delivered in a consistent manner and can not be adversely affected by the performance of any examination team member due to fatigue. The lead examiner needs to carefully consider the examination team workload when determining or modifying the testing schedule.

5. candidates who voluntarily withdraw or cheat at any time during the conduct of the examination, are automatically assigned a fail grade for the examination, and this is documented in the candidate's results form (see appendix C.23)
6. prior to the conduct of a test scenario, a rehearsal of the test scenario is performed in accordance with section 15.1

15.1 Rehearsal of test scenarios prior to conduct

The examination team, under the direction of the lead examiner, shall perform a rehearsal of each approved test scenario (CTS, ATS, DTS and PCTS) with the support team on the NPP simulator for which the approved examination is to be conducted and where initial certification is sought.

This rehearsal shall be performed:

1. at a reasonable time immediately prior to conducting each approved test scenario for the first time
2. to ensure the simulator operator and the members of the support team are fully familiar with their roles and responsibilities during the conduct of the test
3. to ensure that the operator action monitor, the alarm message printer and the parameter recording devices required by the test scenario are operational
4. in real time with a person acting as the candidate who is qualified in the position sought by the candidate to ensure that:
   1. the approved test scenarios unfold as planed in the approved examiner's guides
   2. the simulator response is in accordance with the approved examiner's guides
   3. the support team members respond as specified in sections 3 (candidate action checklist) and 4 (simulator operator instructions) of the approved examiner's guides
5. to ensure that copies of all documents and data related to the examination that may compromise its security are controlled. This includes examiner's guides, alarm printouts, parameter trends, marked up operating procedures, flowsheets, operating and training documentation and personal notes. Ensure no examination material is left unsecured or unattended at any time, including when using copying machines in public areas
6. to ensure that, at the end of each rehearsal, any required minor adjustments made by the lead examiner to an approved examiner's guide are documented in section 1 (test scenario summary) part C of the guide including the reason why each adjustment was required

   Examples of a minor adjustment include the alteration of a support team member's feedback in response to an expected request from the candidate, editorial errors in sections 3 (candidate action checklist) of the examiner's guide, or an incorrect operating procedure referenced in section 3 of the examiner's guide that was not identified during test validation.

7. to ensure that the parameter trends recorded during the rehearsal are kept for reference

   The recorded parameter trends may be used as reference during the assessment and grading of the candidates and may be requested by the CNSC as part of the inspection criteria.

8. the simulator is placed in a secure configuration to avoid compromising the security of the examination

15.2 Conduct of the dynamic portion of a CTS, ATS or DTS

The lead examiner shall coordinate the conduct of each approved test scenario in accordance with the following requirements and instructions:

1. prior to permitting a candidate into the simulator control room for the conduct of a test scenario, ensure that:
   1. the simulator is set in the proper configuration for the test scenario
   2. the operator action monitor, the alarm message printer, the parameter recording devices and the audiovisual systems are in service and synchronized
   3. any equipment out of service is properly identified on the panels in accordance with section 1, part B of the approved examiner's guide
   4. the supporting documentation or information available to the candidates is limited to that approved for use in the control room at the NPP
   5. all support team members participating in the applicable test scenario are present and in their assigned locations
   6. the audiovisual systems for recording communications between the candidate and the support team members are in service
2. when the candidate is permitted to enter into the simulator control room and prior to starting the conduct of the test scenario, ensure:
   1. the audiovisual system is recording
   2. the candidate is given written turnover specifying the existing NPP conditions for the test scenario, as detailed in section 2 of the approved examiner's guide, and these conditions are described to the candidate
   3. the candidate is told that there is no abnormality in the existing NPP conditions other than those just described
   4. the positions and names of the applicable support team members are identified to the candidate
   5. the candidate is given a maximum of five minutes to review the existing conditions prior to commencing the test.
3. when the five minutes have elapsed, or sooner at the request of the candidate, notify the candidate that the test scenario is about to start and start conducting the test by running the scenario
4. during the conduct of the test:
   1. ensure the test scenario proceeds as determined in the approved examiner's guide including, as applicable to the scenario, the sequenced malfunctions, the receipt of the key alarms, the response of the system parameters, operation of the logic control circuits and the performance of the support team
   2. in parallel with the other examiner, record the performance of the candidate in section 3 of the approved examiner's guide by placing a check mark against each check or action, or group of checks or actions, successfully performed by the candidate; document on the checklist any change to the planned evolution of the test scenario, any significant deviation from the checks to be made by the candidate applicable to the scenario and any unexpected action performed by the candidate or any unexpected request to support team members made by the candidate
   3. the lead examiner, in consultation with the other examiner, shall abort a test scenario if, at a minimum, one of the abort conditions specified in part A of appendix C.19 arises at any time during the conduct of the test scenario. If an abort condition is met:
      1. immediately instruct the simulator operator to freeze the simulator
      2. follow the abort instructions provided in part B of appendix C.19
5. at the defined end point of a CTS or an ATS ensure that:
   1. the simulator operator freezes the simulator
   2. the actual duration of the test scenario is recorded in the approved examiner's guide
   3. the candidate and the support team are informed that the end point of the test scenario has been reached and that the simulator has been frozen
   4. the candidate is instructed to remain at the operator desk and prevented from viewing the panels and alarms while the examination team reviews the data collected in a secure location to determine whether any performance clarification questions are required
   5. the support team members are instructed to leave the simulator, dropping off any documentation or notes with the examiners and to remain on standby
   6. in a secure location and away from the candidate's audible range, the examiners shall:
      1. compare the information recorded in the candidate action checklist with that recorded by the other examiner to identify discrepancies in the information gathered
      2. determine if there is a need to ask the candidate any supplementary questions to clarify any aspect of the observed performance based on the discrepancies identified and on the individual recorded examiner notes, which may affect the assessment and grading of the candidate
      3. document all supplementary questions in the lead examiner's approved examiner's guide before proceeding to ask the candidate

         The supplementary questions should be strictly related to the observed performance of the candidate and do not preclude the standard questions that are required to be asked of the CRSS and PSS candidates immediately following the dynamic portion of the test scenarios.

   7. if there is a need to ask the candidate any supplementary questions, that the audiovisual system for recording the questions and the answers are in service, and that the candidate answers are documented in parallel with the other examiner in the approved examiner's guides
   8. for the PSS and CRSS candidates, the non-dynamic portion of the test scenario immediately continues by having the candidate respond to the standard questions as specified in subsection 15.4

      For the RO and U0O candidates, this represents the end of the test scenario. The audiovisual recordings can be stopped and the candidate can be permitted to leave the simulator.

6. at the defined end point of a DTS ensure that:
   1. the simulator operator freezes the simulator
   2. the actual duration of the test scenario is recorded in the approved examiner's guide
   3. inform the candidate and the support team that the end point of the test scenario has been reached and that the simulator has been frozen
   4. the candidate is instructed to remain at the operator desk averted from viewing the panels and alarms
   5. the support team members are instructed to leave the simulator and to leave any documentation or notes with the examiners
   6. the non-dynamic portion of the test scenario, applicable to the PSS and CRSS candidates, immediately continues by having the candidate respond to the standard questions as specified in subsection 15.4

15.3 Conduct of the dynamic portion of a PCTS

The lead examiner shall coordinate the conduct of a PCTS in accordance with the following requirements and instructions:

1. prior to permitting a candidate into the simulator control room for the conduct of a PCTS ensure that:
   1. a suitable test turnover area with a visual barrier is in place to prevent the candidate from seeing the simulator configuration
   2. the simulator is set in the proper configuration for the test scenario
   3. the audiovisual systems are in service and synchronized
   4. any equipment out of service is properly identified on the panels in accordance with section 1, part B of the approved examiner's guide
   5. the supporting documentation or information available to the candidates is limited to that approved for use in the control room at the NPP
   6. the audiovisual system for recording communications between the candidate and the support team members are in service
2. when the candidate is permitted to enter the test turnover area in the simulator control room and prior to starting the conduct of the test scenario ensure:
   1. the audiovisual system is recording
   2. the candidate is given written turnover specifying the existing NPP conditions for the test scenario, as detailed in section 2 of the approved examiner's guide, and these conditions are described to the candidate
   3. the candidate is given a copy of section 5 of the approved examiner's guide, and describe the verification task to be performed by the candidate and include any relevant NPP operating documentation that is necessary for the candidate to complete the task
   4. the candidate is informed of the 15 minutes maximum time requirement for completing the verification task
3. instruct the candidate to start performing the verification task with access to operating documentation
4. during the verification task:
   1. monitor the panel checks the candidate performs, including those required by any supporting operating documentation, in parallel with the other examiner
   2. respond to the expected requests for field verifications immediately following the candidate's request
   3. in consultation with the other examiner, respond in a timely manner to any unanticipated requests made by the candidate for field verifications by providing the accurate information for the existing NPP conditions
   4. note in section 3 of the approved examiner's guide any significant deviations from the required panel checks performed by the candidate and any unexpected requests made by the candidate for field verifications
5. at the end point of the PCTS, when the candidate has completed the verification task or the 15 minute time limit has expired, instruct the candidate to remain at the operator desk, prevented from viewing the panels and alarms, and immediately proceed to the required standard questions as specified in subsection 15.4

15.4 Conduct of the standard questions for PSS and CRSS candidates

The lead examiner, following the dynamic portion of a CTS, ATS, DTS and PCTS, shall:

1. ensure the audiovisual system for recording the questions and the answers are in service
2. for the CTS, ATS and DTS, ask the candidate the standard questions in Part A or Part B of appendix C.15 applicable to the test scenario, as documented in the approved examiner's guide and ensure that:
   1. the candidate is reminded that access to operating documentation is permitted for answering the questions
   2. the questions are conducted one at a time and that the candidate has finished answering a question before proceeding to the next question

      Caution should be taken not to lead the candidate by indicating that the answer is complete, satisfactory or otherwise.

   3. a copy of the question is provided to the candidate and read out loud
   4. in parallel with the other examiner, document the candidate's answers in section 3 of the approved examiner's guide and note any significant differences with the expected answer
   5. any operating documentation the candidate references is recorded in the approved examiner's guide
   6. after the candidate has answered all the standard questions:
      1. in a secure location and away from the candidate's audible range, determine if there is a need to ask the candidate any supplementary questions
      2. document all supplementary questions in the lead examiner's approved examiner's guide before proceeding to ask the candidate

         Supplementary questions are to be strictly related to the observed performance of the candidate or to the answers given by the candidate and can not lead the candidate to the correct response. The purpose of supplementary questions is to obtain clarification on the observed performance or the answers given, which may affect the assessment and grading of the candidate.

   7. if there is a need to ask the candidate any supplementary questions, ensure the audiovisual system is recording the questions and the answers, and that the candidate answers are documented in parallel with the other examiner in the approved examiner's guides
3. for the PCTS, immediately ask the candidate the first standard question in part C of appendix C.15, as documented in the approved examiner's guide and ensure that:
   1. in parallel with the other examiner, the candidate's answers are documented in section 3 of the approved examiner's guide and any significant differences with the expected answer are noted
   2. when the candidate has completed answering the first question, the candidate is given a copy of the remaining questions in part C and is given 10 minutes to prepare for answering these questions orally, with access to operating documentation
   3. when the 10 minutes allocated time period has elapsed, the candidate is asked to answer the remaining questions
   4. after the candidate has answered all the standard questions:
      1. in a secure location and away from the candidate's audible range, determine if there is a need to ask the candidate any supplementary questions to obtain clarification where there is uncertainty on the observed performance or on the answers given, which may affect the assessment and grading of the candidate
      2. document all supplementary questions in the lead examiner's approved examiner's guide before proceeding to ask the candidate

         The supplementary questions are to be strictly related to the observed performance of the candidate or to the answers given by the candidate and cannot lead the candidate to the correct answer.

   5. if there is a need to ask the candidate any supplementary questions, that the audiovisual system is recording the questions and the answers, and that the candidate's answers are documented in parallel with the other examiner in the approved examiner's guides
4. at the end the test scenario, stop the audiovisual recordings, permit the candidate to leave the control room and have the simulator operator collect the data recorded during the test scenario
5. in parallel with the other examiner, document the areas of concern and any significant misconceptions demonstrated by the candidate in the approved examiner's guide
6. complete a data collection checklist similar to that shown in appendix C.16 and ensure that it is attached to the data collected
7. ensure the data collected and the marked-up approved examiner's guides are securely stored
8. ensure there are no controlled copies of the approved examiner's guides or other material around that may compromise the security of the examination
9. ensure to reset the simulator and the control panel devices in preparation for the next candidate

15.5 End of examination conduct

The lead examiner shall, at the completion of a given examination when all the required test scenarios have been conducted, ensure that:

1. any simulator deficiencies or any deficiencies in the NPP documentation encountered during the conduct of the examination is documented and addressed accordingly
2. the necessary modifications to the approved examiner's guides are made to account for any minor adjustments made during the conduct of the examination and ensure the applicable GPEs listed in appendix C.1, C.2 or C.3, as appropriate are assigned to the additional actions or checks

   During the conduct of a simulator-based examination, any changes or adjustments made by the lead examiner to the approved examiner's guides should only be of minor impact. Circumstances may occur during conduct where the candidate performed the expected action according to operational expectations, which was not adequately or was inaccurately reflected in the examiner's guide and was not identified during test validation or rehearsal. As a result, the lead examiner may need to adjust the examination and corresponding candidates' examiner's guides to reflect such unforeseen circumstances. These adjustments should be formally documented in the examiner's guides and used in the crediting and grading of the individual candidate or of all of the candidates.

   An examination that has been validated by a currently certified incumbent should not cause the candidate to deviate from the expected course of action in a manner that results in a significant shift from the approved examiner's guide. Such candidate deviations should result in the examination, or part thereof, to be aborted or invalidated and redesigned.

3. any minor adjustments made as described in 2 above are documented in section 1, part C of each of the candidate's examiner's guide
4. the approved examiner's guides are authorized prior to the assessment of the candidate's performance and the grading of the examination
5. for any candidate for whom a CTS or an ATS was aborted and not resumed, that:
   1. the part of the examination completed by the candidate meets the minimum criteria for a CTS or an ATS and meets the minimum criteria for the examination as specified in section 14, and:
      1. if the part of the examination completed meets those minimum criteria, proceed with the first assessment of the candidate in accordance with section 16
      2. if the part of the examination completed does not meet those minimum criteria and a fail grade was not assigned on the part of the examination that was completed, one or more additional test scenarios needs to be designed, developed and conducted to complete the examination of the candidate
6. for any candidate for whom a DTS was aborted and not resumed, that:
   1. the assessment of the candidate is performed for the part of the examination that was completed
   2. if the candidate is not assigned a failed grade on the part of the examination that was completed, one or more additional DTSs need to be designed, developed and conducted to complete the examination of the candidate
7. a file with all the data collected during the conduct of the examination, as specified in the data collection checklist (appendix C.16), and the marked up authorized examiner's guides from both members of the examination team is prepared for each candidate and used in the assessment and grading as specified in section 16

16. Grading of Examinations

The licensee shall have documented policies, processes and procedures in place regarding the grading of performance-based examinations. This shall include the following:

1. the examinations shall be graded immediately or as soon as practicable after conduct

   The grading of examinations is an integral part of the administration of the certification examination process. Grading should be completed immediately following the conduct of a certification examination. This regulatory document does not stipulate an absolute deadline for grading examinations to allow licensees to deal with unforeseen circumstances. However as part of administering the certification examinations, licensees should have the necessary resources from the onset of planning a given examination.

2. the results of each candidate shall be determined using the data collected during the conduct of the examination
3. the approved examiner's guides shall be authorized by the training manager prior to the assessment of the candidate's performance and the grading of the examination
4. only the examination team involved in the design and conduct of the examination shall perform the first and second assessment of the candidates' performance and perform the grading of the complete examination
5. the examination team shall perform the grading in accordance with sections 16.1-16.3
6. the grading of the complete examination shall be performed on one candidate at a time
7. the first assessment and grading shall be completed on all candidates prior to performing the second assessment
8. the second assessment and grading shall be performed on selected candidates by an examiner who did not perform the first assessment and grading on those candidates
9. a critical or significant error assessment form, which contains the information shown in appendix C.21, shall be completed for each candidate if applicable
10. an examination result form, which contains the information shown in appendix C.23 shall be completed for each candidate

16.1 First assessment

The lead examiner shall ensure that the candidate files are divided between the members of the examination team for the first assessment and that the first assessment of each candidate is performed in accordance with the following:

1. for each test scenario, prepare a new copy of the authorized examiner's guide for each candidate
2. using the file created for each candidate as described in section 16.4 paragraph 7, check off in the new candidate action checklist for each test scenario the items for which both examiners gave credit in their respective candidate action checklists during the conduct of the examination
3. in the case when a test was aborted, assess only those actions and checks that the candidate had an opportunity to perform up to the point where the abort condition occurred
4. grade the complete examination of one candidate at a time
5. whenever a performance item in the candidate action checklist has not been credited or has been credited by only one examination team member or is unclear, refer to the corresponding alarm message printout, the operator action monitor printout and the audiovisual recording to determine whether the credit is warranted

   The parameter trends recorded during the rehearsal of the test scenario performed during the development of the examination may be used as a reference for this determination.

6. record the result of the determination from paragraph 5 above and the rationale in the candidate action checklist
7. for a CRSS or PSS candidate, whenever an answer item to a standard question has not been credited or has been credited by only one examination team member or is unclear, refer to the corresponding alarm message printout, the operator action monitor printout and the audiovisual recording to determine whether the credit is warranted or not
8. record the result of the determination from paragraph 7 above and the rationale in the candidate action checklist
9. for a CRSS or PSS candidate, record any unexpected answers to a standard question given by the candidate at the proper location in the candidate action checklist
10. if a supplementary question was asked, record the question and the candidate's response in the candidate action checklist
11. review all the data in the consolidated new candidate action checklist and for each:
    1. omitted action or check
    2. unexpected action or check
    3. omitted or unexpected answer to a standard question

    determine, considering the answer to any supplementary question, whether it constitutes a critical or a significant error according to the criteria in appendix C.20

12. for each critical or significant error identified, complete a critical or significant error assessment form similar to that shown in appendix C.21
13. compile and record at the end of the consolidated checklist any areas of concern related to the candidate's performance or knowledge found during conduct or the first assessment of the examination, including the responses to any supplementary questions
14. refer to the corresponding audiovisual recording to confirm that the areas of concern are accurately described
15. perform the first grading in accordance with paragraph 16.1.1

First grading of the examination

As part of the first assessment, the examination team shall:

1. calculate, by referring to appendix C.22, the scores obtained by each candidate using all consolidated candidate action checklists and document the following:
   1. the score obtained for each generic performance expectation in the entire examination
   2. the score obtained for each competency area in each test scenario
   3. the score obtained for each competency area in the entire examination
2. assign a pass result or a fail result in accordance with subsection 16.3
3. complete an examination result formfor each candidate that contains the information shown in appendix C.23
4. at the completion of the first assessment, sign and date the cover page of each candidate's consolidated examiner's guide

16.2 Second assessment

Upon completion of the first assessment and grading of all candidates in accordance with subsection 16.1 and paragraph 16.1.1, the second assessment of each candidate shall be performed in accordance with the following:

1. the training manager shall:
   1. approve the candidates selected by the examiners to undergo a second assessment
   2. select a minimum of one candidate graded by each examiner who performed the first assessment
   3. ensure the second assessment of a candidate is performed by the member of the examination team who did not perform the first assessment
   4. ensure the second assessment is performed on one candidate at a time in accordance with the instructions for the first assessment given in section 16.1

      The minimum second grading requirement is to perform a calibration check on the accuracy of the examiner's guide and assessment of the grading process.

2. the lead examiner shall select candidates for a second assessment according to the following criteria:
   1. candidates who do not meet any of the pass criteria in section 16.3
   2. candidates for whom the examiner who did the first assessment recommends a second assessment
   3. candidates for whom significant changes to the planned evolution of a CTS or an ATS occurred
   4. additional candidates at the discretion of the training manager or lead examiner
3. the examination team shall:
   1. perform the second assessment using the signed consolidated examiner's guides of the candidate, any completed critical or significant error assessment form, the scores obtained by the candidate in the examination, the candidate's comment file and any examination files of the candidate from the examiner who performed the first assessment
   2. ensure that any changes made to the information documented and recorded by the first examiner, as a result of this second assessment, are clearly identified in the consolidated examiner's guide
   3. make the necessary changes to the simulator-based examination result forms of the candidates and sign the forms

Second grading of the examination and reconciliation of results

Upon completion of the second assessment of all the candidates selected in accordance with section 16.2, the examiner shall perform the second grading of the examination in accordance with the procedure for the first grading.

Once the scores obtained by each candidate have been calculated, the examiners who performed the first and the second assessment shall:

1. meet to resolve any discrepancies between their two assessments and document how these discrepancies were resolved

   The examiners should refer to the audiovisual recordings of the candidate's performance to resolve discrepancies of actions taken or not taken by the candidate during the conduct of a test scenario.

2. review the complete examination of one candidate at a time
3. document the reconciled examination results and the justifications in the original simulator-based examination result form and the critical or significant error assessment form for each candidate
4. ensure that any final changes made to consolidated examiner's guide for each candidate have the required reviews and approvals

16.3 Assignment of examination results

The examination team shall assign a pass or fail result to a given certification examination in accordance with the following criteria:

1. a pass result shall be assigned to an examination if a candidate has either:
   1. scored 80 percent or higher in each competency area and has not made a critical error or no more than two significant errors with no more than one significant error in the same competency
   2. has scored 70 percent or higher in no more than one competency area with 80 percent or higher for the average of all competency areas, and has not made acritical error or no more than two significant errors with no more than one significant error in the same competency
2. a fail result shall be assigned to a given examination if a candidate has not met the pass criteria above
3. the examination team shall:
   1. document the final examination results and the justifications in the original simulator-based examination result form for each candidate
   2. ensure the results form is reviewed and signed by the training manager

17. Examination Follow-Up

At the completion of the examination, the following steps shall be performed:

1. The examination team shall document and report to the training manager any significant performance deficiencies revealed by a candidate or by an examination.

   The deficiencies revealed by an examination should include those found in the examination design, conduct, grading, simulator fidelity, NPP procedures or training program.

2. The examiners shall ensure that a final assessment package is compiled for each candidate consisting of:
   1. the completed simulator-based examination result form signed by both examiners and authorized by the training manager
   2. the final scores obtained in the examination
   3. the final version of the candidate's comment file, if applicable
   4. any completed critical or significant error assessment formsigned by both examiners
   5. the consolidated examiner's guide signed by both examiners
   6. the signed test scenarios and examination checklists
3. The training manager shall:
   1. after authorizing the results of any given simulator-based certification examination, ensure that a copy of the assessment package specified in 17.2 above is made available to the CNSC upon request and that the CNSC is formally notified of all the results of each candidate
   2. ensure that each candidate who was assigned a pass result receives and completes remedial training and formal evaluations in all areas identified as deficient by the grading process to confirm and document that the candidate has successfully attained the level of knowledge and skills required to competently perform in the certified position
   3. ensure that any deficiencies revealed in 17.1 above are addressed

Appendix A: Simulator Capabilities for Existing Licensed Nuclear Power Plants

Simulators used for certification examinations for RO, U0O, PSS and CRSS candidates must meet the requirements specified below.

A.1 Minimum acceptable simulation capabilities

Simulators must be capable of simulating, realistically and in real time, all significant nuclear power plant (NPP) manoeuvres and transients, including:

• normal
• NPP anticipated operational occurrences
• NPP design-basis accidents

For conditions and failures, such as pipe breaks, loss of inventory, loss of flow, loss of pressure and loss of vacuum, for which unit response and operator actions are a function of the degree of severity of the condition or failure, the simulator must have adjustable rates for the condition or failure covering its entire possible range.

Specifically, simulators must be capable of simulating the abnormal, transient and emergency conditions and failures listed below.

Note In the list, loss of a system includes, but is not limited to, loss of circulation, loss of cooling and loss of inventory.

A1.1 Special safety systems

1. shutdown system (SDS) 1 and SDS 2 trips on all parameters (at the applicable NPPs)
2. reactor protective system (SDSA) and shutdown system enhancement (SDSE) trips on all parameters (at the applicable NPP)
3. neutron overpower detector response to abnormal flux shapes
4. spurious initiation of any special safety system, including for the emergency core cooling system spurious loop isolation, where applicable, and spurious crash cooling
5. safe and unsafe failures of any channel of any special safety system, including failures of start-up instrumentation
6. any special safety system impairment documented in plant procedures that is observable from the main control room

A.1.2 Reactivity effects and reactivity control

1. any shutoff or absorber rod falling partially or fully in core
2. any adjuster or absorber rod driving out of sequence
3. any shutoff, adjuster or absorber rod stuck out of core, or partially or fully in core
4. reactor stepback on all parameters (at the applicable NPPs)
5. reactor setback on all parameters
6. single and dual computer failures
7. drifts and failures of input and output signals of the reactor regulation system control program, including drifts and failures of nuclear instrumentation
8. single and dual failures of the reactor regulation system control program, including failures of the stepback program where applicable
9. loss of reactor regulation
10. reactor flux tilts
11. drifts and failures of input signals of the flux mapping, ZOTPR, FINCH or CTM computer program, as applicable
12. single and dual failures of the flux mapping, ZOTPR, FINCH or CTM computer program, as applicable
13. liquid zone control system failures, including:
    1. loss of helium inventory
    2. loss of balance header pressure control
    3. loss of bubbler header pressure control
    4. loss of water pressure
    5. loss of water inventory, including leak of a zone compartment to the moderator
    6. loss of water flow to any zone compartment

A.1.3 Heat transport system

1. pressure tube leaks into annulus gas
2. reactor fuel channel flow blockages
3. loss-of-coolant accidents (LOCAs) from reactor inlet feeders and headers
4. LOCAs from reactor outlet feeders and headers
5. in-core LOCAs caused by the failure of a pressure tube and its calandria tube
6. boiler tube failures on any boiler
7. preheater tube failures on any preheater (at the applicable NPPs)
8. failure of any seal or combination of seals on any main heat transport pump
9. trip of any one or more main heat transport pumps
10. natural modes of heat transport circulation under abnormal and emergency conditions, including single phase and two phase thermosyphoning
11. operation with the heat transport system partially drained
12. failures associated with inter-unit D2O transfers at multi-unit plants
13. heat transport system pressure and inventory control failures, including:
    1. drifts and failures of input and output signals
    2. single and dual failures of the associated control programs, where applicable
    3. loss of control in normal and in solid mode (at the applicable NPPs)
14. failure open or closed of one or two feed or bleed valves
15. single and dual heat transport feed pump trips
16. failure open of one or more heat transport system liquid relief valves
17. failure open of one or two pressurizer steam bleed valves or relief valves (Pickering NGS A and B excepted)
18. pressurizer steam bleed line failure (at the applicable NPPs)
19. failure open of one or two bleed condenser relief valves (600-MW plants excepted)
20. failure open of one or two degasser condenser relief valves at 600-MW plants
21. bleed condenser tube bundle leaks (600-MW plants excepted)
22. bleed cooler or degasser condenser cooler tube leaks, as applicable
23. loss of shutdown or maintenance cooling system, as applicable, including heat exchanger tube failures
24. high activity in the heat transport system

A.1.4 Heat sinks

1. loss of condenser vacuum
2. loss of condenser cooling water
3. loss of condensate, including pipe breaks
4. deaerator level control failures, including:
   1. drifts and failures of input and output signals
   2. single and dual control program failures, where applicable
5. loss of feedwater to any combination of boilers
6. failures associated with inter-unit feedwater ties, where applicable
7. loss of emergency feedwater to the boilers
8. boiler level control failures, including:
   1. drifts and failures of input and output signals
   2. single and dual failures of the boiler level control program (at the applicable NPPs)
9. symmetric and asymmetric boiler feed line breaks, inside and outside containment
10. boiler pressure control failures, including:
    1. drifts and failures of input and output signals of the boiler pressure control program
    2. single and dual failures of the boiler pressure control program
11. failure open or close of one or more ASDVs or CSDVs (at the applicable NPPs)
12. failure open or close of one or more safety relief valves (SRVs) (at the applicable NPPs)
13. symmetric and asymmetric main steam line breaks outside containment
14. steam line breaks inside containment, where applicable
15. loss of moderator system, including:
    1. pipe breaks inside and outside containment or confinement
    2. heat exchanger tube leaks
    3. calandria tube leaks into annulus gas
16. moderator temperature control failures, including:
    1. drifts and failures of input and output signals
    2. single and dual control program failures, where applicable
    3. spurious crash cooling of the moderator system
17. loss of end shield cooling system, including pipe breaks
18. loss of service water systems, such as low pressure service water, high pressure service water, recirculated cooling water and common service water, including pipe breaks

A.1.5 Electrical systems

1. failure of one or more transmission lines
2. partial and complete main generator load rejections
3. failures of one or more switchyard breakers and buses
4. large main generator load variations due to system frequency disturbances
5. loss of isolated phase bus cooling
6. loss of Class IV power
7. loss of Class III power
8. loss of Class IV and Class III power
9. failure of any Class IV bus
10. failure of any Class IV breaker
11. failure of any Class III bus
12. failure of any Class III breaker
13. effects of the loss of any Class II or Class I power supply, bus, panel and fuse
14. trip and failure to start of one or more standby generators
15. electrical transfer failures
16. transformer failures
17. inverter, converter and rectifier failures
18. emergency power system failures, including trip or failure to start of one or more emergency power generators

A.1.6 Process systems

1. loss of instrument air and service air
2. partial loss of instrument air and service air
3. failures associated with inter-unit instrument air and service air ties, at multi-unit plants
4. loss of component cooling, including, but not limited to, pumps, compressors and heat exchangers
5. process system instrumentation, alarms and control failures

A.1.7 Overall unit

1. turbine trips
2. main generator or exciter trips
3. failure of any main steam valve
4. drifts and failures of input and output signals of the turbine governing system and the turbine tripping system
5. drifts and failures of input and output signals of the unit power regulator control program and turbine run-up control program
6. single and dual failures of unit power regulator control program
7. single and dual failures of turbine run-up control program
8. manual poison prevent operation and pseudo poison prevent operation
9. loss of main steam reheat system
10. failure of the moisture separator drains system
11. main generator seal failures
12. loss of main generator hydrogen cooling, including heat exchanger tube leaks
13. loss of main generator stator cooling, including pipe breaks
14. condenser tube leaks
15. loss of any low pressure or high pressure feedwater heater, or heater bank, including tube breaks
16. deuterium excursions

A.2 Data collection devices

Simulators must be equipped with data recording devices that meet the requirements specified in paragraphs 1 to 2.b. These devices must be capable of being synchronized to within two seconds of each other.

1. simulators must be equipped with an operator action monitor capable of printing in chronological order, with their respective time of occurrence, all malfunctions initiated by the simulator operator and all the actions performed on the control panels during a test scenario
2. simulators must have provisions for either:
   1. tracing, with adequate precision, graphics of any selection of 48 system parameters versus time for up to two hours and for printing those graphics
   2. storing and printing the values versus time of any selection of 48 system parameters sampled at an adequate frequency during a period of up to two hours
3. the simulator must be equipped with an audiovisual system that:
   1. is capable of recording all actions performed by a candidate in the control room during a test scenario
   2. has sufficient resolution to permit the examiners to identify, with the aid of the corresponding control panel photographs, the controls and instruments used by a candidate
   3. is capable of displaying time on the recordings
   4. is capable of recording clearly all verbal communications and telephone conversations between a candidate and other team members during a test scenario
   5. allows for an easy identification of the voices of the different participants

The simulator operating facility must be separated from the control room so that a candidate cannot become aware of the data recorded or of the inputs to the simulator being entered by the simulator operator.

A.3 Other devices

Simulators must be equipped with the following functional devices, which replicate those of the plant's main control room:

1. a telephone system
2. a radiation emergency warning siren
3. a fire emergency warning siren
4. a public address system

Appendix B: Knowledge-based Certification Examinations - Additional Requirements

B.1 Topic groups for the general examinations for RO, PSS and CRSS candidates

These required topic groups are based on the training program in science fundamentals and equipment principles for RO candidates that have been developed in accordance with the principles of a systematic approach to training (SAT).

The various topics that should be covered in the general examinations for RO, PSS and CRSS candidates have been provided for each required group, as shown below in a guidance box, to assist in their selection so that each examination contains an appropriate mix of topics to cover the knowledge that ROs, PSSs and CRSSs are required to have. All topics selected within a topic group shall be linked to specific learning objectives in accordance with subsection 9.3 and made available to the CNSC upon request.

The required topic groups, total marks and marks to be allocated to questions are identified below in bold and shall be used by the examination team to design and develop the General certification examinations.

Total marks: 100

A) Principles of reactor operation and control

1. fission chain reaction, including:
   1. prompt and delayed neutrons and associated effects
   2. neutron life cycle and the six factor formula
2. moderator properties and moderation process
3. neutron interactions with reactor core materials
4. heat production in a reactor
5. spatial dependence of the neutron flux in a reactor
6. critical reactor operation, including:
   1. power response to positive and negative reactivity insertions
   2. reactor period
   3. prompt jump or drop
7. subcritical reactor operation, including:
   1. sources of neutrons
   2. power response to positive reactivity insertions
   3. power doubling rule
8. prompt criticality

1. reactivity effects arising from operation at high power, including:
   1. burn up of U-235
   2. build up of fission products
   3. production and removal of xenon and samarium
   4. xenon and samarium transients
   5. build up and burn up of plutonium isotopes
   6. plutonium transients
   7. reactivity changes due to burn-up
2. addition and removal of chemical neutron poisons in the moderator for reactivity control, including:
   1. poisons used under various operating conditions
   2. limits on poison concentrations
   3. methods of verification and control of poisons
3. xenon oscillations and flux tilts, including:
   1. causes of xenon oscillations
   2. means of controlling xenon oscillations
4. effects of temperature changes on reactivity, including:
   1. temperature coefficients of reactivity
   2. power coefficient and its effect on reactor regulation and protection
   3. moderator temperature limit and consequences of operating above that limit
5. effects of coolant voids on reactivity, including effect of core voiding on neutron flux detectors during a large LOCA
6. moderator and heat transport coolant isotopic limits, including:
   1. reason for these limits
   2. consequences of operating outside the limits
   3. relationship between moderator and heat transport coolant isotopic limits
7. reactivity changes after a reactor shutdown

1. principles of bulk and zone power regulation, including reactivity mechanisms used and their principles of operation
2. principles of reactor protection
3. neutron flux measurements, including types of signals required for reactor regulation and protection
4. neutron flux instrumentation, including:
   1. principles of operation of in-core and out-of-core start-up instrumentation
   2. principles of operation of ion chambers
   3. principles of operation of in-core detectors
   4. factors affecting the accuracy of ion chamber and in-core detector flux measurements
5. thermal power measurements for reactor regulation
6. principles of operation of thermal power instrumentation, including factors affecting the accuracy of thermal power measurements at various power levels
7. calibration of bulk and zone neutron flux measurements for reactor regulation and protection
8. calibration of thermal power measurements for reactor regulation
9. flux shapes, including:
   1. analyzed and unanalyzed flux shapes
   2. flux flattening
   3. effects of fuelling on flux shape
   4. effects of configurations of reactivity mechanisms
   5. channel power peaking factor
   6. reactor overpower protection at high power
10. preferred reactor state for refuelling
11. reactor power rundown after a trip
12. approach to criticality, including:
    1. use of special instrumentation for reactor regulation and protection
    2. methods of approach to criticality
    3. monitoring during an approach to criticality
    4. confirmation of criticality
    5. precautions to be taken during an approach to criticality

B) Principles of heat transfer and thermodynamics

1. principles of fuel cooling, including:
   1. modes of fuel cooling
   2. heat transfer processes from the fuel to coolant
   3. factors affecting critical heat flux
   4. dryout in a fuel channel
   5. coolant boiling
2. fuel cooling by natural circulation, including:
   1. principle of operation of thermosyphoning
   2. single-phase and two-phase thermosyphoning
   3. system parameters monitored during thermosyphoning
   4. system parameters controlled during thermosyphoning and consequences of inadequate control
   5. cooling by intermittent buoyancy induced flow (IBIF)
3. heat removal during small and large LOCAs, including:
   1. need for reactor trip
   2. crash cooling
   3. heat transport system (HTS) blowdown and refill by the emergency coolant injection system
   4. impact of dousing in containment
4. reactor, channel power and fuel bundle operating limits
5. fuel temperature profiles under various conditions
6. coolant temperature and heat flux profiles along a fuel channel
7. abnormal conditions that can lead to fuel overheating
8. fuel damage, including:
   1. contributing factors
   2. fuel sheath behaviour with temperature and irradiation
   3. sheath failure mechanisms
   4. methods of detection and location of fuel failures
9. HTS pressure control, including:
   1. principles of operation of the pressurizer
   2. consequences of operating outside normal pressurizer level and temperature ranges
   3. principles of operation of the bleed condenser
   4. non-condensable gas build up in the bleed condenser

1. principles of operation of the boilers, including:
   1. heat transfer processes from HTS coolant to the boiler water
   2. relationship between boiler pressure and heat transfer
   3. principles of boiler pressure control
   4. boiler pressure changes during HTS warm-up and cool-down
   5. steady state and transient swell and shrink of boiler water
   6. principles of boiler level control at various power levels
   7. causes, operational concerns and consequences of abnormal boiler levels
2. principles of operation of a multi-stage turbine, including:
   1. heat to work conversion process in the turbine
   2. factors affecting turbine efficiency and integrity
   3. control of turbine load
   4. moisture separation and steam reheat
   5. turbine exhaust overheating
3. principles of operation of the condenser, including:
   1. heat transfer processes in the condenser
   2. factors affecting the pressure in the condenser
   3. undesirable conditions in the condenser
4. principles of operation of the feed heaters, including undesirable conditions in feed heaters
5. principles of operation of the deaerator, including causes and consequences of deaerator pressure upsets
6. factors that impact on secondary cycle efficiency
7. precautions necessary when filling or draining a heat exchanger
8. principles of operation of combustion turbines and undesirable operating conditions

C) Principles of CANDU plant equipment

1. vibrations in turbine generators and other rotating machines, including:
   1. major causes and operating conditions that can affect them
   2. operating conditions that may affect critical speeds
2. equipment damage caused by excessive vibrations
3. causes and prevention of cavitation in CANDU plants
4. steam and water hammer in CANDU plants, including:
   1. causes
   2. resulting equipment damage
   3. operating practices to minimize the risk of their occurrence
5. operation of centrifugal pumps, including:
   1. operational changes influencing operation of the pumps
   2. pump cavitation and its consequences
   3. operational changes that may cause pump cavitation
   4. operating conditions that could lead to gas locking and vapour locking
   5. consequences of gas or vapour locking of a pump
   6. major causes and consequences of pump run out
   7. consequences of reverse rotation
   8. pump problem diagnosis
   9. pump start-up sequence and start-up precautions
   10. precautions during pump shutdown and isolation
6. positive displacement pump start-up
7. principles of operation and operational aspects of compressors and vacuum pumps
8. consequences of excessive moisture in compressed air piping and receivers
9. consequences and operational concerns associated with inadequate bearing lubrication, considering the various conditions that may result in inadequate lubrication
10. consequences of abnormal operating conditions in mechanical and labyrinth seals
11. operational concerns associated with valves, including consequences of impurities in hydraulic fluid and moisture in instrument air

1. principles of generator operation and control, including:
   1. generator synchronization
   2. control of terminal voltage, frequency and load for various configurations of generators and loads
   3. changes in generator parameters with changes in turbine steam flow and excitation current
   4. factors affecting generator stability
   5. generator load rejection
   6. automatic response to grid disturbances
   7. heat production and removal in a generator
   8. adverse generator operating conditions and consequences of exceeding operating limits
2. principles of equipment protection, including:
   1. bus protections
   2. transformer protections
   3. motor protections
   4. generator protections
3. operating limitations of transformers, including consequences of exceeding operating limits
4. operation of motors, including causes and consequences of abnormal operating conditions
5. operation of station batteries, including associated operational concerns
6. operation, isolation and de-energization of circuit breakers
7. precautions when operating circuit breakers, motor control centre (MCC) bus breakers and disconnect switches
8. precautions when isolating voltage and current transformers
9. ground faults on DC trip circuits
10. consequences of excessive moisture and temperature on electrical equipment insulation
11. means to achieve and maintain reliability of electrical power supplies, including:
    1. purpose, availability requirements and safety implications of different classes of supply
    2. typical loads for each class of supply with the reasons for their selection
    3. normal and alternate sources of electrical supply
    4. switchyard ring bus

1. basic principles of level, flow, temperature and pressure control loops, including:
   1. principles of operation of detectors
   2. proportional control
   3. proportional control with integral action
   4. proportional control with derivative action
   5. proportional control with integral and derivative actions
   6. feedforward control
   7. cascade and multi element controls
   8. valve actuators and positioners
   9. failure modes of control valves
2. impact of operating environment and process conditions on:
   1. pressure measurements
   2. level measurements
   3. flow measurements
3. effect of failures and abnormal conditions of components on:
   1. pressure measurements
   2. level measurements
   3. flow measurements
   4. temperature measurements
4. principles of operation and failure modes of the following detectors and sensors:
   1. position detectors
   2. speed sensors
   3. vibration sensors
   4. smoke and fire detectors
   5. liquid detectors

1. mechanical and thermal stresses in mechanical equipment, including:
   1. causes and operating practices used to minimize stresses
   2. consequences of excessive stresses
2. effects of radiation on plant materials and components, including problems caused by radiation damage
3. problems affecting pressure tubes, including:
   1. factors affecting creep in pressure tubes
   2. factors contributing to hydrogen embrittlement, delayed hydride cracking and blistering of pressure tubes
   3. operating practices used to minimize delayed hydride cracking of pressure tubes
4. principles of operation of ion exchange (IX) columns, including:
   1. control of pH and conductivity in plant systems by IX columns
   2. control of gadolinium and boron in the moderator
   3. detection of spent IX columns
   4. typical causes, symptoms and correction of IX column problems
5. chemical control in plant systems, including:
   1. causes of corrosion of plant components and ways of minimizing the various corrosion types
   2. effect of pH on corrosion of common plant materials
   3. typical causes of abnormal pH values in plant systems, methods used for control and consequences of operating outside the normal pH range in a given system
   4. typical causes of excessive dissolved oxygen in plant systems, methods used for control and consequences of operating with excessive dissolved oxygen in a given system
   5. typical causes of abnormal conductivity values in plant systems, methods used for control and consequences of operating outside the normal range of conductivity values in a given system
   6. causes of scale and sludge formation in boilers, methods used to minimize scale and sludge formation and consequences of scale and sludge formation in boilers
   7. causes of Chalk River unidentified deposit (CRUD) formation in plant systems, methods used to control CRUD and operational concerns associated with CRUD in plant systems
6. conditions favouring the reaction of zirconium with steam in a CANDU reactor and related operational consequences
7. deuterium or hydrogen excursions in the moderator cover gas, the liquid zone control system and the heat transport system storage tank, including:
   1. reasons for controlling deuterium or hydrogen gas production
   2. factors affecting production
   3. related operational concerns
   4. methods for deuterium or hydrogen reduction and control
   5. recombination unit operation

B.2 Topic groups for the general examinations for U0O candidates

These required topic groups are based on the training program in science fundamentals and equipment principles for U0O candidates that have been developed in accordance with the principles of a SAT.

The various topics that should be covered in the general examinations for U0O candidates have been provided for each required group, as shown below in a guidance box, to assist in their selection so that each examination contains an appropriate mix of topics to cover the knowledge that U0Os are required to have. All topics selected within a topic group shall be linked to specific learning objectives in accordance with subsection 9.3 and made available to the CNSC upon request.

The required topic groups, total marks and marks to be allocated to questions are identified below in bold and shall be used by the examination team to design and develop the General certification examinations.

Total marks : 75

A) Nuclear theory and principles of reactor operation

1. radioactive decay processes
2. induced nuclear reactions, including:
   1. scattering reactions
   2. absorption reactions
   3. photoneutrons
3. nuclear processes important for CANDU operation, including:
   1. neutron thermalization
   2. neutron detection reactions
   3. parasitic absorption and activation
4. fission chain reaction, including:
   1. prompt and delayed neutrons and associated effects
   2. neutron lifecycle
5. moderator properties and moderation process
6. neutron interactions with reactor core materials
7. subcritical, critical and supercritical reactor operation
8. build-up of fission products

1. principles of bulk and zone power regulation, including:
   1. reactivity mechanisms used and their principles of operation
   2. required neutron and thermal power measurements
2. principles of operation of shutdown systems, including:
   1. shutdown mechanisms used
   2. required shutdown system capabilities
   3. requirements for establishing and maintaining a guaranteed shutdown state
3. addition and removal of chemical neutron poisons in the moderator for reactivity control
4. major components of CANDU reactors, including:
   1. reactor shielding
   2. main moderator system and its auxiliaries
   3. main HTS and its auxiliaries
5. heat production and removal in a CANDU reactor
6. moderator and heat transport coolant isotopic

B) Principles of heat transfer and thermodynamics

1. principles of fuel cooling, including:
   1. modes of fuel cooling
   2. heat transfer processes from the fuel to coolant
   3. importance of heat transport system pressure control
   4. dryout in a fuel channel
   5. coolant boiling
2. fuel cooling by natural circulation, including:
   1. principle of operation of thermosyphoning
   2. single-phase and two-phase thermosyphoning
   3. cooling by intermittent buoyancy induced flow
3. heat removal during small and large LOCAs, including:
   1. need for reactor trip
   2. crash cooling
   3. HTS blowdown and refill by the emergency coolant injection system
   4. impact of dousing in containment
   5. containment pressure changes during loss of coolant accidents
4. reactor, channel power and fuel bundle operating limits
5. abnormal conditions that can lead to fuel overheating
6. fuel damage, including:
   1. contributing factors
   2. fuel sheath behaviour with temperature and irradiation
   3. sheath failure mechanisms

1. principles of operation of the boilers, including:
   1. heat transfer processes from HTS coolant to the boiler water
   2. relationship between boiler pressure and heat transfer
   3. principles of boiler pressure control
   4. boiler pressure changes during HTS warm-up and cool-down
2. principles of operation of the condenser, including:
   1. heat transfer processes in the condenser
   2. factors affecting the pressure in the condenser
3. precautions necessary when filling or draining a heat exchanger
4. factors that affect the efficiency of heat exchangers
5. principles of operation of combustion turbines, including:
   1. heat to work conversion process in the turbine
   2. control of turbine load
   3. undesirable operating conditions

C) Principles of CANDU plant equipment

1. vibrations in standby generators and other rotating machines, including:
   1. major causes and operating conditions that can affect them
   2. operating conditions that may affect critical speeds
2. equipment damage caused by excessive vibrations
3. causes and prevention of cavitation in CANDU plants
4. steam and water hammer in CANDU plants, including:
   1. causes
   2. resulting equipment damage
   3. operating practices to minimize the risk of their occurrence
5. operation of centrifugal pumps, including:
   1. operational changes influencing operation of the pumps
   2. pump cavitation and its consequences
   3. operational changes that may cause pump cavitation
   4. operating conditions that could lead to gas locking and vapour locking
   5. consequences of gas or vapour locking of a pump
   6. major causes and consequences of pump run out
   7. consequences of reverse rotation
   8. pump problem diagnosis
   9. pump start-up sequence and start-up precautions
   10. precautions during pump shutdown and isolation
6. positive displacement pump start-up
7. principles of operation and operational aspects of compressors and vacuum pumps
8. consequences of excessive moisture in compressed air piping and receivers
9. consequences and operational concerns associated with inadequate bearing lubrication, considering the various conditions that may result in inadequate lubrication
10. consequences of abnormal operating conditions in mechanical and labyrinth seals
11. operational concerns associated with valves, including consequences of impurities in hydraulic fluid and moisture in instrument air

1. principles of generator operation and control, including:
   1. generator synchronization
   2. control of terminal voltage, frequency and load for various configurations of generators and loads
   3. changes in generator parameters with changes in turbine gas flow and excitation current
   4. factors affecting generator stability
   5. generator load rejection
   6. automatic response to grid disturbances
   7. heat production and removal in a generator
   8. adverse generator operating conditions and consequences of exceeding operating limits
2. principles of equipment protection, including:
   1. bus protections
   2. transformer protections
   3. motor protections
   4. generator protections
3. operating limitations of transformers, including consequences of exceeding operating limits
4. operation of motors, including causes and consequences of abnormal operating conditions
5. operation of station batteries, including associated operational concerns
6. operation, isolation and de-energization of circuit breakers
7. precautions when operating circuit breakers, MCC bus breakers and disconnect switches
8. precautions when isolating voltage and current transformers
9. ground faults on DC trip circuits
10. consequences of excessive moisture and temperature on electrical equipment insulation
11. means to achieve and maintain reliability of electrical power supplies, including:
    1. purpose, availability requirements and safety implications of different classes of supply
    2. typical loads for each class of supply with the reasons for their selection
    3. normal and alternate sources of electrical supply
    4. switchyard ring bus

1. basic principles of level, flow, temperature and pressure control loops, including:
   1. principles of operation of detectors
   2. proportional control
   3. proportional control with integral action
   4. proportional control with derivative action
   5. proportional control with integral and derivative actions
   6. feedforward control
   7. valve actuators and positioners
   8. failure modes of control valves
2. impact of operating environment and process conditions on:
   1. pressure measurements
   2. level measurements
   3. flow measurements
3. effect of failures and abnormal conditions of components on:
   1. pressure measurements
   2. level measurements
   3. flow measurements
   4. temperature measurements
4. principles of operation and failure modes of the following detectors and sensors:
   1. position detectors
   2. speed sensors
   3. vibration sensors
   4. smoke and fire detectors
   5. liquid detectors

1. mechanical and thermal stresses in mechanical equipment, including:
   1. causes and operating practices used to minimize stresses
   2. consequences of excessive stresses
2. effects of radiation on plant materials and components, including problems caused by radiation damage
3. principles of operation of ion exchange (IX) columns, including:
   1. control of pH and conductivity in plant systems by IX columns
   2. detection of spent IX columns
   3. typical causes, symptoms and correction of IX column problems
4. chemical control in plant systems, including:
   1. causes of corrosion of plant components and ways of minimizing the various corrosion types
   2. effect of pH on corrosion of common plant materials
   3. typical causes of abnormal pH values in plant systems, methods used for control and consequences of operating outside the normal pH range in a given system
   4. typical causes of excessive dissolved oxygen in plant systems, methods used for control and consequences of operating with excessive dissolved oxygen in a given system
   5. typical causes of abnormal conductivity values in plant systems, methods used for control and consequences of operating outside the normal range of conductivity values in a given system
   6. causes of scale and sludge formation in boilers, methods used to minimize scale and sludge formation and consequences of scale and sludge formation in boilers
   7. causes of crud formation in plant systems and operational concerns associated with crud
5. conditions favouring the reaction of zirconium with steam in a CANDU reactor and related operational consequences
6. deuterium or hydrogen excursions in the moderator cover gas, including:
   1. reasons for controlling deuterium or hydrogen gas production
   2. factors affecting production
   3. related operational concerns

B.3 Topic groups for the NPP-specific examinations for RO candidates

These required topic groups are based on the training program in NPP-specific integrated systems operations for RO candidates that have been developed in accordance with the principles of a SAT.

The various topics that should be covered in the NPP-specific examinations for RO candidates have been provided for each required topic group, as shown below in a guidance box, to assist in their selection so that each examination contains a balanced mix of topics to cover the knowledge that ROs are required to have. All topics selected within a topic group shall be linked to specific learning objectives in accordance with subsection 9.3 and made available to the CNSC upon request.

The required topic groups, total marks and marks to be allocated to questions are identified below in bold and shall be used by the examination team to design and develop the NPP-specific certification examinations.

All groups Marks: 100

1. Shutdown system (SDS)1 (or SDSA at the applicable NPP), including:
   1. absolute and conditional trip parameters with the reason for their existence, the type of events they protect against, primary and back-up parameters and power dependent trip set points
   2. interlocks between SDS1 and the reactor regulating system
   3. impairments and their consequences on system effectiveness
   4. standard and non-standard operating procedures
2. SDS2 (or SDSE at the applicable NPP), including:
   1. absolute and conditional trip parameters with the reason for their existence, the type of events they protect against, primary and back-up parameters and power dependent trip set points
   2. interlocks between SDS2 and the reactor regulating system
   3. impairments and their consequences on system effectiveness
   4. standard and non-standard operating procedures
3. emergency core cooling system, including:
   1. related systems and subsystems
   2. primary and conditioning initiating parameters, with the reasons for their existence
   3. impairments and their consequences on system effectiveness
   4. standard and non-standard operating procedures
4. negative pressure and containment system, including:
   1. related subsystems
   2. impairments and their consequences on system effectiveness
   3. standard and non-standard operating procedures
5. power house emergency venting system as applicable

1. principles of nuclear safety, including:
   1. defence-in-depth considerations
   2. principles of control, cool and contain under normal and abnormal reactor operating conditions
   3. requirements for establishing and maintaining a guaranteed shutdown state
   4. roles of special safety systems, standby safety support systems and safety related process systems
   5. means to achieve and maintain reliability of systems and equipment
   6. protection against common cause failures
   7. availability requirements and testing of safety related systems
   8. purpose, availability requirements and safety implications of different classes of electrical supplies
   9. plant licensing basis, safety analyses and safe operating envelope
   10. operating licence and Operating Policies and Principles
   11. plant lineup and plant status control
2. heat sinks, including response to non-standard operating conditions during reactor unit outages
3. critical safety parameters and their support parameters

1. unit emergency operating procedures including:
   1. the purpose of the procedure
   2. operational consequences of the event
   3. systems responses to the event
   4. any interactions between the reactor unit and unit 0 as applicable
2. secondary control area operation including impairments and their consequences on system effectiveness

1. normal and abnormal reactivity configurations and flux shapes
2. channel power peaking factor
3. principles of reactor fuelling, fuelling limitations, fuel handling and storage, and irradiated fuel cooling
4. channel temperature monitoring
5. fully instrumented channels
6. channel power mapping, where applicable
7. flux mapping and zone thermal power, as applicable
8. fuelling
   1. the reactor fuelling cycle
   2. fuelling prerequisites and limitations
   3. irradiated fuel cooling

1. reactor regulating system (RRS) program routines
2. stepback routine (at the applicable NPPs)
3. adjuster rods
4. control absorber rods (at the applicable NPPs)
5. moderator liquid poison systems
6. liquid zone control system

1. primary HTS, including:
   1. pressure and inventory control system
   2. heat transport pump gland seal cooling system
   3. heat transport pump trip system
2. shutdown cooling system (maintenance cooling system at applicable NPPs)
3. main moderator system
4. moderator level and temperature control systems
5. end shield cooling system

1. boiler pressure control
2. boiler level control
3. deaerator level control
4. turbine governor
5. general purpose control program or heat transport auxiliaries control program, as applicable

1. condensate systems, including condensate makeup and reject
2. boiler feedwater system
3. emergency boiler makeup systems
4. shutdown cooling system (at the applicable NPPs)
5. boiler steam system, including:
   1. boiler safety valves
   2. condenser and atmospheric control valves (at the applicable NPPs)
6. turbine steam system, including:
   1. emergency stop valves, governor valves
   2. intercept and reheat stop valves, where applicable
7. turbine tripping system
8. unit power regulator control program, where applicable
9. turbine run-up and loading control system
10. turbine generator supervisory

1. main generator system
2. generator static excitation
3. station electrical equipment and distribution, including:
   1. electrical protection
   2. emergency transfer schemes
   3. emergency power system(s)
4. switchyard electrical equipment and distribution system
5. switchyard compressed air system
6. instrument air systems

1. moderator purification system
2. moderator cover gas system
3. heat transfer storage, transfer and recovery systems,
4. heat transfer purification system
5. heat transfer hydrogen addition system
6. annulus gas system
7. failed fuel detection system, where applicable
8. failed fuel location system, where applicable
9. emergency water supply systems (emergency water supply, boiler makeup water)

1. boiler blowdown
2. turbine gland steam system
3. turbine low-pressure exhaust cooling system
4. condenser air extraction system
5. condenser circulating water system
6. generator hydrogen cooling system
7. generator stator cooling system
8. generator seal oil system
9. turbine-generator lubricating oil system
10. governor fluid supply system
11. extraction steam system
12. feedheater and separator drains system
13. condenser leak detection system

1. reactor unit start-up procedures, including:
   1. heat transport system refill
   2. approaches to critical
   3. heat transport system pressurization
   4. unit heatup
   5. turbine-generator and auxiliaries start-up
   6. unit loading
2. reactor unit shutdown procedures, including:
   1. unit unloading
   2. unit cooldown
   3. heat transport system depressurization
   4. heat transport system draining (at the applicable NPPs)
   5. turbine-generator and auxiliaries shutdown
   6. establishment of guaranteed shutdown states
   7. four unit shutdown at multi-unit plants
3. non-standard operating procedures, including:
   1. response to reactor trip
   2. response to reactor setback
   3. response to reactor stepback (at the applicable NPPs)
   4. recovery from a reactor setback or stepback
   5. heat transport pump trip or manual shutdown
   6. generation rejection and recovery
   7. turbine trip and recovery
   8. response to a main generator hydrogen leak or seal failure

1. access control system
2. airlocks
3. radiation monitoring systems
4. emergency filtered air discharge system
5. post accident radiation monitoring system
6. active liquid waste system
7. stack monitoring system, including:
   1. sources of airborne emissions
   2. emission pathways for airborne emissions
8. basic concepts related to the control of the discharge of radioactivity to the environment, including:
   1. derived release limits and associated public dose limits
   2. critical groups
   3. control monitoring
9. the potential sources of radiation hazards from systems operated by the reactor operator
10. possible radiation hazards that may exist after a loss of coolant accident with failed fuel
11. responsibilities of the reactor operator to abnormal radiological conditions

Refer to appendix D.3 for related topics and the breakdown of marks in the group.

1. The responsibilities and authorities of a reactor operator, including:
2. work protection on reactor unit systems
3. personnel safety
4. communications with external groups such as grid operators
5. the licensee safety culture

B.4 Knowledge groups for the NPP-specific examinations for RO candidates

The knowledge groups and marks below shall be used by the examination team to ensure that the sampling of topic groups in appendix B.3 adequately covers the knowledge areas addressed by the NPP SAT-based objectives for ROs.

The knowledge or learning objectives are specific to each NPP and are developed in accordance with a SAT. Single-unit NPPs use the Generic Station System Knowledge Objectives for Control Room Operators document in the development of their NPP-specific learning objectives (refer to appendix D.2 for the corresponding objectives). Multi-unit NPPs use the objective template approved by the CNSC in December 2003 (see References).

B.5 Topic groups for the NPP-specific examinations for U0O candidates

These required topic groups are based on the training program in NPP-specific integrated systems operations for U0O candidates that have been developed in accordance with the principles of a SAT.

The various topics that should be covered in the NPP-specific examinations for U0O candidates have been provided for each group, as shown below in a guidance box, to assist in their selection so that each examination contains a balanced mix of topics to cover the knowledge that U0Os are required to have on the topics covered is tested. All topics selected within a topic group shall be linked to specific learning objectives in accordance with subsection 9.3 and made available to the CNSC upon request.

The required topic groups, total marks and marks to be allocated to questions are identified below in bold and shall be used by examination team to design and develop the NPP-specific certification examinations.

All groups Marks:75

1. emergency coolant injection system, including:
   1. related systems
   2. impairments
   3. standard and non-standard operating procedures
2. negative pressure containment system, including:
   1. related subsystems
   2. impairments
   3. standard and non-standard operating procedures
3. Unit 0 safety system monitoring computer
4. Power house emergency venting system

1. principles of nuclear safety, including:
   1. defence-in-depth considerations
   2. principles of control, cool and contain under normal and abnormal reactor operating conditions
   3. requirements for establishing and maintaining a guaranteed shutdown state
   4. roles of special safety systems, standby safety support systems and safety related process systems
   5. means to achieve and maintain reliability of systems and equipment
   6. protection against common cause failures
   7. availability requirements and testing of safety related systems
   8. purpose, availability requirements and safety implications of different classes of electrical supplies
   9. plant licensing basis, safety analyses and safe operating envelope
   10. plant operating licence and operating policies and principles
   11. plant line-up and plant status control
2. heat sinks, including response to non-standard operating conditions during reactor unit outages
3. critical safety parameters and their support parameters

1. abnormal incident manual procedures
2. interaction between unit 0 systems and those of the reactor units
3. integrated plant operating procedures, including:
   1. quiet mode operation
   2. four unit shutdown
4. non-standard operating procedures, including:
   1. generation rejection
   2. response to grid frequency disturbances
   3. response to loss of grid
   4. switchyard restoration after generation rejection (where applicable)
   5. total loss of 250 VDC power in the switchyard (where applicable)
   6. response to a main generator hydrogen leak or seal failure
   7. loss of common instrument air
5. Unit 0 emergency operating procedures, including:
   1. the purpose of the procedure
   2. operational consequences of the event
   3. any interaction between unit 0 and the reactor unit as applicable
   4. secondary control area operation including impairments and their consequences on system effectiveness
   5. emergency irradiated fuel cooling

1. main power output system, including:
   1. electrical power transformation
   2. switchyard electrical equipment (at applicable NPPs)
   3. switchyard compressed air system (at applicable NPPs)
   4. output system relay protection (at applicable NPPS)
2. AC electrical distribution, including:
   1. 13.8 kV distribution system
   2. 4.16 kV distribution system
   3. 600 VAC Class III and Class IV distribution system
   4. 600 VAC Class II system
   5. 120 VAC Class II system
3. DC electrical distribution, including:
   1. 250 VDC system
   2. 125 VDC system (at applicable NPPs)
   3. 48 VDC system
   4. 45 VDC instrumentation supplies system

1. standby Class III power system, including:
   1. emergency transfer scheme
   2. standby generators
   3. impairments and their consequences on system effectiveness
2. emergency power system, including:
   1. emergency power generators
   2. impairments and their consequences on system effectiveness
3. qualified power supply system (at applicable NPPs)

1. condensate make-up system (at applicable NPPs)
2. common service water system (at applicable NPPs)
3. domestic water system, including site domestic water system(at applicable NPPs)
4. emergency water system (at applicable NPPs)
5. D2O management (at applicable NPPs), including:
   1. D2O transfer and storage system
   2. heavy water leak detection system
6. emergency boiler cooling system (at applicable NPPs)
7. emergency and auxiliary service water systems (at applicable NPPs)
8. frazil ice protection system
9. circulating water discharge control gate (at applicable NPPs)
10. zebra mussel chlorination system
11. service air system
12. instrument air system
13. breathing air system
14. heating and ventilation systems
15. air conditioning systems
16. powerhouse emergency venting system
17. vapour recovery systems
18. fire protection water system
19. site fire protection water system (at applicable NPPs)
20. transformer deluge and sprinkler system (at applicable NPPs)
21. air foam fire protection system
22. smoke detection system
23. CO2 fire protection system

1. access control system
2. airlocks and fuel transfer chambers
3. fixed area gamma monitoring system
4. basic concepts related to the control of the discharge of radioactivity to the environment, including:
   1. derived emission limits and associated dose limits for the station
   2. critical group
   3. station emission target
   4. control monitoring
   5. compliance monitoring
5. stack monitoring system, including:
   1. sources and categories of airborne emissions
   2. emission pathways for airborne emissions
6. active liquid waste system, including:
   1. active liquid waste handling
   2. active liquid waste treatment
   3. liquid effluent monitoring systems
7. off gas management system
8. environmental monitoring program
9. emergency filtered air discharge system
10. post accident radiation monitoring system
11. potential sources of radiation hazards from systems operated by the Unit 0 operator
12. possible radiation hazards that may exist after a loss of coolant accident with failed fuel
13. responsibilities of the Unit 0 operator to abnormal radiological conditions

Refer to appendix D.5 for related topics and the breakdown of marks in the group

1. the responsibilities and authorities of the Unit 0 Operator, including:
2. work protection on unit 0 systems
3. personnel safety
4. communications with external groups such as grid operators
5. the licensee safety culture

B.6 Knowledge groups for the NPP-specific examinations for U0O candidates

The knowledge groups and marks below shall be used by the examination team to ensure that the sampling of topic groups in appendix B.5 adequately covers the knowledge areas addressed by the NPP SAT-based objectives for U0Os.

The knowledge or learning objectives are specific to each NPP and are developed in accordance with a SAT. Multi-unit NPPs who have U0Os use the document entitled Generic Station System Knowledge Objectives for Control Room Operators (refer to References) in the development of their NPP-specific learning objectives (refer to appendix D.4 for the corresponding objectives). NPPs using a different set of generic system knowledge objectives for U0Os may refer to the corresponding generic objectives in their set, once the resulting NPP-specific learning objectives, developed in accordance with a SAT, are approved by the CNSC.

B.7 Topic groups for the supplementary NPP-specific examinations for PSS and CRSS Candidates

These required topic groups for PSS and CRSS candidates are based on the training program that are supplemental to the RO NPP specific integrated systems operations that have been developed in accordance with the principles of a SAT.

The various topics that need to be covered in the supplementary NPP-specific examinations have been grouped as shown below to assist in their selection so that each examination contains a balanced mix of topics to cover the knowledge that PSSs and CRSSs are required to have in addition to that of an RO. All topics selected within a topic group shall be linked to specific learning objectives in accordance with subsection 9.3 and made available to the CNSC upon request.

The marks to be allocated to questions in any topic group are indicated beside the group.

B.8 Sample examination design matrix

B.9 Knowledge-based examination cover page

Initial Knowledge-Based Certification Examination

[Examination title]
[NPP]
[Position]
[Examination conduct date]

Total : [number] marks
Time limit: [number] hours
Estimated required time: [number] hours

Developed by:

_____________________________
[Print name and signature]
Lead examiner

________________
Date

Examination team:

______________________________ ______________________________

______________________________ ______________________________
[Print names]

Approved for conduct by:

_____________________________
[Print name and signature]
Lead examiner

________________
Date

B.10 Sample knowledge-based examination questions

The following are sample examination questions stating the required type of information for each question.

Note: The total mark allotment for a question is the sum of the elements. Key instructions for each question are in bold and underlined.

B.11 Allocation of marks and time versus number of answer elements

Derivation of data:
Marks per question = Round (total marks/total elements * actual elements)
Minutes per question = Round (total minutes/total elements * actual elements)
Example #1 - General examination: 100/300 x 20 elements in a question = 7 marks for that questions
Example #2 - NPP-specific examination: 100/250 x 20 elements in a question = 8 marks for that question

B.12 Knowledge-based marking guide cover page

Marking Guide

Examination title:
NPP:
Candidate's full name:
Examination score:
Examination date:
Marks:

Lead examiner:

____________________________
[Print name and signature]

_________
Date

Approved by

____________________________
[Print name and signature]
Training manager

_________
Date

Marked by:

____________________________
[Print name and signature],
First marker Second marker

___________________________
[Print name and signature],
Second marker

Authorized by:

____________________________
[Print name and signature]
Training manager

_________
Date

B.13 Knowledge-based marking guide answer

Q1 [X] marks
a) [X] elements

i) The purpose of the unloader is to:

• Protect a turbine-generator (under adverse conditions) by limiting its maximum load
  

This is done:

• by limiting the governor valve demand signal
• thereby, limiting the steam flow (limiting turbine load)

Ref.: TM 234-7XXX_XXX [complete titles, Rev. X , p. 4 Knowledge objectives =X]

ii) Unloading in a turbine generating system is initiated by:

• low condenser vacuum (high condenser pressure)
• low boiler pressure
  

Ref.: TM 234-7, p. 4

b) [X] elements

Unloading initiates a runback to prevent:

• turbine load cycling that could occur as follows:
• as a result of the unloading process, the unloading parameter (low condenser vacuum or boiler pressure) could return to an acceptable range
• the load restriction imposed by the unloader would be removed
• this would allow the limit placed on the valve demand signal to be removed
• allowing the valves to return to the unchanged setpoint
• the turbine steam flow would increase
• causing the unloading parameter to again reach an unacceptable value
• and the beginning of another unloading and loading cycle

Ref.: TM 234-7, p. 4 and 5

[NPP-specific: 100/300 x 13 elements = 4 marks for Q1]

B.14 Knowledge-based examination checklist

Examination title: ________________________
Examination date: ________________________

This checklist shall be completed at the end of each examination development.

• The examination and the questions follow the relevant topics, mark allocation and total marks as shown in the examination summary chart below.
• No more than 20 percent of the questions on the certification examination are based on previously used questions, and all reused questions are significantly modified and have never before seen by the candidates.
• At least 80 percent of the questions on the certification examination are newly designed and not based upon questions previously used in any examination, test, candidate evaluation or training.
• The level of difficulty of each question is appropriate to discriminate between those candidates who have an understanding of the subject covered by the question and those candidates who do not.
• Each question is clear, technically accurate and contains all the information necessary to give the required answer.
• The parts and subparts of each question are arranged in a logical order, where applicable.
• The answer to a part of a question should not be completely dependent on the answer to a previous part.
• Questions measure integrated understanding or higher-level knowledge required by persons in the position for which the examination is intended to perform their job competently.
• Questions are operationally focused covering a wide variety of NPP systems, procedures and topics associated with the integrated operation of these systems.
• No excessive emphasis is placed on a particular system, procedure, topic or type of objective.
• None of the questions contain information that could potentially provide the candidate with assistance in answering other questions.
• The information on the cover page of the examination is similar to that shown in appendix B.9.
• The identification codes of equipment and documents used in the questions are correct.
• Any acronym used in a question is defined the first time it appears in that question.
• The questions clearly identify any reference material given to the candidates and give appropriate instructions for its use.
• No reference material is given for a question that covers an area that the candidates have to know from memory.
• Any instruction or other information in a question quoted from the NPP documentation appears verbatim.
• The marks are allocated to each question.
• The formatting of the questions is similar to that shown in appendix B.10.
• The information on the cover page of the marking guide is similar to that shown in appendix B.12.
• Each answer contains only information specifically asked by the question.
• Each answer is complete and technically accurate.
• Each answer reflects the level of knowledge that candidates are expected to have on the subject covered by the question.
• None of the answers conflict with the information contained in the frozenup-to-date documentation.
• The breakdown of each answer into elements is appropriate.
• The formatting of the answers in the marking guide is similar to that shown in appendix B.13.
• The examination has been validated and meets the requirements set out in this document.

B.15 Knowledge-based examination result form

Candidate's full name:
Candidate's position:
Examination title:
Examination date:
NPP:

Justification of the result:

Deficiencies revealed by the candidate:

Appendix C: Performance-Based Certification Examinations - Supplemental Requirements

C.1 Competency areas and generic performance expectations for RO and U0O candidates

The competency areas and generic performance expectations (GPEs) listed in this appendix shall be used for assessing the performance of RO and U0O candidates.

The standard of acceptable performance to be used with each generic expectation is dictated by the corresponding documented specific performance expectations at the NPP for the RO and U0O positions. The number in brackets associated with each generic performance expectation is a weighting factor to be used for calculating the scores obtained by candidates in a competency area. These weighting factors reflect the relative importance of the generic performance expectations.

GPE Competency area

1.0 Monitoring

This competency area addresses the ability to independently maintain an on-going awareness of NPP unit or system conditions, using the appropriate main control room (MCR) devices, control equipment room devices or field reports, either by performing the required checks without reference to operating procedures or by performing the relevant checks not explicitly mentioned in an operating procedure being implemented.

1.1 (5) monitors critical safety parameters (CSPs) and their support parameters (SPs) when necessary

1.2 (4) monitors in a timely manner that major system parameters are trending and stabilize as expected

1.3 (4) monitors annunciations and panel indications, as necessary, to maintain an awareness of changing equipment, system and unit conditions

1.4 (5) checks in a timely manner that major automatic actions (MAAs) occur and have the desired effect, prior to referring to operating procedures

1.5 (3) checks in a timely manner that other automatic actions occur and have the desired effect

1.6 (3) checks in a timely manner that operator actions are having or have had the desired effect

2.0 Actions taken without reference to procedures

This competency area addresses the ability to perform the required actions when reference to operating procedures is not expected, when an operating procedure does not exist or prior to accessing the applicable operating procedures following a transient or an abnormal condition.

2.1 (2) ensures in a timely manner the assembly of the control room response team

2.2 (5) executes in a timely manner the required corrective actions when an MAA fails to occur

2.3 (3) executes in a timely manner the required corrective actions when other automatic actions fail to occur

2.4 (5) executes in a timely manner the required actions, based on diagnosis, prior to accessing the applicable procedures

2.5 (3) executes other required actions, as appropriate, when reference to operating procedures is not expected

2.6 (4) executes, correctly, in a timely manner and following PSS or CRSSS approval, the actions required when a transient, an abnormal condition, an additional malfunction, equipment unavailability or any other unit condition is not addressed by procedures

2.7 (4) ensures in a timely manner that the appropriate corrective or preventive actions are initiated when unit conditions are such that they could threaten NPP personnel or public safety

2.8 (5) executes in a timely manner corrective actions not specified in operating procedures when CSPs or their SPs are deviating unsafely

3.0 Diagnosis and decision-making

This competency area addresses the ability to independently and in a timely manner recognize NPP unit or system abnormalities, to independently diagnose malfunctions, to select the applicable operating procedures and, when no applicable procedures exist, to determine or recommend an appropriate course of actions

3.1 (5) recognizes when CSPs or SPs are deviating unsafely

3.2 (4) recognizes when an NPP unit, system configuration or a system parameter deviates from an acceptable value, configuration or state defined in operating procedures

3.3 (5) recognizes when a requirement specified in the operating policies and principles (OP&Ps) is about to be, or may have been violated

3.4 (5) identifies without procedure, the alarms and indications that characterize the nature of a malfunction and, when applicable, recognizes the entry conditions to the applicable procedure

3.5 (3) identifies the applicable procedure(s) or procedure path based on diagnosis

3.6 (2) with reference to procedures, determines the nature of a malfunction or identifies the specific procedures to be used

3.7 (2) confirms the diagnosis or confirms that the correct procedure has been selected, using other available information such as NPP documentation or information from the field

3.8 (5) determines or recommends an appropriate course of action if procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective

3.9 (4) recognizes in a timely manner when concurrent abnormal NPP conditions need to be addressed and assigns priorities to the actions required to address them, taking into account their relative importance

3.10 (3) when time permits, determines the specific cause of a malfunction using all the information available

4.0 Procedural compliance

This competency area addresses the ability to correctly perform the actions and checks required by any relevant approved procedures.

4.1 (3) executes each step of the applicable procedure(s) correctly, in the correct sequence and in a timely manner

4.2 (4) performs the actions considered as standard operating practices necessary for completing a step of a procedure

5.0 Communications and crew interaction skills

This competency area addresses the ability to communicate with other MCR response team members or crew members clearly and accurately and to interact with them in an efficient and effective manner.

5.1 (5) gives instructions in a timely manner to crew members or MCR response team members, as appropriate

5.2 (3) acknowledges the requests or information received

5.3 (5) communicates the outcome of a request to its originator

5.4 (3) keeps MCR response team members or crew members, as appropriate informed of relevant unit conditions or significant developments through briefings as appropriate

5.5 (2) makes the necessary notifications or requests, as appropriate, in a timely manner

5.6 (3) obtains in a timely manner the necessary approvals from the PSS or CRSS

C.2 Competency areas and generic performance expectations for PSS candidates at single-unit NPPs

The competency areas and GPEs listed in this appendix shall be used for assessing the performance of PSS candidates at single-unit NPPs.

The standard of acceptable performance to be used with each generic expectation is dictated by the corresponding documented specific performance expectations at the NPP for the PSS position. The number in brackets associated with each generic performance expectation is a weighting factor to be used for calculating the scores obtained by candidates in a competency area. These weighting factors reflect the relative importance of the generic performance expectations.

GPE Competency area

1.0 Monitoring

This competency area addresses the ability to independently maintain an on going awareness of NPP conditions, using appropriate MCR devices, either by performing the required checks without reference to operating procedures or by performing relevant checks not explicitly mentioned in an operating procedure being implemented.

1.1 (5) monitors CSPs and their support parameters SPs when necessary

1.2 (4) monitors in a timely manner that major system parameters are trending and stabilize as expected

1.3 (4) monitors annunciations and panel indications, as necessary, to maintain an awareness of changing NPP equipment, system and conditions

1.4 (5) checks in a timely manner that MAAs occur and have the desired effect, prior to referring to operating procedures

1.5 (3) checks that other automatic actions occur and have the desired effect, as appropriate to the situation

1.6 (3) checks in a timely manner that an action performed has the desired effect

1.7 (5) checks relevant system and NPP conditions before approving or initiating an action

1.8 (1) acknowledges and resets alarms or periodically takes alarm summaries as appropriate, when the RO is temporarily absent from the MCR

2.0 Actions taken without reference to procedures

This competency area addresses the ability to direct the execution of the required actions or to perform these actions when reference to operating procedures is not expected, when an operating procedure does not exist or prior to accessing the applicable operating procedures following a transient or an abnormal condition.

2.1 (2) ensures in a timely manner the assembly and the deployment of the MCR response team

2.2 (5) executes or directs the execution of the required corrective actions in a timely manner when an MAA fails to occur

2.3 (3) executes or directs the execution of the required corrective actions in a timely manner when other automatic actions fail to occur

2.4 (5) executes or directs the execution of the required actions in a timely manner, based on diagnosis, prior to accessing the applicable procedures

2.5 (3) executes or directs the execution of other required actions, as appropriate, when reference to operating procedures is not expected

2.6 (4) executes or directs the execution of the actions required in a timely manner when a transient, an abnormal condition, an additional malfunction, equipment unavailability or any other NPP condition is not addressed by procedures

2.7 (4) ensures in a timely manner that the appropriate corrective or preventive actions are initiated when NPP conditions are such that they could threaten NPP personnel or public safety

2.8 (5) executes or directs the execution of corrective actions not specified in operating procedures in a timely manner when CSPs or their SPs are deviating unsafely

3.0 Diagnosis and decision-making

This competency area addresses the ability to independently recognize abnormalities, to independently diagnose malfunctions, to select the applicable operating procedures and, when no relevant procedure exists, to determine an appropriate course of action.

3.1 (5) recognizes in a timely manner when CSPs or their SPs are deviating unsafely

3.2 (4) recognizes in a timely manner when a system parameter, a system configuration or an NPP state deviates from an acceptable value, configuration or state defined in operating procedures

3.3 (5) recognizes in a timely manner when a requirement specified in the OP&Ps is about to be, or may have been violated

3.4 (5) identifies, in a timely manner and without procedure, the alarms and indications that characterize the nature of a malfunction and, when applicable, recognizes the entry conditions to the relevant procedure

3.5 (3) identifies in a timely manner the applicable procedures, based on diagnosis

3.6 (2) with reference to procedures, determines in a timely manner the nature of a malfunction or identifies the specific procedures to be used

3.7 (2) confirms the diagnosis or confirms that the correct procedure has been selected, using other available information such as NPP documentation or information from the field

3.8 (5) determines in a timely manner an appropriate course of actions when procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective

3.9 (4) recognizes in a timely manner when concurrent abnormal NPP conditions need to be addressed and assigns priorities to the actions required to address them, taking into account their relative importance

3.10 (3) when time permits, determines the specific cause of a malfunction using all the information available.

3.11 (5) determines in a timely manner the corrective actions required when CSP or their SPs are deviating unsafely

3.12 (4) determines the desired end state of the NPP, together with any significant intermediate NPP state, and the time frame to establish the end state, following a transient or an abnormal condition

3.13 (5) determines the appropriateness of a request that requires PSS approval, based on a review of the relevant system and NPP conditions, constraints and limits, before approving the action

4.0 Procedural compliance

This competency area addresses the ability to direct the execution of the required actions and checks using a relevant approved procedure. It also addresses the ability to correctly perform the actions and checks required by any relevant approved procedure, when the RO is temporarily absent from the MCR, and the ability to carry out the actions and checks specifically assigned to the PSS using a relevant approved procedure.

4.1 (3) executes or directs the execution of an applicable procedure correctly and in the correct sequence

4.2 (4) performs the actions considered as standard operating practices necessary for completing a step of a procedure

4.3 (5) executes or directs the execution of a CSP restoration procedure in a timely manner when required

5.0 Communications and crew interaction skills

This competency area addresses the ability to communicate with other MCR response team members or crew members clearly and accurately and to interact with them effectively and efficiently.

5.1 (5) gives instructions and approvals in a timely manner to MCR response team members or crew members, as appropriate

5.2 (3) acknowledges the requests and the information received

5.3 (5) communicates, in a timely manner, the outcome of a request to its originator

5.4 (3) communicates, in a timely manner, the required information to MCR response team members or crew members, as appropriate

5.5 (2) makes the necessary notifications in a timely manner

5.6 (3) obtains the necessary approvals from NPP management in a timely manner

C.3 Competency areas and generic performance expectations for CRSS candidates at multi-unit NPPs

The competency areas and GPEs listed in this appendix shall be used for assessing the performance of CRSS candidates at the applicable multi-unit NPPs.

The standard of acceptable performance to be used with each generic expectation is dictated by the corresponding documented specific performance expectations at the NPP for the CRSS position.

Note: The applicability of a GPE in a DTS, a PCTS or both is indicated in brackets.

GPE Competency area

1.0 Monitoring

This competency area addresses the ability to independently maintain an ongoing awareness of NPP unit or system conditions, using appropriate MCR devices, control equipment room devices or field reports, either by performing the required checks without reference to operating procedures or by performing the relevant checks not explicitly mentioned in an operating procedure being implemented.

1.1 (5) monitors CSPs and SPs when necessary (DTS)

1.2 (4) monitors in a timely manner that major system parameters are trending and stabilize as expected (DTS)

1.3 (4) monitors annunciations and panel indications, as necessary, to maintain an awareness of changing equipment, system and unit conditions (DTS)

1.4 (5) checks in a timely manner that MAAs occur and have the desired effect, prior to referring to operating procedures (DTS)

1.5 (3) checks in a timely manner that operator actions are having or have had the desired effect (DTS)

1.6 (5) checks relevant system, unit and NPP conditions before approving or initiating an action

2.0 Actions taken without reference to procedures

This competency area addresses the ability to direct the execution of the required actions when reference to operating procedures is not expected, when an operating procedure does not exist or prior to accessing the relevant operating procedures following a transient or an abnormal condition.

2.1 (2) ensures, in a timely manner, the assembly and the deployment of the control room response team

2.2 (5) directs, in a timely manner, the execution of the required corrective actions when an MAA fails to occur (DTS)

2.3 (5) directs, in a timely manner, the execution of the required actions, based on diagnosis, prior to accessing the applicable procedures

2.4 (3) directs the execution of other required actions, as appropriate, when reference to operating procedures is not expected

2.5 (4) executes, in a timely manner, the actions specifically required of the CRSS when reference to operating procedures is not expected

2.6 (4) directs, in a timely manner, the execution of the actions required when a transient, an abnormal condition, an additional malfunction, equipment unavailability or any other unit condition is not addressed by procedures

2.7 (4) initiates, in a timely manner, the appropriate corrective or preventive actions when NPP conditions are such that they could threaten NPP personnel or public safety.

2.8 (5) directs, in a timely manner, the execution of corrective actions not specified in operating procedures when CSPs or their SPs are deviating unsafely

3.0 Diagnosis and decision-making

This competency area addresses the ability to independently recognize abnormalities, to independently diagnose malfunctions, to select the relevant operating procedures and, when no relevant procedure exists, to determine an appropriate course of action.

3.1 (5) recognizes, in a timely manner, when CSPs or their SPs are deviating unsafely (DTS)

3.2 (4) recognizes, in a timely manner, when a system parameter, a system configuration or a unit state deviates from an acceptable value, configuration or state defined in operating procedures (DTS and PCTS)

3.3 (5) recognizes, in a timely manner, when a requirement specified in the OP&Ps is about to be, or may have been violated (DTS and PCTS)

3.4 (5) identifies, in a timely manner and without procedure, the alarms and indications that characterize the nature of the malfunction and, when applicable, recognizes the entry conditions to the relevant procedure (DTS)

3.5 (3) identifies, in a timely manner, the applicable procedures, based on diagnosis (DTS and PCTS)

3.6 (2) with reference to procedures, determines, in a timely manner, the nature of the malfunction or identifies the specific procedures to be used (DTS and PCTS)

3.7 (2) confirms the diagnosis or confirms that the correct procedure has been selected, using other available information such as NPP documentation or information from the field (DTS)

3.8 (5) determines, in a timely manner, an appropriate course of actions when procedures do not exist, do not fully address the situation, give conflicting directions or are ineffective (DTS and PCTS)

3.9 (4) recognizes in a timely manner when concurrent abnormal NPP conditions need to be addressed and assigns priorities to the actions required to address them, taking into account their relative importance (DTS and PCTS)

3.10 (3) determines the specific cause of a malfunction using all the information available (DTS)

3.11 (5) determines, in a timely manner, the corrective actions required when CSPs or their SPs are deviating unsafely (DTS)

3.12 (4) determines the desired end state of the NPP, together with any significant intermediate NPP state, and the time frame to establish the end state, following a transient or an abnormal condition (DTS and PCTS)

3.13 (5) determines the appropriateness of a request that requires SM approval, based on a review of the relevant system, unit and NPP conditions, constraints and limits, before approving the action

4.0 Procedural compliance

This competency area addresses the ability to direct the execution of the required actions and checks using a relevant approved procedure. It also addresses the ability to carry out the actions and checks specifically assigned to the CRSS using a relevant approved procedure.

4.1 (3) directs the execution of an applicable procedure correctly and in the correct sequence

4.2 (5) directs the execution of a CSP restoration procedure in a timely manner when required

4.3 (3) executes each step of an approved procedure specifically assigned to the CRSS, correctly and in the correct sequence

5.0 Communications and crew interaction skills

This competency area addresses the ability to communicate with other MCR response team members or crew members clearly and accurately and to interact with them in an efficient and effective manner.

5.1 (5) gives instructions and approvals in a timely manner to MCR response team members or crew members, as appropriate

5.2 (3) acknowledges the requests and the information received

5.3 (5) communicates, in a timely manner, the outcome of a request to its originator

5.4 (3) communicates, in a timely manner, the required information to MCR response team members or crew members, as appropriate

5.5 (2) makes the necessary notifications in a timely manner

5.6 (3) obtains the necessary approvals from NPP management in a timely manner

C.4 Design of a CTS for the RO examination

The examination team shall design each CTS for reactor operator candidates in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to the following criteria:
   1. specify the reactor power and generator load of each reactor unit, the state of the major systems of the various units, the equipment out of service, the status of fuelling activities and any other applicable NPP condition, such as routine tests and maintenance activities in progress
   2. some, but not all, of the equipment out of service may have an impact on the response expected from the candidates
   3. the selected initial NPP conditions do not violate requirements in the NPP operating documentation
2. identify a number of primary malfunctions, classify each of them in one of the categories defined in subsection 13.4.1 and arrange them in a credible sequence. Select the primary malfunctions, with their time of occurrence, according to the following criteria:
   1. the estimated CTS dynamic duration does not significantly exceed 50 minutes
   2. there is a category 2 or category 3 primary malfunction that requires a complex intervention by the candidates, or there is a category 4 primary malfunction
   3. there is no more than one category 4 primary malfunction
   4. the total number of category 3 and category 4 primary malfunctions is not greater than three
   5. there are no more than four primary malfunctions
   6. the upper limits specified in paragraphs (c) to (e) may be exceeded when one of the selected malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
   7. select a primary malfunction that results in a degraded reactor unit condition having a significant impact on the actions of the candidates in response to a subsequent primary malfunction

      Preferably, the primary malfunctions are sequenced so that there is a gradual deterioration of unit conditions.

   8. the time allocated between successive primary malfunctions is such that a typically qualified RO and the support team should be able to respond as expected to any given primary malfunction before the next one occurs

      primary malfunctions can be derived by referring to the following documentation:

      • NPP and industry wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
      • list of simulator malfunction options and simulator training scenarios
3. identify a number of secondary malfunctions associated with the primary malfunctions and their time of occurrence. Select these malfunctions according to the following criteria:
   1. each secondary malfunction satisfies the criteria in subsection 16.3.1
   2. the total number of primary and additional malfunctions is at least 4
   3. the total number of primary and secondary malfunctions is not greater than 10
   4. there are no more than five additional malfunctions for a given primary malfunction
   5. there are no more than three monitoring malfunctions
   6. there is no more than one secondary malfunction causing a Level 1 or a Level 2 impairment of a special safety system
   7. there are no more than three secondary malfunctions causing an impairment of a special safety system
   8. there are no more than three secondary malfunctions of poised standby safety support systems
   9. the upper limits specified in paragraphs (c) to (h) may be exceeded when one of the selected primary malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
4. to satisfy criterion 7.a of subsection 14.1, select some combinations of malfunctions and unit conditions that create situations for which the operating procedures give no specific instructions
5. to satisfy criterion 7.b of subsection 14.1, select challenging concurrent malfunctions or unit conditions that require the candidates to assign priorities
6. identify any operating documentation that needs to be used to respond to the primary and secondary malfunctions, using the frozen documentation
7. determine the following conditions that apply to the participation of the members of the control room support team:
   1. the number of members available and their respective roles
   2. where each member will be at the start of the CTS
   3. when each member summoned will be instructed to arrive at the simulated reactor unit
   4. at multi-unit NPPs, whether any certified person on the support team will be instructed to leave the simulated reactor unit during the CTS and, if so:
      1. when the person will be instructed to leave
      2. when the person will be instructed to return to the reactor unit, if applicable
8. define the endpoint of the CTS by specifying the unit conditions to be reached, a step in a procedure to be performed, or a decision or action expected of the candidates. The endpoint must be clearly recognizable by the examiners
9. determine the appropriate response to each primary and secondary malfunction. Ensure that this response is clear and unique
10. estimate the overall complexity of the CTS, considering the number of primary and secondary malfunctions and their timing, the impact of the initial NPP conditions on the required response, and the restrictions placed on the availability of the members of the control room support team; ensure that a typically qualified RO would have sufficient time to respond as expected to each malfunction
11. ensure that the CTS requires the candidates to demonstrate their skills in all competency areas
12. identify the system parameters whose evolution will be recorded during the conduct of the examination for subsequent use during grading of the examination; the selection of these parameters is to be based on their sensitivity to the selected malfunctions and to the expected response of a candidate (as a minimum, the evolution of the parameters listed in appendix C.17 must be recorded)
13. complete a CTS design checklist, as specified in section 14, to confirm the CTS meets the criteria in the checklist
14. complete the development of the certification examination in accordance with the requirements in subsection 14.1
15. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.5 Design of a CTS for the U0O examination

The examination team shall design each CTS for unit 0 operator candidates in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to the following criteria:
   1. specify the reactor power and generator load of each reactor unit, the state of the major systems of the reactor units and of unit 0, the equipment out of service, the status of fuelling activities and any other applicable NPP condition, such as routine tests and maintenance activities in progress
   2. some, but not all, of the equipment out of service may have an impact on the response expected from the candidates
   3. the selected initial NPP conditions do not violate requirements in the NPP operating documentation
2. identify a number of primary malfunctions, classify each of them in one of the categories defined in subsection 13.4.1 and arrange them in a credible sequence; select the primary malfunctions, with their time of occurrence, according to the following criteria:
   1. the estimated CTS dynamic duration does not significantly exceed 60 minutes
   2. there is a category 2 primary malfunction of systems operated by the unit 0 operators that has a potential impact on reactor safety and that requires a complex intervention by the candidates
      or there is a category 2, category 3 or category 4 primary malfunction on the simulated reactor unit that requires a complex intervention on the systems operated by unit 0 operators
   3. there is no more than one category 4 primary malfunction
   4. there are no more than three primary malfunctions that require a complex intervention by the candidates
   5. there are no more than four primary malfunctions
   6. the upper limits specified in paragraphs (c) to (e) may be exceeded when one of the selected malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
   7. select a primary malfunction that results in a degraded unit 0 condition having a significant impact on the actions of the candidates in response to a subsequent primary malfunction
      Preferably, the primary malfunctions are sequenced so there is a gradual deterioration of unit conditions.
   8. the time allocated between successive primary malfunctions is such that a typically qualified unit 0 operator and the support team should be able to respond as expected to any given primary malfunction before the next one occurs

      Primary malfunctions can be derived by referring to the following documentation:

      • NPP and industry-wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
      • list of simulator malfunction options and simulator training scenarios
3. identify a number of secondary malfunctions associated with the primary malfunctions and their time of occurrence. Select these malfunctions according to the following criteria:
   1. each secondary malfunction satisfies the criteria in subsection 13.4.2
   2. the total number of primary and additional malfunctions is at least 4
   3. the total number of primary and secondary malfunctions is not greater than 10
   4. there are no more than 5 additional malfunctions for a given primary malfunction
   5. there are no more than 3 monitoring malfunctions
   6. there is no more than one secondary malfunction causing a Level 1 or a Level 2 impairment of a special safety system
   7. there are no more than 3 secondary malfunctions causing an impairment of a special safety system
   8. there are no more than 3 secondary malfunctions of poised standby safety support systems
   9. the upper limits specified in paragraphs (c) to (h) may be exceeded when one of the selected primary malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
4. to satisfy criterion 6.d of subsection 14.2, select some combinations of malfunctions and conditions that create situations for which the operating procedures give no specific instructions
5. to satisfy criterion 9 of subsection 14.2, select challenging concurrent malfunctions or conditions that require the candidates to assign priorities
6. identify any operating documentation that needs to be used to respond to the primary and secondary malfunctions, using the frozen documentation
7. determine the following conditions that apply to the participation of the members of the control room support team:
   1. the number of members available and their respective roles
   2. where each member will be at the start of the CTS
   3. when each member summoned will be instructed to arrive at unit 0
   4. whether any certified person on the support team will be instructed to leave unit 0 during the CTS and, if so:
      1. when the person will be instructed to leave
      2. when the person will be instructed to return to unit 0, if applicable
8. define the endpoint of the CTS by specifying the unit 0 or NPP conditions to be reached, a step in a procedure to be performed, or a decision or action expected of the candidates. The endpoint must be clearly recognizable by the examiners
9. determine the appropriate response to each primary and secondary malfunction. Ensure that this response is clear and unique
10. estimate the overall complexity of the CTS, considering the number of primary and secondary malfunctions and their timing, the impact of the initial NPP conditions on the required response, and the restrictions placed on the availability of the members of the control room support team; ensure that a typically qualified unit 0 operator would have sufficient time to respond as expected to each malfunction
11. ensure that the CTS requires the candidates to demonstrate their skills in all competency areas
12. identify the system parameters whose evolution will be recorded during the conduct of the examination for subsequent use during grading of the examination; the selection of these parameters is to be based on their sensitivity to the selected malfunctions and to the expected response of a candidate
13. complete a CTS design checklist, as specified in section 14, to confirm that the CTS meets the criteria in the checklist
14. complete the development of the certification examination in accordance with the requirements in subsection 14.2
15. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.6 Design of a CTS for the PSS examination

The examination team shall design each CTS for plant shift supervisor candidates at single-unit NPPs in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to the following criteria:
   1. specify the reactor power, the generator load, the state of major NPP systems, the equipment out of service, the status of fuelling activities and any other applicable NPP condition, such as routine tests and maintenance activities in progress
   2. some, but not all, of the equipment out of service may have an impact on the response expected from the candidates
   3. the selected initial NPP conditions do not violate requirements in the NPP operating documentation
2. identify a number of primary malfunctions, other than category 1 primary malfunctions, classify each of them in one of the categories defined in subsection 13.4.1 and arrange them in a credible sequence; select the primary malfunctions, with their time of occurrence, according to the following criteria:
   1. the estimated CTS dynamic duration does not significantly exceed 50 minutes
   2. there is a category 2 or category 3 primary malfunction that requires a complex intervention by the candidates, or there is a category 4 primary malfunction
   3. there is no more than one category 4 primary malfunction
   4. the total number of category 3 and category 4 primary malfunctions is not greater than three
   5. there are no more than four primary malfunctions
   6. the upper limits specified in paragraphs (c) to (e) may be exceeded when one of the selected malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
   7. select a primary malfunction that results in a degraded NPP condition having a significant impact on the actions of the candidates in response to a subsequent primary malfunction

      Preferably, the primary malfunctions are sequenced so there is a gradual deterioration of NPP conditions.

   8. the time allocated between successive primary malfunctions is such that a typically qualified PSS and the support team should be able to respond as expected to any given primary malfunction before the next one occurs

      Primary malfunctions can be derived by referring to the following documentation:

      • NPP and industry-wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
      • list of simulator malfunction options and simulator training scenarios
3. identify a number of secondary malfunctions associated with the primary malfunctions and their time of occurrence; select these malfunctions according to the following criteria:
   1. each secondary malfunction satisfies the criteria in subsection 13.4.2
   2. the total number of primary and additional malfunctions is at least 4
   3. the total number of primary and secondary malfunctions is not greater than 10
   4. there are no more than 5 additional malfunctions for a given primary malfunction
   5. there are no more than 3 monitoring malfunctions
   6. there is no more than one secondary malfunction causing a Level 1 or a Level 2 impairment of a special safety system
   7. there are no more than 3 secondary malfunctions causing an impairment of a special safety system
   8. there are no more than 3 secondary malfunctions of poised standby safety support systems
   9. the upper limits specified in paragraphs (c) to (h) may be exceeded when one of the selected primary malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
4. select malfunctions that, under the defined initial NPP conditions, create abnormal situations that would be referred to the PSS for resolution, such as malfunctions that cause impairments of special safety systems or standby safety support systems, a requirement in the OP&Ps that is not met, or problems related to the control of reactor power, cooling of the fuel or containment of radioactivity
5. select challenging concurrent malfunctions or NPP conditions that require the candidates to assign priorities to the actions required to address them
6. to satisfy criterion 10 of subsection 14.3, select some combinations of malfunctions and NPP conditions that create situations for which the operating procedures give no specific instructions
7. identify any operating documentation that needs to be used to respond to the primary and secondary malfunctions, using the frozen documentation
8. determine the following conditions that apply to the participation of the members of the control room support team:
   • the number of members available and their respective roles
   • where each member will be at the start of the CTS
   • when each member summoned will be instructed to arrive in the control room
9. define the endpoint of the CTS by specifying the NPP conditions to be reached, a step in a procedure to be performed, or a decision or instruction expected of the candidates; the endpoint must be clearly recognizable by the examiners
10. determine the appropriate response to each primary and secondary malfunction; ensure that this response is clear and unique
11. estimate the overall complexity of the CTS, considering the number of primary and secondary malfunctions and their timing, the impact of the initial NPP conditions on the required response, and the restrictions placed on the availability of the members of the control room support team; ensure that a typically qualified PSS would have sufficient time to respond as expected to each malfunction
12. ensure that the CTS requires the candidates to demonstrate their skills in all competency areas
13. identify the system parameters whose evolution will be recorded during the conduct of the examination for subsequent use during grading of the examination; the selection of these parameters is to be based on their sensitivity to the selected malfunctions and to the expected response of a candidate (as a minimum, the evolution of the parameters listed in appendix C.17 must be recorded)
14. complete a CTS design checklist, as specified in section 14, to confirm that the CTS meets the criteria in the checklist
15. complete the development of the certification examination in accordance with the requirements in subsection 14.3
16. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.7 Design of an ATS for the PSS examination

The examination team shall design each ATS for plant shift supervisor candidates at single-unit NPPs in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to criteria 1 of appendix C.6
2. identify a number of primary malfunctions, other than category 1 primary malfunctions, classify each of them in one of the categories defined in subsection 13.4.1 and arrange them in a credible sequence; select the primary malfunctions, with their time of occurrence, according to the following criteria:
   1. the estimated ATS dynamic duration does not significantly exceed 25 minutes.
   2. there is a category 2 or category 3 primary malfunction that requires a complex intervention by the candidates, or there is a category 4 primary malfunction
   3. there is no more than one category 4 primary malfunction
   4. there are no more than two primary malfunctions
   5. the upper limits specified in paragraphs (c) and (d) may be exceeded when one of the selected malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
   6. select a primary malfunction that results in a degraded NPP condition having a significant impact on the actions of the candidates in response to a subsequent primary malfunction
   7. there is a category 3 or category 4 primary malfunction or there are at least 3 secondary malfunctions of which at least two are additional malfunctions since the ATS does not include a category 3 or a category 4 primary malfunction
   8. when applicable, the time allocated between successive primary malfunctions is such that a typically qualified PSS and the support team should be able to respond as expected to a given primary malfunction before the next one occurs
      A primary malfunction can be derived by referring to the following documentation:

      • NPP and industry-wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
      • list of simulator malfunction options and simulator training scenarios
3. identify a number of secondary malfunctions associated with the primary malfunctions and their time of occurrence; select these malfunctions according to the following criteria:
   1. each secondary malfunction satisfies the criteria in subsection 13.4.2
   2. the total number of primary and additional malfunctions is at least 2
   3. there are at least 3 secondary malfunctions of which at least two are additional malfunctions, whenever the ATS does not include a category 3 or a category 4 primary malfunction
   4. the total number of primary and secondary malfunctions is not greater than five
   5. there are no more than 3 additional malfunctions
   6. there are no more than 2 monitoring malfunctions
   7. there is no more than one secondary malfunction causing a Level 1 or a Level 2 impairment of a special safety system
   8. there is no more than 3 secondary malfunctions causing an impairment of a special safety system
   9. there are no more than 2 secondary malfunctions of poised standby safety support systems
   10. the upper limits specified in paragraphs (d) to (i) may be exceeded when one of the selected primary malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
   11. the secondary malfunctions only affect indicators, equipment, components or control devices that the candidates are expected to check during the ATS dynamic duration
   12. the number and the nature of the secondary malfunctions are such that a typically qualified shift manager can respond as expected to these malfunctions within the ATS dynamic duration
4. select malfunctions that, under the defined initial NPP conditions, create abnormal situations that would be referred to the PSS for resolution, such as malfunctions that cause impairments of special safety systems or standby safety support systems, a requirement in the OP&Ps that is not met, or problems related to the control of reactor power, cooling of the fuel or containment of radioactivity
5. to satisfy criterion 10.a of subsection 14.3, select some combinations of malfunctions and NPP conditions that create situations for which the operating procedures give no specific instructions
6. to satisfy criterion 9 of subsection 14.3, select challenging concurrent malfunctions or NPP conditions that require the candidates to assign priorities
7. identify any operating documentation that needs to be used to respond to the primary and secondary malfunctions, using the frozen documentation
8. determine the following conditions that apply to the participation of the members of the control room support team:
   1. the number of members available and their respective roles
   2. where each member will be at the start of the ATS
   3. when each member summoned will be instructed to arrive in the control room
9. define the endpoint of the ATS by specifying the NPP conditions to be reached, a step in a procedure to be performed, or a decision or instruction expected of the candidates; the endpoint must be clearly recognizable by the examiners
10. determine the appropriate response to each primary and secondary malfunction. Ensure that this response is clear and unique
11. estimate the overall complexity of the ATS, considering the number of primary and secondary malfunctions and their timing as well as the impact of the initial NPP conditions on the required response; ensure that a typically qualified PSS would have sufficient time to respond as expected to each malfunction
12. identify the system parameters whose evolution will be recorded during the conduct of the examination for subsequent use during grading of the examination; the selection of these parameters is to be based on their sensitivity to the selected malfunctions and to the expected response of a candidate (as a minimum, the evolution of the parameters listed in appendix C.17 must be recorded)
13. complete an ATS design checklist, as specified in section 14, to confirm that the ATS meets the criteria in the checklist
14. complete the development of the certification examination in accordance with the requirements in subsection 14.3
15. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.8 Design of a CTS for the CRSS examination

The examination team shall design each CTS for control room shift supervisor candidates at multi-unit NPPs in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to the following criteria:
   1. specify the reactor power and generator load of each reactor unit, the state of the major systems of the various units, the equipment out of service, the status of fuelling activities and any other applicable NPP condition, such as routine tests and maintenance activities in progress
   2. some, but not all, of the equipment out of service may have an impact on the response expected from the candidates
   3. the selected initial NPP conditions do not violate requirements in the NPP operating documentation
2. identify a number of primary malfunctions, other than category 1 primary malfunctions, classify each of them in one of the categories defined in subsection 13.4.1and arrange them in a credible sequence; select the primary malfunctions, with their time of occurrence, according to the following criteria:
   1. the estimated CTS dynamic duration does not significantly exceed 50 minutes
   2. there is a category 2 or category 3 primary malfunction that requires a complex intervention by the candidates, or there is a category 4 primary malfunction
   3. there is no more than one category 4 primary malfunction
   4. the total number of category 3 and category 4 primary malfunctions is not greater than three
   5. there are no more than four primary malfunctions
   6. the upper limits specified in paragraphs (c) to (e) may be exceeded when one of the selected malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
   7. select a primary malfunction that results in a degraded NPP condition having a significant impact on the actions of the candidates in response to a subsequent primary malfunction

      Preferably, the primary malfunctions are sequenced so there is a gradual deterioration of NPP conditions.

   8. the time allocated between successive primary malfunctions is such that a typically qualified CRSS and the support team should be able to respond as expected to any given primary malfunction before the next one occurs
      • Primary malfunctions can be derived by referring to the following documentation:
      • NPP and industry-wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
3. identify a number of secondary malfunctions associated with the primary malfunctions and their time of occurrence; select these malfunctions according to the following criteria:
   1. each secondary malfunction satisfies the criteria in subsection 13.4.2
   2. the total number of primary and additional malfunctions is at least 4
   3. the total number of primary and secondary malfunctions is not greater than 10
   4. there are no more than 5 additional malfunctions for a given primary malfunction
   5. there are no more than 3 monitoring malfunctions
   6. there is no more than one secondary malfunction causing a Level 1 or a Level 2 impairment of a special safety system
   7. there are no more than 3 secondary malfunctions causing an impairment of a special safety system
   8. there are no more than 3 secondary malfunctions of poised standby safety support systems
   9. the upper limits specified in paragraphs c to h may be exceeded when one of the selected primary malfunctions would produce conditions within the NPP that may cause more malfunctions, based on the NPP's safety analyses or on an event that occurred in the nuclear industry and that could credibly occur at the NPP
4. select malfunctions that, under the defined initial NPP conditions, create abnormal situations that would be referred to the CRSS for resolution, such as malfunctions that cause impairments of special safety systems or standby safety support systems, a requirement in the OP&Ps that is not met, or problems related to the control of reactor power, cooling of the fuel or containment of radioactivity
5. select challenging concurrent malfunctions or NPP conditions that require the candidates to assign priorities to the actions required to address them
6. to satisfy criterion 12 of subsection 14.4, select some combinations of malfunctions and NPP conditions that create situations for which the operating procedures give no specific instructions
7. identify any operating documentation that needs to be used to respond to the primary and secondary malfunctions, using the frozen documentation
8. determine the following conditions that apply to the participation of the members of the control room support team:
   1. the number of members available and their respective roles
   2. where each member will be at the start of the CTS
   3. when each member summoned will be instructed to arrive at the simulated reactor unit or at unit 0
   4. whether any certified individual on the support team will be instructed to leave the simulated reactor unit or unit 0 during the CTS and, if so:
      1. when the individual will be instructed to leave
      2. when the individual will be instructed to return to the relevant unit, if applicable
9. for the CTS during which the candidates have to direct in detail the execution of a complex sequence of operator actions to satisfy criterion 9 of subsection 14.4, determine the particular sequence of actions that the candidates have to direct
10. define the endpoint of the CTS by specifying the NPP conditions to be reached, a step in a procedure to be performed, or a decision or instruction expected of the candidates. The endpoint must be clearly recognizable by the examiners
11. determine the appropriate response to each primary and secondary malfunction. Ensure that this response is clear and unique
12. estimate the overall complexity of the CTS, considering the number of primary and secondary malfunctions and their timing, the impact of the initial NPP conditions on the required response, and the restrictions placed on the availability of the members of the control room support team; ensure that a typically qualified CRSS would have sufficient time to respond as expected to each malfunction
13. ensure that the CTS requires the candidates to demonstrate their skills in all competency areas
14. identify the system parameters whose evolution will be recorded during the conduct of the examination for subsequent use during grading of the examination; the selection of these parameters is to be based on their sensitivity to the selected malfunctions and to the expected response of a candidate (as a minimum, the evolution of the parameters listed in appendix C.17 must be recorded)
15. complete a CTS design checklist, as specified in section 14, to confirm that the CTS meets the criteria in the checklist
16. complete the development of the certification examination in accordance with the requirements in subsection 14.4
17. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.9 Design of a DTS for the CRSS examination

The examination team shall design each DTS for control room shift supervisor candidates at multi-unit NPPs in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to the following criteria:
   1. specify the reactor power and generator load of each reactor unit, the state of the major systems of the various units, the equipment out of service, the status of fuelling activities and any other applicable NPP condition, such as routine tests and maintenance activities in progress
   2. some, but not all, of the equipment out of service may have an impact on the diagnosis expected or on the course of actions to be taken
   3. the selected initial NPP conditions do not violate requirements in the NPP operating documentation
2. select primary and secondary malfunctions that, under the defined initial NPP conditions, create abnormal NPP conditions for which there is at least one significant concern regarding one or more of the following:
   1. control of reactor power
   2. cooling of the fuel
   3. containment of radioactivity
   4. impairment of special safety systems
   5. impairment of standby safety support systems
   6. compliance with OP&Ps
   7. safe operation of NPP systems and equipment
   8. safety of NPP personnel
   9. protection of the environment
3. select one primary malfunction according to the following criteria:
   1. the estimated DTS dynamic duration does not significantly exceed 15 minutes
   2. the primary malfunction is a category 2, category 3 or category 4 primary malfunction that, under the defined initial NPP conditions, requires a sufficiently complex analysis of the available information to make the correct diagnosis and to determine the appropriate course of actions
   3. the primary malfunction does not create abnormal NPP conditions that would require prompt implementation of more than two complex procedures or courses of actions
   4. if the primary malfunction were to occur at the NPP, sufficient information would be available within the DTS dynamic duration to make the correct diagnosis
      • Primary malfunctions can be derived by referring to the following documentation:
      • NPP and industry-wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
      • list of simulator malfunction options and simulator training scenarios
4. define the specific diagnosis that the candidates are expected to make and determine the control room and field information required to make this diagnosis
5. identify a number of secondary malfunctions associated with the primary malfunction and their time of occurrence; select these malfunctions according to the following criteria:
   1. each secondary malfunction satisfies the criteria in subsection 13.4.2
   2. there is at least 1 secondary malfunction
   3. there are no more than 4 secondary malfunctions
   4. there is no more than one secondary malfunction causing a Level 1 or a Level 2 impairment of a special safety system
   5. there are no more than 3 secondary malfunctions causing an impairment of a special safety system
   6. there are no more than 3 secondary malfunctions of poised standby safety support systems
   7. the secondary malfunctions only affect indicators, equipment, components or control devices that the candidates are expected to check during the DTS dynamic duration
   8. the number and the nature of the secondary malfunctions are such that a typically qualified CRSS would be able to respond as expected to these malfunctions and to formulate the correct diagnosis within the DTS dynamic duration
6. to satisfy criterion 12 of subsection 14.4, select some combinations of malfunctions and NPP conditions that create situations for which the operating procedures give no specific instructions
7. to satisfy criterion 10 of subsection 14.4, select challenging concurrent abnormal NPP conditions that require the candidates to assign priorities
8. identify any operating documentation that needs to be used to respond to the primary and secondary malfunctions, using the frozen documentation
9. estimate the time that will be given to the candidates to make the correct diagnosis, based on the performance expectations of the NPP for the CRSS position; this time defines the endpoint of the dynamic portion of the DTS when the simulator is frozen and questioning of the candidates begins
10. determine the appropriate course of actions to respond to the selected malfunctions
    1. ensure that this course of actions is clear
    2. ensure that any appropriate course of actions to respond to a failure of a major automatic action prior to diagnosing the primary malfunction does not have an impact on unit conditions that would prevent, or excessively interfere with, making the correct diagnosis
11. identify the significant concerns associated with the NPP conditions existing at the end of the DTS and arrange them in the categories listed in paragraph 2 above
12. estimate the overall complexity of the DTS, considering the number of malfunctions and their timing as well as the impact of the initial NPP conditions on the required response; ensure that a typically qualified CRSS would have sufficient information and time to respond as expected to the secondary malfunctions and to make the correct diagnosis within the DTS dynamic duration
13. identify the system parameters whose evolution will be recorded during the conduct of the examination for subsequent use during grading of the examination; the selection of these parameters must be based on their sensitivity to the selected malfunctions and to the expected response of a candidate (as a minimum, the evolution of the parameters listed in appendix C.17 must be recorded)
14. complete a DTS design checklist, as specified in section 14, to confirm that the DTS meets the criteria in the checklist
15. complete the development of the certification examination in accordance with the requirements in subsection 14.4
16. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.10 Design of a PCTS for the CRSS examination

The examination team shall design the PCTS for control room shift supervisor candidates at multi-unit NPPs in accordance with the following criteria:

1. define the initial NPP conditions in section 2 of the examiner's guide according to the following criteria:
   1. initial conditions include reactor power and generator load of each reactor unit, the state of the major systems of the various units, the equipment out of service, the status of fuelling activities and any other applicable NPP condition, such as routine tests and maintenance activities in progress
   2. some, but not all, of the equipment out of service may have an impact on the significance of the panel anomalies or on the course of actions to be taken
   3. the selected equipment out of service does not create conditions that violate requirements in the NPP operating documentation
2. identify a verification task to be performed by the candidates and a number of panel anomalies associated with the task. Select the task and the panel anomalies according to the following criteria:
   1. the duration of the selected task does not significantly exceed 15 minutes
   2. the task is related to the responsibilities of the CRSS such as an independent verification of system operating configurations and unit state; verification of system, unit and NPP conditions before granting an approval; verification of available heat sinks
   3. there are at least 3 panel anomalies that indicate abnormal situations that would be referred to the CRSS for resolution, such as abnormal system configurations, impairments of special safety systems or of standby safety support systems, a violation of a requirement in the OP&Ps, or problems related to the control of reactor power, cooling of the fuel or containment of radioactivity
   4. there are no more than 10 panel anomalies
   5. the panel anomalies can be detected from the control room panels during the execution of the verification task
   6. to satisfy criterion 12 of subsection 14.4, select some panel anomalies that represent situations for which the operating procedures give no specific instructions
   7. a typically qualified CRSS would be able to complete the selected task within the time available
      • The verification task and the panel anomalies can be derived by referring to the following documentation:
      • NPP and industry-wide significant event reports
      • operating manuals and operating memos
      • emergency operating procedures and abnormal incident manuals
      • design manuals and other technical basis documents
      • NPP safety report
      • training manuals
      • list of simulator malfunction options and simulator training scenarios
3. identify the control room indications associated with each panel anomaly and any field information required to assess the corresponding abnormal condition
4. identify any operating documentation that needs to be used to perform the verification task, and to identify the existing panel anomalies and to address them, using the frozen documentation
5. determine the appropriate course of actions to address each panel anomaly; ensure that this course of actions is clear
6. determine the time that will be given to the candidates to review and perform the verification task; also determine the time that will be given to the candidates to assess the significance of the panel anomalies and to decide on the course of actions to address them, based on the performance expectations of the NPP for the CRSS position
7. estimate the overall complexity of the PCTS, considering the number of panel anomalies as well as the difficulty of determining their significance and the course of actions to address them; ensure that a typically qualified CRSS would have sufficient time to complete the verification task and to decide how to address the panel anomalies as expected
8. complete a PCTS design checklist, as specified in section 14, to confirm that the PCTS meets the criteria in the checklist
9. complete the development of the certification examination in accordance with the requirements in subsection 14.4
10. develop each examiner's guide in accordance with the requirements in subsection 14.5

C.11 Cover page for a performance-based examiner's guide

Examiner's guide

NPP:
Examination date:
Test scenario title:
Test scenario type:
Test scenario no.:

Estimated duration:
Actual duration:

Candidate's full name:
Candidate's position:

Examination team members: [Print names]