2017 Audit of Contracting and Procurement - Office of Audit and Ethics - August 2018

Executive summary

1.1 Background

The audit of contracting and procurement was included in the approved Canadian Nuclear Safety Commission’s (CNSC) Risk-Based Audit Plan (RBAP) for 2016-17 to 2018-19.

Contracting and procurement activities support the CNSC in achieving its objectives, and are subject to continuous scrutiny across the Government of Canada. A key challenge for the CNSC is to establish contracting and procurement policies and practices that are flexible enough to meet organizational needs, while being fair, open and transparent.

The Accounting, Reporting and Contracting Division (titled the Accounting and Contract Management Division during the period under audit) was assigned contracting and procurement accountability, with operational responsibility assigned to the Contract Management Services (CMS) unit as the CNSC’s contracting authority. These accountabilities and responsibilities included the creation of contracting strategies, development of tools (such as training and guidelines for program managers), and administration and processing of all CNSC contracts and purchase orders.

1.2 Audit objective, scope and approach

The objective of the audit was to provide reasonable assurance that management controls in relation to contracting and procurement activities were in place and operating as intended to ensure compliance with approved contracting and procurement policies, guidelines, directives, activities and procedures, and any other related legislation, guidelines or directives. At the November 21, 2016 Departmental Audit Committee (DAC) meeting, a decision was made to limit the scope of the audit and address contracting and procurement activities not governed by the CNSC Contracting Policy at a later time.

For the purposes of this audit, contracting and procurement were generally defined as the process to acquire goods and services that were not acquired through other business processes, such as acquisition card, travel and accommodation, human resources and grants and contributions. The audit focused on contracting and procurement activities between April 1, 2014 and March 31, 2016.

The audit approach included:

• review of the CNSC’s contracting and procurement processes, policies, standards and guidance in the audit scope period
• interviews with selected individuals from the Corporate Services Branch (CSB), primarily within the Accounting, Reporting and Contracting Division and CMS, and other CNSC stakeholders in their capacity as project authority and bid evaluator
• examination of contract files on a sample basis for evidence in support of contracting and procurement compliance
• review of supporting documentation for contracting activities
• analyses of contracting and procurement data
• assessment of the effectiveness and adequacy of processes and internal controls related to contracting and procurement activities

All audit observations and findings relate to contracting and procurement activities in the scope of the audit between April 1, 2014 and March 31, 2016.

Summary of observations

1.1 – Contracting management

The audit found that the contract planning, documentation and training processes were designed to mitigate contracting risks. CMS had undertaken contract planning activities to assist with the processing of contracts. Contract files were well organized and complete. The CNSC had an adequate contract and procurement training regime for project authorities and others responsible for procurement.

The audit found there were operational-level procurement controls in place, such as the Contract Request Form (CRF), contract checklists and the contracting log, that were not consistently used or not intended to be relied upon as documentation of the formal operational controls. There was a risk that the inconsistent use and informal nature of some controls did not allow management to adequately mitigate contracting risks.

1.2 – Monitoring

The audit found there were contract monitoring processes in place. The CNSC developed several risk-based approaches to monitoring contracts to ensure fairness in vendor selection, including a contracting risk management framework and a Contract Review Committee (CRC).

However, the audit found that CMS did not capture and was not able to effectively provide contract-related information. Monitoring on a more timely and frequent basis and opportunities for continuous improvement were constrained by the manual and inefficient nature of report compilation. A formal cost-benefit analysis of a procurement software application had not been undertaken.

The audit found that the types/nature of contracting and procurement risks were not consistently identified.

The audit found that the CRC had a mandate to review assessments or audit reports, findings or recommendations related to CNSC procurement and contracting, however the CRC did not review the audited financial statements to determine whether the respective audit reports had any observations, findings or recommendations that related to contracting such as FAA sections 32, 33 and 34, contingent liabilities, etc.

The audit also found there were some gaps in the design of the contract peer review process. Further, the audit found that CMS did not have a documented process to identify and escalate potential contract disputes and non-compliance to senior management or to seek advice from the CRC. There was a risk that the control environment and key contracting processes were not adequately monitored as required by the Treasury Board Secretariat (TBS) Contracting Policy, section 5.1.1.

2.1 – Contract award

The audit found that there were documented processes and controls in place to award competitive and non-competitive contracts designed to ensure compliance with policy and regulations. This included guidance and tools for CMS officers, project authorities and bid evaluators for the bid process as well as the selection of the appropriate contracting vehicle. The audit found that the CMS organization structure was conducive to addressing risks and evidence indicating that contracting officers had escalated higher-risk procurements.

However, the audit found that improvements were required to controls that provided assurance of compliance with guidance and policies related to interviews during the bidding process. There was an increased risk that the bidding process would not stand the test of public scrutiny in terms of the fair and transparent communication of all information to all bidders, and it was not in compliance with the TBS Contracting Policy or the CNSC’s contracting guidance.

The audit found there was a high volume of sole-source contracts (with a $25,000 limit). The audit found that during the audit scope period there were 33 contracts of $24,800 to $25,200 put in place. The use of sole-source contracts increased the risk that work was not adequately funded or planned and did not stand the test of public scrutiny that the contracting process was fair and transparent.

The audit found in the contract files that were reviewed that there was no evidence that CMS analyzed the contract price to substantiate the price as required in the TBS Contracting Policy, section 10. The audit found two contracts with one vendor valued at US$311,900, where the contract daily rate was approximately US$13,000. There is a risk that CNSC controls were not effective to ensure that the price of contracts was in compliance with policy.

The audit found that there was no documentation indicating: which contracts were deemed higher-risk; the potential contracting issues; the escalation path; and the basis on which the issues, if any, were resolved. There is a risk that management did not effectively identify, manage, escalate and resolve higher-risk contracts.

2.2 – Contract administration

The audit found that there were contract administration processes and controls in place to provide assurance of compliance with policy and regulations. The CNSC had guidance and processes for contracting officers, project authorities and others responsible for procurement. There was reasonable assurance that the controls related to the justification and approval of contracts and amendments were effective. The audit found that CMS had guidance and processes for CMS officers and project authorities regarding security requirements and there was reasonable assurance that security clearances were in place before contracts were issued.

However, the audit found that the documented assessment of employer-employee relationship was not adequate. The lack of documented assessment of employer-employee relationships with contractors did not provide assurance that the CNSC was in compliance with policy.

The audit found that the CNSC did not have processes to define and manage expectations and timelines related to the evaluation and resolution of potential conflicts of interest. The audit found that there was no evidence to indicate that CMS consistently escalated contracts with vendors that had potential conflict of interest issues to appropriate levels of senior management. The audit also found that CMS did not have a process to identify licensees in order to assess potential contracting conflicts of interest. Further, the audit found opportunities to improve the consultation process with the CNSC Ethics Office and found that the CNSC had made inconsistent conclusions regarding conflicts of interest in support of contracting decisions. There was a risk that the processes and controls for managing contracting conflicts of interest did not ensure compliance with the CNSC Contracting Policy and guidance, the TBS Policy on Contracting and the Government Contracts Regulations. There was also a risk that the evaluation of contracting conflicts of interest would not stand the test of public scrutiny of fairness in procurement processes.

3.1 – Delegation of financial authority

The delegation of authority under the FAA section 32, commitment of funding, section 33, expenditure authority, and section 34, certification of contract deliverables was documented appropriately and exercised properly, in a timely manner. There was an opportunity to clarify the approval of contract options in the delegation of financial signing authorities and in guidance to contracting officers, project authorities and others responsible for procurement.

Overall conclusion

The audit found that the CNSC has contracting processes and controls in place that were designed to ensure compliance with approved contracting and procurement policies, guidelines, directives and procedures and related legislation, guidelines and directives.

The audit identified opportunities for improvement in governance areas specifically related to the contract management and monitoring processes, as well as improvements in the contract award and administration processes. This included improving the control environment by documenting existing controls and ensuring their consistent application, assessing the benefit of a software application specific to contracting and procurement, clarifying and improving the design of some of its existing processes with respect to selecting a contracting vehicle, risk management and escalating higher-risk contracts and vendors that have a potential conflict of interest.

The audit findings have been communicated to Finance and Administration Directorate management, and if addressed, will help provide reasonable assurance that procurement and contracting activities are in place and operating as intended to ensure compliance with contracting and procurement policies, guidelines, directives, activities and procedures, and any other related legislation, guidelines or directives. Detailed information is included in Appendix E.

The audit team would like to acknowledge and thank management and staff for their support throughout the conduct of this audit.

Statement of conformance

This audit conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Office of Audit and Ethics quality assurance and improvement program.

Introduction

1.1. Background

Contracting and procurement activities support the CNSC in achieving its objectives and are subject to continuous scrutiny across the Government of Canada. The CNSC is a designated departmental corporation for the purposes of the Financial Administration Act (FAA)^{Footnote 1} and therefore must carry out contracting and procurement activities according to government regulations, central agency policies and directives, and applicable trade agreements. A key challenge for the CNSC is to establish contracting and procurement policies and practices that are flexible enough to meet organizational needs, while being fair, open and transparent.

The Accounting, Reporting and Contracting Division was assigned contracting and procurement accountability, with operational responsibility assigned to the CMS unit as the CNSC’s contracting authority. The Accounting, Reporting and Contracting Division functionally reports under the CSB. These accountabilities and responsibilities include the creation of contracting strategies, development of tools (such as training and guidelines for program managers), and administration and processing of all CNSC contracts and purchase orders.

Contracting and procurement is an area of high materiality and public visibility within the federal government. An audit of the CNSC’s contracting and procurement provides assurance to the President, the DAC members and senior management that management controls are working effectively to ensure sound stewardship and compliance with contracting and procurement policies, guidelines and procedures.

The CNSC introduced an updated Contracting Policy effective April 1, 2016 as part of cyclical policy renewal initiatives across the Government of Canada. The CNSC also developed and published a Procurement and Contracting Handbook in October 2016, six months after the end of the scope period of the audit of March 31, 2016 intended to inform clients of risks and considerations as part of contracting for goods and services.

1.2. Authority

The 2017 Audit of Contracting and Procurement was part of the approved CNSC RBAP for 2016-17 to 2018-19.

1.3. Audit objective, scope and approach

The objective of the audit was to provide reasonable assurance that management controls in relation to contracting and procurement activities were in place and operating as intended to ensure compliance with approved contracting and procurement policies, guidelines, directives and procedures, and any other related legislation, guidelines or directives  (e.g., Values and Ethics Code, Conflict of Interest and Post-employment Policy, etc.).

The audit included an examination of the contracting and procurement management controls in place over the contracting and procurement function that help ensure compliance with approved contracting and procurement policies, guidelines, directives, activities and procedures. At the November 21, 2016 DAC meeting, a decision was made to address contracting and procurement activities not governed by the CNSC’s Contracting Policy at a later time. Contracting and procurement activities not governed by the CNSC’s Contracting Policy included the acquisition of legal services,memoranda of understanding, service-level agreements and contractual arrangements with service organizations and government organizations.

For the purposes of this audit, contracting and procurement were generally defined as the process followed to acquire goods and services that were not acquired through other business processes, such as acquisition card, travel and accommodation, human resources and grants and contributions.

The audit considered management inputs and contracting and procurement deliverables to the extent that they informed contracting and procurement processes and controls. The financial reporting and administrative aspects of the purchase-to-pay business process were out of scope.

The audit focused on contracting and procurement activities between April 1, 2014 and March 31, 2016. The CNSC issued approximately 900 contracts during that period with an estimated value of $27.1 million and approximately 600 amendments with an estimated cumulative value of $10.3 million.

The audit approach included:

• a review of the CNSC’s contracting and procurement processes, policies, standards and guidance in the audit scope period
• interviews with selected individuals from the CSB, primarily within the Accounting, Reporting and Contracting Division and CMS, and other CNSC stakeholders in their capacity as project authority and bid evaluator; a list of interviewees is included in Appendix D
• the examination of contract files on a sample basis for evidence in support of contracting and procurement compliance
• a review of supporting documentation in regards to contracting activities
• analyses of contracting and procurement data
• an assessment of the effectiveness and adequacy of processes and internal controls related to contracting and procurement activities

1.4. Analysis of risks and fraud, lines of enquiry and audit criteria

During the audit planning phase, an analysis was conducted to identify the risks and any potential fraud or inappropriate actions in the contracting and procurement area, and to assess and prioritize their relevance to the audit objectives. Risks were identified by reviewing relevant documentation and procurement data and through interviews. The lines of enquiry were identified for examination during the audit. Appendix A provides a list of the lines of enquiry and related audit criteria.

1.5. File sampling methodology

It was expected that management documented instances where policies, guidelines, directives, activities and procedures were not followed and included sufficient context and appropriate approvals. Management asserted that contracting and procurement processes and controls were applied consistently throughout the audit scope period. Based on this assertion, the audit team reviewed contract files on a sample basis. A risk-based statistical approach was used to select files for review such that observations from the sample could be projected across the population of contracts from which the sample was drawn.

CMS provided an Excel spreadsheet (the contracting log) as its source for CNSC contracts. The contracting log contained 1,728 individual in-scope contract and amendment records. The contracting data were analyzed to establish a set of records for each relevant audit procedure. For example, all amendments were identified as a single set from which a sample could be drawn and against which audit procedures associated with amendments could be applied. This analysis resulted in eight sets of records.

A risk-based statistical approach was used to define a number of samples for each set of records. This approach resulted in a sample of 244 records related to 166 contract files. Appendix B presents and explains the relationship between the number of contracting records in the audit scope period and the number of records sampled for file review.

1.6. Conformance with professional standards

This audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the Office of Audit and Ethics Quality Assurance and Improvement Program.

Audit observations and recommendations

All audit observations and findings relate to contracting and procurement activities in the scope of the audit between April 1, 2014 and March 31, 2016.

Line of enquiry 1 – Governance

La surveillance de la passation de marchés et de l’approvisionnement fournit une assurance raisonnable du fonctionnement et du contrôle de la fonction de passation de marchés et d’approvisionnement.

Contracting and procurement oversight provides reasonable assurance of operation and control of the contracting and procurement function

Audit criteria 1.1 – Contracting management: Management of the contracting and procurement function is designed to mitigate contracting risks.

The audit found that CMS was engaged with the federal government contracting community to keep abreast of changes and evolving issues related to contracting and procurement. The CMS unit was co-located with seven full-time staff members including a single manager, with a direct reporting structure into the Director, Accounting, Reporting and Contracting Division of the CSB. It was evident that this created an environment that facilitated collaboration on potential issues and minimized procurement-related risks.

Audit criteria 1.1.1 – Policy renewal: The CNSC’s Contracting Policy is compliant with the Treasury Board Secretariat’s Policy on Contracting and Government Contracts Regulations

The CNSC created its own contracting policy to clarify the CNSC’s roles and responsibilities as they related to the TBS Contracting Policy. It was evident that the content of the CNSC’s Contracting Policy would not supersede or create any conflict with the TBS Contracting Policy. The CNSC also documented contracting and procurement guidance available to contracting officers, project authorities and others responsible for procurement.

The CMS structure, practices and approach within the CSB was conducive to responding to policy and guidance changes in a timely manner.

Ad hoc guidance and changes communicated by TBS typically impacted CMS procedures (day-to-day operations) rather than impacting the CNSC Contracting Policy. The CNSC introduced the CSB Policy Instrument Process in December 2016, eight months after the end of the scope period of this audit. The CSB Policy Instrument Process was intended to establish a consistent approach to documenting and monitoring CSB policy instruments throughout their lifecycle, including contracting and procurement.

Audit criteria 1.1.2 – Contract planning: Consider CMS planning efforts with management and the extent to which a managed process mitigates contracting and procurement risks

The audit found that CMS had undertaken contract planning activities to assist with the volume, type, timing and processing of contracts, and to improve client communication.

CMS initiated a planning process in the third quarter of 2015, conducting quarterly meetings with directors, financial advisors, and project authorities.

Audit criteria 1.1.3 – Contract file documentation: Management takes appropriate actions to ensure the adequacy of all contract-related documentation, as per policy requirements.

The audit found that contracting processes and controls were designed to mitigate contracting risks. Contract files were well organized, complete and consistent with documentation requirements on each file’s checklist. Procurement and contracting processes and controls were manual in nature (use of scanned documents, hard copy, Microsoft Word and Excel, pdf, eDocs).

CMS contracting processes and controls were documented in guidance available to contracting officers, project authorities and others responsible for procurement. The CNSC also developed and published the Procurement and Contracting Handbook in October 2016, six months after the end of the scope period of this audit. The Procurement and Contracting Handbook was intended to inform clients of risks and considerations as part of contracting for goods and services.

Mandatory documented CMS controls were limited to obtaining approval for the commitment of funding under section 32 of the FAA and contract approval based on the delegation of authorities. The audit found there were operational-level procurement controls, for example:

• Contract Review Form (CRF) – CRFs were used to document FAA section 32 commitment approval. The CRF was not consistently used. The audit found that the CRF was not used in 4 of 30 files (13%) sampled; approval of the FAA section 32 commitment was in the form of an email, two of which were for a contract value greater than $25,000.
• Contract checklists – Checklists were used to manage the procurement process.

A checklist was observed on each contract file sampled, and each checklist observed was tailored to the contract vehicle being used for the contract. The audit found that CMS management did not intend the checklists to be relied upon as documentation of the formal operational control.

• Contracting log – The contracting log was a management tool used to support corporate reporting and monitoring. The contracting log was generated from information captured in the financial system of record. CMS used the contracting log to track the assignment of contract files and contract end dates. The audit found that CMS management did not intend the contracting log to be relied upon as documentation of the formal operational controls. The audit concluded that there was reasonable assurance the contracting log was complete and accurate for the purposes of audit sampling (see 1.5 File sampling methodology).

There was an opportunity to further enable management’s ability to monitor the adequacy of controls, as per the monitoring requirements of the TBS Contracting Policy section 5.1.1., by clarifying and formalizing documentation and expectations for operational-level controls.

Audit criteria 1.1.4 – Contract training: Management ensures training for each project authority is adequate.

The audit found that the CNSC had an adequate contract and procurement training regime for project authorities and others responsible for procurement. Extensive training material was provided on contracting and delegation of authorities. The material was readily available and provided a depth of details on expectations and explanations.

A managed process was in place to ensure that mandatory contract-related training was successfully completed. CMS targeted project authorities who were expected to take on a larger volume of contracting.

Project authorities and bid evaluators were comfortable with CMS as a resource to ask questions and seek clarification regarding contracting processes and issues.

Summary of audit criteria 1.1

The audit found that the contract planning, documentation and training processes were designed to mitigate contracting risks. However, there was an opportunity to improve the control environment by clarifying and formalizing the documentation and expectations for operational-level controls such as the CRF, contract checklists and the contracting log.

Recommendation 1

It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to ensure controls are designed to mitigate contracting risks by reviewing and documenting formal operational-level controls, including expectations of how the controls are to be performed and how evidence of the controls should be documented and consistently applied. Controls should be designed to ensure compliance with contracting policy and guidance, with a focus on contracting areas that management deems to be higher-risk.

Management response and action plan

Agreed – To further ensure that CNSC contracting controls are effective, the Director General of the Finance and Administration Directorate will review contracting policy, guidance and operational-level controls to clarify and formalize the documentation and execution expectations, with a focus on contracting areas that management deems to be higher-risk. Target completion date: February 2019.

Audit criteria 1.2 – Monitoring: The Accounting, Reporting and Contracting Division and the CRC have adequate processes in place to monitor key contracting processes.

The audit assessed management’s monitoring processes and found that the CNSC’s contracting data repository was a financial software application, which was not intended for and not efficient or effective at facilitating monitoring of the contracting function. The CNSC did not have a software application specific to contracting and procurement. CMS did not capture and was not able to effectively provide contract-related information, such as a list of contracts by procurement vehicle (e.g., ACAN), a list of all contracts referred to the Ethics Officer of the CNSC’s Office of Ethics for conflict of interest review, or a reconciliation of contracts to source data to ensure completeness.

Monitoring on a more timely and frequent basis and opportunities for continuous improvement were constrained by the manual and inefficient nature of report compilation. Management indicated that an investment in technology would be required to improve the efficiency of contracting beyond the current capabilities; however, a formal cost-benefit analysis of a procurement software application had not been undertaken.

There was an opportunity to further enable management’s ability to monitor the adequacy of controls, as per the monitoring requirements of the TBS Contracting Policy section 5.1.1.,
by assessing the benefit of a software application specific to the management, monitoring and reporting needs of contracting and procurement.

Audit criteria 1.2.1 – Fairness in vendor selection (directed contracts/favouritism/repetition) Contracts issued over time are monitored to ensure fairness in the evaluation process and to mitigate the risk of repetitive contracts with the same vendor; the evaluation process is periodically re-evaluated, as required.

The audit found that the CNSC had several risk-based approaches to monitoring contracts to ensure fairness, including a contracting risk management framework and a CRC.

Contracting risk management framework

CMS had a documented contracting risk management framework focused on operational risks. The purpose of the risk management framework was to provide assurance that the CNSC complied with government policy and risk managed the procurement and contracting process. The contracting risk management framework included an assessment of risk for various procurement activities and controls and escalation expectations for each of the procurement sub-activities identified in the contracting risk management framework.

However, the audit found there was an opportunity to improve the control environment by ensuring consistency in the treatment of contracting risks. The types and nature of risks identified in the contracting risk management framework were not consistent with the risks addressed in contracting guidance and not consistent with the risks monitored by management of the Finance and Administration Directorate, as follows:

• There was evidence that CMS monitored contracts based on the information requested by the CRC. The audit did not find evidence to indicate that CMS monitored contracts for the purposes of evaluating risks documented in the contracting risk management framework.
• CMS provided guidance to management on “Unacceptable Contracting Processes” and other guidance that were not assessed in the documented contracting risk management framework, including:
  • contracting with licensees
  • privacy of information
  • protection of information
  • payrolling
• CMS’s documented contracting risk management framework and guidance did not address risks related to audited financial statements, such as FAA sections 32, 33, and 34, contingent liabilities, etc. The Director, Accounting, Reporting and Contracting Division, had oversight of CMS responsibility for audited financial statements including external reporting, and the Internal Controls and Policy Section, the group responsible for testing the effectiveness of internal controls related to audited financial statements.

The audit found that CMS identified and managed contracting and procurement risks and that the CRC provided effective oversight and advice on the fairness of vendor selection. There was an opportunity to improve monitoring processes and activities to ensure they were consistent with risks identified and reported by management.

The audit found that the CRC had a mandate to review assessments or audit reports, findings or recommendations related to CNSC procurement and contracting; however, the CRC did not review the audited financial statements to determine whether the respective audit reports had any observations, findings or recommendations that related to contracting, such as FAA sections 32, 33 and 34, contingent liabilities, etc.

There was an opportunity to document guidance and procedures for the peer review process to mitigate contract risks. There was also an opportunity to include the work of the CRC as an entity-level control over CMS in the CNSC’s financial control framework.

Audit criteria 1.2.2 – Service standards: Performance is evaluated against standards, there is a process to action individual contracts, and the standards are periodically re-evaluated.

Contract service standards were developed after the audit scope period and approved by Management Committee.

For the period of the audit scope, a practical process to monitor performance on a continuous basis was absent. Performance monitoring was limited by the manual and inefficient nature of report compilation. Management Committee did not require CMS to report on service standard performance during the scope period of the audit. Subsequent to the audit scope period, CMS was required to report to Management Committee on an annual basis.

Audit criteria 1.2.3 – Contract disputes and non-compliance: There is evidence of a documented process available to the project authority and/or vendor to escalate, report and resolve contract disputes and internal and external non-compliance.

A documented process was available to project authorities and vendors to escalate, report and resolve contract disputes and non-compliance.

The contracting risk management framework provided an opportunity for management in the Finance and Administration Directorate to escalate contracting files to the CRC for review or challenge, although there was no requirement. It was evident that the CRC requested additional information from CMS and recommended process changes in response to potential issues. The audit found that CMS management informed the CRC of specific contractor disputes, which were minimal in the audit scope period.

The audit revealed that CMS did not have a documented process to identify and escalate potential contract disputes and non-compliance to senior management or to seek advice from the CRC. The audit identified potential issues that were not raised with the CRC, including, but not limited to:

• potential conflict of interest with former public servants in receipt of a Public Service Superannuation Act (PSSA) pension
• potential conflict of interest with licensees

Effective oversight and advice from the CRC was limited to the information provided by CMS in accordance with the CRC’s terms of reference.

Summary of audit criteria 1.2

The audit found that processes and practices in place included a CRC and a contracting risk management framework to monitor key contracting processes. However, there were opportunities to improve the control environment by ensuring that monitoring activities were adequate, as per the requirements of the TBS Contracting Policy section 5.1.1, by assessing the need for a software tool to support the management, monitoring and reporting needs of contracting and procurement, and through improvements to the contracting risk management framework, monitoring by the CRC, the peer review process and the contract escalation process.

Recommendation 2

It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to:

1. assess the benefit of automating tools that will facilitate the management, monitoring and reporting needs of contracting and procurement
2. assess the CNSC’s contracting risk management framework ensuring it is consistent with changes in contracting policies and ensuring risks are reflected in contracting processes, controls and guidance to CMS officers, project authorities and others responsible for procurement. The assessment should include risks related to audited financial statements in the contracting risk management framework and whether the CRC is fulfilling its mandate to review assessments, audit reports, findings and recommendations related to contracting and procurement consistent with the CNSC’s contracting risk management framework
3. review, develop where necessary and document guidance and procedures for the peer review process to mitigate risks consistent with the contracting risk management framework
4. revise, formalize and communicate a process to identify, escalate and report potential contract disputes and non-compliance to senior management and seek advice from the CRC

Management response and action plan

Agreed

To further improve the CNSC contract monitoring environment, the Director General of the Finance and Administration Directorate will:

1. carry out a cost-benefit analysis for a software application in phase 2 of the SAP project
2. assess the Contracting Risk Management Framework to ensure the risk management guidance, processes and controls are appropriate, consistent, integrated and aligned with contracting policies
3. review, develop where necessary, and document procedures for the conduct of peer reviews
4. revise, formalize and communicate the process included in the existing Contracting Risk Management Framework to identify, escalate and report potential contract disputes and non‐compliance to senior management

Target completion date: February 2019

Line of enquiry 2 – Compliance

Contracting and procurement operational processes and controls provide reasonable assurance of compliance with the CNSC Contracting Policy, the Treasury Board Secretariat Policy on Contracting and the Government Contracts Regulations.

The CNSC had guidance for both CMS officers and clients to assess and document risks associated with contracting, including but not limited to:

• the bid solicitation and bid evaluation process
• the sole-source process
• amendments
• contract splitting
• employer-employee relationships
• conflicts of interest
• former public servants in receipt of a PSSA pension
• security requirements

Audit criteria 2.1 – Contract award: The process to award competitive and non-competitive contracts ensures compliance.

Audit criteria 2.1.1 – Bid process: Competitive contracts are issued based on CNSC competitive bidding process policy requirements, including appropriate independent evaluations.

The audit found that the CNSC had documented guidance and tools for CMS officers, project authorities and bid evaluators that provided a clear depth of detail regarding the bidding process, including evaluations. The guidance and tools set clear expectations to protect the integrity of the solicitation process, which began with the CNSC developing a procurement strategy and included developing procurement requirements, criteria for vendor selection, issuing a solicitation document, collecting and evaluating bids, selecting a successful bidder and issuing a contract.

The segregation of responsibilities between CMS and the funding authority, project authority and bid evaluators was appropriate and effectively managed. Project authorities and bid evaluators had a positive perspective of CMS as a valuable resource both in terms of process and knowledge. The segregation of responsibilities was evident and consistent with the CNSC Contracting Policy and guidance.

The audit found during file reviews that communication between the project authority and bidders during the bid evaluation process was not adequately transparent, specifically as it related to interviewing bidder resources. Interviews with bidder resources were not documented in the contract files that were sampled. Based on discussions with management, for certain contracts, interviews of bidder resources were conducted during the bid evaluation process by the CNSC project authority for the purposes of validating the bid submission. Audit interviews revealed that CMS provided inconsistent advice to the project authority on whether it was permissible to interview bidder resources during the bid evaluation period. The CMS officers did not play any role in interviews with bidders, contrary to guidance from CMS that communications with bidders should have been directed through the contracting authority during the bid evaluation process.

CMS did not provide detailed guidance on roles, responsibilities, expectations and required documentation for interviews of resources during the bid evaluation process.

CMS contracting guidance to CNSC managers and staff required that bids must always be assessed in a manner that is fair and transparent, based solely on the evaluation methodology stated in the solicitation documents and known to bidders from the outset. Further, the guidance provided that documents pertaining to the evaluation of bids were to be preserved and documented in the procurement file, and further, that the contracting authority was responsible for ensuring that the confidentiality of the bids was preserved.
The TBS Contracting Policy provided the following:

• 4.2.15 Departments must ensure that adequate management controls are in place to protect the integrity of the bidding process.
• 5.2.2 Contracting authorities are to ensure that contract files are properly documented.
• 12.3.1 Procurement files shall be established and structured to facilitate management oversight with a complete audit trail that contains contracting details related to relevant communications and decisions including the identification of involved officials and contracting approval authorities.

The absence of documentation of interviews with bidder resources increased the risk that the bidding process would not stand the test of public scrutiny of fair and transparent communication of all information to all bidders, and it was not in compliance with the
TBS Contracting Policy or the CNSC’s contracting guidance.

4 of 18 files (22%) sampled did not have a signed bid Evaluator(s) Acceptance of Evaluation Procedures, Conflict of Interest and Non-disclosure Certification. CMS management asserted that the lack of documented certification was not an indication of the existence of a conflict of interest. The absence of documented conflict of interest certification was not in compliance with the TBS Contracting Policy or the CNSC’s contracting guidance, and adequate assurance was not provided to management and stakeholders that a conflict of interest did not exist

Audit criteria 2.1.2 – Contract vehicle: The contract vehicle (competitive, including Advance Contract Award Notice (ACAN), non-competitive, and Standing Offer processes) being used is aligned with policy requirements and legislation (e.g., Values and Ethics Code, Conflict of Interest and Post-employment Policy, etc.), and each contract has an adequate and clear statement of work in which the deliverables are clearly identified in support of the contract objectives.

The audit found an extensive amount of contract-related information available to funding authorities and project authorities on the CMS intranet portal. The portal had links to contracting guidance and tools for individuals at the CNSC, and it was logically structured and easy to understand. There were guidance and tools to assist the project authority in selecting a contracting vehicle and reinforced the importance of developing an adequate statement of work. The portal contained links to templates and examples to assist the project authority with their contracting responsibilities.

Key contracting thresholds

The audit procedures included analyses of contracting data with values around key contracting thresholds. An analysis of contracts in the $10,000-to-$50,000 range was conducted, where the value of each contract was plotted. Based on assertions by CMS management, a larger number of low-dollar value contracts was expected, with a decrease in the number of contracts as the contract value increased.

The audit observed a pattern that was not consistent with this expected relationship, corresponding with the $25,000 sole-source contracting limit (see Appendix C). Further analysis revealed that there were 33 contracts valued at between $24,800 and $25,200, of which 29 were valued at between $24,800 and $25,000. The expected number of contracts in the $24,800-to-$25,200 range was four, based on the assumption that there would be an even distribution of a larger number of low-dollar value contracts with a decrease in the number of contracts as the contract value increased. CNSC guidance and the
TBS Contracting Policy required that every effort should be made to avoid inadequate funding and pre-planning, resulting in amendments, and stated that contract splitting was not permissible.

During audit interviews, CMS and CSB management indicated they encouraged project authorities to make use of sole-source contracting for low-dollar procurement and conversely to make use of a competitive process with options for longer-term resource-related procurement.

CNSC guidance indicated that contracts with a value below $25,000 could set aside the competitive process and be directed to a pre-selected supplier; however, contracting authorities were expected to call for bids whenever it was cost effective to do so, as per the TBS Contracting Policy and to use Public Services and Procurement Canada’s procurement vehicles wherever feasible. Further, justification for a non-competitive contract exceeding $25,000 but below $100,000 had to clearly identify and support one of three noted exceptions (i.e., an emergency, a bid solicitation was not in the public interest, or only one capable supplier existed), and contracts greater than $100,000 required TBS approval or be processed and awarded through Public Services and Procurement Canada to forgo a competitive process.

Reliance on sole-source contracting increased the risk that work was not adequately funded or planned, potentially resulting in either contract splitting or avoidable amendments, both of which were not in compliance with CNSC guidance and the TBS Contracting Policy. While the use of sole-source contracts was permissible, there was a risk that sole-source contracts would not stand the test of public scrutiny, as required in the TBS Contracting Policy statement that indicated that government contracting shall be conducted in a manner that will stand the test of public scrutiny in matters of prudence and probity, facilitate access, encourage competition, and reflect fairness in the spending of public funds.

ACAN contracts

In the contract files that were sampled, there was no evidence that CMS substantiated the price of bids or contracts. In the sample, one vendor had two contracts that were directed through the ACAN contracting process, where there was no evidence that the cost of the services was not analyzed by CMS to substantiate the price. The vendor was a licensee of the CNSC, which increased the risk of a conflict of interest. The contracts were each one year plus two option years (six years) with a cumulative value of US$311,900 for four days of service per year (approximately US$13,000 per day). The contract files did not include a documented rationale or analysis to substantiate the price. CMS management alleged that the cost of service included the value of the vendor’s CNSC license. The contract files did not disclose that the value of the CNSC license was included in the value of the contract. CMS did not adequately substantiate the price of these contracts and the project authority did not adequately disclose the content of these contracts in the statement of work as required by the TBS Contracting Policy, section 10.

Notwithstanding the noted exceptions related to reliance on sole-source contracting and the substantiation of contract prices, the selection and use of contract vehicles was consistent with policy requirements and legislation, and each contract had an adequate statement of work that was clear and whose deliverables were clearly identified in support of the contract objectives.

Audit criteria 2.1.3 – Risk-based contract assessment: Management ensures that individual contracts deemed to be high-risk are appropriately identified and escalated for review, as per the CNSC policy requirements, including any contracts not covered by the CNSC’s Contracting Policy.

It was evident during file reviews and discussions that contract officers were aware of and had escalated some higher-risk procurement. The CMS structure within the CSB was conducive to addressing risks at a contract level in a timely manner. However, during the audit scope period, CMS did not consistently escalate higher-risk procurements. Based on
40 files sampled, some procurement issues were not escalated, for example:

• Out of the four contracts with potential contract splitting identified in the sample, CMS identified and escalated three.
• CMS obtained advice from the CNSC’s Ethics Office on conflict of interest on an ad hoc basis; however, advice was not sought for contracts with CNSC licensees.
• CMS did not identify a potential conflict of interest with a licensee that had two contracts with the CNSC where the price was not analyzed (observation in 2.1.2).

Contracts were not formally assigned a level of risk as indicated in CMS’s process documentation, including contracting guidance, the contracting risk management framework and the peer review process. There was no documentation indicating which contracts were deemed higher-risk, the potential contracting issues, the escalation path and the basis upon which the issues, if any, were resolved. The absence of formal risk assessment could result in ineffective monitoring and mitigation of higher-risk procurement, with a potential for non-compliance with policies.

Summary of audit criteria 2.1

The audit found there were documented processes and controls in place to award competitive and non-competitive contracts designed to ensure compliance with policy and regulations. However, controls that provided assurance of compliance with guidance and policies on interviews during the bidding process, use of sole-source contracts below $25,000 as a contracting vehicle and the process to substantiate the contract price were not designed effectively. There was no documentation indicating which contracts were deemed higher-risk, the potential contracting issues, the escalation path and the basis on which the issues, if any, were resolved. There was a risk that the process to award competitive and non-competitive contracts did not ensure compliance with the CNSC Contracting Policy, and the TBS Policy on Contracting and the Government Contracts Regulations.

Recommandation 3

It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to:

1. clearly define expectations for contracting officers, project authorities and others responsible for procurement regarding the process to conduct and document interviews with bidders in advance of awarding a contract that ensures compliance with policy
2. assess the effectiveness of controls that provide assurance that the use of sole-source contracts below $25,000 as a contracting vehicle is in compliance with policy, the Values and Ethics Code, is fair and transparent and would stand the test of public scrutiny
3. assess the effectiveness of controls that ensure substantiation of the contract price for the service to be provided and do not result in an excess profit to the vendor
4. have CMS develop a methodology to formally assign a level of risk to each contract consistent with the contracting risk management framework, so there is transparency in the escalation and resolution of contracts that pose a relatively higher risk

Management response and action plan

To ensure that CNSC contracting procedures are clearer and more comprehensive, the Director General of the Finance and Administration Directorate will:

1. enhance bid evaluation guidance for project authorities that wish to conduct interviews so that a consistent approach is used that includes documentation and ensures compliance with policy
2. assess the use of sole-source contracts below $25,000 as a contracting vehicle to ensure it is in compliance with policy. The Government Contract Regulations exempts departments from having to solicit bids when the value of the requirement is under $25,000. Additionally, the CRC routinely monitors contracts in the under $25,000 range to ensure that there is no contract splitting or inappropriate amendments. The audit did not identify significant concerns relating to contract splitting or inappropriate amendments
3. review and document the process to ensure that fees being charged are fair (no higher than what is normally charged to other customers)
4. review the Contracting Risk Management Framework to ensure the risk management framework is aligned with contracting policies

Target completion date: February 2019

Audit criteria 2.2 – Contract administration: The process and controls ensure contracts comply with approved government policy and regulations.

Audit criteria 2.2.1 – Employer-employee relationship: There is evidence of a process to identify contracts with individuals requiring escalated approval, as per policy requirements, due to a high risk of an employer-employee relationship.

The audit found that the CNSC had guidance and processes for CMS officers and project authorities to document risks associated with creating an employer-employee relationship. Employer-employee relationships were assessed in the contracting risk management framework. CMS management and the CRC identified and mitigated risks associated with employer-employee relationships for selected contracts.

In 23 of 25 files (92%) sampled, a documented assessment of employer-employee relationship was not included.

The TBS Contracting Policy and CNSC guidance required documentation that facilitated management oversight with an audit trail that contains relevant contracting decisions.

CMS officers judgementally assessed the risk of employer-employee relationship for each contract, and a formal assessment was documented only in instances where a risk was identified.

There was a risk that the lack of documented assessment of employer-employee relationships with contractors did not provide assurance that the CNSC was in compliance with policy.

Audit criteria 2.2.2 – Contracting with former public servants: There is evidence of a process to identify contracts with former public servants and an independent review of conflict of interest conducted by the Office of Audit and Ethics, as per the CNSC’s Conflict of Interest and Post-employment Policy.

The audit found that the CNSC had guidance and a process for project authorities to identify and document risks associated with contracting with former public servants in receipt of a PSSA pension. Former public servants in receipt of a PSSA pension were required to
self-identify any conflict of interest. CMS was not made aware of any conflicts of interest in contracts with former public servants in receipt of a PSSA pension during the audit scope period.

Procurement processes did not effectively mitigate risks associated with potential conflicts of interest when contracting with a former public servant in receipt of a PSSA pension.
CMS requested advice from the CNSC’s Ethics Officer on 16 contracts during the audit scope period.

• In 4 of 16 files (25%) sampled, CMS proceeded with contracting:
  • without providing justification for the decision to proceed with contracting, which in 2 of 4 files included information that had not been made available to CNSC’s Ethics Officer in advance; and
  • for 2 of 4 files, provided a limited amount of time for the Ethics Officer to provide advice, and some instances where the advice was sought after the contract was in place.
• CMS was not made aware of any conflicts of interest in contracts with former public servants in receipt of a PSSA pension during the audit scope period.
• There was no documentation to indicate whether the project authority was aware of the perceived conflict or whether they made an informed decision to proceed with the contract.

Section 16.8.1 of the TBS Contracting Policy indicates, “As stated in article 4.2,
Related requirements, contracts for the services of former public servants in receipt of a pension or of a lump sum payment (the definition of "former public servant" can be found in Appendix A) are to bear the closest public scrutiny and reflect fairness in spending public funds. Contracting authorities should exercise extreme discretion when contracting with former employees in receipt of a pension or of a lump sum payment. These contracts should be subject to the usual review and approval procedures required by the Contracts Directive and departmental policy.” CNSC guidance also emphasized the need for additional scrutiny, making reference to specific TBS Contracting Policy sections.

The CNSC did not have processes to define and manage expectations and timelines related to the evaluation and resolution of potential conflict of interest associated with contracts with former public servants in receipt of a PSSA pension.

There was a risk that the evaluation of contracting conflicts of interest would not stand the test of public scrutiny of fairness in procurement processes.

For audit criteria 2.2.3 Contract amendments, 2.2.4 Non-competitive contracts and 2.2.5  Contract splitting, the audit found that the CNSC had guidance and tools for CMS officers and project authorities to document justification for contracts and amendments and guidance that clearly articulated the potential for negative consequences with contract splitting. The audit found that work related to amendments commenced after the amendment was approved per policy requirements, based on the contract files that were sampled.

Audit criteria 2.2.3 – Contract amendments: There is evidence that amendments are fully justified and documented in the contract file, as per policy requirements.

During file reviews, the audit found documented justification for amendments from the project authority; however, the audit observed instances where the justification for amendments did not address the underlying reasons for a change in contract scope, value or extension of time. A sample of 25 contract records were selected to consider audit procedures specific to amendments, of which five (20%) did not address the underlying reason for a change. Of these five files, one was identified and remediated by CMS during the audit scope period. A sample of 40 contract records were selected to consider audit procedures specific to contracts assessed as high-risk, of which two (5%) involved inadequate planning by the project authority to minimize amendments.

While the documented justification for some amendments was deficient, the audit found a limited number of instances where this was happening. There was reasonable assurance that the controls related to the justification of amendments was effective. Audit management discussed these issues with CMS management and suggested they consider evaluating the effectiveness of controls that provide assurance to management that documented justification for amendments was in full compliance with the TBS Contracting Policy and CNSC guidance.

Audit criteria 2.2.4 – Non-competitive contracts: There is evidence of documented justification, as per policy requirements, to ensure reasonable control over non-competitive contract award.

The audit sampled 45 contract files to assess audit procedures specific to non-competitive contracts and any related amendments. It focused on contracts around contracting thresholds (sole-source contract threshold of $25,000, amendment signing authority threshold of $10,000 and the North American Free Trade Act threshold of $89,600) and vendors with multiple contracts in the audit scope period. The audit found that the justification in 6 of 45 files (13%) sampled may not have addressed the need for the contract, the use of a sole-source contract vehicle or adequately mitigated the need for future amendments. CMS identified deficient justification on three of these contracts, two of which were effectively remediated; for one of these three contracts, the project authority subsequently issued a sole-source contract contrary to the documented direction of CMS for a value close to the sole-source limit ($24,905).  
While the documented contract justification was deficient in some files, the audit found a limited number of instances where this was happening. There was reasonable assurance that the controls related to the justification of contracts were effective. Audit management discussed these issues with CMS management and suggested they consider evaluating the effectiveness of controls that provide assurance to management that documented justification for contracts was in full compliance with the TBS Contracting Policy and CNSC guidance.

Audit criteria 2.2.5 – Contract splitting: There is evidence of a process to ensure that management follows the CNSC’s contracting guidelines regarding contract splitting for each non-competitive contract and amendment.

There was potential contract splitting in 4 of 45 files (9%) sampled, three of which were identified and remediated by CMS, as noted in Audit criteria 2.2.4.

The TBS Contracting Policy required that every effort should be made to avoid inadequate pre-planning, resulting in amendments to change the design, specifications or quantity involved. CNSC guidance indicated that contract amendments should demonstrate that they were not due to lack of funding or project planning, but more due to unforeseen or unanticipated factors, and that contract amendments should have been on an exception basis. The TBS Contracting Policy and CNSC guidance strictly forbid splitting contracts or making contract amendments in order to circumvent appropriate management approval.

While there was a risk that amendments could result in circumventing certain controls, such as sole-source, contract splitting or delegation of authority, or be the result of inadequate funding or poor project planning at the time contracting requirements were developed, the audit found a limited number of instances where this was happening. There was reasonable assurance that the controls related to approval of contracts and amendments was effective. Audit management discussed these issues with CMS management and suggested they consider evaluating the effectiveness of controls that provide assurance to management that contracts and the justification for contracts and amendments are in full compliance with the TBS Contracting Policy and CNSC guidance and stand the test of public scrutiny, including the escalation of contracts and amendments that pose a risk of non-compliance.

Audit criteria 2.2.6 – Conflict of interest and non-disclosure: There is evidence of a process including guidance, tools and controls to ensure reasonable control over conflicts of interest and non-disclosure.

The audit found that CMS monitored potential conflict of interest with vendors. CMS guidance clearly outlined roles and responsibilities for identifying, avoiding and mitigating conflicts of interest in contracting.

Bidders and vendors were required to self-identify any conflict of interest. There were no instances in which a vendor identified a conflict of interest; however, the audit found there was a risk that contracting conflicts of interest were not adequately mitigated. Based on the files sampled, the audit found that there was no evidence to indicate that CMS consistently escalated contracts with potential conflict of interest issues to appropriate levels of senior management.

Licensees

CMS did not have a process to identify licensees in order to assess potential contracting conflict of interest. Negotiations and contracts with licensees represented a higher inherent risk of conflict of interest for which there should have been consultations with the CNSC’s Office of Audit and Ethics, Legal Services and/or other CNSC stakeholders.

Bids and contracts with licensees during the audit scope period were not referred to the CNSC’s Ethics Officer for advice as indicated in CMS guidance.

There was a risk that contracting with licensees would not stand the test of public scrutiny of fairness and independence. An analysis of the impact on the licensee and the licensing process of any real or potential conflict of interest was not evident.

Office of Ethics

As noted in 2.2.2, the CNSC did not have processes to define and manage expectations and timelines related to the evaluation and resolution of potential conflict of interest associated with contracts with former public servants in receipt of a PSSA pension. The consultation process with the CNSC Ethics Office was not documented to ensure that adequate information was provided to the Ethics Officer so that the Ethics Officer could provide informed advice in a timely manner. There was a risk that the CNSC had inconsistent conclusions regarding conflicts of interest in support of contracting decisions.

Role of the project authority

It was not evident from the files sampled whether the project authority was aware of potential conflicts in order to make an informed decision to proceed with the contract. As noted in 2.1.1, bid evaluator conflict of interest certifications were missing from 4 of 18 files (22%) sampled. None of the missing conflict of interest certifications from contract files that were sampled related to contracts with licensees. During audit interviews, the CMS’ and project authorities’ understanding of accountability regarding conflict of interest was not consistent.

There was a risk that CNSC contracting decisions regarding conflicts of interest were not adequately informed.

Role of the project authority

It was not evident from the files sampled whether the project authority was aware of potential conflicts in order to make an informed decision to proceed with the contract. As noted in 2.1.1, bid evaluator conflict of interest certifications were missing from 4 of 18 files (22%) sampled. None of the missing conflict of interest certifications from contract files that were sampled related to contracts with licensees. During audit interviews, the CMS’ and project authorities’ understanding of accountability regarding conflict of interest was not consistent.

There was a risk that CNSC contracting decisions regarding conflicts of interest were not adequately informed.

Purchase orders

Audit criteria 2.2.7 –Security requirements: There is evidence of a process including guidance, tools and controls to ensure reasonable control over contract security requirements.

The audit found that CMS had guidance and processes for CMS officers and project authorities regarding security requirements. The CNSC’s Corporate Security had a role in validating contract security. The audit reviewed a sample of 30 files for which 2 files (6%) did not have the required security documentation. During the course of file reviews and interviews, it was evident that security contract clearance was prioritized at an individual contract level. There was reasonable assurance that CMS officers ensured that security clearance were in place before contracts were issued. Missing documentation was considered administrative in nature with an inherently low risk.

Management response and action plan

The audit found that there were contract administration processes and controls in place to provide assurance of compliance with policy and regulations. The CNSC had guidance and processes for contracting officers, project authorities and others responsible for procurement. However, there were opportunities to improve the documentation of employer-employee relationship, clarify processes to manage and document contracting conflicts of interest, and address potential conflicts of interest in specific relationships, including contracts with licensees and former public servants in receipt of a PSSA pension.

Recommendation 4

It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to:

1. ensure that a documented assessment of employer-employee relationships with contractors is prepared and kept with contract files
2. assess and resolve potential conflicts of interest by:
   1. formalizing and communicating a process that includes roles, responsibilities, expectations and timelines of key stakeholders that includes project authorities and the CNSC’s Ethics Office to ensure timely and informed advice
   2. formally documenting the assessment of conflict of interest that includes an escalation path for instances where there are unresolved potential conflicts of interest
   3. ensuring that the declaration of conflicts of interest by bidders, contractors, project authorities and bid evaluators is prepared and kept with contract files

Management response and action plan

Agreed

1. The Director General of the Finance and Administration Directorate will assess and document the risk-based measures to ensure the risk of an employer-employee relationship is adequately mitigated. The CNSC has chosen to mitigate the risk of an employer-employee relationship by providing training for project authorities and publishing guidance in procurement handbooks and on its procurement website. In addition to these proactive measures, the CNSC also monitors for the development of an employer-employee relationship during the contract execution phase. On an annual basis, a report on contracts that may result in the development of an employer-employee relationship is reviewed by the CRC and recommendations are made to adequately mitigate the risk.
2. The Director General of the Finance and Administration Directorate will further reduce the low risk of potential conflicts of interest by:
1. formalizing and communicating roles and responsibilities, expectations and timelines regarding conflict of interest
2. documenting the escalation path for unresolved potential conflict of interests
3. reinforcing the need to complete and maintain declarations of conflicts of interests from the project authorities and evaluators on contract files.

Target completion date: February 2019

Line of enquiry 3 – Contracting Authority

There is adequate control over the exercise of financial signing authorities (FAA sections 32, 33 and 34) and the delegation of financial signing authorities.

Audit criteria 3.1 – Delegation of financial authority: Delegations of authority under the FAA sections 32, 33 and 34 are appropriately documented and properly exercised in a timely manner for all financial transactions.

The audit found that the CNSC had documented guidance related to the delegation of financial authority, including the commitment of funds, expenditure authority and certification of contract deliverables.

Internal Controls and Policy Section

The CNSC’s Internal Controls and Policy Section was tasked with assessing the CNSC’s Internal Control Management Framework. The focus was on internal controls over financial reporting and was not intended to address all management controls (i.e., contracting controls that mitigated risk at a lower or operational level).

The Internal Controls and Policy Section reported in December 2015 on the effectiveness of design and operating effectiveness of controls to Management Committee identifying four deficiencies, one of which related to FAA section 33 expenditure authority approval.
The Internal Controls and Policy Section validated the implementation of the management action plans to address the deficiencies and reported to Management Committee.

The CNSC evaluated and reported on the operating design and effectiveness of key financial reporting controls related to the purchase-to-payment business cycle.

Audit criteria 3.1.1 – Commitment control: Provide assurance that the responsibility centre manager authorized the expenditure and that CMS ensured the timely and appropriate commitment of funds.

CMS had guidance and processes regarding the commitment of funds. The commitment of funds was reasonably assured, based on an assessment of work performed by the Internal Controls and Policy Section.

Contract options

In 6 of 85 files (7%) sampled, the approval of the contract amendment extending the original contract exceeded the individual’s delegated authority for the cumulative dollar value of the contract. The lack of clarity in applying the delegation of financial signing authorities was compounded by the fact that the amendment simultaneously exercised or otherwise adjusted an option to extend the original contract.

CMS provided written guidance on contract amendments, which considered administrative amendments as posing less risk than financial amendments. While the written guidance was silent on options, CMS management considered options to be administrative amendments.

The audit further assessed the six files noted above, which included additional documentation that mitigated the risks associated with approvals that exceeded the delegated authority. All six files had appropriate FAA section 32 approval based on the delegation of financial signing authorities at the time the contract was originally signed, for the cumulative value of the contract, including options. Further, it was evident that the individual who gave the approval (for which that individual did not have delegated authority) had continuous and ongoing involvement throughout the contract period for each contract file sampled. There was a low risk that the project authority did not have accountability for the contracting process, notwithstanding a technical application of the delegation of financial signing authorities.

Audit criteria 3.1.2 – Expenditure authorization: Finance ensures compliance with the contract terms of payment (FAA section 33).

As indicated in section 2.3.1.1, the Internal Controls and Policy Section reported on the effectiveness of design and operating effectiveness of expenditure authority controls related to the FAA section 33. Contract terms of payment were reasonably assured, based on an assessment of the work performed by the Internal Controls and Policy Section.

Audit criteria 3.1.3 – Certification of contract deliverable: Documentation provided by the vendors supports the contract terms, and deliverables are certified by CNSC project authority managers and retained on file (FAA section 34).

As indicated in section 2.3.1.1, the Internal Controls and Policy Section reported on the effectiveness of design and operating effectiveness of the certification of contract deliverables as they related to the FAA section 34. Certification of contract deliverables was reasonably assured, based on an assessment of work performed by the Internal Controls and Policy Section.

Summary of audit criteria 3.1

The audit found that the delegation of FAA section 32 commitment of funding, section 33 expenditure authority, and section 34 certification of contract deliverables was appropriately documented and properly exercised in a timely manner. There was an opportunity to clarify the approval of contract options. Audit management brought this issue to the attention of CMS management for their consideration.

Overall conclusion

The audit found that the CNSC has contracting processes and controls in place that were designed to ensure compliance with approved contracting and procurement policies, guidelines, directives and procedures and related legislation, guidelines and directives.

The audit identified opportunities for improvement in governance areas specifically related to the contract management and monitoring processes, as well as improvements in the contract award and administration processes. This included improving the control environment by documenting existing controls and ensuring their consistent application, assessing the benefit of a software application specific to contracting and procurement, clarifying and improving the design of some of its existing processes with respect to selecting a contracting vehicle, risk management and escalating higher-risk contracts and vendors that have a potential conflict of interest.

The audit findings have been communicated to Finance and Administration Directorate management, and if addressed, will help provide reasonable assurance that procurement and contracting activities are in place and operating as intended to ensure compliance with contracting and procurement policies, guidelines, directives, activities and procedures, and any other related legislation, guidelines or directives. Detailed information is included in Appendix E.

The audit team would like to acknowledge and thank management and staff for their support throughout the conduct of this audit.

Appendix A: Lines of enquiry and audit criteria

The following lines of enquiry and audit criteria were developed to address the risks identified in the audit risk and fraud assessment.

• 1. Governance – Contracting and procurement oversight provides reasonable assurance of operation and control of the contracting and procurement function.
  • 1.1 Contracting management: Management of the contracting and procurement function is designed to mitigate contracting risks.
    • 1.1.1 Policy renewal: The CNSC Contracting Policy is compliant with the Treasury Board Secretariat’s Policy on Contracting and the Government Contracts Regulations.
    • 1.1.2 Contract planning: Consider CMS planning efforts with management and the extent to which a managed process mitigates contracting and procurement risks.
    • 1.1.3 Contract file documentation: Management takes appropriate actions to ensure the adequacy of all contract-related documentation, as per policy requirements.
    • 1.1.4 Contract training: Management ensures training for each project authority is adequate.
  • 1.2 Monitoring: The Accounting, Reporting and Contracting Division and the CRC have adequate processes in place to monitor key contracting processes.
    • 1.2.1 Fairness in vendor selection (directed contracts/favouritism/repetition): Contracts issued over time are monitored to ensure fairness in the evaluation process and to mitigate the risk of repetitive contracts with the same vendor; the evaluation process is periodically re-evaluated, as required.
    • 1.2.2 Service standards: Performance is evaluated against standards, there is a process to action individual contracts, and the standards are periodically re-evaluated.
    • 1.2.3 Contract disputes and non-compliance: There is evidence of a documented process available to the project authority and/or vendor to escalate, report and resolve contract disputes and internal and external non-compliance.
• 2. Compliance – Contracting and procurement operational processes and controls provide reasonable assurance of compliance with the CNSC Contracting Policy and the Treasury Board Secretariat’s Policy on Contracting and the Government Contracts Regulations.
  • 2.1 Contract award: The process to award competitive and non-competitive contracts ensures compliance.
    • 2.1.1 Bid process: Competitive contracts are issued based on CNSC competitive bidding process policy requirements, including appropriate independent evaluations.
    • 2.1.2 Contract vehicle: The contract vehicle (competitive, including ACAN), non-competitive, and Standing Offer processes) being used is aligned with policy requirements and legislation (e.g., Values and Ethics Code, Conflict of Interest and Post-employment Policy, etc.), and each contract has an adequate and clear statement of work in which the deliverables are clearly identified in support of the contract objectives.
    • 2.1.3 Risk-based contract assessment: Management ensures that individual contracts deemed to be high-risk are appropriately identified and escalated for review, as per CNSC policy requirements, including any contracts not covered by the CNSC’s Contracting Policy.
  • 2.2 Contract administration: The process and controls ensure contracts comply with approved government policy and regulations.
  • 2.2.1 Employer-employee relationship: There is evidence of a process to identify contracts with individuals requiring escalated approval, as per policy requirements, due to a high risk of an employer-employee relationship.
  • 2.2.2 Contracting with former public servants: There is evidence of a process to identify contracts with former public servants and to have an independent review of conflicts of interest conducted by the Office of Audit and Ethics, as per the CNSC’s Conflict of Interest and Post-employment Policy.
  • 2.2.3 Contract amendments: There is evidence that amendments are fully justified and documented in the contract file, as per policy requirements.
  • 2.2.4 Non-competitive contracts: There is evidence of documented justification, as per policy requirements, to ensure reasonable control over non-competitive contract award.
  • 2.2.5 Contract splitting: There is evidence of a process to ensure that management follows the CNSC’s contracting guidelines regarding contract splitting for each non-competitive contract and amendment.
  • 2.2.6 Conflict of interest and non-disclosure: There is evidence of a process, including guidance, tools and controls to ensure reasonable control over conflicts of interest and non-disclosure.
  • 2.2.7 Security requirements: There is evidence of a process, including guidance, tools and controls to ensure reasonable control over contract security requirements.
• 3. Contracting Authority – There is adequate control over the exercise of financial signing authorities (FAA sections 32, 33 and 34) and delegation of financial signing authorities.
  • 3.1 Delegation of financial authority: Delegations of authorityunder the FAA sections 32, 33 and 34 are appropriately documented and properly exercised in a timely manner for all financial transactions.
    • 3.1.1 Commitment control: Provide assurance that the responsibility centre manager authorized the expenditure and that CMS ensured the timely and appropriate commitment of funds.
    • 3.1.2 Expenditure authorization: Finance ensures compliance with the contract terms of payment (FAA section 33).
    • 3.1.3 Certification of contract deliverable: Documentation provided by the vendors supports the contract terms, and deliverables are certified by CNSC project authority managers and retained on file (FAA section 34).

The audit criteria were developed based on an assessment of the inherent risks associated with this audit and draw upon the Office of the Comptroller General’s Audit Criteria Related to the Management Accountability Framework.

Appendix B: Contract file samples

The following table presents the relationship between the number of contracting records and the number of records sampled for file review.

Sample number	Number of records	Number of records sampled
1 – General	1 728	30
2 – Tendered	415	25
3 – Non-Competitive – Thresholds	886	45
4 – Non-Competitive – Low-dollar	882	10
5 – Amendments	620	25
6 – Employer-Employee Relationship	1 368	25
7 – Conflict of Interest	39	44
8 – Risk-based Contract Assessment	227	40
Total	1 728	244

In sample 7, 39 organizations were identified on the CNSC's customer list (including but not limited to all licensees) with which the CNSC contracted (representing vendors) during the audit scope period. A total of 44 contract files were selected, including one contract file for each of the 39 organizations identified, plus five contract files for a vendor that had a significantly larger number of contracts with the CNSC.

In total, 244 records were sampled as drawn from 166 contract files.

Appendix C: Graph of contract values

Support for audit observations are presented in the following graph, which plots all contracts in the audit scope period with a value ranging from $17,500 to $32,500.

Appendix D: List of interviewees

The following table presents the individuals interviewed during the audit.

Organization/role	Interviewee and organizational title
Corporate Services Branch	Stéphane Cyr, Vice-President, Corporate Services Branch and Chief Financial Officer
Finance and Administration Directorate	Daniel Schnob, Director General, Finance and Administration Directorate
	Pierre Souligny, Director, Accounting, Reporting and Contracting Division
	Cara Cosgrove, Chief, Internal Control, Internal Controls and Policy Section
	Kristann Rose, Senior Policy Analyst, Internal Controls and Policy Section
	Lise Fiallos, Chief, Corporate Accounting, Reporting and Financial Services
Contract Management Services	Alex Cassol, Chief, Contract Management Services
	Dan Simard, Senior Contracting Officer, Contract Management Services
	Nathalie Périard, Purchasing Officer, Contract Management Services
Contract Review Committee	Kathleen Heppell-Masys, Contract Review Committee Chair, Director General, Directorate of Security and Safeguards
	Pascale Bourassa, Former Contract Review Committee Member, Director, Training Program Evaluation Division
	Clea Mittag, Member, Contract Review Committee Member, Director, Application Services Division
Project Authority	Peter Button, Project Authority, Research Program Officer, Regulatory Research and Evaluation Division
	Joanne Faucher, Project Authority, Director, Project Management Office Division
	Richard Dubois, Project Authority, Director, IT Security and Services Division
	Michelle Sigouin, Project Authority, Administration Officer, Client Relations and Administrative Services
Bid Evaluator	Lili Hajiesmaili, Bid Evaluator, Team Leader, IM/IT Projects, Project Management Office Division
	Megan Ledgerwood, Bid Evaluator, Senior Programmer Analyst, Application Development and Support, Application Services Division
Secretariat	Nouhad Hammad, Ethics Officer, Office of Audit and Ethics

Appendix E: Audit recommendations and management action plans

The following table presents recommendations made in Audit observations and recommendations (i.e., section 2 of this report) and the associated MAPs provided by CMS.

Recommendations	Management response and action plan	Target completion date
1. It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to ensure controls are designed to mitigate contracting risks by reviewing and documenting formal operational-level controls, including expectations of how the controls are to be performed and how evidence of the controls should be documented and consistently applied. Controls should be designed to ensure compliance with contracting policy and guidance, with a focus on contracting areas that management deems to be higher risk.	The CNSC contracting and procurement control environment is multi-dimensional and mitigates risks through a series of complementary formal and informal controls to ensure compliance with the appropriate policies. Management has also taken steps beyond the requirements of the Treasury Board Contracting Policy to enhance procurement controls. The complementary nature of the controls mitigate the overall risks and provide a high level of assurance that the CNSC complies with the principles of the Financial Administration Act, Treasury Board Contracting Policy and CNSC Contracting Policy. Agreed. To further ensure that CNSC contracting controls are effective, the Director General of the Finance and Administration Directorate will review contracting policy, guidance and operational-level controls to clarify and formalize the documentation and execution expectations, with a focus on contracting areas that management deems to be higher risk.	February 2019
2. It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to: assess the benefit of automating tools that will facilitate the management, monitoring and reporting needs of contracting and procurement assess the CNSC’s contracting risk management framework ensuring it is consistent with changes in contracting policies and ensuring risks are reflected in contracting processes, controls and guidance to CMS officers, project authorities and others responsible for procurement. The assessment should include risks related to audited financial statements in the contracting risk management framework and whether the CRC is fulfilling its mandate to review assessments, audit reports, findings and recommendations related to contracting and procurement consistent with the CNSC’s contracting risk management framework review, develop where necessary and document guidance and procedures for the peer review process to mitigate risks consistent with the contracting risk management framework revise, formalize and communicate a process to identify, escalate and report potential contract disputes and non-compliance to senior management and seek advice from the CRC	Management has previously taken steps beyond the requirements of the Treasury Board Contracting Policy to enhance the nature of procurement controls. For example, CMS developed and introduced service standards which were approved by Management Committee. The CNSC has also established a Contract Review Committee (CRC) to serve as a post-contract award challenge function. The CRC meets quarterly and reviews reports on contracts that were awarded to ensure that contracting activities are compliant with policy. In addition to the CRC, the CNSC has established a peer review process to monitor contracts prior to award. A Contracting Risk Management Framework is applied that outlines key contracting risks, proposes risk mitigation measures, and identifies an escalation process to be followed when the risk cannot be completely mitigated. The CNSC has also developed procurement handbooks and training to provide guidance and advice to contracting officers and project authorities on specific contracting practices (e.g., privacy of information, protection of information, etc.) to minimize risks. The above processes complement one another. The risk management framework identifies an escalation process for some risks and the CRC acts as a post-contract award review committee. The CRC reviews procurements over $10K regardless of risk level. Assessments on financial reporting are reviewed by the Internal Controls Section and their findings are presented at the Departmental Audit Committee. Though the existing processes monitor contracting activities to mitigate risks and ensure compliance with policies, there is an opportunity to evaluate how the individual processes can better work together to enhance the contract monitoring framework. Agreed. To further improve the CNSC contract monitoring environment, the Director General of the Finance and Administration Directorate will: carry out a cost-benefit analysis for a software application in phase 2 of the SAP project assess the Contracting Risk Management Framework to ensure the risk management guidance, processes and controls are appropriate, consistent, integrated and aligned with contracting policies review, develop where necessary, and document procedures for the conduct of peer reviews revise, formalize and communicate the process included in the existing Contracting Risk Management Framework to identify, escalate and report potential contract disputes and non‐compliance to senior management	February 2019
3. It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to: clearly define expectations for contracting officers, project authorities and others responsible for procurement regarding the process to conduct and document interviews with bidders in advance of awarding a contract that ensures compliance with policy assess the effectiveness of controls that provide assurance that the use of sole-source contracts below $25,000 as a contracting vehicle is in compliance with policy, the Values and Ethics Code, is fair and transparent and would stand the test of public scrutiny assess the effectiveness of controls that ensure substantiation of the contract price for the service to be provided and do not result in an excess profit to the vendor have CMS develop a methodology to formally assign a level of risk to each contract consistent with the contracting risk management framework, so there is transparency in the escalation and resolution of contracts that pose a relatively higher risk	Management has taken steps to ensure that contracting processes and controls provide assurance of compliance with the CNSC’s Contracting Policy, and the Treasury Board Secretariat’s Policy on Contracting and the Government Contracts Regulations. Overall, the CNSC presents a well-balanced approach to the use of processes, controls and tools to ensure compliance with policies and other guidance documents. Furthermore, it relies on the judgement, knowledge and expertise of its contract officers and the trustworthiness and values of its project authorities. When these findings are combined with the limited complexity of the CNSC contracting environment, there is a high level of assurance that the CNSC minimizes contracting risk and complies with the principles of the TBS and CNSC contracting policies. Agreed. To ensure that CNSC contracting procedures are clearer and more comprehensive, the Director General of the Finance and Administration Directorate will: enhance bid evaluation guidance for project authorities that wish to conduct interviews so that a consistent approach is used that includes documentation and ensures compliance with policy assess the use of sole-source contracts below $25,000 as a contracting vehicle to ensure it is in compliance with policy. The Government Contract Regulations exempts departments from having to solicit bids when the value of the requirement is under $25,000. Additionally, the CRC routinely monitors contracts in the under $25,000 range to ensure that there is no contract splitting or inappropriate amendments. The audit did not identify significant concerns relating to contract splitting or inappropriate amendments review and document the process to ensure that fees being charged are fair (no higher than what is normally charged to other customers) review the Contracting Risk Management Framework to ensure the risk management framework is aligned with contracting policies	February 2019
4. It is recommended that the Director General of the Finance and Administration Directorate take appropriate action to: a) ensure that a documented assessment of employer-employee relationships with contractors is prepared and kept with contract files b) assess and resolve potential conflicts of interest by: formalizing and communicating a process that includes roles, responsibilities, expectations and timelines of key stakeholders that includes project authorities and the CNSC’s Ethics Office to ensure timely and informed advice formally documenting the assessment of conflict of interest that includes an escalation path for instances where there are unresolved potential conflicts of interest ensuring that the declaration of conflicts of interest by bidders, contractors, project authorities and bid evaluators is prepared and kept with contract files	Agreed. The Director General of the Finance and Administration Directorate will assess and document the risk-based measures to ensure the risk of employer-employee relationship is adequately mitigated. The CNSC has chosen to mitigate the risk of employer-employee relationship by providing training for project authorities and publishing guidance in procurement handbooks and on its procurement website. In addition to these proactive measures, the CNSC also monitors for the development of an employer-employee relationship during the contract execution phase. On an annual basis, a report on contracts that may result in the development of an employer-employee relationship is reviewed by the CRC and recommendations are made to adequately mitigate the risk. The Director General of the Finance and Administration Directorate will further reduce the low risk of potential conflicts of interest by: formalizing and communicating roles and responsibilities, expectations and timelines regarding conflict of interest documenting the escalation path for unresolved potential conflict of interests reinforcing the need to complete and maintain declarations of conflicts of interests from the project authorities and evaluators on contract files	February 2019

Annexe F : Acronyms

The following table presents acronyms used in this document.

ACAN	Advance Contract Award Notice
CMS	Contract Management Services
CNSC	Canadian Nuclear Safety Commission
CRC	Contract Review Committee
CRF	Contract Review Form
CSB	Corporate Services Branch
DAC	Departmental Audit Committee
FAA	Financial Administration Act
PSPC	Public Services and Procurement Canada
PSSA	Public Service Superannuation Act
RBAP	Risk-Based Audit Plan
TBS	Treasury Board Secretariat

Appendix G: Audit time frame and team members

The following is the timeline for the audit planning, examination and reporting.

Audit milestones	Completion dates
Planning phase – audit planning report and audit program	November 28, 2016
Examination phase	May 8, 2017
Audit findings and recommendations to program management	June 15–16, 2017
Draft report to program management	August 16, 2017
Departmental Audit Committee tabling	March 19, 2018

The audit team is composed of the following members of the Office of Audit and Ethics.

Team member	Contact
Joe Anton, Chief Audit Executive	613-947-8220
Rolf Krantz, Audit Team Lead	613-995-3379
Daniel Murphy, Senior Internal Auditor	613-943-5366
Ron Chuchryk, Senior Internal Auditor (examination phase)	613-995-8264
David Holder, Senior Internal Auditor (planning phase)	613-943-0264