Compliance audit for non-complex established operations

SMS Evaluation Tool for Non Complex organisations

INSERT COMPANY NAME HERE

Audit reference from Safety Team:

Signature:

Position:

Print Name:

Date of signing:

SMS Manual Revision:

To be completed and signed for by the Safety Manager or Accountable Manager

SMS Evaluation Tool Instructions for use

This Tool is designed to be used by non-complex organisations and is a simplified version of the Complex organisation established SMS evaluation Tool. It evaluates the effectiveness of the SMS at 3 different levels of SMS maturity based on the ICAO SMS Framework and Management Guidance for non-complex organisations. References to the CASA SMS Framework are detailed for each SMS element. For each of the 15 elements of the CASA Framework there are a series of markers that are assessed to determine whether they are present and effective using the definitions detailed below. Once the individual markers are assessed by the organisation the management will then sample and verify each marker and assess the overall effectiveness assessment of each CASA SMS element. The overall effectiveness for each element will be used by Management to determine the overall level of the SMS maturity and effectiveness using the Summary Evaluation matrix.

NOTE: As this tool is designed for an established SMS, Element 1.4 – SMS implementation Plan is no longer applicable

Y/N/P Definitions for individual markers (assessed and completed initially by organisation and verified by Management)

Yes

There is evidence that the ‘marker’ is present and is effectively achieving the desired outcome.

There is no evidence that the ‘marker’ is effectively achieving the desired outcome.

Partial

There is evidence that the organisation has developed or is developing satisfactory processes to support the marker, but these are not yet effective or the output has not been tested for effectiveness.

Completing the self assessment : Organisations should carry out a self assessment to determine for each marker whether they feel it is effective or not by placing a Y, N or P in the column and justify that determination in the ‘how its is achieved’ column and be prepared to show evidence of during the evaluation.

1.0 SAFETY POLICY AND OBJECTIVES

1.1 MANAGEMENT COMMITMENT AND RESPONSIBILITY

The organisation should define its safety policy which shall be signed by the Accountable Manager of the organisation. The safety policy should set a clear high level direction for the organisation to follow in order to manage safety effectively. The safety policy should include the safety reporting procedures and clearly indicate which types of behaviours are unacceptable and shall include the conditions under which disciplinary action would not apply. The safety policy should be periodically reviewed to ensure its remains relevant and appropriate to the organisation.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
1.1.1	There is a safety policy that includes a commitment towards achieving the highest safety standards signed by the Accountable Manager
1.1.2	The safety policy is communicated to all employees with the intent that they are made aware of their individual contributions and obligations with regard to Safety.
1.1.3	The safety policy includes a commitment to continuous improvement, observe all applicable legal requirements, standards and best practice providing appropriate resources and defining safety as a primary responsibility of all Managers.
1.1.4C	The safety policy actively encourages safety reporting
1.1.5	The safety policy is reviewed periodically to ensure it remains current
1.1.6C	A Just Culture policy has been defined that clearly identifies the conditions under which punitive action would be considered (e.g. illegal activity, negligence or wilful misconduct)

Safety Team review

1.1 EFFECTIVENESS is achieved when the organisation has defined its safety policy that clearly states its intentions, safety objectives and philosophies and there is visible evidence of safety leadership and Management ‘walking the talk’ and demonstrating by example.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

1.2 SAFETY ACCOUNTABILITIES

The organisation shall identify the accountable manager who shall have ultimate responsibility and accountability, on behalf of the organisation, for the implementation and maintenance of the SMS. Safety responsibilities, accountabilities and authorities should be documented and communicated throughout the organisation so everyone is aware of their roles and responsibilities.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
1.2.1	An Accountable Manager has been appointed with full responsibility and ultimate accountability for the SMS to ensure it is properly implemented and performing effectively.
1.2.2	The Accountable Manager is fully aware of their SMS roles and responsibilities in respect of the safety, policy, safety standards and safety culture of the organisation.
1.2.3	Staff at all levels are aware of and understand their safety accountabilities, authorities and responsibilities regarding all safety Management processes, decisions and actions.

Safety Team review

1.2 EFFECTIVENESS is achieved when there are clear lines of safety accountabilities throughout the organisation including an accountable person who has ultimate accountability for the SMS and the Accountable Manager and Management team fully understand the risks faced by the organisation.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

1.3 APPOINTMENT OF KEY PERSONNEL

The organisation should identify a safety manager to be the responsible individual and focal point for the implementation and maintenance of an effective SMS. Where appropriate a safety committee should support the Accountable Manager and the Safety Manager in delivering an effective SMS.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
1.3.1	A competent person with the appropriate knowledge, skills and experience has been nominated to manage the operation of the SMS and fulfils the required job functions and responsibilities.

Safety Team review

1.3 EFFECTIVENESS is achieved when the SMS is facilitated by the responsible individual and there is a safety structure of key personnel from the various operational areas of the organisation. Business area heads are actively engaged in the safety Management system.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

NOTE: ELEMENT 1.4 – SMS IMPLEMENTATION PLANNING – IS NO LONGER APPLICABLE

1.4 THIRD PART INTERFACE (TPC)

The organisation shall identify all Third Party Contractor (TPC)s requiring to be aware of, and participate in the SMS. Each TPC will be appropriately trained and encouraged to participate in all aspects of the SMS. The company accepts that it holds the responsibility of all workers including those not directly employed, but are reliant on for continued operations.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
1.4.1	All TPCs have been identified and appropriately trained in the Management of safety at the organisation
1.4.2	The organisation has a process to communicate and distribute SMS information to TPCs.
1.4.3	There are SMS training records for all TPCs and these are reviewed annually for accuracy and compliance

Safety Team review

1.4 EFFECTIVENESS is achieved when TPCs are appropriately trained and aware of the organisation’s SMS. There is evidence that TPCs are actively participating in the SMS and two-way communication is apparent.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

1.5 CO-ORDINATION OF EMERGENCY RESPONSE PLANNING

The organisation shall ensure that an emergency response plan that provides for the orderly and efficient transition from normal to emergency operations and the return to normal operations is properly coordinated with the emergency response plans of those organisations it must interface with during the provision of its service.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
1.5.1	An emergency response plan that reflects the size, nature and complexity of the operation has been developed and defines the procedures, roles, responsibilities and actions of the various organisations and key personnel.
1.5.2	The organisation has a process to communicate and distribute the ERP procedures and key personnel in an emergency have easy access to the ERP at all times.
1.5.3	The ERP is periodically tested for the adequacy of the plan and the results reviewed to improve its effectiveness

Safety Team review

1.5 EFFECTIVENESS is achieved when the organisation has an emergency response plan that is appropriate to the organisation and is regularly tested and updated including coordination with other organisations as appropriate.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

1.6 SMS DOCUMENTATION

The organisation shall develop and maintain SMS documentation describing the safety policy, processes and procedures and appropriate safety records as hazard logs and risk assessments. The organisation may incorporate the SMS documentation into its existing operational documentation, or develop and maintain a stand SMS manual.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
1.6.1	There is documentation that describes the safety Management system and the interrelationships between all of its elements
1.6.2	SMS documentation, including SMS related records, are regularly reviewed and updated with appropriate version control in place
1.6.3	SMS documentation is readily available to all personnel

Safety Team review

1.6 EFFECTIVENESS is achieved when the organisation has SMS documentation that describes their approach to the Management of safety that is used throughout the organisation and is regularly reviewed and updated. The documentation meets the safety objectives of the organisation.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

2.0 SAFETY RISK MANAGEMENT

2.1 HAZARD IDENTIFICATION

The organisation has processes in place enable aviation safety hazards to be identified. This should include expert judgement, reporting system and the investigation of incidents and accidents to identify potential hazards.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
2.1.1	The organisation has a reporting system to captures errors, hazards and near misses that is simple to use and accessible to all staff.
2.1.2	The organisation has proactively identified all the major hazards and assessed the risks related to its current activities.
2.1.3	Safety investigations are carried out to identify underlying causes and potential hazards for existing and future operations.
2.1.4	Safety reports are acted on in a timely manner.
2.1.5	Hazard identification is an ongoing process and involves all key personnel and appropriate stakeholders.
2.1.6C	Personnel express confidence and trust in the organisations reporting policy and process.
2.1.7	Human performance related hazards are being identified.

Safety Team review

2.1 EFFECTIVENESS is achieved when aviation safety hazards are being identified and reported throughout the organization. Hazards are captured in a hazard register and assessed in a systematic and timely manner.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

2.2 RISK ASSESSMENT AND MITIGATION

The organisation shall develop and maintain a process that ensures identified hazards are risk assessed and managed to an acceptable level.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
2.2.1	There is a structured process for the Management of risk that includes the assessment of risk associated with identified hazards, expressed in terms of likelihood and severity
2.2.2	There are criteria for evaluating the level of risk the organisation is willing to accept and risk assessments and risk ratings are appropriately justified.
2.2.3	The organisation has risk control strategies that deliver effective and robust mitigations and controls and where applicable corrective action.
2.2.4	Corrective actions resulting from the risk assessment, including timelines and allocation of responsibilities are documented.
2.2.5	Senior Management have visibility of medium and high risk hazards and their mitigation and controls.
2.2.6	The MANAGEMENT significant 7 are being considered and addressed as appropriate. (Refer to risk picture spreadsheet)

Safety Team review

2.2 EFFECTIVENESS is achieved when the organisation understands and is managing its most significant safety risks through a formal process that ensures analysis, assessment and control of the safety risks in operations to an acceptable level.

Effectiveness Not Achieved

MANAGEMENT REMARKS

Effectiveness Achieved

3.0 SAFETY ASSURANCE

3.1 SAFETY PERFORMANCE MONITORING AND MEASUREMENT

The organisation shall develop and maintain the means to verify the safety performance of the organisation and to validate the effectiveness of safety risks controls. This is achieved through audits and the establishment and monitoring of safety performance indicators.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
3.1.1	Safety performance indicators have been defined, promulgated and are being monitored and analysed for trends.
3.1.2	Risk mitigations and controls are being verified/audited to confirm they are working and effective.
3.1.3	Safety Assurance and Compliance Monitoring activities feed back into the hazard identification and risk Management process.
3.1.4	Safety assurance takes into account activities carried out in all directly contracted / sub-contracted organisations.
3.1.5	The organisation is monitoring its current, future and third party safety risks and is taking action to address unacceptable safety risks

Safety Team review

3.1 EFFECTIVENESS is achieved when the organisation has developed a series of safety performance indicators that are appropriate to the type of operation. There is a means to measure and monitor trends and take appropriate action when necessary.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

3.2 INTERNAL SAFETY INVESTIGATION

The organisation shall develop and maintain an ability to internally investigate occurrences, incidents and accidents where there are valuable lessons to be learnt. Not all occurrences will be investigated. The Safety Manager in collaboration with the SRC will determine which will be worthwhile investigations by considering a cost benefit analysis against the safety gains achieved.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
3.2.1	The organisation has established personnel capable and trained to conduct internal safety investigation.
3.2.2	There is evidence that appropriate investigations have be competently investigated and lessons have been learnt
3.2.3	The results of safety investigations have be communicated to the workforce as an educational and safety tool.

Safety Team review

3.2 EFFECTIVENESS is achieved when the organisation uses internal safety investigation effectively to gain training and education, conduct root cause analysis and implement corrective actions	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

3.3 THE MANAGEMENT OF CHANGE

The organisation shall develop and maintain a process to identify changes within the organisation and its operation. It should identify any related hazards which should be risk assessed and managed to ensure that the change does not have an adverse effect on safety performance.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
3.3.1	The organisation has established a process and conducts formal hazard analyses/risk assessment for major operational changes, major organisational changes and changes in key personnel.
3.3.2	Key stakeholders are involved in the change Management process
3.3.3	During the change Management process previous risk assessments and existing hazards are reviewed for possible effect.

Safety Team review

3.2 EFFECTIVENESS is achieved when the organisation uses the safety risk Management system to proactively assess all major changes to the organisation and its operations.

Effectiveness Not Achieved

MANAGEMENT REMARKS

Effectiveness Achieved

3.4 CONTINUOUS IMPROVEMENT OF THE SMS

The organisation shall strive for continuous improvement through regular Management reviews , a process which may include formal and informal cross organisational meetings, employee continuous improvement processes to enhance the organisation’s safety performance.

COMPLIANCE + PERFORMANCE MARKERS

Y/N/P

How it is achieved

Remarks

3.4.1

The SMS is periodically reviewed (at least annually) for improvements in safety performance.

Safety Team review

3.4 EFFECTIVENESS is achieved when the organisation routinely monitors the SMS performance to identify potential areas of improvement and the outcomes of this process lead to improvements to the safety Management system.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

4.0 SAFETY PROMOTION

4.1 TRAINING AND EDUCATION

All personnel are trained and competent to perform their SMS related duties and are aware of the organisation’s safety policy, procedures and processes.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
4.1.1	There is a documented process to identify Safety Management training requirements, including initial and recurrent training, so that personnel are competent to perform their duties, including appropriate training records.

Safety Team review

4.1 EFFECTIVENESS is achieved when all personnel are trained and competent to perform their SMS related duties and the training programme is monitored for its effectiveness and updated.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

4.2 SAFETY COMMUNICATION

The organisation shall communicate safety critical information to its own personnel and other parties where appropriate.

COMPLIANCE + PERFORMANCE MARKERS		Y/N/P	How it is achieved	Remarks
4.2.1	Significant events and investigation outcomes are communicated to staff, including contracted organisations where appropriate.

Safety Team review

4.2 EFFECTIVENESS is achieved when personnel are aware of the SMS, safety critical information and their role in respect of aviation safety.	Effectiveness Not Achieved		MANAGEMENT REMARKS
	Effectiveness Achieved

SMS Evaluation Risk Picture

Risk Description (Top 5 Risks)	Mitigation actions

Significant 7 Risk	How is it being addressed?	How is being measured?
Loss of Control
Runway Excursion
Runway Incursion
Airborne Conflict
Controlled Flight into Terrain
Ground Handling
Fire

SMS Evaluation Summary

Safety Team review

Effectiveness Index	Initiating	Present and suitable	Operating	Effective	Excellence
The SMS as a whole	The SMS is still at the implementation stage	Achieved a successful phase 1 assessment	The systems and processes of the SMS are operating.	The SMS is working in an effective way and is striving for continuous improvement	The organisation is an industry leader and embraces and shares its best practice
Safety Risk Management	The SMS is still at the implementation stage	Achieved a successful phase 1 assessment	The hazard and risk registers are being built up and risks are starting to be managed in proactive manner.	The organisation is continuously identifying hazards and understands it biggest risks and is actively managing them and this can be seen in their safety performance. Safety Risk Management is proactive and predictive.	Key Personnel throughout the organisation are aware and understand the risks relative to their responsibilities and are continuously searching out new hazards and risks and re-evaluating existing risks
Safety Assurance	The SMS is still at the implementation stage	Achieved a successful phase 1 assessment	The Organisation has established SPIs that it is monitoring and is auditing and assessing its SMS and its outputs	The organisation assures itself that is has an effective SMS and is managing its risk through audit, assessment and monitoring of its safety performance.	The organisation is continuously assessing it approach to safety Management and is continuously improving its safety performance and seeking out and embracing best practice
Safety Policy and Objectives	The SMS is still at the implementation stage	Achieved a successful phase 1 assessment	There is a safety policy in place and Senior Management are committed to making the SMS work and is providing appropriate resources to safety Management .	Senior Management are clearly involved in the SMS and the Safety Policy sets out the organisations intent to manage safety and is clearly evident in the day to day operations	The organisation is an industry leader and embraces best practice
Safety Promotion	The SMS is still at the implementation stage	Achieved a successful phase 1 assessment	The organisation has trained its people and has several mediums for safety promotion that it uses for passing on safety information	The organisation puts a considerable resource and effort into training its people and publicising its safety culture and other safety information and monitors the effectiveness of its safety promotion	In addition the organisation provides training and safety promotion to its contracted service providers and assesses the effectiveness of its safety promotion