Article citation information:

Dudziak, D., Buczkowska-Murawska, T., Żokowski, M. Assessment of an unmanned aircraft system’s airworthiness for certification. Scientific Journal of Silesian University of Technology. Series Transport. 2020, 108, 27-36. ISSN: 0209-3324. DOI: https://doi.org/10.20858/sjsutst.2020.108.3.

 

 

Dominika DUDZIAK[1], Teresa BUCZKOWSKA-MURAWSKA[2], Mariusz ŻOKOWSKI[3]

 

 

 

ASSESSMENT OF AN UNMANNED AIRCRAFT SYSTEM’S AIRWORTHINESS FOR CERTIFICATION

 

Summary. This article shows that there is a need to carry out the certification process of the system of Unmanned Aircraft Systems (UAS) as an element conditioning the safety of their operation. The published statistics related to the operation of this type of technical objects were presented and analysed stating that one of the causes of incidents and accidents involving the unmanned aircraft mostly includes failure of components of the unmanned aircraft systems. Considering the lack of definitive formulated legislation and procedures in this area, it gains particular importance. Bearing in mind the review nature of this article, it also introduces basic information on the certification of unmanned aircraft systems, according to standardisation agreements of the North Atlantic Treaty Organisation (NATO). Furthermore, this article presents the characteristics of unmanned aircraft systems, especially for this class of devices, in case of their failure there is no ultimate level of safety guarantee, which is the human factor – operator action.

Keywords: certification, standardisation agreement, safety, unmanned aircraft system

1. INTRODUCTION

 

Unmanned Aircraft, commonly called drones, are gaining popularity by the year. Initially, like most devices,  this technology was available for the Armed Forces only. The development of manufacturing technologies made these solutions widely available and was subsequently applied in addition to the military class solutions. They are currently used for various purposes in uniformed services, health service, energy, geology, photography, cartography, agriculture, environmental protection, recreation and many others. Manufacturers, noticing the potential of unmanned aircraft systems, made attempts to build them for transporting courier parcels, medicines and ultimately organs for transplantation purposes. In the future, unmanned aircraft systems would be applied in areas that are presently unlikely, such as fire extinguishing, rescuing people, and also serve as “sky taxis”. These applications are the ambitious plans of today’s manufacturers, however, it should be noticed that work on these concepts goes far beyond the sphere of science-fiction. An example may be the Australian company, Wing, which launched a parcel delivery service up to 1.5 kg [15] using drones in 2019. In these packages, medicines, food and other items, books, for example, can be transported. This service is available for inhabitants of the suburbs of Crace, Palmerston and Franklin in Australia. These and other concepts of applications are associated with numerous challenges. In the case of parcel delivery, these involve, among others, guaranteeing the safe parcel delivery while considering their weight, weather conditions, adaptation to a type of building, safety issues of performing operations or system reliability. Every concept has its individual limitations. However, the exception here is the operating safety area of unmanned aircraft systems that should be subject to special regulations. In our opinion, one of the most important ways to increase operating safety is the certification of unmanned aircraft systems as a method for allowing the use of systems that meet given criteria.

 

 

1.    CERTIFICATION

 

Certification is a recognition that a product, part, device, organisation or a person meets the applicable requirements of airworthiness, which was confirmed in the declaration of conformity [4]. However, airworthiness is considered as the ability of the aircraft or other equipment or on-board system to operate in flight and on the ground without a significant threat to the flight crew, ground crew, passengers or other third parties [4]. Conclusively, certification is a process primarily aimed at increasing the safety of unmanned aircraft system operation by conducting the UAS system tests verifying its parameters.

 

 

2.    NEED FOR CERTIFICATION

 

Analysing the available tests and reports on the causes of the occurrence of the UAS system failures, it can be noticed that the main failure sources are component failures, human factor, service and others.


2.1.  Causes of unmanned aircraft system failures in the U.S. Army, Navy and Air Force

 

Kevin W. Williams found that available reports on the UAS monitored prove that an accident rate for the UAS is generally much higher than for manned aircraft [14]. Understanding the causal factors associated with these accidents is important because it determines the unmanned aircraft system reliability improvement to a level comparable with manned aircraft. According to Williams, the most reliable source of data on unmanned aircraft system accidents are those provided by the U.S. Armed Forces. This is justified by the fact that the U.S. Armed Forces have a relatively long history of use of the unmanned aircraft system and accurately record information on accidents and incidents.  He analysed the causes of the unmanned aircraft system’s incidents that took place in 1986-2004. However, accidents and incidents without sufficient information as recorded in the U.S. Armed Forces were not considered by William. Finally, he analysed a total of 320 accidents and incidents in the UAS area. He proved that in the case of the majority of tested systems, the electrical and mechanical reliability is at least the same factor causing accidents as the human factor. Based on Williams’s report, it can be concluded that even 59% of accidents were caused by the failures of the UAS components, similarly, the human factor represented only 30% (Fig. 1). It indicates that there is an inverse relationship to that of manned aircraft, where the human factor is consistently the main cause of accidents.

 

 

Fig. 1. Accident causes of Unmanned Aircraft Systems in the U.S. Army, Navy and Air Force from 1986 to 2004 [14]

 

2.2.  Causes of events involving unmanned aircraft systems in 2006-2015

 

The issue of the UAS’s incident and accident causes became a point of interest after the unmanned aircraft collision with British Airways Airbus A320 at the Heathrow airport in 2016. It was decided that understanding the causes of these events will result in increased safety [13]. The authors of this article, in their work, analysed 152 accidents and incidents from unmanned aircraft system’s area from around the world in 2006-2015. These studies as in K. Williams’s report showed that technological issues are important, not human factors. This conclusion is significant because as confirmed by research, it is contrary to the view of the aviation industry, which for the last quarter of the century maintained that the human factor is the main cause of aviation incidents. Their research proved that this statement is true only in the field of manned aviation. Consequently, over the last years of the unmanned aircraft system industry, the greatest emphasis was placed on education and licensing of operators and raising awareness among the users. As a result, the number of events caused by the human factor decreased, but it did not affect the design of air systems. These requirements imposed on unmanned aircraft system operators, however, failed to eliminate the main cause of incidents. It results from the conducted analyses [13] that the cause of 64% of events are failures of the unmanned aircraft system components (Fig. 2), which is consistent with Williams’s results. Finally, it was recommended that regulatory authorities should lay down provisions regulating the issues of requirements on airworthiness and other technical issues.

 

 

Fig. 2. Causes of incidents of unmanned aircraft systems in 2006-2015 [13]

 

According to this research, it can be seen that for more than 25 years, the main factor of the UAS failure was incorrectly identified, which consequently did not allow in minimising the number of incidents. From this perspective, it should be expected that normalisation of this area by introducing regulations and standards imposing requirements in the field of airworthiness and the mandatory UAS certification will probably minimise the number of accidents and incidents. The basis of this claim is the fact that the certification process is inherent to the need to perform a series of tests aimed at verification of the unmanned aircraft system safety.

 

2.3.  EASA safety report

 

The unmanned aircraft system safety issues constitute an area of interest for the European Aviation Safety Agency (EASA), which states in the annual safety reports that technical failures are one of the causes of the UAS incidents and accidents (Tab.) [3]. In its statistics, technical factors are not dominant. The reason for this state can be observed in the absence of reporting accidents and incidents involving unmanned aircraft systems. In addition, these incidents are not tested as thoroughly as in the case of manned aircraft. According to the data published by EASA, it is clear that the number of incidents and accidents involving unmanned aircraft increases yearly (Tab. 1).


Tab.1

Causes of events and incidents of unmanned aircraft in 2011-2016 according to EASA [3]

 

SAFETY PROBLEMS

 

Key areas risks

Incident

Serious incidents

Accidents

Aircraft disruption

Air collision

Obstacle collision in flight

Ground collision

TECHNICAL

System reliability

20

1

11

OPERATIONAL

Control of unmanned aircraft system Flight path and automatic use

3

1

5

 

Airspace violation

185

5

1

 

 

Collision with a bird/wild animal

1

-

1

 

 

 

Flight planning and preparation

3

-

-

Landing management

1

-

-

 

Separation in the air

42

-

-

 

 

HUMAN FACTOR

Navigation and airspace, knowledge and skills

102

4

-

Knowledge of the aircraft, systems and procedures

-

-

-

 

Experience, training, individual competences

-

-

-

ORGANISATIONAL

 

 

 

 

 

 

 

Development and application of regulations and procedures

-

-

-

 

Change and new situation management

-

-

-

 

 

 

Tab. 1

The increase in the number of accidents and incidents in 2011-2016 according to EASA [3]

 

Fatal accidents

Other accidents

Serious accidents

2011-2015

0

2.6

0.3

2016

0

15

7

Change

0

470%

2230%

 


2.4.  The number of incidents in unmanned aircraft systems area according to the Civil Aviation Authority (CAA)

 

The basic institution operating in Poland in this area is the Civil Aviation Authority (CAA). The Civil Aviation Authority informs about the annual increase in the number of events involving unmanned aircraft systems [1]. The Civil Aviation Authority declared in 2018 that the total number of events was 17(Fig. 3). However, according to the Civil Aviation Authority, 60 events were reported in August 2019 [2]. The increase in the number of these incidents is explained by the result of the popularisation of the unmanned aircraft system solutions.

 

Fig. 1. Number of events involving unmanned aerial systems in 2011-2018 according to the Civil Aviation Authority [1]

 

2.5.  Safe impact energy

 

Regardless of stating the fact that the technical aspect determines the safety of using the aircraft system, the challenge is to provide protection against the uncontrolled fall effects as a result of a failure. The studies showed that the impact energy of a falling object should not exceed 66Jso as not to cause harm to man. For example, to make the energy less than 66 J, the object falling at a speed of 20 m/s (72km/h) should have a weight of not more than:

 

                                                                                               (1)

 

Hence, it is necessary to conduct activities to make manufacturers and operators aware of the importance of airworthiness requirements in safety. The manufacturers and operators must be aware that even the so-called “drone toy” may pose a threat.

 

 

3.    CERTIFICATION OF UNMANNED AIRCRAFT SYSTEMS IN THE ARMED FORCES

 

Given that unmanned aircraft systems application began with military applications, the safety issues were equally recognised by the NATO member states. To increase the operational safety of drones, they recognised the need for unmanned aircraft system certification, which results in standardisation documents on unmanned aircraft system airworthiness [7]. These documents are a standardisation agreement (STANAG). This is an agreement of NATO member states on the implementation of the alliance standard in whole or in part, with or without restrictions, to meet the interoperability requirement [6]. By 2019, NATO issued three standardisation documents on unmanned aircraft system airworthiness [9-11], and the fourth is in the phase arrangements [8]. These documents were divided according to the unmanned aircraft type and take-off weight. The requirements contained in the standardisation agreements were based on manned aviation regulations, such as CS-22, CS-23, CS-VLA, etc. These agreements, as interdisciplinary standardisation agreements, are ratified and implemented by the Armed Forces of the NATO alliances and constitute the basis of unmanned aircraft system certification process in military applications.

 

3.1.  Certification process

 

The certification process consists of four main stages [5]. Initially, the applicant applying for the certification of his/her product submits the application and the required documents to the institution conducting the certification process. Thereafter, the certifying authority verifies the quality, substantiveness and adequacy of the provided documentation to the scope of the proposed activity and its compliance with the requirements. Then, the practical verification of compliance with the requirements is conducted. It involves verifying the applicant and his/her subcontractors. As a result, the certifying authority can determine whether the entity applying for the certificate is able to manufacture aviation products in accordance with the requirements and provided instructions. The entire process ends with the acceptance or non-acceptance of the application.

The certification process is time-consuming and restrictive. For example, according to STANAG 4703, the object must meet at least 86% of the requirements, and the rest should be met to obtain a positive certification result [10]. Experience from the certification process carried out by a team of specialists operating at the Aircraft Composite Structures Division of Air Force Institute of Technology shows that in spite of such stringent requirements, it can be concluded that the STANAG guidelines are flexible in terms of the way of presenting proof of compliance with airworthiness. The mentioned airworthiness can be proven by presenting qualitative and quantitative evidence, as well as those resulting from the analysis and process ones, technical description of the construction, design review, those of risk management [12] and in any other form if it is properly described.

 

3.2.  Certification process challenges

 

The manufacturer’s inseparable involvement is connected with the certification process. Already at the stage of the product (unmanned aircraft system) preparation, it should take into account, the costs relevant to the necessary quality and number of the conducted tests. The planned time for conducting the unmanned aircraft system tests is also significant. In addition, conducting the tests and confirming airworthiness constitute a great challenge for the research body. Despite 25 years of history, the unmanned aircraft system is still a relatively new technology, mainly due to its very intensive development. Because of structural limitations, the components are usually parts that are primarily dedicated to use in modelling. Although these components are characterised by low price and high parameters (for example, power in case of engines), they are not as repeatable as in manned aviation, they have a wide range of manufacturing tolerance, and consequently, they may be unreliable. Currently, there are no components dedicated to unmanned aircraft systems on the market, which would meet the aviation requirements. This is attributable to the dynamic development of technology, which motivates manufacturers to frequently introduce new, improved, and thus, not fully tested products to the market. This kind of race results from failure to carry out time-consuming and costly tests of components, as an unprofitable activity. It is worth noting that the manufacturers address their offer to modellers. In turn, the modelling applications do not require the product to meet high requirements in reliability. In this area, the operation of models significantly differs from the operation of unmanned aircraft systems, on which the requirements are imposed from the perspective of manned aviation.

 

3.3.  Airworthiness requirements on the example of a piston engine

 

In contrast to the models that are used only within the modeller’s sight, the unmanned aircraft system usually performs tasks beyond the operator’s sight. This makes it possible to conduct the UAS’s tasks from several hundred kilometres from the operator. Hence, the requirements for unmanned aircraft systems engines are definitely higher in order to ensure the required reliability and adequate operation time. For example, STANAG 4703 requires conducting the engine strength test, which covers a total of 50 hours of operation [10]. For one of the air targets tested in Air Force Institute of Technology, the used engine provided failure-free operation only for about 5 hours. The manufacturer apparently assumed that the engine would be used at full power only for short periods of time, which is typical for modelling applications. However, the air target operation required the engine to run at full power for most of the mission time, which significantly affected its service life. Aiming to extend the operation time, the specialists of Air Force Institute of Technology conducted tests on engines fuelled with higher oil content in relation to 1:50 proposed by the manufacturer. The increase in the oil content to the value from 1:25 to 1:30 ensured the extension of operation time to 25 hours. In addition, the number of engine failures decreased from the initial 1 per 10 flights to approx. 1 per 100 flights. This case perfectly demonstrates the differences in the way of operation of flying models and unmanned aircraft systems. The applied construction and material solutions provide high performance (high power, low weight) sufficient in modelling applications, however, they do not provide the required 50 hours of operation. Due to differences in operation, the modelling components in the selection process for use in an unmanned aircraft system should be tested and operated in a rational way (for example, by limiting the service life resources) to finally meet the expected safety and reliability requirements.

 

3.4.  Airworthiness requirements on the example of the Jet type engine

 

Another example can be experience gained during the development of a set of jet air targets with a programmed flight route (ZOCP-JET2), which was equipped with miniature JetCat type miniature turbine jet engines (MTJE). During the system tests, it occurred that the differences in the production of individual engine units affected real reliability and durability. In addition, these engines were operated in a manner unpredicted by the manufacturer in the modelling applications. One of the main causes of failure in the engine was damage to the bearings during operation. To reduce the frequency of engine failure, and thus, minimise the risk of loss of costly unmanned aircraft system, Air Force Institute of Technology with the Military University of Technology within the framework of the work “Jet Air Targets with a programmed flight route”, developed the solution in the form of a controlled operation. Consequently, an engine preparation and service station was designed. The engine preparation and service station was designed to determine and archive the basic initial characteristics of the (model) new JetCat type miniature turbine jet engines before their installation on a jet air target (JAT) and to monitor the basic engine characteristics in the operation process, taking into consideration the critical external loads resulting from the implementation of air tasks. Finally, a station that allows for testing the selected class of jet engines in terms of safety and guaranteeing airworthiness for this component was created. The process of controlling the engine parameters implemented through its assistance allows to objectively determine the moment of its safe decommissioning. As an effect of the project implementation, it should be stated that ensuring reliable operation of components is a complicated process and requires knowledge, experience, time and instrumentation. However, it should be emphasised that obtaining reliable operation of all components is the only guarantee for achieving the intended increase in unmanned aircraft system operation safety.

 

3.5.  Certification result

 

The above-presented examples are just one aspect of unmanned aircraft system testing, which basically covers every area of its manufacture and operation. Hence, ultimately, the certification result of the entire product can be positive, positive with limitations, and negative. The certification objective is to increase safety; therefore, a defective product cannot be put into service as it poses a threat. However, in such a case, the applicant receives feedback and can improve the unmanned aircraft system and proceed to the certification process again.

 

 

4.    CONCLUSION

 

Certification is a time consuming, but necessary process that involves numerous challenges. However, it is necessary as it guarantees an increase in safety due to the unmanned aircraft system quality. Obtaining the certificate increases confidence in the technology and the manufacturer, which can contribute to his/her revenues. Although the standardisation requirements on the unmanned aircraft system airworthiness are still being developed, following the principles adopted in manned aviation can support manufacturers in the construction of safe unmanned aircraft systems.

 

 

References

 

1.        Civil Aviation Authority. 2019. Biuletyn bezpieczeństwa w lotnictwie cywilnym 3(9) [In Polish: Safety bulletin in civil aviation 3(9)].

2.        Civil Aviation Authority. 2019. Conference: Drony – Prawo, Technologia, Usługi [In Polish: Drones – Law, Technology, Services]. Katowice, 29 August 2019.

3.        EASA. 2017. Annual safety review 2017.

4.        European Defence Agency. 2017. EMAD 1 - acronyms and definitions document.

5.        Ministry of Infrastructure and Construction. 2017. Regulation of the Minister of Infrastructure and Construction of 7 July 2017 on the certification of activities in civil aviation.

6.        MON. 2016. Decision No. 311/Mon Mnister of National Defence of 16 November 2016 regarding the introduction in the Ministry of National Defence of the “Instructions on how to deal with standardisation documents of the North Atlantic Treaty Organisation”.

7.        NATO. 2007. Draft STANAG 4671. Unmanned aircraft systems airworthiness requirements (USAR). NATO Standardization Office.

8.        NATO. Draft STANAG 4738. Unmanned Aerial Vehicle (UAV) Systems Airworthiness Requirements (USAR) for Light Vertical Take-Off and Landing (VTOL) Aircraft. NATO Standardization Office.

9.        NATO. 2016. STANAG 4702. Rotary wing unmanned aircraft systems airworthiness requirements. NATO Standardization Office.

10.    NATO. 2016. STANAG 4703. Light unmanned aircraft systems airworthiness requirements. NATO Standardization Office.

11.    NATO. 2019. STANAG 4671. Unmanned aircraft systems airworthiness requirements (USAR), NATO Standardization Office.

12.    Simba S., W. Niemann, T. Kotzé, A. Agigi. 2018. “Supply chain risk management processes for resilience: a study of South African grocery manufacturers”. Journal of Transport and Supply Chain Management 11 (A325): 1-13. DOI: https://doi.org/10.4102/jtscm.v11i0.325.

13.    Wild G., J. Murray, G. Baxter. 2016. Exploring Civil Drone Accidents and Incidents to Help Prevent Potential Air Disasters. Aerospace.

14.    Williams Kevin. 2004. A Summary of Unmanned Aircraft Accident/Incident Data: Human Factors Implications.

15.    Wing Aviation PTY LTD. Available at: https://wing.com.

 

 

Received 11.03.2020; accepted in revised form 12.06.2020

 

 

Scientific Journal of Silesian University of Technology. Series Transport is licensed under a Creative Commons Attribution 4.0 International License



[1] Aircraft Composite Structures Division, Air Force Institute of Technology, Księcia Bolesława 6,
01-494 Warsaw, Poland. Email: dominika.dudziak@itwl.pl

[2] Aircraft Composite Structures Division, Air Force Institute of Technology, Księcia Bolesława 6,
01-494 Warsaw, Poland. Email: teresa.buczkowska@itwl.pl

[3] Aircraft Composite Structures Division, Air Force Institute of Technology, Księcia Bolesława 6,
01-494 Warsaw, Poland. Email: mariusz.zokowski@itwl.pl