Article
citation information:
Dudziak, D., Buczkowska-Murawska,
T., Żokowski, M. Assessment of an unmanned aircraft system’s airworthiness for
certification. Scientific Journal of
Silesian University of Technology. Series Transport. 2020, 108, 27-36. ISSN: 0209-3324. DOI: https://doi.org/10.20858/sjsutst.2020.108.3.
Dominika
DUDZIAK[1], Teresa BUCZKOWSKA-MURAWSKA[2], Mariusz ŻOKOWSKI[3]
ASSESSMENT
OF AN UNMANNED AIRCRAFT SYSTEM’S AIRWORTHINESS FOR CERTIFICATION
Summary. This article shows that there is a need to carry out
the certification process of the system of Unmanned Aircraft Systems (UAS) as
an element conditioning the safety of their operation. The published statistics
related to the operation of this type of technical objects were presented and analysed
stating that one of the causes of incidents and accidents involving the
unmanned aircraft mostly includes failure of components of the unmanned
aircraft systems. Considering the lack of definitive formulated legislation and
procedures in this area, it gains particular importance. Bearing in mind the
review nature of this article, it also introduces basic information on the
certification of unmanned aircraft systems, according to standardisation
agreements of the North Atlantic Treaty Organisation (NATO). Furthermore, this
article presents the characteristics of unmanned aircraft systems, especially
for this class of devices, in case of their failure there is no ultimate level
of safety guarantee, which is the human factor – operator action.
Keywords: certification, standardisation agreement,
safety, unmanned aircraft system
1. INTRODUCTION
Unmanned Aircraft, commonly called drones,
are gaining popularity by
the year. Initially, like most
devices, this technology was
available for the Armed Forces only. The development of manufacturing
technologies made these solutions widely available and was subsequently
applied in addition to the military class solutions. They are currently used
for various purposes in uniformed services, health service, energy, geology,
photography, cartography, agriculture, environmental protection, recreation and
many others. Manufacturers, noticing the potential of unmanned aircraft
systems, made attempts to build them for transporting courier parcels, medicines
and ultimately organs for transplantation purposes. In the future, unmanned
aircraft systems would be applied in areas that are presently unlikely, such as fire
extinguishing, rescuing people, and also serve as “sky taxis”.
These applications are
the ambitious plans of today’s manufacturers,
however, it should be noticed that work on these concepts goes far beyond
the sphere of science-fiction. An example may be the Australian company, Wing,
which launched a parcel delivery service up to 1.5 kg [15] using drones in
2019. In these packages, medicines, food and other items, books, for example, can be transported. This service is available for inhabitants of
the suburbs of Crace, Palmerston and Franklin in Australia. These and other concepts of
applications are associated with numerous challenges. In the case of parcel
delivery, these involve, among others, guaranteeing the safe parcel delivery
while considering their weight, weather conditions, adaptation to a type of
building, safety issues of performing operations or system reliability. Every
concept has its individual limitations. However, the exception here is the
operating safety area of unmanned aircraft systems that should be subject to
special regulations. In our opinion, one of the most important ways to increase operating
safety is the certification of unmanned aircraft systems as a method for
allowing the use of systems that meet given criteria.
1.
CERTIFICATION
Certification is a recognition that
a product, part, device, organisation or a person meets the applicable requirements
of airworthiness, which was confirmed in the declaration of conformity [4].
However, airworthiness is considered as the ability of the aircraft or other
equipment or on-board system to operate in flight and on the ground without a
significant threat to the flight crew, ground crew, passengers or other third
parties [4]. Conclusively, certification is a process primarily aimed at
increasing the safety of unmanned aircraft system operation by conducting the
UAS system tests verifying its parameters.
2.
NEED FOR CERTIFICATION
Analysing the available tests and
reports on the causes of the occurrence of the UAS system failures, it can be
noticed that the main failure sources are component failures, human factor,
service and others.
2.1.
Causes of unmanned aircraft system
failures in the U.S. Army, Navy and Air Force
Kevin W.
Williams found that available reports on the UAS monitored prove that an accident rate for the UAS is
generally much higher than for manned aircraft [14]. Understanding the causal
factors associated with these accidents is important because it determines the
unmanned aircraft system reliability improvement to a level comparable with
manned aircraft. According to Williams, the most reliable source of data on
unmanned aircraft system accidents are those provided by the U.S. Armed Forces.
This is justified by the fact that the U.S. Armed Forces have a relatively long
history of use of the unmanned aircraft system and accurately record
information on accidents and incidents.
He analysed the causes of the unmanned aircraft system’s incidents
that took place in 1986-2004. However, accidents and incidents without
sufficient information as recorded in the U.S. Armed Forces were not considered
by William. Finally, he analysed a total of 320 accidents and incidents in the
UAS area. He proved that in the case of the majority of tested systems, the
electrical and mechanical reliability is at least the same factor causing
accidents as the human factor. Based on Williams’s report, it can be concluded
that even 59% of accidents were caused by the failures of the UAS components,
similarly, the human factor represented only 30% (Fig. 1). It indicates that there is an
inverse relationship to that of manned aircraft, where the human factor is
consistently the main cause of accidents.
Fig. 1. Accident causes of Unmanned
Aircraft Systems in the U.S. Army, Navy and Air Force from 1986 to 2004 [14]
2.2. Causes of events
involving unmanned aircraft systems in 2006-2015
The issue of the UAS’s incident and accident
causes became a point of interest after the unmanned aircraft collision with
British Airways Airbus A320 at the Heathrow airport in 2016. It was decided
that understanding the causes of these events will result in increased safety
[13]. The authors of this article, in their work, analysed 152 accidents and
incidents from unmanned aircraft system’s area from around the world in
2006-2015. These studies as in K. Williams’s report showed that
technological issues are important, not human factors. This conclusion is
significant because as confirmed by research, it is contrary to the view of the
aviation industry, which for the last quarter of the century maintained that the
human factor is the main cause of aviation incidents. Their research proved
that this statement is true only in the field of manned aviation. Consequently,
over the last years of the unmanned aircraft system industry, the greatest
emphasis was placed on education and licensing of operators and raising
awareness among the users. As a result, the number of events caused by the
human factor decreased, but it did not affect the design of air systems. These
requirements imposed on unmanned aircraft system operators, however, failed to
eliminate the main cause of incidents. It results from the conducted analyses
[13] that the cause of 64% of events are failures of the unmanned aircraft
system components (Fig. 2), which is consistent with Williams’s results.
Finally, it was recommended that regulatory authorities should lay down
provisions regulating the issues of requirements on airworthiness and other
technical issues.
Fig. 2. Causes of incidents of
unmanned aircraft systems in 2006-2015 [13]
According to this research, it can
be seen that for more than 25 years, the main factor of the UAS failure was
incorrectly identified, which consequently did not allow in minimising the
number of incidents. From this perspective, it should be expected that
normalisation of this area by introducing regulations and standards imposing
requirements in the field of airworthiness and the mandatory UAS certification
will probably minimise the number of accidents and incidents. The basis of this
claim is the fact that the certification process is inherent to the need to
perform a series of tests aimed at verification of the unmanned aircraft system
safety.
2.3. EASA safety report
The unmanned aircraft system safety
issues constitute an area of interest for the European Aviation Safety Agency
(EASA), which states in the annual safety reports that technical failures are
one of the causes of the UAS incidents and accidents (Tab.)
[3]. In its statistics, technical factors are not dominant. The reason for this
state can be observed in the absence of reporting accidents and incidents
involving unmanned aircraft systems. In addition, these incidents are not
tested as thoroughly as in the case of manned aircraft. According to the data
published by EASA, it is clear that the number of incidents and accidents
involving unmanned aircraft increases yearly (Tab. 1).
Tab.1
Causes of
events and incidents of unmanned aircraft in 2011-2016 according to EASA [3]
|
SAFETY PROBLEMS |
|
Key areas risks |
|||||
|
Incident |
Serious
incidents |
Accidents |
Aircraft
disruption |
Air
collision |
Obstacle collision in flight |
Ground collision |
|
|
TECHNICAL |
|||||||
|
System reliability |
20 |
1 |
11 |
● |
● |
● |
● |
|
OPERATIONAL |
|||||||
|
Control of unmanned aircraft system Flight path and automatic use |
3 |
1 |
5 |
● |
● |
|
● |
|
Airspace violation |
185 |
5 |
1 |
● |
● |
|
|
|
Collision with a bird/wild animal |
1 |
- |
1 |
● |
|
|
|
|
Flight planning and preparation |
3 |
- |
- |
● |
● |
● |
● |
|
Landing management |
1 |
- |
- |
● |
|
● |
● |
|
Separation in the air |
42 |
- |
- |
|
● |
● |
|
|
HUMAN FACTOR |
|||||||
|
Navigation and airspace, knowledge and skills |
102 |
4 |
- |
● |
● |
● |
● |
|
Knowledge of the aircraft, systems and procedures |
- |
- |
- |
● |
● |
● |
|
|
Experience, training, individual competences |
- |
- |
- |
● |
● |
● |
● |
|
ORGANISATIONAL |
|
|
|
|
|
|
|
|
Development and application of regulations and procedures |
- |
- |
- |
|
● |
● |
● |
|
Change and new situation management |
- |
- |
- |
|
● |
● |
|
Tab. 1
The increase in the number of
accidents and incidents in 2011-2016 according to EASA [3]
|
Fatal accidents |
Other accidents |
Serious accidents |
|
|
2011-2015 |
0 |
2.6 |
0.3 |
|
2016 |
0 |
15 |
7 |
|
Change |
0 |
470% |
2230% |
2.4. The number of
incidents in unmanned aircraft systems area according to the Civil Aviation
Authority (CAA)
The basic institution operating in
Poland in this area is the Civil Aviation Authority (CAA). The Civil Aviation
Authority informs about the annual increase in the number of events involving
unmanned aircraft systems [1]. The Civil Aviation Authority declared in 2018 that
the total number of events was 17(Fig. 3). However, according to the Civil
Aviation Authority, 60 events were reported in August 2019 [2]. The increase in
the number of these incidents is explained by the result of the popularisation
of the unmanned aircraft system solutions.
Fig. 1. Number of events involving
unmanned aerial systems in 2011-2018 according to the Civil Aviation Authority [1]
2.5. Safe impact energy
Regardless of stating the fact that
the technical aspect determines the safety of using the aircraft system, the
challenge is to provide protection against the uncontrolled fall effects as a
result of a failure. The studies showed that the impact energy of a falling
object should not exceed 66Jso as not to cause harm to man. For example, to
make the energy less than 66 J, the object falling at a speed of 20 m/s
(72km/h) should have a weight of not more than:
(1)
Hence, it is necessary to conduct
activities to make manufacturers and operators aware of the importance of
airworthiness requirements in safety. The manufacturers and operators must be
aware that even the so-called “drone toy” may pose a threat.
3. CERTIFICATION OF
UNMANNED AIRCRAFT SYSTEMS IN THE ARMED FORCES
Given that unmanned aircraft systems
application began with military applications, the safety issues were equally
recognised by the NATO member states. To increase the operational safety of
drones, they recognised the need for unmanned aircraft system certification,
which results in standardisation documents on unmanned aircraft system
airworthiness [7]. These documents are a standardisation agreement (STANAG).
This is an agreement of NATO member states on the implementation of the
alliance standard in whole or in part, with or without restrictions, to meet
the interoperability requirement [6]. By 2019, NATO issued three
standardisation documents on unmanned aircraft system airworthiness [9-11], and
the fourth is in the phase arrangements [8]. These documents were divided
according to the unmanned aircraft type and take-off weight. The requirements
contained in the standardisation agreements were based on manned aviation
regulations, such as CS-22, CS-23, CS-VLA, etc. These agreements, as
interdisciplinary standardisation agreements, are ratified and implemented by
the Armed Forces of the NATO alliances and constitute the basis of unmanned
aircraft system certification process in military applications.
3.1.
Certification process
The certification process consists
of four main stages [5]. Initially, the applicant applying for the
certification of his/her product submits the application and the required
documents to the institution conducting the certification process. Thereafter,
the certifying authority verifies the quality, substantiveness and adequacy of
the provided documentation to the scope of the proposed activity and its
compliance with the requirements. Then, the practical verification of
compliance with the requirements is conducted. It involves verifying the
applicant and his/her subcontractors. As a result, the certifying authority can
determine whether the entity applying for the certificate is able to
manufacture aviation products in accordance with the requirements and provided
instructions. The entire process ends with the acceptance or non-acceptance of
the application.
The certification process is
time-consuming and restrictive. For example, according to STANAG 4703, the
object must meet at least 86% of the requirements, and the rest should be met
to obtain a positive certification result [10]. Experience from the
certification process carried out by a team of specialists operating at the
Aircraft Composite Structures Division of Air Force Institute of Technology
shows that in spite of such stringent requirements, it can be concluded that
the STANAG guidelines are flexible in terms of the way of presenting proof of
compliance with airworthiness. The mentioned airworthiness can be proven by
presenting qualitative and quantitative evidence, as well as those resulting
from the analysis and process ones, technical description of the construction,
design review, those of risk management [12] and in any other form if it is
properly described.
3.2. Certification process challenges
The manufacturer’s inseparable
involvement is connected with the certification process. Already at the stage
of the product (unmanned aircraft system) preparation, it should take into
account, the costs relevant to the necessary quality and number of the
conducted tests. The planned time for conducting the unmanned aircraft system
tests is also significant. In addition, conducting the tests and confirming
airworthiness constitute a great challenge for the research body. Despite 25
years of history, the unmanned aircraft system is still a relatively new
technology, mainly due to its very intensive development. Because of structural
limitations, the components are usually parts that are primarily dedicated to
use in modelling. Although these components are characterised by low price and
high parameters (for example, power in case of engines), they are not as
repeatable as in manned aviation, they have a wide range of manufacturing
tolerance, and consequently, they may be unreliable. Currently, there are no
components dedicated to unmanned aircraft systems on the market, which would
meet the aviation requirements. This is attributable to the dynamic development
of technology, which motivates manufacturers to frequently introduce new,
improved, and thus, not fully tested products to the market. This kind of race
results from failure to carry out time-consuming and costly tests of
components, as an unprofitable activity. It is worth noting that the
manufacturers address their offer to modellers. In turn, the modelling
applications do not require the product to meet high requirements in
reliability. In this area, the operation of models significantly differs from
the operation of unmanned aircraft systems, on which the requirements are
imposed from the perspective of manned aviation.
3.3.
Airworthiness requirements on the
example of a piston engine
In contrast to the models that are
used only within the modeller’s sight, the unmanned aircraft system
usually performs tasks beyond the operator’s sight. This makes it
possible to conduct the UAS’s tasks from several hundred kilometres from
the operator. Hence, the requirements for unmanned aircraft systems engines are
definitely higher in order to ensure the required reliability and adequate
operation time. For example, STANAG 4703 requires conducting the engine
strength test, which covers a total of 50 hours of operation [10]. For one of
the air targets tested in Air Force Institute of Technology, the used engine
provided failure-free operation only for about 5 hours. The manufacturer
apparently assumed that the engine would be used at full power only for short
periods of time, which is typical for modelling applications. However, the air
target operation required the engine to run at full power for most of the
mission time, which significantly affected its service life. Aiming to extend
the operation time, the specialists of Air Force Institute of Technology
conducted tests on engines fuelled with higher oil content in relation to 1:50
proposed by the manufacturer. The increase in the oil content to the value from
1:25 to 1:30 ensured the extension of operation time to 25 hours. In addition,
the number of engine failures decreased from the initial 1 per 10 flights to
approx. 1 per 100 flights. This case perfectly demonstrates the differences in
the way of operation of flying models and unmanned aircraft systems. The
applied construction and material solutions provide high performance (high
power, low weight) sufficient in modelling applications, however, they do not
provide the required 50 hours of operation. Due to differences in operation, the
modelling components in the selection process for use in an unmanned aircraft
system should be tested and operated in a rational way (for example, by
limiting the service life resources) to finally meet the expected safety and
reliability requirements.
3.4. Airworthiness requirements on the example of the Jet type engine
Another example can be experience
gained during the development of a set of jet air targets with a programmed
flight route (ZOCP-JET2), which was equipped with miniature JetCat type
miniature turbine jet engines (MTJE). During the system tests, it occurred that
the differences in the production of individual engine units affected real
reliability and durability. In addition, these engines were operated in a
manner unpredicted by the manufacturer in the modelling applications. One of
the main causes of failure in the engine was damage to the bearings during
operation. To reduce the frequency of engine failure, and thus, minimise the
risk of loss of costly unmanned aircraft system, Air Force Institute of
Technology with the Military University of Technology within the framework of
the work “Jet Air Targets with a
programmed flight route”, developed the solution in the form of a
controlled operation. Consequently, an engine preparation and service station
was designed. The engine preparation and service station was designed to
determine and archive the basic initial characteristics of the (model) new
JetCat type miniature turbine jet engines before their installation on a jet
air target (JAT) and to monitor the basic engine characteristics in the
operation process, taking into consideration the critical external loads
resulting from the implementation of air tasks. Finally, a station that allows
for testing the selected class of jet engines in terms of safety and
guaranteeing airworthiness for this component was created. The process of
controlling the engine parameters implemented through its assistance allows to
objectively determine the moment of its safe decommissioning. As an effect of
the project implementation, it should be stated that ensuring reliable
operation of components is a complicated process and requires knowledge,
experience, time and instrumentation. However, it should be emphasised that
obtaining reliable operation of all components is the only guarantee for
achieving the intended increase in unmanned aircraft system operation safety.
3.5.
Certification result
The above-presented examples are
just one aspect of unmanned aircraft system testing, which basically covers
every area of its manufacture and operation. Hence, ultimately, the
certification result of the entire product can be positive, positive with
limitations, and negative. The certification objective is to increase safety;
therefore, a defective product cannot be put into service as it poses a threat.
However, in such a case, the applicant receives feedback and can improve the
unmanned aircraft system and proceed to the certification process again.
4.
CONCLUSION
Certification is a time consuming, but necessary
process that involves numerous challenges. However, it is necessary as it
guarantees an increase in safety due to the unmanned aircraft system quality.
Obtaining the certificate increases confidence in the technology and the
manufacturer, which can contribute to his/her revenues. Although the
standardisation requirements on the unmanned aircraft system airworthiness are
still being developed, following the principles adopted in manned aviation can
support manufacturers in the construction of safe unmanned aircraft systems.
References
1.
Civil Aviation Authority. 2019. Biuletyn bezpieczeństwa w lotnictwie
cywilnym 3(9) [In Polish: Safety bulletin in
civil aviation 3(9)].
2.
Civil Aviation Authority. 2019. Conference: Drony –
Prawo, Technologia, Usługi [In Polish:
Drones – Law, Technology, Services]. Katowice, 29
August 2019.
3.
EASA. 2017. Annual safety review 2017.
4.
European Defence
Agency. 2017. EMAD 1 - acronyms and
definitions document.
5.
Ministry of
Infrastructure and Construction. 2017. Regulation
of the Minister of Infrastructure and Construction of 7 July 2017 on the certification
of activities in civil aviation.
6.
MON. 2016. Decision No. 311/Mon Mnister of
National Defence of 16 November 2016 regarding the introduction in the Ministry
of National Defence of the “Instructions on how to deal with
standardisation documents of the North Atlantic Treaty Organisation”.
7.
NATO. 2007. Draft STANAG 4671. Unmanned
aircraft systems airworthiness requirements (USAR). NATO Standardization Office.
8.
NATO. Draft STANAG 4738. Unmanned Aerial Vehicle
(UAV) Systems Airworthiness Requirements (USAR) for Light Vertical Take-Off and
Landing (VTOL) Aircraft. NATO
Standardization Office.
9.
NATO. 2016. STANAG 4702. Rotary wing
unmanned aircraft systems airworthiness requirements. NATO Standardization Office.
10. NATO. 2016. STANAG
4703. Light
unmanned aircraft systems airworthiness requirements. NATO Standardization Office.
11. NATO. 2019. STANAG
4671. Unmanned
aircraft systems airworthiness requirements (USAR), NATO Standardization Office.
12. Simba S., W.
Niemann, T. Kotzé, A. Agigi. 2018. “Supply chain risk management
processes for resilience: a study of South African grocery
manufacturers”. Journal of
Transport and Supply Chain Management 11 (A325): 1-13. DOI:
https://doi.org/10.4102/jtscm.v11i0.325.
13. Wild G., J. Murray, G.
Baxter. 2016. Exploring Civil Drone Accidents and Incidents to Help Prevent
Potential Air Disasters. Aerospace.
14. Williams Kevin. 2004. A
Summary of Unmanned Aircraft Accident/Incident Data: Human Factors Implications.
15. Wing Aviation PTY LTD. Available at: https://wing.com.
Received 11.03.2020; accepted in revised form 12.06.2020
Scientific
Journal of Silesian University of Technology. Series Transport is licensed
under a Creative Commons Attribution 4.0 International License