Analysing in-flight emergencies using big data

Corresponding publication

X. Olive, A. Tanner, M. Strohmeier, M. Schäfer, M. Feridun, A. Tart, I. Martinovic and V. Lenders.
OpenSky Report 2020: Analysing in-flight emergencies using big data.
(paper)
Proceedings of the 39th Digital Avionics Systems Conference (DASC), 2020

The dataset presented in the paper is available as a direct import in the library.

from traffic.data.datasets import squawk7700

squawk7700

Traffic

with 832 identifiers

	count
flight_id
ASR172B_20190715	22825
UAL275_20191110	19812
UAE15_20190208	18220
UAL1827_20190903	18150
AAL2807_20180428	17987
AAL343_20191119	17319
AAL163_20190317	16589
UAL102_20180919	16438
UAL1566_20181026	16125
PIA709_20200124	16060

Metadata

Associated metadata is merged into the Traffic structure, but also available as an attribute. The table includes information about:

flight information: callsign, number (IATA flight number), origin , destination (where the aircraft intended to land), landing (where the aircraft actually landed, if available), diverted (where the aircraft actually landed, if applicable);
aircraft information: icao24 transponder identifier, registration (the tail number) and typecode;
information about the nature of the emergency, from Twitter and The Aviation Herald.

squawk7700.metadata.iloc[:10, :10]  # just a preview to fit this page

	flight_id	callsign	number	icao24	registration	typecode	origin	landing	destination	diverted
0	ARG1511_20180101	ARG1511	AR1511	e06442	LV-FQB	B738	SACO	SABE	SABE	NaN
1	DAL14_20180101	DAL14	DL14	a14c29	N183DN	B763	KATL	NaN	EDDF	NaN
2	JBU263_20180108	JBU263	B6263	aa600a	N768JB	A320	KJFK	NaN	KSEA	NaN
3	DAL65_20180108	DAL65	DL65	ab2855	N818NW	A333	KATL	KLAX	KLAX	NaN
4	EDW24_20180111	EDW24	WK24	4b1901	HB-JMF	A343	LSZH	LSZH	MMUN	LSZH
5	ASA111_20180112	ASA111	AS111	a602ab	N487AS	B739	KLAS	NaN	PANC	NaN
6	ONE8511_20180113	ONE8511	O68511	e4919d	PR-OCX	A332	KMIA	NaN	SBGR	NaN
7	BAW882_20180115	BAW882	BA882	406532	G-EUYM	A320	EGLL	NaN	UKBB	EDDT
8	ENY3342_20180117	ENY3342	AA3342	a272b8	N257NN	E75L	KDFW	NaN	KLIT	NaN
9	WJA506_20180122	WJA506	WS506	c0499b	C-GBWS	B736	CYXE	NaN	CYYZ	NaN

squawk7700.metadata.iloc[:10, 10:]  # just a preview to fit this page

	tweet_problem	tweet_result	tweet_fueldump	avh_id	avh_problem	avh_result	avh_fueldump
0	NaN	NaN	NaN	NaN	NaN	NaN	NaN
1	NaN	NaN	NaN	NaN	NaN	NaN	NaN
2	NaN	NaN	NaN	NaN	NaN	NaN	NaN
3	NaN	NaN	NaN	NaN	NaN	NaN	NaN
4	engine	return	unknown	4b382175	engine	return	unknown
5	NaN	NaN	NaN	NaN	NaN	NaN	NaN
6	NaN	NaN	NaN	NaN	NaN	NaN	NaN
7	medical	diverted	unknown	NaN	NaN	NaN	NaN
8	NaN	NaN	NaN	NaN	NaN	NaN	NaN
9	NaN	NaN	NaN	NaN	NaN	NaN	NaN

Data exploration

Simple queries provide subsets of the trajectories:

diverted aircraft: diverted == diverted selects flights where diverted is not empty (NaN);
returning aircraft, when the diversion airport is the origin airport

squawk7700.query("diverted == diverted") | squawk7700.query("diverted == origin")

Traffic

with 295 identifiers

	count
flight_id
UAE15_20190208	18220
AAL2807_20180428	17987
BAW269_20181106	15145
ROU1901_20190809	15051
ANA232_20180622	14712
SWA3708_20191027	13757
AFR1145_20190820	13566
ETD87W_20180324	13474
EDW24_20180111	12272
UAL41_20180730	11532

Traffic

with 111 identifiers

	count
flight_id
BAW269_20181106	15145
EDW24_20180111	12272
DLH501_20180626	10889
NAX7075_20180606	10119
FDX5392_20190221	9215
CLX4327_20191028	7995
AFR090_20180302	7801
AFR1196_20180303	7712
BGA114D_20190311	7563
AFR724_20190201	6976

For example, we can pick the following emergency situation:

squawk7700["AFR1196_20180303"]

Flight AFR1196_20180303

callsign: AFR1196 (LFPG to GMMN diverted to LFPG)
aircraft: 392af2 · 🇫🇷 F-GKXS (A320)
start: 2018-03-03 15:22:07+00:00
stop: 2018-03-03 17:32:24+00:00
duration: 2:10:17
sampling rate: 1 second(s)

The highlight keyword helps identifying parts of the trajectory where the 7700 squawk code was activated.

from ipywidgets import Layout

squawk7700["AFR1196_20180303"].map_leaflet(
    zoom=7,
    highlight=dict(red=lambda f: f.emergency()),
    layout=Layout(height="500px", max_width="800px"),
)

Information about the nature of the emergencies have been collected from two sources of information: Twitter and The Aviation Herald. The following categories have been created:

nan means no information was found;
unclear means that we found an entry about the flight, but that the reason remains unknown;
misc means that the explanation does not fit any category.

tweet_issues = set(squawk7700.metadata.tweet_problem)
avh_issues = set(squawk7700.metadata.avh_problem)
tweet_issues | avh_issues

{'air_condition',
 'bird',
 'bomb_threat',
 'brakes',
 'cabin_pressure',
 'cracked_windshield',
 'door',
 'engine',
 'flaps',
 'fuel_leak',
 'heating',
 'hot_air_leak',
 'hydraulics',
 'instrument',
 'landing_gear',
 'maintenance',
 'medical',
 'misc',
 nan,
 'operational_issue',
 'slats',
 'smoke_burn_smell_flames',
 'technical',
 'tyre',
 'unclear',
 'weather_damage'}

Cabin depressurisation

Since the metadata has been merged into the Traffic structure, we can select flights meeting certain requirements:

we found 31 flights related to cabin pressure or cracked windshields;
among them, 27 flights were diverted

pressure_pbs = ["cabin_pressure", "cracked_windshield"]
pressure = squawk7700.query(
    f"tweet_problem in {pressure_pbs} or avh_problem in {pressure_pbs}"
)
pressure | pressure.query("diverted == diverted")

Traffic

with 31 identifiers

	count
flight_id
ANA232_20180622	14712
TJK646_20180701	11350
AUA463_20190714	8297
UAL135_20180520	6485
LLX5963_20180526	6310
ROU1522_20190909	5396
SVA768_20180427	5224
JZR306_20190626	4823
GMI32FG_20180504	4591
EZY64NL_20190904	4588

Traffic

with 27 identifiers

	count
flight_id
ANA232_20180622	14712
TJK646_20180701	11350
UAL135_20180520	6485
LLX5963_20180526	6310
ROU1522_20190909	5396
SVA768_20180427	5224
JZR306_20190626	4823
GMI32FG_20180504	4591
EZY64NL_20190904	4588
RPA4599_20190719	4556

These flights are usually characterised by a rapid descent to around 10,000ft.

squawk7700["RPA4599_20190719"]

Flight RPA4599_20190719

callsign: RPA4599 (KPHL to KSAV diverted to KPHL)
aircraft: a00929 · 🇺🇸 N101HQ (E75L)
start: 2019-07-19 19:38:28+00:00
stop: 2019-07-19 20:56:16+00:00
duration: 1:17:48
sampling rate: 1 second(s)

import altair as alt

base = (
    squawk7700["RPA4599_20190719"]
    .chart()
    .encode(
        alt.X(
            "utchoursminutesseconds(timestamp)",
            axis=alt.Axis(title=None, format="%H:%M"),
        ),
        alt.Y(
            "altitude",
            title="altitude (in ft)",
            axis=alt.Axis(titleAngle=0, titleY=-15, titleAnchor="start"),
        ),
    )
)

chart = (
    alt.layer(
        base,
        base.transform_filter("datum.squawk == 7700").mark_line(color="#f58518"),
    )
    .properties(height=250)
    .configure_axis(labelFontSize=12, titleFontSize=13)
)
chart

Dumping fuel to reduce landing weight

Emergencies are sometimes associated to dumping fuel in order to reduce landing weight:

tweet_fueldump = set(squawk7700.metadata.tweet_fueldump)
avh_fueldump = set(squawk7700.metadata.avh_fueldump)
tweet_fueldump | avh_fueldump

{'fueldump', 'hold_to_reduce', nan, 'unknown'}

fuel = ["fueldump", "hold_to_reduce"]
squawk7700.query(f"tweet_fueldump in {fuel} or avh_fueldump in {fuel}")

Traffic

with 32 identifiers

	count
flight_id
BAW269_20181106	15145
TJK646_20180701	11350
UAL986_20191215	9222
BAW57K_20180319	8623
DAL445_20180303	7871
BGA114D_20190311	7563
AAL1897_20180603	7484
RYR77FJ_20190807	7479
VIR651Y_20181109	6879
UAL135_20180520	6485

squawk7700["AFL2175_20190723"] | squawk7700["BAW119_20190703"]

Flight AFL2175_20190723

callsign: AFL2175 (ENGM to UUEE diverted to ENGM)
aircraft: 42434f · 🇧🇲 VP-BFF (A321)
start: 2019-07-23 11:37:31+00:00
stop: 2019-07-23 12:23:44+00:00
duration: 0:46:13
sampling rate: 1 second(s)

Flight BAW119_20190703

callsign: BAW119 (EGLL to VOBL diverted to EGLL)
aircraft: 405bfe · 🇬🇧 G-YMMU (B772)
start: 2019-07-03 14:27:29+00:00
stop: 2019-07-03 15:39:33+00:00
duration: 1:12:04
sampling rate: 1 second(s)

Landing attempts

Also, sometimes emergency situations are associated to several landing attempts, at the same or at different airports. aligned_on_ils() and landing_attempts() are two methods available to detect these events:

squawk7700["AFR1145_20190820"].last("45 min")

Flight AFR1145_20190820

callsign: AFR1145 (UUEE to LFPG diverted to ELLX)
aircraft: 3944e1 · 🇫🇷 F-GRHB (A319)
start: 2019-08-20 09:04:02+00:00
stop: 2019-08-20 09:49:01+00:00
duration: 0:44:59
sampling rate: 1 second(s)

squawk7700["AFR1145_20190820"].landing_attempts()

FlightIterator

Flight AFR1145_20190820

callsign: AFR1145 (UUEE to LFPG diverted to ELLX)
aircraft: 3944e1 · 🇫🇷 F-GRHB (A319)
start: 2019-08-20 09:34:02+00:00
stop: 2019-08-20 09:37:07+00:00
duration: 0:03:05
sampling rate: 1 second(s)

Flight AFR1145_20190820

callsign: AFR1145 (UUEE to LFPG diverted to ELLX)
aircraft: 3944e1 · 🇫🇷 F-GRHB (A319)
start: 2019-08-20 09:44:53+00:00
stop: 2019-08-20 09:48:26+00:00
duration: 0:03:33
sampling rate: 1 second(s)

squawk7700["AFR1145_20190820"].map_leaflet(
    zoom=9,
    airport="ELLX",
    highlight=dict(red=lambda f: f.landing_attempts()),
)

Explanation about this particular situation is available:

Incident: France A319 near Luxembourg on Aug 20th 2019, hot brakes indication

squawk7700.metadata.query('flight_id == "AFR1145_20190820"').iloc[0].to_dict()

{'flight_id': 'AFR1145_20190820',
 'callsign': 'AFR1145',
 'number': 'AF1145',
 'icao24': '3944e1',
 'registration': 'F-GRHB',
 'typecode': 'A319',
 'origin': 'UUEE',
 'landing': 'ELLX',
 'destination': 'LFPG',
 'diverted': 'ELLX',
 'tweet_problem': 'unclear',
 'tweet_result': 'diverted',
 'tweet_fueldump': 'unknown',
 'avh_id': '4cbcbfb7',
 'avh_problem': 'brakes',
 'avh_result': 'diverted',
 'avh_fueldump': 'unknown'}