Analysing in-flight emergencies using big data

https://zenodo.org/badge/DOI/10.5281/zenodo.3937483.svg

Corresponding publication

X. Olive, A. Tanner, M. Strohmeier, M. Schäfer, M. Feridun, A. Tart, I. Martinovic and V. Lenders.
OpenSky Report 2020: Analysing in-flight emergencies using big data. (paper)
Proceedings of the 39th Digital Avionics Systems Conference (DASC), 2020

The dataset presented in the paper is available as a direct import in the library.

from traffic.data.datasets import squawk7700

squawk7700

Traffic

with 832 identifiers
  count
flight_id  
ASR172B_20190715 22825
UAL275_20191110 19812
UAE15_20190208 18220
UAL1827_20190903 18150
AAL2807_20180428 17987
AAL343_20191119 17319
AAL163_20190317 16589
UAL102_20180919 16438
UAL1566_20181026 16125
PIA709_20200124 16060

Metadata

Associated metadata is merged into the Traffic structure, but also available as an attribute. The table includes information about:

  • flight information: callsign, number (IATA flight number), origin , destination (where the aircraft intended to land), landing (where the aircraft actually landed, if available), diverted (where the aircraft actually landed, if applicable);

  • aircraft information: icao24 transponder identifier, registration (the tail number) and typecode;

  • information about the nature of the emergency, from Twitter and The Aviation Herald.

squawk7700.metadata.iloc[:10, :10]  # just a preview to fit this page
flight_id callsign number icao24 registration typecode origin landing destination diverted
0 ARG1511_20180101 ARG1511 AR1511 e06442 LV-FQB B738 SACO SABE SABE NaN
1 DAL14_20180101 DAL14 DL14 a14c29 N183DN B763 KATL NaN EDDF NaN
2 JBU263_20180108 JBU263 B6263 aa600a N768JB A320 KJFK NaN KSEA NaN
3 DAL65_20180108 DAL65 DL65 ab2855 N818NW A333 KATL KLAX KLAX NaN
4 EDW24_20180111 EDW24 WK24 4b1901 HB-JMF A343 LSZH LSZH MMUN LSZH
5 ASA111_20180112 ASA111 AS111 a602ab N487AS B739 KLAS NaN PANC NaN
6 ONE8511_20180113 ONE8511 O68511 e4919d PR-OCX A332 KMIA NaN SBGR NaN
7 BAW882_20180115 BAW882 BA882 406532 G-EUYM A320 EGLL NaN UKBB EDDT
8 ENY3342_20180117 ENY3342 AA3342 a272b8 N257NN E75L KDFW NaN KLIT NaN
9 WJA506_20180122 WJA506 WS506 c0499b C-GBWS B736 CYXE NaN CYYZ NaN 
squawk7700.metadata.iloc[:10, 10:]  # just a preview to fit this page
tweet_problem tweet_result tweet_fueldump avh_id avh_problem avh_result avh_fueldump
0 NaN NaN NaN NaN NaN NaN NaN
1 NaN NaN NaN NaN NaN NaN NaN
2 NaN NaN NaN NaN NaN NaN NaN
3 NaN NaN NaN NaN NaN NaN NaN
4 engine return unknown 4b382175 engine return unknown
5 NaN NaN NaN NaN NaN NaN NaN
6 NaN NaN NaN NaN NaN NaN NaN
7 medical diverted unknown NaN NaN NaN NaN
8 NaN NaN NaN NaN NaN NaN NaN
9 NaN NaN NaN NaN NaN NaN NaN

Data exploration

Simple queries provide subsets of the trajectories:

  • diverted aircraft: diverted == diverted selects flights where diverted is not empty (NaN);

  • returning aircraft, when the diversion airport is the origin airport

squawk7700.query("diverted == diverted") | squawk7700.query("diverted == origin")

Traffic

with 295 identifiers
  count
flight_id  
UAE15_20190208 18220
AAL2807_20180428 17987
BAW269_20181106 15145
ROU1901_20190809 15051
ANA232_20180622 14712
SWA3708_20191027 13757
AFR1145_20190820 13566
ETD87W_20180324 13474
EDW24_20180111 12272
UAL41_20180730 11532

Traffic

with 111 identifiers
  count
flight_id  
BAW269_20181106 15145
EDW24_20180111 12272
DLH501_20180626 10889
NAX7075_20180606 10119
FDX5392_20190221 9215
CLX4327_20191028 7995
AFR090_20180302 7801
AFR1196_20180303 7712
BGA114D_20190311 7563
AFR724_20190201 6976

For example, we can pick the following emergency situation:

squawk7700["AFR1196_20180303"]

Flight AFR1196_20180303

  • callsign: AFR1196 (LFPG to GMMN diverted to LFPG)
  • aircraft: 392af2 · 🇫🇷 F-GKXS (A320)
  • start: 2018-03-03 15:22:07+00:00
  • stop: 2018-03-03 17:32:24+00:00
  • duration: 2:10:17
  • sampling rate: 1 second(s)

The highlight keyword helps identifying parts of the trajectory where the 7700 squawk code was activated.

from ipywidgets import Layout

squawk7700["AFR1196_20180303"].map_leaflet(
    zoom=7,
    highlight=dict(red=lambda f: f.emergency()),
    layout=Layout(height="500px", max_width="800px"),
)

Information about the nature of the emergencies have been collected from two sources of information: Twitter and The Aviation Herald. The following categories have been created:

  • nan means no information was found;

  • unclear means that we found an entry about the flight, but that the reason remains unknown;

  • misc means that the explanation does not fit any category.

tweet_issues = set(squawk7700.metadata.tweet_problem)
avh_issues = set(squawk7700.metadata.avh_problem)
tweet_issues | avh_issues

{'air_condition',
 'bird',
 'bomb_threat',
 'brakes',
 'cabin_pressure',
 'cracked_windshield',
 'door',
 'engine',
 'flaps',
 'fuel_leak',
 'heating',
 'hot_air_leak',
 'hydraulics',
 'instrument',
 'landing_gear',
 'maintenance',
 'medical',
 'misc',
 nan,
 'operational_issue',
 'slats',
 'smoke_burn_smell_flames',
 'technical',
 'tyre',
 'unclear',
 'weather_damage'}

Cabin depressurisation

Since the metadata has been merged into the Traffic structure, we can select flights meeting certain requirements:

  • we found 31 flights related to cabin pressure or cracked windshields;

  • among them, 27 flights were diverted

pressure_pbs = ["cabin_pressure", "cracked_windshield"]
pressure = squawk7700.query(
    f"tweet_problem in {pressure_pbs} or avh_problem in {pressure_pbs}"
)
pressure | pressure.query("diverted == diverted")

Traffic

with 31 identifiers
  count
flight_id  
ANA232_20180622 14712
TJK646_20180701 11350
AUA463_20190714 8297
UAL135_20180520 6485
LLX5963_20180526 6310
ROU1522_20190909 5396
SVA768_20180427 5224
JZR306_20190626 4823
GMI32FG_20180504 4591
EZY64NL_20190904 4588

Traffic

with 27 identifiers
  count
flight_id  
ANA232_20180622 14712
TJK646_20180701 11350
UAL135_20180520 6485
LLX5963_20180526 6310
ROU1522_20190909 5396
SVA768_20180427 5224
JZR306_20190626 4823
GMI32FG_20180504 4591
EZY64NL_20190904 4588
RPA4599_20190719 4556

These flights are usually characterised by a rapid descent to around 10,000ft.

squawk7700["RPA4599_20190719"]

Flight RPA4599_20190719

  • callsign: RPA4599 (KPHL to KSAV diverted to KPHL)
  • aircraft: a00929 · 🇺🇸 N101HQ (E75L)
  • start: 2019-07-19 19:38:28+00:00
  • stop: 2019-07-19 20:56:16+00:00
  • duration: 1:17:48
  • sampling rate: 1 second(s)
import altair as alt

base = (
    squawk7700["RPA4599_20190719"]
    .chart()
    .encode(
        alt.X(
            "utchoursminutesseconds(timestamp)",
            axis=alt.Axis(title=None, format="%H:%M"),
        ),
        alt.Y(
            "altitude",
            title="altitude (in ft)",
            axis=alt.Axis(titleAngle=0, titleY=-15, titleAnchor="start"),
        ),
    )
)

chart = (
    alt.layer(
        base,
        base.transform_filter("datum.squawk == 7700").mark_line(color="#f58518"),
    )
    .properties(height=250)
    .configure_axis(labelFontSize=12, titleFontSize=13)
)
chart

Dumping fuel to reduce landing weight

Emergencies are sometimes associated to dumping fuel in order to reduce landing weight:

tweet_fueldump = set(squawk7700.metadata.tweet_fueldump)
avh_fueldump = set(squawk7700.metadata.avh_fueldump)
tweet_fueldump | avh_fueldump

{'fueldump', 'hold_to_reduce', nan, 'unknown'}

fuel = ["fueldump", "hold_to_reduce"]
squawk7700.query(f"tweet_fueldump in {fuel} or avh_fueldump in {fuel}")

Traffic

with 32 identifiers
  count
flight_id  
BAW269_20181106 15145
TJK646_20180701 11350
UAL986_20191215 9222
BAW57K_20180319 8623
DAL445_20180303 7871
BGA114D_20190311 7563
AAL1897_20180603 7484
RYR77FJ_20190807 7479
VIR651Y_20181109 6879
UAL135_20180520 6485 
squawk7700["AFL2175_20190723"] | squawk7700["BAW119_20190703"]

Flight AFL2175_20190723

  • callsign: AFL2175 (ENGM to UUEE diverted to ENGM)
  • aircraft: 42434f · 🇧🇲 VP-BFF (A321)
  • start: 2019-07-23 11:37:31+00:00
  • stop: 2019-07-23 12:23:44+00:00
  • duration: 0:46:13
  • sampling rate: 1 second(s)

Flight BAW119_20190703

  • callsign: BAW119 (EGLL to VOBL diverted to EGLL)
  • aircraft: 405bfe · 🇬🇧 G-YMMU (B772)
  • start: 2019-07-03 14:27:29+00:00
  • stop: 2019-07-03 15:39:33+00:00
  • duration: 1:12:04
  • sampling rate: 1 second(s)

Landing attempts

Also, sometimes emergency situations are associated to several landing attempts, at the same or at different airports. aligned_on_ils() and landing_attempts() are two methods available to detect these events:

squawk7700["AFR1145_20190820"].last("45 min")

Flight AFR1145_20190820

  • callsign: AFR1145 (UUEE to LFPG diverted to ELLX)
  • aircraft: 3944e1 · 🇫🇷 F-GRHB (A319)
  • start: 2019-08-20 09:04:02+00:00
  • stop: 2019-08-20 09:49:01+00:00
  • duration: 0:44:59
  • sampling rate: 1 second(s)
squawk7700["AFR1145_20190820"].landing_attempts()

FlightIterator

Flight AFR1145_20190820

  • callsign: AFR1145 (UUEE to LFPG diverted to ELLX)
  • aircraft: 3944e1 · 🇫🇷 F-GRHB (A319)
  • start: 2019-08-20 09:34:02+00:00
  • stop: 2019-08-20 09:37:07+00:00
  • duration: 0:03:05
  • sampling rate: 1 second(s)

Flight AFR1145_20190820

  • callsign: AFR1145 (UUEE to LFPG diverted to ELLX)
  • aircraft: 3944e1 · 🇫🇷 F-GRHB (A319)
  • start: 2019-08-20 09:44:53+00:00
  • stop: 2019-08-20 09:48:26+00:00
  • duration: 0:03:33
  • sampling rate: 1 second(s)
squawk7700["AFR1145_20190820"].map_leaflet(
    zoom=9,
    airport="ELLX",
    highlight=dict(red=lambda f: f.landing_attempts()),
)
Explanation about this particular situation is available:
Incident: France A319 near Luxembourg on Aug 20th 2019, hot brakes indication
squawk7700.metadata.query('flight_id == "AFR1145_20190820"').iloc[0].to_dict()

{'flight_id': 'AFR1145_20190820',
 'callsign': 'AFR1145',
 'number': 'AF1145',
 'icao24': '3944e1',
 'registration': 'F-GRHB',
 'typecode': 'A319',
 'origin': 'UUEE',
 'landing': 'ELLX',
 'destination': 'LFPG',
 'diverted': 'ELLX',
 'tweet_problem': 'unclear',
 'tweet_result': 'diverted',
 'tweet_fueldump': 'unknown',
 'avh_id': '4cbcbfb7',
 'avh_problem': 'brakes',
 'avh_result': 'diverted',
 'avh_fueldump': 'unknown'}