9,500 Canceled Flights and Other Fallout Caused by Bad Data

It is not uncommon for technical issues to cause disruptions in transportation systems. The logistics systems that power air traffic, shipping routes, trains, and other forms of mass transportation operate with razor-thin margins for departure, arrival, and transit. With a single disturbance, the ripple effect could be devastating. This week, we all witnessed when the U.S. Federal Aviation Administration (FAA) grounded flights for more than three hours, causing the delay or cancellation of almost 9,500 flights in the United States.

As per the FAA, it appears that a damaged database file in the Notice to Air Missions (NOTAM) system led to an outage which caused the delays and cancellations. The FAA will undoubtedly conduct an investigation to determine the cause of the issue and how to prevent similar problems in the future. But the unfortunate situation illustrates the importance of clean, reliable, accurate data – with it, systems run smoothly and according to plan. If data is bad, slow, or incomplete, the systems and decision-making that relies on it will be adversely affected. When those systems are at the scale of something like mass transportation, the fallout – economically and socially – is massive.

This week’s travel disruption could have been avoided if there had been awareness of the damaged database file before it created a repercussive effect. With awareness, remediation could have been implemented and work-arounds could have been put in place. The only way to truly get that awareness is with clear, actionable data insights, which are clearly required elements for modern data environments. Modern solutions, like Data Observability not only would provide visibility into the quality of data but also alerts of mitigation and remediation before such outages.

(Gorodenkoff/Shutterstock)

For infrastructure like mass transportation, data can be used to optimize the performance of its underlying systems. For example, data on passenger traffic patterns can be used to schedule trains and buses more efficiently, reducing delays and improving overall service. That’s just for starters, but it clearly provides a huge economic benefit. In fact, one could argue that the organization with the best data can deliver the best product.

But can also be used to identify and predict potential problems before they occur. The data being shared and integrated through a variety of data pipelines can include insights into everything from identifying maintenance issues with vehicles and equipment, to identifying potential safety hazards. Identify where there is a threat or issue, and it can be avoided.

Accurate and reliable data is essential to make better decisions on organizational investments, planning, and operations. By having accurate and timely data, transportation agencies can make more informed decisions on where to allocate resources and how to improve their services.

Specifically as it relates to airline traffic and control, data insights, delivered through data observability, can be used now to avoid flight delays and cancellations:

Predictive maintenance: By analyzing data from aircraft systems, airlines can identify potential maintenance issues before they become major problems, which can prevent delays and cancellations caused by equipment failure.
Crew scheduling: Data on flight schedules and crew availability can be used to optimize crew scheduling and reduce the number of delays and cancellations caused by crew shortages.
Weather forecasting: Data on weather patterns can be used to plan routes and schedules that avoid severe weather conditions, reducing the number of delays and cancellations caused by bad weather.
Traffic management: Data on flight traffic patterns can be used to optimize the use of airspace and reduce delays caused by congestion.
Passenger data: Airlines can use data on passenger travel patterns, preferences and booking history to optimize flight schedules and routes, reducing the number of delays and cancellations caused by low passenger demand.
Real-time monitoring: By monitoring the status of flights in real-time, airlines can quickly identify and respond to potential delays and cancellations, minimizing their impact on passengers.
Historical data analysis: By analyzing historical data, airlines can identify patterns and trends that contribute to delays and cancellations, allowing them to take proactive measures to prevent them in the future.

Data observability can be implemented by airlines, regulators, and other mass transit organizations to improve the efficiency and reliability of their operations, which can help to reduce the number of delays and cancellations experienced by passengers.

About the author: Rohit Choudhary is the CEO and co-founder of Acceldata, a San Jose-based startup that has developed an end-to-end data observability cloud to help enterprises observe and optimize modern data systems and maximize return on data investment. Prior to Acceldata, Choudhary served as director of engineering at Hortonworks, where he led development of Dataplane Services, Ambari and Zeppelin among other products. While at Hortonworks, Rohit was inspired to start Acceldata after repeatedly witnessing his customers’ multi-million dollar data initiatives fail despite employing the latest data technologies and experienced teams of data experts.