On this page · 9 sections
"Have you tried turning it off and on again?" is the oldest advice in tech support. In 2015, the U.S. Federal Aviation Administration (FAA) made a version of it a legal requirement for one of the world's most advanced jetliners. The reason was a software counter inside the Boeing 787 that could run out of room.
Here's what went wrong, what airlines had to do, and why counters like this break in the first place.
What the FAA found
On May 1, 2015, the FAA published Airworthiness Directive (AD) 2015-09-07 in the Federal Register. An AD is a legally binding order to fix a safety problem. This one covered every Boeing Model 787.
The FAA described the problem in one sentence: "a Model 787 airplane that has been powered continuously for 248 days can lose all alternating current (AC) electrical power due to the generator control units (GCUs) simultaneously going into failsafe mode."
The cause, the FAA said, was "a software counter internal to the GCUs that will overflow after 248 days of continuous power." GCUs manage the plane's electrical generators.
Here's why that's serious. The 787 has four main GCUs, one for each engine-mounted generator. If all four were powered up at the same moment, they would all hit 248 days at the same moment too. Then, the FAA said, "all four GCUs will go into failsafe mode at the same time, resulting in a loss of all AC electrical power regardless of flight phase." The FAA issued the AD "to prevent loss of all AC electrical power, which could result in loss of control of the airplane."
Found in a lab, not in the air
The FAA said Boeing told it about "an issue identified during laboratory testing." Neither FAA order we read reports any 787 losing power because of this counter. This was a problem caught on the ground, with a fix ordered before it could cause one.
What airlines had to do
The 2015 order was simple: power the plane down on a schedule. Its main requirement says operators must "accomplish electrical power deactivation on the airplane," following steps from a Boeing message to operators, and then "repeat the electrical power deactivation thereafter at intervals not to exceed 120 days." The main and backup (APU) batteries didn't need to be disconnected.
The first shutdown was due at the latest of these times:
- Within 120 days after the plane's last power-down done under Boeing's instructions
- Within 120 days after the plane's original certificate of airworthiness (or export certificate) was issued
- Within 7 days after the AD took effect on May 1, 2015
An estimated 28 U.S.-registered planes were affected. Each power-down was estimated at 1 work-hour, or $85 per plane, for a total of $2,380 per cycle for U.S. operators.
The FAA called this "interim action." Meanwhile, Boeing was developing a GCU software upgrade.
The permanent fix
That upgrade arrived in a second order. AD 2018-20-15, published October 16, 2018, and effective November 20, 2018, replaced the 2015 order. It covers 787-8 and 787-9 airplanes listed in a Boeing service bulletin.
It kept the 120-day power-down "until the software installation required by paragraph (h) of this AD is done." Paragraph (h) gave operators 12 months to "install new operational program software (OPS), or later-approved version, into each of the six GCUs" and check it. This order was estimated to cover 55 U.S. planes, with the software install estimated at 5 work-hours, or $425 per plane.
Why counters overflow
Explainer: when a number runs out of room
Computers often keep time by counting up: one tick, two ticks, three ticks. Each count is stored in a space with a fixed number of digits.
Think of an old car odometer with six wheels. It can show 999,999 miles, but one more mile and it rolls over to 000,000. A computer counter works the same way. When it fills up, it wraps around to zero, or to a negative number, and software that wasn't built for that can get confused.
A restart resets the counter to zero, so the clock starts again from the beginning.
The FAA order doesn't say how big the 787 counter was or how fast it counted. Our read: 248 days lines up with a familiar computer number. A counter that ticks 100 times a second and tops out at 2,147,483,647 (the largest number a common 32-bit signed counter can hold) fills up after just over 248 days (about 248.55). That's our math, not the FAA's.
Your PC once had the same kind of bug
Windows had a famous version of this. A Windows-based computer could stop responding after 49.7 days of continuous operation. The cause could be a timing problem in a system file, Vtdapi.vxd. The bug affected Windows 95 and Windows 98, and Microsoft released a fix.
Our read: 49.7 days is how long it takes a counter that ticks once every millisecond to fill a 32-bit space. That's 4,294,967,296 milliseconds, or about 49.71 days. Again, that's our math. Microsoft's support article on the bug doesn't spell it out.
Why it matters: what it means for you
Long uptime can break things. Few people tested what happened after 248 days of nonstop power, because most systems get switched off long before that. Software bugs that only show up after a long time are hard to catch.
Restarting is a real fix, not a superstition. The FAA's interim answer to a jetliner bug was a scheduled power-down. Your phone, laptop, router and smart TV keep their own counters and memory, and a restart clears out a lot of built-up state.
Updates are the permanent fix. The FAA's final answer was new software. The same goes for your devices.
Cheat codes: what to do with your own gear
These are our general tips for everyday devices, not anything from the FAA.
- Restart your phone and computer now and then. Weekly is a good habit, and right away if something starts acting strange.
- Reboot your router. It's often on for months at a time. Unplug it for about 30 seconds, then plug it back in.
- Install updates. A restart is a stopgap. Updates fix the underlying bug.
- Let updates finish restarting. If your computer asks to restart to finish an update, let it. The fix may not take effect until it does.