Requirements Kill
Apr. 11th, 2007 12:10 am![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
selenite0Aerospace and software engineering have many tales of huge projects that ate incredible amounts of resources and then failed, sometimes completely, sometimes producing something with a fraction of the capability originally desired. There's a ton of case studies on them, sometimes whole books of them. These get blamed on forcing the technology beyond achievable limits, requirements creep, poor management, lack of resources, etc. I think all of those come down to one problem—too many requirements.
The requirements I'm talking about here are:
Even if all the conflicts are carefully ironed out of the top-level requirements the project's chance of success will depend on the number of requirements:
Every requirement added reduces the chance of success.
Everone accepts that you can't get everything you want. Lots of engineers have a sign saying "Cheap, Fast, Good: Pick Two" in their cubicles. But even if you "pick two" you haven't capped the number of requirements for your system.
Technical performance—"good"—can go in many directions. The more different tasks the system has to do the less the designers can focus on any one of them. In practice each task gets assigned to a different design team, with the project management and/or systems engineers trying to make sure they don't interfere with each other's performance.
Let's look at an example: the often longed-for flying car. Even if you don't set any limits on the cost or schedule, there's going to be a lot of requirements describing what you want it to do: Speed X in the air. Speed Y on the ground. Road handling characteristics. What kind of fuel it takes. Cargo capacity. How many passengers and how much room they get. Essentially you'll be adding the requirements for a Cessna 150 to those of a Mazda Miata, with only a small percentage being eliminated in the overlap.
Doubling the number of requirements doesn't just double the complexity of your design. When you design a part to meet a particular requirements, you have to look at its effect on every other requirement on the system. The number of interactions increases roughly with the square of the total number of requirements [I = (x^2 - x)/2 ].
In the flying car example, look at what the air bag designer has to deal with. For a car it has to meet NHTSA safety requirements, interface with the car's electrical system, and not interfere with the steering wheel's function (various other requirements such as acceleration and range aren't affected but still have to be checked to make sure). Adding the requirements for a flying forces changes to the design to comply with FAA regulations, keep the airbag from going off during a rough landing, and make it lower weight to support flying performance.
Some requirements don't directly conflict but push the design in different directions. Take the flying car's wheels. Improving the traction and responsiveness on roads requires wider tires and heavier brakes and suspension. For better performance in the air the tires need to be narrow (to reduce drag) and all components must be low weight. In situations like this the performance allocations to the subsystems must be made carefully to ensure both requirements can be met.
Since engineers tend to advocate for their favorite requirements, allocations and flow-down to lower-level requirements can be contentious. The more top level requirements pull in different directions, the harder it is to find an acceptable compromise. The compromise may be vague wording that postpones the conflict to the next level of detail.
With conflicting agendas among the different development teams, each protecting their pet requirements, engineers have to take time away from their own tasks to check up on other teams. It's easy for someone to make a change to his design that impacts another team's performance. Avoiding that requires studying the whole project's requirements, annoying supervisors who want their tasks done on time. Frequently Team B won't realize they've been affected by Team A's change until after it's been approved and incorporated into the baseline. This can produce thrashing—A's change forces B to redesign their subsystem, causing a problem for C. If C's proposed fix affects A's subsystem the cycle will repeat until the system-level management recognizes the problem and imposes a common solution (or the system is delivered with flaws intact). This can be averted by each team reviewing proposed changes before they're approved—as required by every project's CM process—but analyzing piles of change requests is always a low priority task, discarded when the team is behind schedule.
This illustrates what the most precious resource of a project is:
Concentrating attention is how the Apollo Program succeeded at its incredibly difficult task. "Man, Moon, decade" was a requirements set that everyone could grasp and apply to the problem in front of them. With a common goal and priorities engineers could trust each other and focus on their own jobs. Apollo proves even a very large project can be at the top of the success curve.
Projects at the bottom of the curve fail in multiple directions. Time spent arguing priorities or struggling with new processes causes schedule delays. The salary hours used add to the cost overruns. Thrashing from repeated changes drives up both. Technical performance suffers because no part of the design gets the attention needed to optimize total system performance. Worse, a conflict between requirements or subsystem designs could be missed, producing a product that doesn't meet the original need. Actual flying cars have been built but suffer from these requirement conflicts. Their performance is inferior to cars and light planes while costing more than the combined cost of both. Flying car makers have repeatedly gone bankrupt while nearly all Cessna owners also have a car.
When managers realize their projects are in trouble they start climbing the success curve. Cost and schedule requirements fall off the list once they've been passed but the design choices that gave up performance for low cost or quick manufacturing are still part of the design. Process requirements are frequently removed informally by paying lip service while bypassing the process steps. Deleting a performance requirement can be essential for getting back on track but can threaten the viability of the project. Removing a feature drives away the customers who wanted that capability. Government projects have their funding cut as stakeholders lose interest. If that doesn't leave enough money to meet the remaining requirements the project can go into a "death spiral".
A system can be built with a huge number of requirements if you go about it the right way—get a basic version working and only then add the advanced features. The C-130 Hercules cargo plane has been expanded to land on skis, fly into hurricanes, refuel other planes, infiltrate hostile territory, and be a platform for radio stations and artillery pieces. Developing all of those capabilities at once would have pulled the engineers in so many different directions they may not have delivered a working system at all.
In the software world this is called Spiral Development. The define requirements-design-build-test process is repeated several times, adding more requirements on each loop. The cycles continue until all requirements are implemented or the project runs out of time or money (This works in other industries. The author used it on an aerospace project.) Introducing a new development process should be done as a "zero" iteration, practicing the new methods on a "build one to throw away" prototype.
Not every project can be a spiral development. Some requirement sets can't be split into useful packages. More resistance comes from corporate (and government) cultures which believe a "Great Leap Forward" is faster and cheaper. They can still succeed if they don't let their requirements list get too long. Deciding on spiral vs. waterfall development is the job of project managers and chief engineers but all developers can push back on a new requirement. The success curve can be a useful tool for making it clear that someone's "tiny little new feature" can have an out of proportion effect on the whole project.
Too many requirements can kill a project.
Kill them first.
The requirements I'm talking about here are:
1. Top level. They describe what the system must do, not just a part of it. A complicated vehicle can have many, many derived requirements allocating the top level ones to each subsystem, but the derived ones aren't part of this discussion.The more requirements there are on a system the more likely it is that two or more of them will directly conflict. Ideally such conflicts will be recognized before the project gets go-ahead, but there are projects that never resolve those conflicts and instead will thrash about trying to square the circle without realizing (or admitting) what the problem is. But fixing them isn't enough to assure success.
2. Top priority. Any requirement that can be sacrificed to meet another one isn't really a requirement, it's a goal, something optional. Of course, how the requirement is designated officially may not match that definition. If a "goal" has to be met or the project loses funding, then it's really a requirement and equal to all the other "top" priority ones.
3. Not just technical. Any constraints on the schedule or budget for a project count as requirements. If the project has to stay under $1.5M/yr, deliver the production version for $3000 each, and be shipping by 6/1/2009, that's three more requirements. Other constraints count too—if a specific development methodology or tool set must be used, that's a requirement. So is a security regime imposed on the project staff (classified, ITAR, proprietary—they all complicate things).
4. Not just formal. The informal requirements on a project can be the most difficult ones. US government projects are infamous for these . . . despite thousands of pages of requirement studies, the design can be driven by an "understanding" that work will be done in a particular Congressional district, or the employment level at a NASA facility will not be cut.
Even if all the conflicts are carefully ironed out of the top-level requirements the project's chance of success will depend on the number of requirements:
Every requirement added reduces the chance of success.
Everone accepts that you can't get everything you want. Lots of engineers have a sign saying "Cheap, Fast, Good: Pick Two" in their cubicles. But even if you "pick two" you haven't capped the number of requirements for your system.
Technical performance—"good"—can go in many directions. The more different tasks the system has to do the less the designers can focus on any one of them. In practice each task gets assigned to a different design team, with the project management and/or systems engineers trying to make sure they don't interfere with each other's performance.
Let's look at an example: the often longed-for flying car. Even if you don't set any limits on the cost or schedule, there's going to be a lot of requirements describing what you want it to do: Speed X in the air. Speed Y on the ground. Road handling characteristics. What kind of fuel it takes. Cargo capacity. How many passengers and how much room they get. Essentially you'll be adding the requirements for a Cessna 150 to those of a Mazda Miata, with only a small percentage being eliminated in the overlap.
Doubling the number of requirements doesn't just double the complexity of your design. When you design a part to meet a particular requirements, you have to look at its effect on every other requirement on the system. The number of interactions increases roughly with the square of the total number of requirements [I = (x^2 - x)/2 ].
In the flying car example, look at what the air bag designer has to deal with. For a car it has to meet NHTSA safety requirements, interface with the car's electrical system, and not interfere with the steering wheel's function (various other requirements such as acceleration and range aren't affected but still have to be checked to make sure). Adding the requirements for a flying forces changes to the design to comply with FAA regulations, keep the airbag from going off during a rough landing, and make it lower weight to support flying performance.
Some requirements don't directly conflict but push the design in different directions. Take the flying car's wheels. Improving the traction and responsiveness on roads requires wider tires and heavier brakes and suspension. For better performance in the air the tires need to be narrow (to reduce drag) and all components must be low weight. In situations like this the performance allocations to the subsystems must be made carefully to ensure both requirements can be met.
Since engineers tend to advocate for their favorite requirements, allocations and flow-down to lower-level requirements can be contentious. The more top level requirements pull in different directions, the harder it is to find an acceptable compromise. The compromise may be vague wording that postpones the conflict to the next level of detail.
With conflicting agendas among the different development teams, each protecting their pet requirements, engineers have to take time away from their own tasks to check up on other teams. It's easy for someone to make a change to his design that impacts another team's performance. Avoiding that requires studying the whole project's requirements, annoying supervisors who want their tasks done on time. Frequently Team B won't realize they've been affected by Team A's change until after it's been approved and incorporated into the baseline. This can produce thrashing—A's change forces B to redesign their subsystem, causing a problem for C. If C's proposed fix affects A's subsystem the cycle will repeat until the system-level management recognizes the problem and imposes a common solution (or the system is delivered with flaws intact). This can be averted by each team reviewing proposed changes before they're approved—as required by every project's CM process—but analyzing piles of change requests is always a low priority task, discarded when the team is behind schedule.
This illustrates what the most precious resource of a project is:
The attention of the workersIf the engineers try to pay attention to too many requirements, they won't be able to focus on creating their designs. So they pick the ones that matter the most to them and ignore the rest. This is where process and security requirements do their damage. Time spent on learning new change forms, adding required markings to drawings, or evaluating data to see if it's classified or ITAR-restricted takes away from designing. Hiring more people can help with the technical workload but adding them for process/security enforcement just adds interruptions without reducing anyone's workload.
Concentrating attention is how the Apollo Program succeeded at its incredibly difficult task. "Man, Moon, decade" was a requirements set that everyone could grasp and apply to the problem in front of them. With a common goal and priorities engineers could trust each other and focus on their own jobs. Apollo proves even a very large project can be at the top of the success curve.
Projects at the bottom of the curve fail in multiple directions. Time spent arguing priorities or struggling with new processes causes schedule delays. The salary hours used add to the cost overruns. Thrashing from repeated changes drives up both. Technical performance suffers because no part of the design gets the attention needed to optimize total system performance. Worse, a conflict between requirements or subsystem designs could be missed, producing a product that doesn't meet the original need. Actual flying cars have been built but suffer from these requirement conflicts. Their performance is inferior to cars and light planes while costing more than the combined cost of both. Flying car makers have repeatedly gone bankrupt while nearly all Cessna owners also have a car.
When managers realize their projects are in trouble they start climbing the success curve. Cost and schedule requirements fall off the list once they've been passed but the design choices that gave up performance for low cost or quick manufacturing are still part of the design. Process requirements are frequently removed informally by paying lip service while bypassing the process steps. Deleting a performance requirement can be essential for getting back on track but can threaten the viability of the project. Removing a feature drives away the customers who wanted that capability. Government projects have their funding cut as stakeholders lose interest. If that doesn't leave enough money to meet the remaining requirements the project can go into a "death spiral".
A system can be built with a huge number of requirements if you go about it the right way—get a basic version working and only then add the advanced features. The C-130 Hercules cargo plane has been expanded to land on skis, fly into hurricanes, refuel other planes, infiltrate hostile territory, and be a platform for radio stations and artillery pieces. Developing all of those capabilities at once would have pulled the engineers in so many different directions they may not have delivered a working system at all.
In the software world this is called Spiral Development. The define requirements-design-build-test process is repeated several times, adding more requirements on each loop. The cycles continue until all requirements are implemented or the project runs out of time or money (This works in other industries. The author used it on an aerospace project.) Introducing a new development process should be done as a "zero" iteration, practicing the new methods on a "build one to throw away" prototype.
Not every project can be a spiral development. Some requirement sets can't be split into useful packages. More resistance comes from corporate (and government) cultures which believe a "Great Leap Forward" is faster and cheaper. They can still succeed if they don't let their requirements list get too long. Deciding on spiral vs. waterfall development is the job of project managers and chief engineers but all developers can push back on a new requirement. The success curve can be a useful tool for making it clear that someone's "tiny little new feature" can have an out of proportion effect on the whole project.
Too many requirements can kill a project.
Kill them first.
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2007-04-11 05:38 am (UTC)-goes off to think
no subject
Date: 2007-04-11 06:33 am (UTC)Ack!!!!
**Runs screaming out into the night**
no subject
Date: 2007-04-11 06:42 am (UTC)On the other end is the Shuttle, which is almost a textbook case of spiral development in which they put so many requirements into it that the end result was too complex for the intended job.
no subject
Date: 2007-04-11 02:38 pm (UTC)no subject
Date: 2007-04-11 06:49 pm (UTC)Very special case. Of course the fun is about to continue. More manned vehicles with solid rocket motors coming up...
no subject
Date: 2007-04-11 08:08 pm (UTC)Then again, if you were an exceptionally clever USAF budget writer, perhaps you'd leave out all the requests that could be pork from your proposal and rely on Congress to add them in later. (Thus keeping the initial proposed budget cost down.)
no subject
Date: 2007-04-11 08:30 pm (UTC)no subject
Date: 2007-04-15 10:29 pm (UTC)no subject
Date: 2007-04-11 07:20 am (UTC)Speaking of flying cars, I have a question I'm not mathematically equipped to handle: what if I want to build a flying car that hops instead of flies? In other words, imagine a "flying car" that does not use aerodynamic forces to remain aloft during travel from A to B, but rather travels ballistically — it boosts up for X number of seconds, cuts jets, coasts upward in a parabolic arc, then "falls" towards the destination using its body shape to create lift to control the fall. The "personal ballistic transport vehicle" (PBTV), or "hopper" I have in mind would be a lifting body shape with room for eight — a combination minivan and NASA HL-10. Passengers would get in, strap down, input the desired destination, and hold on. The hopper would then boost upwards using jet or rocket power, coast along a parabolic arc, descend in free fall to the desired destination using GPS guidance, fire jets again to slow itself, then soft-land at the target. In case of emergency, a parachute-like inflatable wng would lower craft and crew safely to earth.
My questions: is such a vehicle even possible? If so, what kind of engines would be best suited to it, and how much onboard fuel would be required per trip? Also, how long would the launch burn be, at what altitude would the craft's trajectory top out for a give horizontal distance traveled, and how long would the landing burn be?
Sorry to hit you with complex questions. Please don't feel obligated to answer them. It's just -- well, you know, flying cars.
no subject
Date: 2007-04-11 12:54 pm (UTC)Whether machinery can be designed to handle the strain is immaterial if the purpose (the human machine) cannot be re-designed to match.
no subject
Date: 2007-04-11 02:37 pm (UTC)Sure. You're looking at a DCX or Blue Origins New Shepard. You'd probably want to go with hybrids for the safety advantage with a passenger mission. The rest of your questions come out as a set of equations. For any given horizontal distance you want to cover you've got a range of launch angles, each with a different burn time and max altitude. Landing burn time will be the same as take-off, minus a few seconds for speed lost to drag.
no subject
Date: 2007-04-11 02:49 pm (UTC)no subject
Date: 2007-04-11 06:54 pm (UTC)no subject
Date: 2007-04-11 08:51 pm (UTC)no subject
Date: 2007-04-11 09:20 pm (UTC)Also, don't assume that an exception will be made to the way things are done now for every new vehicle that comes along. We do things the way we do them in airplanes for a reason. Usually we have learned something from seeing how someone else got killed and we plan to avoid that particular mistake.
You want to add a whole new layer of stuff. There will be resistance. For good reason.
And is the 2000m apogee a typo ? That's very flat for a 100km trip. And also a VERY inconvenient altitude for operations in a high population area. Lots of air traffic there and down.
Sorry to rain on your parade...
no subject
Date: 2007-04-11 11:26 pm (UTC)As for those aspects, here are two graphics that I hope will clear up the concept:
no subject
Date: 2007-04-12 12:02 am (UTC)Think last part of the space shuttle approach profile. As in falling like a stone. Also, lifting bodies get efficient only when they supersonic if I recall correctly. Not good over population centers.
For a subsonic glide of 100km, you need roughly 50km of altitude and then enough delta V to accelerate to near mach 1 horizontally. ( assumes a 25 degree glide slope. )
I'd have to call this one busted.
Here's the closest we've come to this so far BTW:
http://en.wikipedia.org/wiki/McDonnell_Douglas_DC-X
It worked fine until NASA took it over. Broke it and killed it in short order. Not invented here syndrome.
Food for thought
Date: 2007-04-11 01:05 pm (UTC)Though with a quibble. Remember that I'm coming from the other direction. ITAR / classification / security et al shouldn't be a roadblock. If viewed as a base requirement, not a goal, but a deal stopper, and built in from the beginning of a project, it should (not saying will, but thinking in theory and perfect worlds here) be dealt with and then released during the process.
When does it become a roadblock? When it's given lip service (goal, not requirement) and shoved into a corner until the end of the process, when suddenly, it's a wall that must be dealt with (hopefully) or evaded (which is what puts companies on black lists. Everybody saw the ITT press release? Yeah.). Treated as a requirement with moderately low input needs, ITAR or EAR compliance is simply part of a successfully functioning process, as opposed to being a separate process that's duct-taped to the very end of a development project.
Re: Food for thought
Date: 2007-04-11 01:24 pm (UTC)My thought is requirements can be ranked. We do use foundation assumptions, absolutely necessities. Atht eh most basic level are the obvious ones: working in base ten maths, the laws of physics, 32 feet per second, that sort of thing. Then there are company imposed non-project-specific requirements, boundaries placed by corporate policy or departmental nonsense or union laws: six sigma processes or Q9001 or whatever they are calling it these days. Next level up would be the project-specific across the board requirements, and that's where I'm thinking stuff like facility security come in. Then project-specific requirements which are, from an engineering standpoint, the important ones, the purpose of the exercise. But they sit on a foundation of all those others.
Instead of a tangled spaghetti mess of overlapping requirements, maybe setting them as a pyramid built of apples? Take away a bottom one (Does anyone but Heinlein work math in base twelve save for the late-night revelers at the Math Department Christmas party?) and all the apples roll away. Let the top level of apples outnumber the base ones and the you have a bushel's worth of apples that roll over the table (requiring a project-specific software program in addition to or in conflict with the company-standard/default operating system). But build the specific requirements on top of the general ones, and you have a stack of apples.
ye gods, I really do relate everything to food, don't I? On a conference call with the DoC yesterday, I resorted to a peanut butter sandwich analogy to make a point. Sigh.
Re: Food for thought
Date: 2007-04-17 02:45 pm (UTC)I think developers can handle any number of requirements as long as they have time to understand and internalize them a few at a time. The laws of physics usually get taught before they're hired. Ideally a company with lots of formal processes would put employees through instead of having them learn on the job. They they could concentrate on the requirements for their work assignment when they started on it. So I think you could have a tall stack of apples instead of a short pyramid . . . as long as you build it slowly.
no subject
Date: 2007-04-11 05:18 pm (UTC)Yeah, baby! LC-130H, AF84-0490! Built with pride by a team that included me!
..ahem...I'm sorry. Got carried away there...
no subject
Date: 2007-04-11 06:55 pm (UTC)Any idea how folks regard the Systems Engineering Masters program at George Mason? I'm thinking of enrolling this year.
no subject
Date: 2007-04-17 02:48 pm (UTC)no subject
Date: 2007-04-11 10:55 pm (UTC)no subject
Date: 2007-04-12 12:43 am (UTC)Too many requirements = everything is top priority: It's impossible to make intelligent design decisions when everything you want is given equal weight. As you say, the list of requirements should be short. The list of goals should be long, and most of what is currently a requirement should probably be re-cast as a goal, and ranked in order or priority. Is there something fundamental about the culture that results in all important needs being stated as requirements?
Communication breakdowns. This is a huge issue in my industry too. Once the problem has been partitioned, the different pieces of the puzzle can easily get out of sync. The only way to avoid this is to invest time in keeping everyone on the same page. The more bits you break the problem into, the worse it gets, and having people in different offices, different companies, etc just makes it worse. Eventually you reach the point where the necessary communications overhead, all by itself, exceeds the time available.
Even if you avoid this fate, it still seems that communication is never given the necessary priority. The problem solving process is fundamentally social (http://www.touchstone.com/tr/wp/wicked.html).
There's another issue which you allude to, but don't actually talk about: morale and motivation. If the engineers view security protocols as pointless (whether or not they are), they'll tend to ignore them. If they are cynical about the chances of the project ever seeing the light of day, they will ignore the change management process. You are absolutely right that the attention of the workers is the most precious resource on a project, but it's not a fixed resource. Motivated workers pay more attention; dismotivated ones pay less. Vision provides motivation. Bureaucracy, political requirements masquerading as engineering requirements, everchanging priorities, and pretty much anything else that gets in the way of getting stuff done saps it. And once the apathy gets a certain toehold in an organization, it grows, by direct transmission (watercooler conversations) and indirect effects (someone else screws up because they're not paying attention, causing me to waste my time).
I'm sure that the simple, clean requirements of Apollo had alot to do with its success. But I would submit that the compelling vision (which was strongly related to the simple requirements) also had alot to do with it. I've worked in organizations on both ends of the spectrum, and I'm absolutely convinced that the most important factor to success and productivity is whether or not your workers think their job matters. And somehow, management doesn't seem to understand this.
no subject
Date: 2007-04-12 09:38 am (UTC)Your writeup is definitely on target though. I saw most, if not all, of your examples played out during my 6 years on the B-2 program -a program that had a fairly significant design change due to the end of the Cold War. The flight requirements changed from a high-altitude bomber to a terrain-following bomber and almost completely re-designed most of the flight control surfaces. Alas, the people that should be reading this and taking it to heart are just the people I believe are incapable of understanding it...
no subject
Date: 2007-04-12 09:47 pm (UTC)MY enemies are in the Middle East, China, N. Korea, and the Congress ( specifically Mme. Speaker's office - among others - of late ).
The White House is on our side, at least in terms of defense.
( Ok - except for the open south border... )
That said, I hear you on the "fun" in this business. And I know exactly what
no subject
Date: 2007-04-12 10:00 pm (UTC)no subject
Date: 2007-04-12 10:06 pm (UTC)Back to rocket science... :-)