"Explosive" spacewalk: preventing the Soyuz separation/reentry problem

[DrRocket]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>It does seem like a long shot to me too.&nbsp; A manufacturing defect seems more likely to me, but that must either have been confined to the last two Soyuz or be too subtle to spot, since IIRC they've been inspecting pyrobolts from the same lot.&nbsp; But I suppose when you're desperate, any clue seems worth looking into.As far as the fleet being grounded, failures like this have not been common.&nbsp; It could be that there is a very rare condition lurking in the design that chance alone caused to happen twice in a row, confounding engineers who are naturally looking for a commonality between Soyuz TMA 10 and Soyuz TMA 11. Look at Columbia; it was taken down by something that had happened on every single flight since the beginning, yet the fleet had never been grounded.&nbsp; That's the nightmare of every spacecraft engineer, of course -- a latent design defect. <br />Posted by CalliArcale</DIV></p><p>Columbia should have&nbsp;been grounded.&nbsp; The rationale presented for flying despite loss of foam -- that the foam was not known to hit anything -- I will charitably call suspect.&nbsp; That the rationale was, unusually, presented by the main tank program manager rather than the chief engineer (as I was told by someone in the room), is more suspect.&nbsp; One ought not accept a condition that is other than what it was designed to be, and anyone with any appreciation for fluid dynamics ought to know that the trajectory of something like a piece of foam breaking off of the main tank in flight will not be be reproducible.</p><p>I have seen launch schedules put in jeopardy for a lot less, and for good reason.&nbsp; I forced replacement of a pyrotechnic component on launcher for the Cassini mission, despite the fact that it was apparently acceptable to the component engineer, and "in spec" but out-of-family.&nbsp; Further inspection determined that it was about to fail.</p><p>And&nbsp;I dont't thnk&nbsp;that the foam&nbsp;condition had existed from the very beginning.&nbsp; The sprayable insulation used on Columbia was a bit different from that used at the beginning of the program.&nbsp; The original formulation used a large amount of methylene chloride as a carrier solvent, and I believe that the new formulation is somewhat different.&nbsp; I have been lead to believe that it does not adhere as well.</p><p>I have been on quite a few failure investigations, and this techique of pulling a single bolt from a flight vehicle is a completely new one on me.&nbsp; I don't understand it from a physics point of view, I don't understand it from a statistical point of view,&nbsp;I don't understand it from a configuration control point of view, and I don't understand it from a manufacturing point of view.&nbsp; </p><p>What bothers me more is the Russian tendancy to focus on whether there is a latent design defect&nbsp;or whether there is a quality control issue.&nbsp; And if the issue is quality control to decide that the design is OK and proceed with use relying on future quality&nbsp;inspections to catch the defect that they missed earlier.&nbsp; The U.S. would drive for a "root cause" and implementation of targeted corrective action, and correction of other&nbsp;potential problems that might be uncovered during the investigation.&nbsp; Duplication of the failure in controlled tests is also typical of a U.S. approach.&nbsp; Of course, not always is it possible to meet this ideal.&nbsp; But for a small component like an explosive bolt, it ought to be quite doable, assuming that the configuration control is good and that quality records are maintained.</p><p>I would&nbsp;tend to doubt that there is a&nbsp;very rare defect lurking in the design.&nbsp; Design disciplines and solid analysis and qualification testing can address such issues, and explosive bolts are not all that complicated anyway.&nbsp; More likely there is a problem with the manufacturing process and with control of that process.&nbsp; Those sorts of problems can also be handled in a straightforward manner using statistical process control on the manufacturing floor.</p><p>I have seen quie a few design defects, but no latent ones.&nbsp; Rockets tend to be pretty unforgiving of design problems and reveal them rather quickly -- and often loudly.&nbsp; I have seen many more problems with&nbsp;deviations in the manufacturing process.&nbsp; Most are caught in acceptance testing, but one can certainly run into problems with untestable items, like explosive bolts.&nbsp; However, there are some pretty good non-destructive inspection techniques that can help to assure that the bolts are good.&nbsp; Where you get into trouble is with notions like an AQL (acceptable quality level) that permits a certain number of defects in a lot and still allows acceptance of the lot.&nbsp; AQLs are common in munitions but not common at all with aerospace parts.&nbsp; Nothing fails by chance alone, it fails for a reason.&nbsp; Physics at a macroscopic level is deterministic.&nbsp; If you make a part the same way every time it will perform the same way every time.&nbsp; Variation in performance is traceable to variation in manufacturing. </p><p>I'm not sure what a "common" failure might be in this business.&nbsp; But in an earlier post in this thread I found reference to explosive bolt problems going back quite some time on Soyuz.&nbsp; Explosive bolts do not have a particularly good reputation in the industry, and in my opinion other options are often preferable.&nbsp; There are some pretty good and extremely reliable pyrotechic devices for separation of stages and such&nbsp;--&nbsp;mild detonating fuses and the "super zip" system leap to mind, and shaped charges are often used if the shock can be tolerated.&nbsp; The shuttle has had explosive bolt failures.&nbsp; I recall a launch in the last few years (before Columbia) in which at least one of the explosive bolts that release the SRBs did not fire -- the SRBs left the pad anyway.</p> <div class="Discussion_UserSignature"> </div>

 

[CalliArcale]

<p>Ask shuttle_guy.&nbsp; He said they saw foam loss way back on STS-1, and on absolutely every flight since then. But they fell prey to the logic of "if it hasn't failed, it must not be broken, and if it ain't broke, don't fix it."&nbsp; I would not be surprised if a similar logic has persisted in the Soyuz program.&nbsp; Finding a defect that hasn't caused a failure is very difficult, and you can never be sure you've found them all since it's logically impossible to prove a negative. </p> <div class="Discussion_UserSignature"> <p> </p><p><font color="#666699"><em>"People assume that time is a strict progression of cause to effect, but actually from a non-linear, non-subjective viewpoint it's more like a big ball of wibbly wobbly . . . timey wimey . . . stuff."</em>  -- The Tenth Doctor, "Blink"</font></p> </div>

 

[Zipi]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>I have been lead to believe that it does not adhere as well.I have been on quite a few failure investigations, and this techique of pulling a single bolt from a flight vehicle is a completely new one on me.&nbsp; I don't understand it from a physics point of view, I don't understand it from a statistical point of view,&nbsp;I don't understand it from a configuration control point of view, and I don't understand it from a manufacturing point of view.</DIV></p><p>Maybe a little bit screwy comment but: What if we take the conspiracy theory point of view? Now when they have removed the bolt and hacked the Soyuz on orbit there are lots of room to hide possible future events behind that... For example if the components of the Soyuz will totally fail to separate at this time and the crew will die in the process there is now a good excuse because the ship has been modified at pretty uncontrolled conditions.</p><p>If there is a quality control issue it is quite possible that they try to hide it. And if there is quality issues, they know of those by now and they have fixed the problem already. By modifying the ship they have now an insurance where to claim if the next descent fails (and possibly by this way they are able to avoid grounding of the fleet for a long time).<br /><br />Hopefully you got the point... I'm not saying this is the case, but it just popped into my weird mind. <img src="http://sitelife.space.com/ver1.0/content/scripts/tinymce/plugins/emotions/images/smiley-tongue-out.gif" border="0" alt="Tongue out" title="Tongue out" /></p> <div class="Discussion_UserSignature"> </div>

 

[DrRocket]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>Maybe a little bit screwy comment but: What if we take the conspiracy theory point of view? Now when they have removed the bolt and hacked the Soyuz on orbit there are lots of room to hide possible future events behind that... For example if the components of the Soyuz will totally fail to separate at this time and the crew will die in the process there is now a good excuse because the ship has been modified at pretty uncontrolled conditions.If there is a quality control issue it is quite possible that they try to hide it. And if there is quality issues, they know of those by now and they have fixed the problem already. By modifying the ship they have now an insurance where to claim if the next descent fails (and possibly by this way they are able to avoid grounding of the fleet for a long time).Hopefully you got the point... I'm not saying this is the case, but it just popped into my weird mind. <br />Posted by Zipi</DIV></p><p>I am generally skeptical about conspiracy theories.&nbsp; They are too hard to pull off.&nbsp; Physics tends to win in the end.&nbsp; The problem with engineers is that if you put their back to the wall they tend to blurt out the truth.<br /></p> <div class="Discussion_UserSignature"> </div>

 

[erioladastra]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>Maybe a little bit screwy comment but: What if we take the conspiracy theory point of view? Now when they have removed the bolt and hacked the Soyuz on orbit there are lots of room to hide possible future events behind that... For example if the components of the Soyuz will totally fail to separate at this time and the crew will die in the process there is now a good excuse because the ship has been modified at pretty uncontrolled conditions.If there is a quality control issue it is quite possible that they try to hide it. And if there is quality issues, they know of those by now and they have fixed the problem already. By modifying the ship they have now an insurance where to claim if the next descent fails (and possibly by this way they are able to avoid grounding of the fleet for a long time).Hopefully you got the point... I'm not saying this is the case, but it just popped into my weird mind. <br />Posted by Zipi</DIV></p><p>I am pretty confident they have no idea what has caused the problems.&nbsp; They are grasping for straws.&nbsp; Either whatever the problem was has been fixed so everything they have tested is now good, or it is a very insideous failure.&nbsp;&nbsp;&nbsp; But while I trust the Russians to be as forthcoming about as far as I can toss a Soyuz (e.g., although 5 things were lost or jettisoned during the EVA, they insist the crew didn't lose any thing - even when shown the video of it clearly leaving them) I do believe them in this case because of the access.&nbsp; And of course they are freaked because they want to sell us Soyuzs!<br /></p>

 

[Zipi]

<p>Nice wrapup of the first space walk:&nbsp;&nbsp; (video)</p><p>http://www.space.com/php/video/player.php?video_id=080711-soyuz-pyrobolt</p><p>You can also see how the Strela (Russian hand operated crane) works.</p> <div class="Discussion_UserSignature"> </div>

 

[Arion]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>*chuckles*&nbsp; Cosmonauts and astronauts tend to have good senses of humor.&nbsp; &nbsp; Thanks for confirming that the Strela is hand-operated.&nbsp; Still hope some of our Russian members will wander in, but I think it's the middle of the night over there.This has been a fun spacewalk to follow on NASA TV.&nbsp; Kononenko just finished with the camera.&nbsp; He was having a lot of trouble with it for a while -- the glare of the sun made it impossible to see the viewfinder, and impossible for him to tell what Volkov could see -- that the camera was still switched off.&nbsp; &nbsp; That makes me feel much better about my own photographic endeavors! <br />Posted by CalliArcale</DIV><br /><br />You are right. <div class="Discussion_UserSignature"> </div>

 

[MeteorWayne]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>*chuckles*&nbsp; Cosmonauts and astronauts tend to have good senses of humor.&nbsp; &nbsp; Thanks for confirming that the Strela is hand-operated.&nbsp; Still hope some of our Russian members will wander in, but I think it's the middle of the night over there.This has been a fun spacewalk to follow on NASA TV.&nbsp; Kononenko just finished with the camera.&nbsp; He was having a lot of trouble with it for a while -- the glare of the sun made it impossible to see the viewfinder, and impossible for him to tell what Volkov could see -- that the camera was still switched off.&nbsp; &nbsp; That makes me feel much better about my own photographic endeavors! <br />Posted by CalliArcale</DIV><br /><br />By coincidence, today, while cleaning through my late father-in-law's posessions I recovered a copy of the NASA Astronaut's Photography Manual produced by Hasselblad. He worked on all the Hasslebalds used during the Apollo days. <div class="Discussion_UserSignature"> <p><font color="#000080"><em><font color="#000000">But the Krell forgot one thing John. Monsters. Monsters from the Id.</font></em> </font></p><p><font color="#000080">I really, really, really, really miss the "first unread post" function</font><font color="#000080"> </font></p> </div>

 

[MeteorWayne]

<p>Russia thinks they have found the problem</p><p>From Spaceflightnow: http://www.spaceflightnow.com/shuttle/sts126/080925pyrobolt/</p><p>&nbsp;</p><p>Russian troubleshooters believe electrical arcing between the space environment and the international space station most likely caused recent problems with explosive bolts used to separate Soyuz re-entry vehicles just before atmospheric entry</p><p>"Our Russian colleagues have taken a number of steps," said space station Program Manager Michael Suffredini. "For the ascent vehicle, they've actually changed the pyros to a configuration that's less sensitive to this. ... In addition to that, the vehicle that's due to return home (in October) has, as you know, through a pair of EVAs, a pyro was removed therefore freeing up the mechanism." </p><p>The Russians plan to launch Soyuz TMA-13 on Oct. 12 to ferry two fresh station crew members - Expedition 18 commander Michael Fincke and flight engineer Yuri Lonchakov - to the lab complex, along with Richard Garriott, a U.S. space tourist. Volkov, Kononenko and Garriott then will return to Earth in the TMA-12 vehicle on Oct. 23. </p><p>Suffredini said the TMA-13 spacecraft has been modified to minimize the threat of arcing. </p><p>"Our Russian colleagues have spent an enormous amount of time, a very structured effort, to create a fault tree and then work off the fault tree to come to a most probable cause," Suffredini said. "They have concluded the most probable cause to be related to the plasma environment and the delta potential between the plasma environment, the environment around space station, and the space station itself. </p><p>"The way the space station is wired together, there actually is a voltage, a delta voltage potential, between the environment around the ISS and the ISS itself and what happens over time is occasionally that potential tries to equalize. And when it tries to equalize, you get what essentially is a spark to the ISS. This is an environment we've been living in since we starting flying ISS, it's why we have what we refer to as the plasma contactor units that we operate during EVAs to prevent that from occurring during EVAs." </p><p>The plasma contactor units use xenon gas to equalize electrical potential around the station. The xenon supply is limited and the PCUs are only operated during spacewalks to prevent any chance of arcing while astronauts are working outside. Testing showed no obvious threat to the station's systems when the PCUs were turned off. </p><p>"What our Russian colleagues have determined is because of the very specific configuration of the pyros ... on the Soyuz, where it's failed to separate each time, they've determined that there's been some arcing, if you will, or equalizing of voltage in this area," Suffredini said. "And this over time has caused the pyros to be ineffective." </p><p>The culprit apparently involves insulation on the Soyuz near the pyro in question that includes an ungrounded aluminized layer. </p> <div class="Discussion_UserSignature"> <p><font color="#000080"><em><font color="#000000">But the Krell forgot one thing John. Monsters. Monsters from the Id.</font></em> </font></p><p><font color="#000080">I really, really, really, really miss the "first unread post" function</font><font color="#000080"> </font></p> </div>

 

[silylene old]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>Russia thinks they have found the problem..... <br />Posted by MeteorWayne</DIV><br /><br />That is a rather obscure cause.&nbsp; But let's hope they are correct, and the problem is fixed. <div class="Discussion_UserSignature"> <div class="Discussion_UserSignature" align="center"><em><font color="#0000ff">- - - - - - - - - - - - - - - - - - - - - -</font></em> </div><div class="Discussion_UserSignature" align="center"><font color="#0000ff"><em>I really, really, really miss the "first unread post" function.</em></font> </div> </div>

 

[MeteorWayne]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>That is a rather obscure cause.&nbsp; But let's hope they are correct, and the problem is fixed. <br />Posted by silylene</DIV><br /><br />We'll find out pretty soon, since the next Soyuz landing is only a week or so away. (IIRC) <div class="Discussion_UserSignature"> <p><font color="#000080"><em><font color="#000000">But the Krell forgot one thing John. Monsters. Monsters from the Id.</font></em> </font></p><p><font color="#000080">I really, really, really, really miss the "first unread post" function</font><font color="#000080"> </font></p> </div>

 

[shuttle_guy]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>We'll find out pretty soon, since the next Soyuz landing is only a week or so away. (IIRC) <br />Posted by MeteorWayne</DIV> believe that the next manned Sopyuz is set for Oct 12th&nbsp;with 2 new ISS crew members. The returning Soyuz is the one which has been at the iSS for about 6 months and which was&nbsp;had the suspect pyro bolt removed&nbsp;during a EVA. That will be the next Soyuz landing in mid OCT.<br /> <div class="Discussion_UserSignature"> </div>

 

[MeteorWayne]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'> believe that the next manned Sopyuz is set for Oct 12th&nbsp;with 2 new ISS crew members. The returning Soyuz is the one which has been at the iSS for about 6 months and which was&nbsp;had the suspect pyro bolt removed&nbsp;during a EVA. That will be the next Soyuz landing in mid OCT. <br />Posted by shuttle_guy</DIV><br /><br />Ok 2 weeksish. I was intentionally vague <div class="Discussion_UserSignature"> <p><font color="#000080"><em><font color="#000000">But the Krell forgot one thing John. Monsters. Monsters from the Id.</font></em> </font></p><p><font color="#000080">I really, really, really, really miss the "first unread post" function</font><font color="#000080"> </font></p> </div>

 

[erioladastra]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>Ok 2 weeksish. I was intentionally vague <br />Posted by MeteorWayne</DIV></p><p>&nbsp;</p><p>Actually to be clear.&nbsp; A single pyro was removed.&nbsp; It is quite likely that we won't know anything about wther they have the cause right until well after the landing, if then.&nbsp; If something is spoiling the bolts on orbit then having that one back might show something (of course it will be hard to prove it didn't happen before launch or if on orbit that its not a cumaltive thing that they retrieved it too soon).&nbsp; The Soyuz going up has modified bolts based on this theory.&nbsp; So we really wont' liekly know for sure until 17S lands in the&nbsp;spring.</p>

 

[trailrider]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>&nbsp;Actually to be clear.&nbsp; A single pyro was removed.&nbsp; It is quite likely that we won't know anything about wther they have the cause right until well after the landing, if then.&nbsp; If something is spoiling the bolts on orbit then having that one back might show something (of course it will be hard to prove it didn't happen before launch or if on orbit that its not a cumaltive thing that they retrieved it too soon).&nbsp; The Soyuz going up has modified bolts based on this theory.&nbsp; So we really wont' liekly know for sure until 17S lands in the&nbsp;spring. <br />Posted by erioladastra</DIV><br /><br />Good luck and godspeed to those riding this one down!

 

[silylene old]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>&nbsp;Actually to be clear.&nbsp; A single pyro was removed.&nbsp; It is quite likely that we won't know anything about wther they have the cause right until well after the landing, if then.&nbsp; If something is spoiling the bolts on orbit then having that one back might show something (of course it will be hard to prove it didn't happen before launch or if on orbit that its not a cumaltive thing that they retrieved it too soon).&nbsp; The Soyuz going up has modified bolts based on this theory.&nbsp; So we really wont' liekly know for sure until 17S lands in the&nbsp;spring. <br />Posted by erioladastra</DIV></p><p>Is it posible to see evidence of arcing in the pydro which had been removed&nbsp;during the spacewalk&nbsp;and returned already?</p> <div class="Discussion_UserSignature"> <div class="Discussion_UserSignature" align="center"><em><font color="#0000ff">- - - - - - - - - - - - - - - - - - - - - -</font></em> </div><div class="Discussion_UserSignature" align="center"><font color="#0000ff"><em>I really, really, really miss the "first unread post" function.</em></font> </div> </div>

 

[shuttle_guy]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>Is it posible to see evidence of arcing in the pydro which had been removed&nbsp;during the spacewalk&nbsp;and returned already? <br />Posted by silylene</DIV></p><p>&nbsp;</p><p>I am certain that they do not have any hard evidence of the arcing since the arcing was one of a large number of possible causes. The arcing was just the "most likly" cause.<br /></p> <div class="Discussion_UserSignature"> </div>

 

[DrRocket]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>&nbsp;I am certain that they do not have any hard evidence of the arcing since the arcing was one of a large number of possible causes. The arcing was just the "most likly" cause. <br />Posted by shuttle_guy</DIV></p><p>'Most likely" based on what evidence ?&nbsp; Has the failure been recreated on some sort of reasonable simulation, or it is the "most likely" based on elimination of originally competing failure modes ?</p><p>This ideas of a rather nebulous "electrical discharge" between the environment and the ISS strikes me as a failure mode that would not be particularly repeatable, and one that would defy verification of the effectiveness of corrective action.&nbsp; If this phenomena were affecting explosive bolts one would also expect it to&nbsp;affect sensitive electronic equipment on both the Soyuz and the ISS.&nbsp; Have any such effects or failures been experienced ?&nbsp; One might also expect that triboelectric effects in flight would be a concern.&nbsp; How do they handle triboelectric charging in flight, and why do those design features not provide protection when docked at the ISS ?<br /></p> <div class="Discussion_UserSignature"> </div>

 

[shuttle_guy]

<p><BR/>Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>'Most likely" based on what evidence ?&nbsp; Has the failure been recreated on some sort of reasonable simulation, or it is the "most likely" based on elimination of originally competing failure modes ?This ideas of a rather nebulous "electrical discharge" between the environment and the ISS strikes me as a failure mode that would not be particularly repeatable, and one that would defy verification of the effectiveness of corrective action.&nbsp; If this phenomena were affecting explosive bolts one would also expect it to&nbsp;affect sensitive electronic equipment on both the Soyuz and the ISS.&nbsp; Have any such effects or failures been experienced ?&nbsp; One might also expect that triboelectric effects in flight would be a concern.&nbsp; How do they handle triboelectric charging in flight, and why do those design features not provide protection when docked at the ISS ? <br />Posted by DrRocket</DIV></p><p>I certainly agree that the "most likely" (as defined by the Russian's)&nbsp;cause is probably not based on any direct evidence. The next Soyuz entry in a few weeks will be interesting.</p> <div class="Discussion_UserSignature"> </div>

 

[MeteorWayne]

Replying to:<BR/><DIV CLASS='Discussion_PostQuote'>I certainly agree that the "most likely" (as defined by the Russian's)&nbsp;cause is probably not based on any direct evidence. The next Soyuz entry in a few weeks will be interesting. <br />Posted by shuttle_guy</DIV><br /><br />Yeah, as in "May you live in interesting times" <div class="Discussion_UserSignature"> <p><font color="#000080"><em><font color="#000000">But the Krell forgot one thing John. Monsters. Monsters from the Id.</font></em> </font></p><p><font color="#000080">I really, really, really, really miss the "first unread post" function</font><font color="#000080"> </font></p> </div>