06 December 2018

Hidden Damage

     Of course, it took one of the Power Room electricians to find it.

     And of course, it was the ternary degausser.  The degaussers don’t run very often but they’re absolutely essential.  They’re after the output of the [CLASSIFIED], five hundred feet right after the big ion trap, and cycle on whenever the induced magnetism of the support structure exceeds a preset level. There’s a neat little magnostrictive oscillator to detect it, and—  But that’s too much geekery already.  The blamed thing kept tripping the breaker and the riggers and I hadn’t been able to figure out why.  I was sure the problem had to be in the wiring or hardware out on the boom; there’s a lot of stress on everything out there and we’d replaced the transformers for it just last year. The most recent breaker trip was nasty enough that it glitched the stardrive -- and it was pure luck it didn't happen at a critical time.

     Dr. Schmid, the Chief's boss, decided we needed an electrician and I figured he was right.  The degausser secondaries run a couple of hundred Amps through the coils.  Power Room’s got test gear that’ll read that high; we don’t.  The primary current’s only 19 or 20 Amps per leg, at 480 Volts, and even that’s enough energy to make plasma arcs on a bad day.

     Ron, the electrician, was whistling a tune I didn’t recognize when he showed up at the Drive Room in one of the little electric service vehicles, loaded down with tools and supplies.  There were a couple of riggers waiting with me, already in their pressure suits except for gloves and helmets.  Ron’s first words after introductions were, “So what’s going on with this thing?”

     “It keeps blowing the breaker,” I told him.  “We haven’t been able to find any wiring problems and every check I’ve had the riggers make out at the transformer looks okay.  I’m mystified.”

     “Well, let’s set up to see what happens and give it a try.”

     The riggers finished suiting up, Ron handed out clamp-on ammeters and we got set up: riggers in vacuum out on the boom, checking at the first junction box and the transformer, me inside watching the degausser controller and contactor in their explosion-proof box, with the door open, and the electrician at the breaker panel around the corner and a few steps down the passageway.  I had the controller off, so the contactor wasn’t going to pull in and pass any juice to the degaussers.  We’d just be checking the wiring from the breaker to the contactor for this first step.  Just turn the circuit breaker on and right back off, unless it tripped first.

     He shouted “Here we go,” reset the breaker and something went POP! like a firecracker.

     I yelled, “Dammit!” right afterwards. 
    
     My radio clicked and on of the riggers asked, “What was that thump?  I felt it through the conduit!”

     Ron had come over to where I was by then, and asked, “What was it?”

     “I don’t know.   Not the contactor closing; it’s pretty quiet.”

     “You want to try again?”

     I said okay, told the riggers to listen close, and sure enough, the same thing happened.  Pop!

     So, we’re clever: we checked the wiring from the breaker to the contactor, unhooked it from the breaker and pulled it out of the conduit, laid it out on the floor.  Nothing wrong.  Twenty years old and the wires looked brand new.

     Ron looked at me.  “You sure that contactor’s off?”

     “Should be.”

     He said, "Mm," and we traded a look.  “Should be” often isn’t and we both knew it.

     On close examination, the contactor -- a big relay -- seemed fine.  The contacts are fully enclosed for safety, so you can’t see them, but the moving part, the armature, has a little tab that sticks out and that was clearly in the de-energized position and moved freely.

     Ron whistled a couple of bars of something vaguely classical while getting an ohmmeter out of his toolbar.  “I wonder….,” he said, more to himself than to me.

     Three measurements later, we weren’t wondering.  Three sets of contacts in the thing and one was open just like it should be, one was closed…and one read a few hundred Ohms instead of zero or infinity.
    
      I went to look for a spare contactor on the parts inventory.  He started taking the old one apart.

* * *
     Ten minutes later, having come up dry on a replacement, I walked back to see how things were going and Ron handed me a set of contacts.  No bigger than dimes, scarred and blackened.  “They weren’t quite welded together.  I pried them apart.”

     I looked them over.  “Ugly.  Inventory shows we should have three replacement contactors, the whole thing, new in the box; note says the last one got used in ’97, right before I started.”

     “These’ll clean up.  How’re you fixed for sandpaper?” 

     In the end, it took fine files to clear up the worst set. In a half-hour, we had them all smooth, shiny, and moving freely.  With the contactor back together and wired up, he returned to the breaker, I radioed the riggers to watch their meters, and told Ron, “Power up!”

     He hit the breaker — and there was no pop.  I waited a minute and tapped the manual override button.  The contactor pulled in with a muted “clunk,” followed by the riggers checking in on the radio:

     “Twenty Amps at the J-box, over!”

     “Degausser’s humming like normal.  Twenty Amps. Over.”

     “Thanks, guys.  Stand by.” I raised my voice.  “Ron, you hear that?  Looking good!”

     He walked around the corner smiling.  “It must have single-phased every time you reset the breaker!”

     I gave him a rueful grin, “Yeah, and arced like a son of a gun.”

     He nodded.  “The worse set of contacts were wedged in at a kind of funny angle.  It can’t have been good.”

     I told the riggers to button everything up and head for the airlock.  I cleared the manual override, but the degaussers stayed on, with the flickering red light of the “magnetization detected” indicator lit up.   Ron and I closed things up — gutter covers on the breaker box, the lid of the explosion-proof enclosure for the degausser contactor and controller — and about the time we finished, the red light went out and the contactor released with a soft thump.  No arc, no popping sounds.

     It’s easy sometimes to leap to a conclusion and become too attached to it.  If I’d suspected the contactor myself, it wouldn’t’ve taken much effort to open it up and check, or put an ohmmeter across it.  But I hadn’t.  It took a fresh pair of eyes — and a lack of preconceptions — to find the problem.