1. Medical Center case in Chicago
Before and After – WDC power conditioner second installation
History: From the installation of the HiArt at a Medical Center in Chicago, there were a
number of power monitors and investigations into the power. Everything pointed to bad,
unstable power from the facility.
Initial power monitoring showed minor sags in the power. This was signed off and the
installation commenced without correction to the power. Machine was ATP’d and went
clinical.
Right from the beginning, errors were a problem. Never the same error twice. Improved
performance with few to no errors, after treatment hours when the electrical loading was
light. The site started to lose nodes on the cluster. Evaluating the Tripplite monitor on
the Dataserver revealed that power would drop out and fluctuate frequently.
This prompted a second monitor to be installed. The ringing of the 480 Volt was
discovered. A 70 Vp-p wave modulated into the 480 Volt at a low frequency that was
used to sync the hospital clocks and turn on/off the parking lot lights. This induced
transient caused computers, such as the STC to be scrambled and cause connection
problems.
By the beginning of April, three months after going clinical, there was a series of
component failures that prompted management to get involved. Within a week
timeframe a few solid state relays, 3 gun boards and finally a linac were replaced.
The hospital finally did there own power survey and found more issues. Larger problems
were specific to grounding of substations and power sags. They installed TVSS (surge
suppressors), at the recommendation of their contractor, which did nothing to help the
situation.
By the beginning of June the decision was made to install the first WDC power
conditioner. By August, it had arrived and was installed. It was left in bypass. About
two weeks later, the WDC rep arrived to “tune it up” and put it into AUTO. When he put
it in AUTO, it generated a lot of smoke and filled the department. It was never put in
AUTO again.
Several trips by the WDC rep later, it was determined that the unit needed to be replaced.
It was determined that the power conditioner needed to have more front end protection
against the severe transients on the hospital supply.
The second unit was installed mid December. Since then there had been a significantly
noticeable reduction in errors. As an example: December 6, 7, and 10th
there were 84
errors that were either RTT or Linac hardware. Since the installation of the second power
conditioner, a timeframe of Jan 8, 9, 10, 11, 14, and 15 there only 5 RTT events with no
Linac hardware errors.
2. Over the next few months the degraded components from the bad power were isolated
and replaced.
In summary: a reduction from 84 errors in 3 days to 5 errors in 6 days. Also, a reduction
in the type of errors.
Background information:
The power conditioner that was installed was a WDC without a UPS. Had a UPS been
purchased with the unit, the 5 errors in 6 days would probably been reduced to zero.
The site has phase loss or complete power loss issues.
Below are 2 captures of a power loss associated with this site. The top picture is of
unconditioned power. Needless to say, it creates errors and damage to the machine when
this type of power is applied to the HiArt. The bottom picture is a similar event AFTER
the installation of a power conditioner. The pink line is smaller than the other 2 phases,
indicating a phase loss. Had the site purchased a UPS, the pink waveform would
conform to the other 2 and the event would never have been recorded.
During power outage at the facility before the second power conditioner was installed.
Power outage with the second power conditioner installed.
3. Below are the uptime reports from ATP through 2009.
Note: Red listed uptimes are below the TomoTherapy goal for that year. Red listed
comments are either confirmed or suspected components damaged and replaced as a
result of facility power.
RTT (labor hours) without a part, indicates that the machine operated properly after the
normal hospital treatment hours.
2007 Uptime percentage (96%
uptime goal)
Comments
1 100
2 100
3 98
4 94 Power losses caused loss of 3
gun boards and linac.
Multiple Solid State Relays.
5 100
6 98
7 97
8 100
9 90 • Status Console
• Gun Board
• Union (not power
related)
10 90 • MLC Errors replaced
• MLC and MLC
Controller
• RTT (6.25hr labor)
• RTT (9.3 hr labor)
• Pulse Shaper
11 92 • Air Conditioning issues
• Artifact
• Machine Hangs on
Startup
• OS issue
• RTT
• Solid State Relay
12 95 • Heater Open/Short
• Facility power losses
• Heater Pulsing Failure
• Linac SSM
• RTT (6.0 hrs labor)
• RTT (12 hrs labor) +
Mag + HVPS Controller
• RTT (3 hrs labor)
• RTT (1 hr labor)
• Pulse Shaper
Working power conditioner
installed Mid December.
