ITSS reports on data center outages

The Information Technology Systems and Services data center has had three data center outages in the past week (two unplanned on Monday and one planned on Sunday) that caused all ITSS services to be unavailable for hours each time. This has had a large impact on both NDUS (ConnectND Campus Community and Ancillary systems and ODIN) and UND (e-mail, web, file, print, imaging, directory, and many other) services.

ITSS data center has made significant investments in uninterruptible power supply (UPS) and electrical generator upgrades and additions. Initially, ITSS management was perplexed as to why these systems didn't protect the data center from this week’s failures. Because of the location of the failures, neither the UPS nor the generator could help. ITSS works closely with UND electricians regarding the data center electrical distribution. The electricians found, in trouble-shooting the outages, that the data center had a circuit switch that was failing and replaced it Monday evening. Further review later that week found other potential single-point-of-failure for which the electricians recommended immediate repair. ITSS didn’t give notice according to standard practice but, because of the potential for another unplanned failure, the replacement was scheduled as quickly as possible. The work was done early Sunday morning. ITSS staff spent many extra hours, most out of normal work hours, recovering systems following outages. Everyone is interested in preventing these outages.

ITSS and UND facilities are actively planning for an increase of capacity for our UPS and generator to meet needs for the next five to 10 years. Facilities will further review the data center electrical distribution to detect other repairs/replacements that should be made to assure electrical availability. The ITSS data center hosts both critical UND and NDUS systems which students, faculty and staff expect to be available. ITSS staff plan and work hard to match that expectation as closely as possible.

ITSS management understand that these events caused frustration as some student and faculty work had to be re-done and there were service, communication and business delays. Thank you for your patience. -- Dorette Kerian, director, ITSS.