UK NHS 850k Reply-all email fail: State health service blames Accenture

The UK National Health Service Digital has blamed IT outsourcer Accenture for its ongoing email Reply-all woes.

The cause of the cockup is almost certainly a combination of user error and a trouble-inviting system config update, though.

On Monday morning, the NHS' internal email system was brought down by a test message sent by "R," a senior IT facilitator at South East Commissioning Support Unit.

R managed to send a blank message intended for the CroydonPractices distribution list to all 850,000 people with a registered NHS.net email address.

Irritated folk then began responding to the list address, which promptly generated huge volumes of email traffic and caused severe delays in messages being sent and received.

An NHS Digital spokesman told The Register: "A number of email accounts have been operating slowly. This was due to an NHS Mail user setting up an email distribution list which, because of a bug in the supplier's system, inadvertently emailed everyone in the NHS mailing list."

The NHS.net outsourced email supplier is Accenture, which the NHS Digital spokesman confirmed to The Register. Accenture Digital's UK arm had not, by the time of publication, responded to The Register's enquiries.

As for the cause of the accidental mass mailing, NHS Digital were keen to suggest that R was not to blame for the email ping-pong hell. One alert Reg reader of the original story, however, spotted this NHS.net changelog entry dated yesterday, 13 November:

Add contacts from other organisations to a distribution list... hmmm

One of the main headaches for sysadmins is the horror of a Reply-all email chain. Indeed, putting the term through Google (other search engines exist) returns this result about how to block email distribution lists from receiving emails sent by list users.

If you go to that link, it's got a screencap from Exchange Server 2010 in which a simple radio button sets the difference between "only senders inside my organisation" [can receive emails from all users on the list, not just ones sent by the list admin] and "senders inside and outside my organisation."

+Comment

While the true cause is doubtless known within the NHS and to Accenture, The Register can only make an informed guess for now. What follows is speculation as we do not know the internal setup of NHS.net.

Certain configuration sets can cause group emails sent to an incorrectly configured address to be sent to the entire group. If the "group" happens to be the entirety of NHS.net due to a config snafu ... you see where this is going.

At 1545 GMT, NHS sources were telling The Register that emails with 0950 timestamps were only just beginning to arrive in their mailboxes. ®