Health NZ deploying AI scribe Heidi to 16 emergency departments: But what’s the story with errors and privacy?

The AI can take notes, which it can also summarise, add to a patient’s electronic health file or format into a template for a letter to the patient’s GP.

Hawke’s Bay District chief medical officer Dr Benjamin Pearson said the East Coast AI scribe trial, involving 10 clinicians, saw note-taking and form-filing time reduce from 17 minutes to just over four minutes per patient.

In an ED environment, notes were usually written up between seeing patients.

The administration time-saving allowed registrars (junior doctors) and nurse practitioners to each see an average of three extra patients per shift, Pearson told the Herald.

Senior doctors – who were usually involved in more complex cases and spent some of their time overseeing junior doctors – saw an average one extra patient per shift.

Pearson – who was effusive about the technology – said 100% of clinicians at the Hawke’s Bay Hospital ED were using an AI scribe by the end of the trial.

He also saw it as a recruitment and retention tool. He said going into the trial, he talked to a psychiatrist who said their stress level was 10 out of 10. They were on the verge of walking away.

After the AI scribe trial, they said their stress load was two out of 10, Pearson said.

Everyone who uses AI for transcription or summaries knows that it can make mistakes (which could be down to a thick Kiwi accent or a badly positioned microphone as much as software coding shortcomings) and that artificial intelligence is prone to “hallucinate” (to use the technical industry term) or just make stuff up when it’s not sure of something.

In the medical context, any mistake could have huge consequences.

Otago University Professor of bio-ethics Angela Ballantyne criticised the speed and relatively small scale of the Hawke’s Bay trial. She told errors could be catestropic in an ED environment and that they could not necessarily be easy to correct.

She told RNZ she had spoken to one doctor using a similar tool - not Heidi and outside the Hawke’s Bay trial - where a slight change in how a patient’s response had been recorded could have had serious implications for their treatment.

“The tool made a very subtle error. The patient had said they experienced this symptom for the first time ‘in the morning’ and the tool had changed it to ‘every morning’. So a very subtle difference but [it] has really significant clinical implications if someone else comes along and reads and tries to interpret that note,” Ballantyne said.

“This doctor said it was only on the second read through that they realised that the error had been made.”

Heidi Health clinical director Dr Ben Condon told the Herald: “We strongly advocate as part of our terms of service that all of our users review their notes contemporaneously for errors, because they can occur.”

He added: “The benefit of reviewing the note immediately after the consultation is finished – unlike at the end of the day, where you might be quickly writing 10-plus notes – is you’ve just seen that patient and it’s fresh in your mind. So you’re more likely to identify those errors.

“We think that the risks of perpetuating errors is less than the benefits that the scribe provides. But we’re really open with our users that the scribe can make mistakes, like humans can, and that it’s important to check every note. Ultimately, the clinician is still responsible for the accuracy of the note.”

READ MORE: New Apple Watch feature addresses “the silent killer”

Condon – once an emergency registrar at St Vincent’s Hospital in Melbourne – said while an emergency department can be raucous, the thinking was that, “if it works in an ED, it’ll work anywhere”.

Heidi is trained for medical jargon and also comes in versions refined for different specialiaties. There’s also a version for vets.

A clinician captures what you say using a webcam or other microphone-enabled hardware, but Heidi does not record any audio, only a live transcription (that is, just text).

It’s up to a healthcare provider how long they keep the transcription. If they take it up, there are automated options to delete a transcript in between one to 90 days.

Heidi touts that data is “stored locally” as an additional privacy safeguard, but Condon said his company was still in the process of organising local storage in New Zealand.

For now, New Zealand data are stored in Sydney on encrypted servers. With whom? Condon refused to say, apart from: “It’s a third party. We don’t own any servers ourselves.”

No patient notes are used for training Heidi, Pearson said.

His firm’s procedures had been approved by Health NZ’s National Artificial Intelligence and Algorithm Expert Advisory Group.

By default, only your doctor or other authorised clinicians within your healthcare provider can see any notes about you produced by Heidi.

But if there’s some kind of recurring error or glitch, then a healthcare provider could choose to send a transcription to Heidi trouble-shooters in Australia.

Condon says they’re bound by privacy laws and strict in-house privacy protocols. He added that Heidi was independently certified for the ISO 27001 information privacy standard.

Pearson said the Hawke’s Bay trial was set up so that if notes were sent to Heidi’s team, a header containing the patient’s name and NHI number and other identifying details was not included – although it was possible the patient’s name, or part of their name, would be said in a conversation captured by the transcription.

Health NZ’s HealthX team began trialling three AI scribes: Heidi, iMedX (made by another Australian start-up) and an in-house product dubbed Tuhi.

Tuhi was culled early on. Pearson said digital cuts saw the number of programmers reduced, but that there were already questions over the local effort.

He said one clinician hit pause then lost 45 minutes’ worth of notes. “It’s hard to build interest when that sort of thing happens,” he said.

Meanwhile, Heidi had been winning hearts and minds across the private sector with its freemium mode.

It was effective, popular with clinicians and low-cost, Pearson said.

Health NZ and the Health Minister’s office would not put a total on the cost of the rollout, other than saying it’s accommodated within the agency’s existing budget.

It also helped that Heidi – founded in 2019 by the Melbourne-based vascular surgeon Thomas Kelly – raised $110 million in venture capital (VC) in early October (taking its total VC backing to $170m) on the back of becoming Australasia’s most-used AI scribe.

It has also made inroads into the UK, the US and other offshore markets. All up, Condon says Heidi now handles some two million consults a week.

Heidi has just signed Mount Sinai Beth Israel hospital in New York, among other international customers. It’s advertising for 97 new staff.

Health NZ was happy to piggyback on the kind of R&D heft that sort of capital provided.

Pearson said iMedX would remain in the picture, despite Health NZ going with Heidi for the emergency department rollout. iMedX was being piloted at two hospitals. Health NZ wants to keep a number of alternatives in the mix.

Coincidentally, I had occasion to visit two specialists using AI scribes in the fortnight ahead of writing this article (fear not, readers, it was routine with no complications).

Both told me up front they were using the technology, which was good.

An Otago University survey of 197 health practitioners – carried out in February and March last year and released in August this year – said 40% of those surveyed reported using AI scribes to take patient notes.

But of those using an AI scribe, only 66% had read the terms and conditions on the use of the software and 59% reported seeking patient consent.

Dr Emily Cavana, a Lower Hutt GP and senior lecturer at the University of Otago’s Wellington campus, earlier told me she had mixed views on AI scribes – but she did like how they had made medical professionals more “present”.

I found this too. Not only did the two specialists spend 100% of their time in front of me, with no detours to a keyboard (one used Heidi, one a rival product) but they had to narrate everything they did, for Heidi’s benefit, as they took notes as they went - which was interesting and informative for me as a patient and led me to ask questions I probably would not have otherwise. I felt more in the picture.

In terms of Prof Ballantyne’s critic about the “in the morining”/“every morning” almost-error: It is concerning but my own experience in EDs and with various family members has been that you’re asked to give an account of the your condition and the circumstances leading up to it multiple times - to the point where it can drive you bonkers, but also maximising the odds that the correct version of events is the one that makes it to the record.

One of the specialists told me he had realised he had blundered by buying an omni-directional microphone – for all of his presumably off-the-charts medical expertise, audio was new to him. It was not a particularly big deal in a quiet consultation room in Remuera.

For the more blaring public hospital emergency department environment, Heidi’s Condon said his firm is working with local partner Hendrix Health to get the right gear deployed.

Overall, both of the specialists I saw were hugely in favour of AI scribes.

Both used the scribe’s notes as a starting point, but it was a huge time-saver to be reviewing ready-made notes, rather than starting from scratch.

One told me: “I used to do my letters [to GPs] at the end of the day. Now I just go home.”

Cavana – though still not 100% convinced of the technology – said many doctors had told her they had their lunch breaks back, which was better for their overall focus.

Chris Keall is an Auckland-based member of the Herald’s business team. He joined the Herald in 2018 and is the technology editor and a senior business writer.