Skin cancer apps fall short – forskning.se

Early detection of skin cancer from malignant melanoma is important, and today there are many mobile phone applications that promise quick and easy analysis of self-taken photos of liver spots and birthmarks.

But it is difficult for the individual consumer to know whether the apps are up to the mark, says Asa Ingvar, a senior physician in dermatology at Skane University Hospital and associate professor at Lund University.

In two studies, she investigated AI-based applications for self-examination of skin changes.

– In order for the user to be able to determine whether the quality of an app can be trusted, the user needs to have information about how it is organised – something that is often missing today, says Åsa Ingvar.

The restrictions are not clear.

The studies were conducted in Australia. The aim was to verify the type of labelling and information provided by manufacturers, but also to come up with guidelines on how information on applications should appear.

In one study, researchers reviewed 21 AI-based mobile dermatology apps. They were able to determine that none of the apps met standard rating recommendations.

– It's worrying because as a user you don't get the knowledge about the restrictions on the apps, says Åsa Ingvar.

AI is trained on fair skin.

For example, she notes that many apps are often trained on images of skin changes among people with lighter skin tones. This can make them worse at analyzing images of people with different skin tones.

Applications may also be limited by only being trained on images from a certain type of camera. They may also have been trained on very few images of certain skin changes, or not trained at all.

– It is well known among developers that AI applications are sensitive to deviations from the data they are trained on. But this knowledge must also be disseminated to users, regardless of whether they are healthcare professionals or not, says Åsa Ingvar.

Low threshold increases burden on health care

Another limitation is that apps often have a low threshold when it comes to alerting users to skin changes.

– App makers don’t want to risk missing a malignant melanoma, for example, so they prefer to advise seeking care more often rather than less often. This could mean increasing the burden on traditional care by increasing the proportion of unnecessary visits that don’t provide added medical value in the form of improved medical condition, says Asa Ingvar.

Help from the expert committee

In the second study, the researchers recruited a panel of experts to identify requirements and benchmarks that should be applied to AI-based dermatology applications.

The committee concluded, among other things, that it should be clear how the apps were tested and trained, what clinical studies may have been conducted, and what limitations the app faces.

“Like many others, I have high hopes for the use of AI in healthcare. But AI-based applications have limitations when it comes to detecting malignant melanoma. We hope that clearer guidelines will lead to increased quality,” says Åsa Ingvar.

Text: Rebecca Sjoberg

The article was originally published in the journal Science & Health.

Scientific studies:

Examining guidelines for labelling AI-based software as a medical device: a review and analysis of mobile dermatology applications in AustraliaAustralian Journal of Dermatology.

Minimum requirements for labeling AI-based software for dermatology as a medical device: a consensus statement, Australian Journal of Dermatology.