IEEE Technology and Society Magazine - September 2022 - 42

Technological Stewardship and Responsible Innovation
over an A/IS, more willing to pay for healthcare provided
by a human relative to an A/IS, and preferred a
human healthcare provider relative to an A/IS. Qualifying
these results, they additionally found evidence
that participants' preferences were a consequence
of their belief in the uniqueness of their medical conditions
relative to others. Thus, health information
seekers might believe that A/ISs are generally accurate,
but cannot identify the complex conditions that
are defined by idiosyncratic features.
Even if users experience a form of algorithmic aversion
initially, a growing familiarity with technology
might alter this pattern, resulting in algorithmic preference.
Perceptions of consensus within communities
of experts (i.e., authority) or peers (i.e., social proof)
that technology has utility can reduce user uncertainty.
In the case of folkmedicine, users might look to
expert endorsements (e.g., " doctor recommended " ),
application ratings, and/or total downloads from websites
and app stores. Endorsements and peer ratings
illustrate an important feature of folkmedical technology
adoption: rather than using knowledge of the
technical affordances of an A/IS or their accuracy and
reliability, users employ heuristics (i.e., social proof
and source credibility) to determine whether they
adopt the technology. Thus, when a developer or distributor
claims that an app performs a specific function,
users might reasonably infer that it performs an
EPA. Similarly, users might also infer that social media
or video-sharing websites would not allow media that
presents misinformation or disinformation.
Subjective estimation of intelligence
If A/ISs can autonomously perform EPAs, we must
then consider how reliable these systems are (or can
be). Reliability for any given agent will be determined
by both the accuracy of the data set (or experience)
used to train the agent and the consistency of
an agent's operations. However, estimating reliability
can prove challenging. Humans are notoriously
bad at recognizing random patterns [42], estimating
the probability of lethal events (e.g., the prevalence
of disease) [43], understanding mortality based on
statistics [44], and identifying the co-occurrence of
a symptom and disease [45]. Estimates of likelihood
tend to be informed by judgments of similarity (representativeness),
affect, or the information's ready
availability in the environment or memory [1]. Thus,
healthcare technologies are as, if not more, likely to
be adopted for their prestige value rather than the
42
reliability of technical and functional affordances.
Here, I consider four brief case studies to illustrate
how applications are validated and how this promotes
perceptions of trust and distrust in users.
Formally, reliability information should be based
on validation studies and, ideally, this information
should undergo peer review within one or more
expert communities. It should also be publicly available.
This is not always the case, with studies often
reflecting " black data " that is inaccessible, leading
to a need to rely on alternative sources. For instance,
Binah.ai uses mobile phone-based photoplethysmography
to analyze how light is reflected off the face to
measure heart rate, blood pressure, oxygen saturation,
and heart rate variability. Although an internal
validation was conducted with 264 participants, this
information was not published, and it benchmarked
the device against low-accuracy device standards
[47].6 Moreover, the application does not use a calibration
method (e.g., blood pressure cuff). Despite
being an honoree of the Consumer Technology
Association (CES) 2022 Innovation Award Product,
negative app store feedback for such applications
notes a lack of reliability in the readings as well as
the presence of advertisements. In the absence of
publicly accessible data and scientific peer review,
users must rely on user ratings that might not correspond
to the " intelligence " of the system.
Healthcare technologies can also straddle the
boundary of institutionalized and folkmedicine
using both expert endorsement and social proof.
For instance, the Miiskin skin and mole dermatology
tracking application boast that it has over 500,000
downloads and is HIPPA-compliant, has earned a
4.2-star rating in the Google App store (retrieved on
27 March 2022), is accredited by the Skin Health Alliance,
and provides financial support to skin cancer
organizations (e.g., it provides the Canadian Skin
Cancer Foundation (CSCF) with " 15% of [its] subscriptions "
[revenue]). This support is bidirectional.
On its website, the CSCF description of the application
proclaims: " Privacy & Independence: You don't
need anyone else's help to photograph your skin. The
Miiskin app will do that for you, " and " Automatic Skin
Imaging is powered by Computer Vision and Augmented
Reality helping you consistently take photos "
(italics added).7 This endorsement promotes trust in
6The author contacted the organization in March 3, 2022, and did not receive a copy
of the report as requested at the time of publishing.
7Retrieved March 27, 2022, from: https://www.canadianskin cancerfoundation.com/
prevention/miiskin-app/
IEEE Technology and Society Magazine
http://www.Binah.ai https://www.canadianskincancerfoundation.com/prevention/miiskin-app/ https://www.canadianskincancerfoundation.com/prevention/miiskin-app/
IEEE Technology and Society Magazine - September 2022

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - September 2022
