Over the past decade, artificial intelligence (AI) has emerged as a pivotal technology across numerous sectors, including healthcare. Algorithms are now capable of detecting anomalies in medical imaging, predicting pathological risks, and optimizing care pathways. This promise of more precise, faster, and more efficient medicine appeals to healthcare systems grappling with issues of rationalization, workforce shortages, and rising costs
[1]
-
[3]
. In hospitals, AI is increasingly integrated into diagnostic imaging
[4]
, predictive analytics
[5]
, and clinical decision-support systems
[6]
, suggesting a profound reconfiguration of medical practice and organizational governance.
However, behind this technological promise lies another, quieter yet equally crucial reality: the subjective, relational, and ethical transformations of medical work
[7]
-
[9]
. While many studies have focused on AI’s technical performance and its potential for efficiency
[1]
[10]
, fewer have examined how healthcare professionals—particularly physicians—experience, perceive, and internalize these tools in their daily practice. Recent literature underscores these dynamics across both organizational and patient-centered perspectives. For instance,
[2]
highlight AI’s capacity to enhance hospital workflow and clinical decision-making, while Siau and Wang
[11]
examine the challenges of trust and interpretability. Similarly, Longoni et al.
[12]
and Cahan
[13]
explore patients’ ambivalence toward algorithmic medicine, emphasizing the relational and ethical tensions at play. The literature often refers to “resistance to change” or “barriers to adoption,” but rarely addresses the felt vulnerability arising from the increasing automation of clinical judgment.
At the same time, the field of healthcare management has largely concentrated on performance metrics, data governance, and organizational optimization
[14]
[15]
, often overlooking the relational and identity-based dynamics triggered by this digital transformation. This theoretical silence represents a troubling blind spot in a field—medicine—that is fundamentally rooted in human relationships, ethical deliberation, and a unique relationship to knowledge and care.
In this context, the notion of vulnerability offers a powerful lens to analyze how AI’s promises reshape medical practice. Drawing from consumer vulnerability theory
[16]
, we approach vulnerability not as a fixed trait but as a situated condition, emerging from asymmetries of power, information, and control between humans and algorithmic systems. We identify three interrelated, yet distinct, dimensions of vulnerability that structure physicians’ and patients’ experiences in the age of AI:
By articulating these three dimensions, this article aims to clarify how AI operates as a performative force that not only reconfigures workflows and roles but also reshapes the moral and emotional architecture of care. AI does not merely assist medical reasoning—it acts as a new epistemic and relational actor, influencing how knowledge, trust, and responsibility are distributed in clinical practice.
Through a qualitative inquiry based on semi-structured interviews with physicians across diverse specialties, this study examines how these professionals navigate the tensions between efficiency and ethics, standardization and autonomy, and technological mediation and human connection. In doing so, it contributes to a more nuanced understanding of AI’s integration in healthcare—not as a purely technical innovation, but as a sociotechnical transformation that generates new forms of vulnerability, negotiation, and adaptation within medical organizations.
2. Literature Review: AI, Vulnerability, and Medical Decision-Making
Increasingly used for diagnosis and patient management, Artificial Intelligence (AI) is poised to reshape the dynamics of medical decision-making
[10]
[17]
[18]
. In particular, machine learning algorithms enable the analysis of vast medical datasets and the detection of complex correlations that might elude human reasoning
[1]
[4]
.
However, the automation of certain clinical decisions tends to diminish the role of human judgment in favor of advanced statistical models
[19]
. While some scholars argue that this evolution allows for more objective, evidence-based medicine
[15]
, others highlight the risk of excessive reliance on algorithms and the erosion of physicians’ autonomy
[20]
, with consequences for the quality of patient care.
Historically, physicians have relied on clinical experience and personal judgment to make diagnostic decisions. AI introduces a new form of rationality—algorithmic—which may conflict with medical expertise
[1]
. Several studies have pointed out the tension between the algorithmic rationality of AI and the holistic reasoning typical of human clinical practice
[15]
[19]
, but few have examined how healthcare professionals perceive a form of alienation when constrained by AI tools, leading to a sense of vulnerability
[21]
[22]
. While AI can reduce diagnostic uncertainty, it often standardizes practice and downplays clinical intuition and accumulated physician experience
[20]
.
In the field of management, the work of Baker
[16]
on consumer vulnerability provides a complementary lens for analyzing physicians’ vulnerability in the context of AI. Their framework highlights several dimensions of vulnerability that are also relevant to the medical domain.
Baker
[16]
emphasize that vulnerability is not a fixed characteristic of an individual but rather a state that can be heightened or reduced depending on context and available resources. Physicians are not inherently vulnerable to AI, but the introduction of this technology into their practice can place them in a state of contextual vulnerability. For example, an experienced radiologist may feel comfortable using AI in image interpretation, whereas another physician less familiar with new technologies might feel a loss of control in the face of automated decisions.
Baker
[16]
also note that vulnerability can be intensified when individuals lose autonomy and the capacity to make choices, especially in contexts marked by high information asymmetry. The integration of AI into medical decision-making creates a situation in which physicians may lose autonomy to algorithms they are expected to follow. If an algorithm recommends a diagnosis or treatment based on probabilistic calculations without clear explanation, the physician may feel compelled to accept the suggestion by default, unable to exercise clinical judgment fully.
This parallels consumer vulnerability in digital marketing, where recommendation algorithms influence choices without users fully understanding the underlying criteria (e.g., Amazon, Netflix, or Facebook recommendations).
Furthermore, Baker
[16]
point out that vulnerability can provoke intense emotional responses, ranging from frustration and anxiety to outright rejection of technology perceived as threatening. Some physicians may experience a loss of self-confidence as their role is reduced to merely validating automated diagnoses. This can result in professional stress and burnout, particularly if practitioners feel they no longer control their own expertise or play a key role in the caregiver–patient relationship. Just as some consumers avoid online shopping due to mistrust of recommendation algorithms, some physicians may resist adopting AI out of fear that technology could take over their profession.
One frequently cited risk in the literature is excessive dependence on algorithms, potentially leading to automation bias—a phenomenon well-documented in cognitive psychology and behavioral economics. Here, humans tend to follow automated recommendations unquestioningly, even when contradictory evidence is present
[23]
[24]
. In other words, physicians may be less inclined to challenge AI suggestions, thereby diminishing their critical role and their ability to tailor decisions to the patient’s specific context
[22]
.
Recent studies also raise concern about shifting legal responsibility. If a diagnosis is incorrect due to a biased algorithm, who is accountable—the physician or the AI developer?
[25]
. This legal ambiguity further reinforces physicians’ sense of vulnerability
[26]
.
Another major issue highlighted in the literature is the perceived transparency of algorithmic decisions. Several researchers in marketing and healthcare management have shown that patient trust depends heavily on understanding the medical decisions that affect them
[27]
-
[32]
. If AI-generated recommendations are perceived as opaque or arbitrary, patients may lose trust in the healthcare system
[33]
-
[35]
. This phenomenon is often referred to as the “black-box problem,” highlighting the fact that even algorithm developers cannot always fully explain how AI reaches certain diagnostic conclusions
[36]
.
In healthcare management, it is crucial to adopt a human-centered approach that combines the benefits of AI with the preservation of the physician’s role as a key actor in the therapeutic relationship
[32]
[37]
.
Strategies from behavioral economics could also be applied to encourage responsible AI use in medicine. For instance, nudges
[38]
designed to prompt physicians to manually validate AI suggestions could help mitigate automation bias. Similarly, greater transparency in algorithm development and explanation is essential for maintaining trust among both healthcare professionals and patients.
The challenge, therefore, is not merely technological but also organizational and ethical. To ensure that AI in medicine does not become an additional source of fragility for caregivers and patients alike, its integration must be rethought in a more collaborative way—one that is sensitive to the realities of medical practice.
3. Methods
This study adopts a qualitative design based on semi-structured interviews conducted with 20 French physiciansworking across a diverse range of medical specialties, including psychiatry, genetics, radiology, pathology, anesthesiology, and other clinical domains such as oncology, nephrology, cardiology, urology, gastroenterology, and surgery (see
Table 1
).
Sampling Strategy and Participants
A purposive sampling approach was employed to ensure the inclusion of participants with varied professional backgrounds, levels of seniority, and degrees of exposure to artificial intelligence (AI). This strategy was chosen to capture a diversity of perspectives across different medical contexts while remaining feasible within the project’s time constraints. Selection criteria included:
The final sample consisted of 10 male and 10 female physicians, aged 30 to 70, practicing mainly in Paris (n = 14) and other regions of France (n = 6). A majority worked in public institutions (n = 15), with a smaller subset in private practice (n = 5). Eighteen participants expressed a generally favorable attitude toward AI, while eight reported direct professional experience with AI tools (see
Table 1
for full details).
<xref ref-type="bibr" rid="scirp.146722-"></xref>Table 1. Characteristics of participants.
Data collection was conducted between November 2022 and March 2023 by the lead investigator. Participation was voluntary, and informed consent was obtained via email or written message prior to each interview. Physicians were first contacted by telephone to explain the research objectives and confirm their willingness to participate. Each semi-structured interview lasted between 60 and 90 minutes and was audio-recorded with participants’ permission.
The interview guide explored three core themes:
1) Physicians’ perceptions of AI integration in their daily practice;
2) The impact of AI on clinical decision-making and autonomy;
3) The relational and ethical implications of AI for the doctor–patient relationship.
The analysis of the interviews reveals that in France, the introduction of artificial intelligence (AI) into medical practice continues to raise deep concerns among physicians, particularly across three essential dimensions: decision-making, therapeutic, and relational. While these concerns are shared across various medical specialties, they manifest differently depending on the clinicians’ specific field of practice (see
Figure 1
).
<xref ref-type="bibr" rid="scirp.146722-"></xref>Figure 1. Context of AI use and typology of vulnerability.
4.1. Decision-Making Vulnerability
One of the key findings from the interviews is the fear of excessive delegation of medical decision-making to AI algorithms, which could reduce the physician’s role to that of a mere validator. Several practitioners expressed concern that AI might gradually distance them from their own clinical reasoning: “Personally, I wouldn’t feel comfortable letting an AI system make a diagnosis 100% on its own. Without a physician’s co-signature, we completely lose control.” (F5, Gastroenterologist) This concern reflects a fear that the physician’s role could be reduced to a mere technical executor of algorithmic decisions: “AI is useful, but I refuse to become a simple operator who presses a button and follows a protocol dictated by a machine.” (F10, Pathologist) The expression “simple operator” conveys a fear of deskilling the clinical act—where human expertise, grounded in experience, reasoning, and contextual judgment, is replaced by the mechanical application of AI-generated protocols.
Some practitioners emphasize that AI is not yet reliable enough to make definitive diagnoses and must always be accompanied by medical judgment: “AI works well with standard cases, but when it comes to complex or rare situations, it quickly reaches its limits. I still prefer to rely on my clinical intuition and experience.” (F4, Cardiologist)
This statement highlights perceived limitations of AI, particularly its inability to manage clinical uncertainty and complexity. While AI excels in typical, well-documented cases, it struggles in atypical, rare, or evolving situations—contexts where human expertise remains indispensable. The cardiologist underlines the importance of clinical judgment, shaped by intuition and experience, as a fundamental pillar of decision-making. Algorithmic tools cannot fully grasp the contextual, emotional, and intersubjective nuances of real-life cases.
Other testimonials point to an emerging facet of decision-making vulnerability: the erosion of the physician’s authority in the face of algorithmically generated information accessible to the general public: “With the internet and AI, patients come to consultations already having their own diagnosis based on online algorithms. In the end, who gets the final say?” (F5, Gastroenterologist) This physician voices concern about the proliferation of “pre-packaged” diagnoses generated by digital tools or online searches, which may disrupt the traditional hierarchy of medical knowledge. This situation could dilute the practitioner’s decision-making power, create tension in the doctor–patient relationship, and increase uncertainty around responsibility. The rhetorical question “who gets the final say?” reflects anxiety about a potential loss of legitimacy and the need to redefine the physician’s role in an era of hyper-accessible medical information.
Many physicians also fear that their role may evolve into that of mere validators of algorithmic decisions, potentially impacting the prestige and value of the profession. The progressive deprofessionalization of medicine in the face of AI’s rise is evident in the testimony of oncologists: “I’m afraid we’ll become technicians who just apply the recommendations of an algorithm instead of clinicians who make complex decisions.” (F7, Oncologist) This specialist fears a gradual replacement of the clinician’s cognitive autonomy—essential for handling complex and evolving cases—by a technical role, in which the physician simply executes algorithmic suggestions with limited room for critical analysis. This shift is especially troubling in oncology, where patient experience, comorbidities, and personal preferences play a crucial role in therapeutic decision-making. Another specialist directly questions the future role of the physician in tomorrow’s medical ecosystem: “If AI becomes too dominant, won’t the physician’s role be reduced to just validating or signing off on decisions made by the machine?” (H6, Pathologist)
This pathologist voices concern about a reversed decision-making process: rather than producing informed decisions based on clinical and scientific analysis, physicians may be relegated to a control or certification role for algorithmically generated decisions. This shift raises deep ethical and professional issues, particularly regarding the dilution of medical responsibility and the loss of individual critical thinking. The sense of becoming a mere “validator” reflects an identity-based vulnerability in the face of automation and underscores the urgent need for safeguards to preserve the physician’s decision-making sovereignty in interpreting and applying AI-generated recommendations.
Another aspect of decision-making vulnerability concerns the fear of overly rigid medicine, where algorithms impose standardized care protocols that may conflict with the uniqueness of individual patients. “AI forces excessive standardization of protocols, but every patient is unique. Sometimes you need to step outside the box to truly provide good care.” (F5, Gastroenterologist)
This perspective echoes concerns shared by physicians in medical genetics, where algorithms can detect predispositions to diseases, but interpreting those results is fundamentally a matter of human explanation and support: “AI can give you a probability of genetic risk, but a diagnosis cannot be delivered by a machine. There’s a human dimension that algorithms just can’t handle.” (F11, Geneticist)
Some physicians also highlight a paradoxical shift: while AI is supposed to reduce their workload by automating certain tasks, it may actually burden them with new administrative and technical responsibilities, distancing them even further from their patients. “They tell us AI will save us time, but in reality, I’m already spending more time verifying software-generated reports than talking to my patients. If this continues, I’ll become a doctor who looks at screens instead of people.” (F12, Radiologist)
Cardiologists express similar concerns: “As a cardiologist, I’ve seen diagnostic support tools introduced that were supposed to speed up our work. What ends up happening is we spend more time analyzing and validating machine-produced results instead of examining the patient. In the end, I don’t see any time savings.” (F4, Cardiologist)
Rather than freeing time to enhance the quality of the caregiver–patient relationship, AI can lead to the hyper-rationalization of clinical processes, where physicians become more like data managers than engaged clinicians practicing individualized, human-centered medicine. This testimony expresses a sense of professional disillusionment, emphasizing how the promise of efficiency clashes with a reality in which the complexity of clinical cases and the constant need to validate algorithmic outputs hinder the spontaneity and fluidity of medical practice. The cardiologist’s account also illustrates a persistent tension: although diagnostic support tools are introduced to streamline medical work, they can in fact create cognitive and technical overload, shifting the physician’s focus away from the patient and toward the screen.
Nevertheless, some practitioners also view AI as an opportunity to refocus their work on the most human aspects of care: “In the end, AI could free up time so we can focus on what really matters: talking to patients, supporting them. But only if we’re not completely overwhelmed by managing all the data these tools generate.” (F5, Gastroenterologist)
The two main characteristics of decision-making vulnerability have been synthesized in
Figure 2
.
<xref ref-type="bibr" rid="scirp.146722-"></xref>Figure 2. The main dimensions of decision-making vulnerability.
4.2. Relational Vulnerability: Concern about the Dehumanization of Care
One of the most frequently mentioned concerns by the physicians interviewed for our study is that AI could progressively diminish human contact in the caregiver–patient relationship (see
Figure 3
). While medicine has historically been grounded in direct interaction between practitioner and patient, the increasing automation of diagnostic and decision-making tasks may relegate the physician to a more secondary role, reducing time spent on communication and patient support. This transformation is perceived as a threat to the quality of care. “A consultation is not just a data exchange. There’s eye contact, body language, even silent moments that help build trust. If tomorrow everything goes through machines, something essential will be lost.” (H9, Psychiatrist) This psychiatrist’s statement reminds us that medical consultations are spaces of subtle communication that go beyond factual information exchange. The gaze, silence, and posture of the doctor are all micro-relational cues that help build therapeutic trust. This intersubjective bond—especially vital in fields such as psychiatry—cannot be captured or replicated by a machine. The testimony implicitly criticizes the risk of AI impoverishing the clinical encounter by reducing it to a cold, mechanical interaction devoid of the human warmth essential to a therapeutic alliance.
Other psychiatrists express even deeper concerns about the impact of AI on therapeutic listening and non-verbal communication. For them, the very essence of psychiatry lies in attentive listening, nuance, and the subjectivity of the patient’s speech—elements no algorithm can fully grasp: “Psychiatry is about human connection—what happens between two people in a closed space. If tomorrow I’m told that an AI can analyze a patient’s emotions based on language or micro-expressions, I’ll be skeptical. Understanding psychological suffering isn’t just about data analysis.” (H10, Psychiatrist) This statement calls attention to a truth often overshadowed by tech-centric discourse: psychiatry is fundamentally about human encounter, in an intersubjective space where non-verbal cues, silences, ambivalence, and implicit emotions are as important as spoken words. The idea that AI could “analyze” emotions based on data is seen here as an overly reductive view of human complexity. This quote denounces the fetishization of data, which risks overlooking the fact that understanding psychological distress involves co-constructed meaning, empathetic presence, and intuitive capacities that only human beings can mobilize within a therapeutic framework.
Another testimony illustrates this critique through a concrete example: algorithmic quantification of anxiety disorders. “If an anxious patient tells me about their fears and I reply, ‘Your anxiety level is 72% according to the algorithm,’ we’re heading straight for disaster. Care is about connection and presence—not just algorithmic interpretation.” (H13, Psychiatrist). Responding to a patient with a percentage from an algorithm, according to this psychiatrist, denies the subjective and relational nature of care. While objective, such logic undermines the therapist’s containing function and may even exacerbate distress by invalidating the patient’s emotional experience. As he reaffirms, care is not just a technical assessment—it rests on presence, listening, empathy, and connection.
These testimonies underscore the structural inability of AI to replace the human depth of psychological care. They reinforce the idea that, if not integrated with caution, AI could undermine the very foundations of the therapeutic relationship in psychiatry—one based on co-presence, subjectivity, and mutual recognition.
This perception is echoed by oncologists, who reinforce this view from another angle: the need for human contextualization in communicating diagnoses and treatment plans. While AI may offer probabilities of treatment efficacy, it cannot address the existential, emotional, and social dimensions of medical decision-making. What matters is not only the accuracy of the prognosis, but the physician’s ability to adapt information to the patient’s lived experience, environment, values, and preferences.
“A cancer patient doesn’t just want to hear they have an 85% chance of responding to chemotherapy. They need to understand what that means for their daily life, for their family. AI can give us numbers, but it can’t replace the work of support.” (H8, Oncologist)
“In oncology, we must personalize treatment and take into account the patient’s overall situation. An algorithm might say there’s a 90% chance of success, but it can’t replace the discussion with the patient about their wishes or their tolerance for side effects.” (F7, Oncologist)
Between the lines, these statements reveal that AI, by taking over certain medical tasks, may disrupt the protective asymmetry of the care relationship—where the physician guides, supports, and humanizes the diagnostic or therapeutic process. Excessive technicization could thus break the affective and communicational bond that is essential, particularly in serious conditions like cancer.
Together, these voices call for a rethinking of the physician’s role in the age of AI—not as a mere interpreter of algorithms, but as a vital intermediary between data and meaning, between statistics and the unique lived experience of each patient.
<xref ref-type="bibr" rid="scirp.146722-"></xref>Figure 3. The main dimensions of relational vulnerability.
4.3. Therapeutic Vulnerability of the Patient: Loss of Reference Points, Sense of Isolation, and Potential Disengagement
Beyond the experiences of healthcare providers, the analysis of interview transcripts highlights an increased vulnerability on the patient’s side (see
Figure 4
), driven by several factors: the lack of understanding surrounding automated medical decisions, which reinforces feelings of powerlessness; the loss of the physician as an embodied point of reference in an increasingly complex technical environment; and the fear of being judged or overlooked by dehumanized systems—particularly in situations involving anxiety, uncertainty, or the need for reassurance.
“When a patient no longer trusts the human interlocutor, they feel alone in front of the machine.” (H13, Psychiatrist)
This therapeutic vulnerability can manifest in various ways: patient disengagement from their own care, blind adherence to AI-generated recommendations perceived as indisputable, or growing distrust toward a technocentric form of medicine.
Several practitioners also noted that patients themselves express reluctance when their care is delegated to algorithms, especially when they experience a lack of explanation or miss the direct contact with a physician.
“Patients want to see a doctor—not a screen telling them their risk score is 85%.” (F4, Cardiologist)
“Patients want dialogue. When someone is anxious about a diagnosis, they want time and explanation. They don’t want a number thrown at them with no context.” (F5, Gastroenterologist)
In summary, these interviews reveal several major concerns about the role of AI in the caregiver–patient relationship:
Reduced dialogue time: There is a recurring fear that physicians will become increasingly detached from their patients as they shift toward monitoring digital tools. Some express frustration at seeing their profession evolve into data management rather than human connection.
Erosion of the medical role: The transformation of the physician into a mere validator of AI-generated diagnoses raises concerns, especially among specialists whose practice relies heavily on interaction and communication (psychiatry, gastroenterology, oncology).
Negative patient perception: Testimonies suggest that some patients already perceive a growing distance between themselves and their doctors. Overreliance on screens and digital tools at the expense of human interaction may erode patient trust and satisfaction.
A paradoxical evolution: Although AI is expected to free up clinical time, in practice it may instead add new validation and supervision tasks. This creates a sense of incoherence and fatigue among physicians, who feel caught between a technologized model of care and their original vocation as healers.
Overall, the findings of this section show that while AI can be a valuable tool, it must be used carefully to avoid compromising the quality of the doctor–patient relationship. The interviewed physicians stress the importance of preserving human interaction and ensuring that AI does not distance them from their primary role: to support and listen to their patients.
The collected testimonies highlight a strong tension between the perceived benefits of AI and the fears associated with its use. While physicians acknowledge the technological advances and improved accuracy AI offers, they also emphasize the need to preserve the human dimension of care and maintain decision-making autonomy in the face of algorithmic recommendations.
AI should not be a substitute for clinical reasoning but rather a decision-support tool—one that leaves the final judgment to the physician and safeguards the integrity of the doctor–patient relationship, which remains the cornerstone of medical practice.
5. Discussion and Managerial Implications
This study set out to explore physicians’ perceptions of AI and to identify the multidimensional vulnerabilities it generates—particularly within decision-making,
<xref ref-type="bibr" rid="scirp.146722-"></xref>Figure 4. The main dimensions of therapeutic vulnerability.
relational, and therapeutic domains. Existing literature on AI in healthcare (Topol, 2019; Jiang et al., 2021) emphasizes the anticipated benefits of these technologies, particularly regarding diagnostic precision and administrative task automation. However, it only marginally addresses the indirect effects AI may have on the doctor–patient relationship and on clinical decision-making—two dimensions that are central to medical practice.
Our study provides a novel contribution by highlighting the tensions physicians perceive in relation to AI and by deepening our understanding of their sense of vulnerability. More specifically, we show that these vulnerabilities do not stem from technophobic resistance, but from legitimate concerns about losing decision-making control and about the dehumanization of care.
Our findings reveal a strong fear of losing decision-making autonomy in the face of increasingly powerful algorithms. This fear is particularly pronounced given that medical practice historically relies on decisions that combine objective data (e.g., symptoms, imaging, lab results) with subjective and contextual factors (e.g., patient history, clinical intuition, practitioner experience). This finding resonates with the research of Perrin (2023), which shows that automation in clinical decisions may diminish physicians’ capacity to exercise professional judgment. Moreover, AI could exacerbate a phenomenon of responsibility displacement, where practitioners might feel inclined to follow algorithmic recommendations without critically assessing their relevance.
Thus, our study suggests that AI, far from remaining a mere decision-support tool, can progressively frame and guide medical decisions, to the extent that it limits physicians’ autonomy. This increased standardization could deeply alter the cognitive dynamics of medical practice, reducing the space for clinical intuition and pushing toward more protocol-driven care. Such dynamics illustrate the emergence of a decision-making vulnerability, where clinical reasoning risks being constrained by algorithmic rationalities.
The second key vulnerability emerging from our study is the fear of deterioration in the doctor–patient relationship, exacerbated by the growing presence of AI in the care process. This concern aligns with broader debates on telemedicine and algorithmic healthcare, which tend to reduce human interaction in favor of more automated management (Matuchansky, 2019).
Our findings show that the doctor–patient relationship is perceived as a fundamental component of care—one that is increasingly at risk in the digital age. AI is seen as a powerful tool, but one that may transform the physician’s role into that of a technician, simply validating algorithm-generated decisions. Several physicians express deeper concerns about the progressive erasure of the human element in caregiving:
“A human being in contact with another human being co-creates care. The danger with AI is that all of that could disappear.” (Psychiatrist)
“If patients no longer come to see a doctor but instead a machine that gives them a diagnosis, where is the care relationship?” (Nephrologist)
This perspective underlines the growing salience of relational vulnerability as a core concern in the algorithmic transformation of healthcare. This perception echoes the work of Pelluchon (2009) on bioethics and the care relationship, which argues that medical acts are not merely technical interventions but rest on trust and listening, which can be crucial for therapeutic effectiveness.
While management and health systems literature often frames AI around performance and care rationalization (Jiang et al., 2021), our study shifts the perspective by emphasizing subjective and relational dimensions of vulnerability.
These findings can be further illuminated through the conceptual framework of consumer vulnerability proposed by Baker et al. (2005), which defines vulnerability as a state of powerlessness resulting from imbalances in marketplace interactions. Transposed to healthcare, this framework reveals three interrelated dimensions:
By mapping our results onto this framework, we show that AI-induced vulnerabilities extend beyond technical or ethical concerns—they reshape the experiential and relational fabric of medical practice.
In line with Actor–Network Theory (ANT) (Latour, 2005; Callon, 1986), we conceptualize AI as an actant—that is, a non-human entity that participates in shaping social and organizational practices. As such, AI is not a neutral instrument but an active agent that redistributes power, responsibility, and authority among physicians, patients, and healthcare institutions. This performative capacity of AI—as both promise and constraint—explains why vulnerability emerges not as a static state, but as an evolving relational condition embedded in socio-technical assemblages.
Managerial Implications
The identification of decision-making, relational, and therapeutic vulnerabilities calls for a strategic, ethical, and organizational response. Hospital managers and AI developers must move beyond technical optimization to embrace human-centered governance models that reinforce autonomy, empathy, and transparency.
Preserving Clinical Autonomy and Accountability: To preserve physicians’ clinical autonomy, AI systems should be designed according to Explainability by Designprinciples. Explainable AI (XAI) interfaces that provide contrastive explanations (“why A rather than B”) can support physicians’ understanding and accountability. Continuous ethical training programs should reinforce critical reasoning, bias awareness, and responsibility-sharing between humans and machines.
Safeguarding Empathy and Communication: Relational vulnerability requires strategies that safeguard empathy and communication. Hospital leaders should promote hybrid care models that combine digital efficiency with human interaction, and establish communication protocols enabling physicians to translate algorithmic recommendations into patient-friendly language, thus maintaining trust and shared decision-making.
Empowering Patients Through Participatory Design: Therapeutic vulnerability, affecting patients’ capacity to understand and engage in their care, calls for participatory and inclusive approaches. Managers should foster co-design processes involving physicians, patients, and engineers, ensuring tools are accessible, comprehensible, and aligned with ethical standards. Establishing ethical oversight committees can ensure that AI integration respects fairness, privacy, and patient agency.
Turning Vulnerability into Strategic Reflexivity: Finally, vulnerability should not be viewed solely as a constraint, but as a strategic resource. By fostering reflexive governance—regular feedback loops, interdisciplinary dialogue, and participatory design—health organizations can turn these vulnerabilities into drivers of resilience, trust, and innovation.
Together, these implications contribute to a broader understanding of how AI reconfigures professional identities, governance logics, and ethical responsibilities in healthcare organizations. Such an approach not only strengthens physicians’ autonomy and patient trust but also provides a foundation for theoretical advancement.
Building on these managerial insights, our study contributes to the broader literature in three significant ways:
A New Conceptualization of Physicians’ Decision-Making Vulnerability
While many studies approach AI through the lens of efficiency or diagnostic performance, our research sheds light on physicians’ contextual and experiential vulnerability—the fear of seeing their clinical autonomy weakened by tools whose logic and limits they do not fully understand. Unlike classical technology adoption models (Davis, 1989; Venkatesh et al., 2003), we do not treat AI as a neutral tool but as an actant that reconfigures power and responsibility in medical decision-making.
The Emergence of Relational Vulnerability as a Managerial Issue
By identifying the fear of diminishing human connection in favor of technicized processes, our study brings to light a relational vulnerability that is often ignored in traditional models of digital transformation. This contribution aligns with an ethical and humanistic vision of healthcare management, in line with Zielinski’s (2011) work on vulnerability as an ethical lever.
Repositioning the Physician’s Role in a Hybrid System
Our results show that physicians do not reject AI outright, but rather question how their role will be redefined within a logic of human–machine hybridization. Far from the myth of “disruption,” they call for a cooperative model in which AI serves as a cognitive assistant or co-pilot (Wahl et al., 2018), not a replacement. This idea extends work in strategic management on human–machine complementarity (Parker & Wills, 2022).
Our findings suggest that physicians are currently in a phase of partial acceptance—many express hesitations, and actual adoption remains limited due to fears of autonomy loss. Large-scale adoption seems distant, underscoring the need to rethink support strategies for a more harmonious integration of AI in clinical practice.
Given the challenges posed by AI, it is essential to ensure complementarity between artificial and human intelligence. Rather than opposing AI to physicians, we should work toward ethically and pragmatically integrating these technologies into clinical routines. Our results suggest that AI should not replace, but rather enhance medical autonomy and the quality of the therapeutic relationship.
The future of digital medicine will depend on how physicians appropriate these tools—and on the ability of health systems to preserve the humanistic essence of care.
Far from demonizing or glorifying AI, our study proposes a nuanced analytical framework grounded in physicians’ lived experiences. It reminds us that the acceptability of a technology cannot be separated from the subjective, ethical, and relational conditions of its integration. As such, it calls on policymakers, AI designers, and hospital managers to treat the implementation of AI as not just a technical challenge, but also a human and organizational one, a challenge that demands continuous reflection on autonomy, equity, and the humanistic essence of care.
6. Limitations and Future Research
This study has certain limitations. Most notably, our analysis reflects physicians’ perceptions of patient vulnerability, which provides an indirect perspective. Future research should incorporate patients’ voices through interviews, focus groups, or ethnographic observation to capture lived experiences of algorithmic care.
Moreover, the sample—while diverse in specialty—was geographically concentrated in France. Comparative studies across healthcare systems could reveal how cultural and institutional contexts shape perceptions of AI-related vulnerability.
7. Conclusion
Our study underscores that AI integration in medicine cannot be reduced to technical deployment—it demands relational, ethical, and organizational adaptation. Physicians’ experiences reveal that vulnerability is not merely a side effect of innovation but a structural consequence of AI’s performative promises. A balanced approach—anchored in explainability, shared decision-making, and co-construction of care—will be key to ensuring that AI enhances, rather than erodes, the human dimension of medicine.
References
Topol, E.J. (2019) High-Performance Medicine: The Convergence of Human and Artificial Intelligence. Nature Medicine, 25, 44-56. >https://doi.org/10.1038/s41591-018-0300-7
Rajkomar, A., Dean, J. and Kohane, I. (2019) Machine Learning in Medicine. New England Journal of Medicine, 380, 1347-1358. >https://doi.org/10.1056/nejmra1814259
Davenport, T. and Kalakota, R. (2019) The Potential for Artificial Intelligence in Healthcare. Future Healthcare Journal, 6, 94-98. >https://doi.org/10.7861/futurehosp.6-2-94
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., et al. (2019) A Guide to Deep Learning in Healthcare. Nature Medicine, 25, 24-29. >https://doi.org/10.1038/s41591-018-0316-z
Miotto, R., Wang, F., Wang, S., Jiang, X. and Dudley, J.T. (2018) Deep Learning for Healthcare: Review, Opportunities and Challenges. Briefings in Bioinformatics, 19, 1236-1246. >https://doi.org/10.1093/bib/bbx044
Jiang, Y., Qiu, B., Xu, C. and Li, C. (2017) The Research of Clinical Decision Support System Based on Three-Layer Knowledge Base Model. Journal of Healthcare Engineering, 2017, Article ID: 6535286. >https://doi.org/10.1155/2017/6535286
de Margerie-Mellon, C. and Chassagnon, G. (2023) Artificial Intelligence: A Critical Review of Applications for Lung Nodule and Lung Cancer. Diagnostic and Interventional Imaging, 104, 11-17. >https://doi.org/10.1016/j.diii.2022.11.007
Sasseville, M., et al. (2023) Risk of Bias Mitigation for Vulnerable and Diverse Groups in Community-Based Primary Health Care Artificial Intelligence Models: Protocol for a Rapid Review. JMIR Research Protocols, 12, e46684.
Pellegrini, E., Ballerini, L., Hernandez, M.d.C.V., Chappell, F.M., González‐Castro, V., Anblagan, D., et al. (2018) Machine Learning of Neuroimaging for Assisted Diagnosis of Cognitive Impairment and Dementia: A Systematic Review. Alzheimer’s&Dementia: Diagnosis, Assessment&Disease Monitoring, 10, 519-535. >https://doi.org/10.1016/j.dadm.2018.07.004
McKinney, S.M., et al. (2020) International Evaluation of an AI System for Breast Cancer Screening. Nature, 577, 89-94.
Siau, K. and Wang, W. (2020) Artificial Intelligence (AI) Ethics: Ethics of AI and ethical AI. Journal of Database Management, 31, 74-87. >https://doi.org/10.4018/jdm.2020040105
Longoni, C., Bonezzi, A. and Morewedge, C.K. (2019) Resistance to Medical Artificial Intelligence. Journal of Consumer Research, 46, 629-650. >https://doi.org/10.1093/jcr/ucz013
Cahan, E.M., Hernandez-Boussard, T., Thadaney-Israni, S. and Rubin, D.L. (2019) Putting the Data before the Algorithm in Big Data Addressing Personalized Healthcare. npj Digital Medicine, 2, Article No. 78. >https://doi.org/10.1038/s41746-019-0157-2
Lee, S.M. and Trimi, S. (2021) Convergence Innovation in the Digital Age and in the COVID-19 Pandemic Crisis. Journal of Business Research, 123, 14-22. >https://doi.org/10.1016/j.jbusres.2020.09.041
Obermeyer, Z. and Emanuel, E.J. (2016) Predicting the Future—Big Data, Machine Learning, and Clinical Medicine. New England Journal of Medicine, 375, 1216-1219. >https://doi.org/10.1056/nejmp1606181
Baker, S.M., Gentry, J.W. and Rittenburg, T.L. (2005) Building Understanding of the Domain of Consumer Vulnerability. Journal of Macromarketing, 25, 128-139. >https://doi.org/10.1177/0276146705280622
Fogel, A.L. and Kvedar, J.C. (2018) Artificial Intelligence Powers Digital Medicine. npj Digital Medicine, 1, Article No. 5. >https://doi.org/10.1038/s41746-017-0012-2
Zhang, K., Liu, X., Shen, J., Li, Z., Sang, Y., Wu, X., et al. (2020) Clinically Applicable AI System for Accurate Diagnosis, Quantitative Measurements, and Prognosis of COVID-19 Pneumonia Using Computed Tomography. Cell, 181, 1423-1433.E11. >https://doi.org/10.1016/j.cell.2020.04.045
Char, D.S., Shah, N.H. and Magnus, D. (2018) Implementing Machine Learning in Health Care—Addressing Ethical Challenges. New England Journal of Medicine, 378, 981-983. >https://doi.org/10.1056/nejmp1714229
Verghese, A., Shah, N.H. and Harrington, R.A. (2019) What This Computer Needs Is a Physician. JAMA, 319, 19-20. >https://doi.org/10.1001/jama.2017.19198
Bankins, S. and Formosa, P. (2023) The Ethical Implications of Artificial Intelligence (AI) for Meaningful Work. Journal of Business Ethics, 185, 725-740. >https://doi.org/10.1007/s10551-023-05339-7
Walsh, T., Levy, N., Bell, G., Elliott, A., Maclaurin, J., Mareels, I.M.Y. and Wood, F.M. (2019) The Effective and Ethical Development of Artificial Intelligence: An Opportunity to Improve Our Wellbeing. Report for the Australian Council of Learned Academies. >https://acola.org/wp-content/uploads/2019/07/hs4_artificial-intelligence-report.pdf
Berner, E.S. and Graber, M.L. (2008) Overconfidence as a Cause of Diagnostic Error in Medicine. The American Journal of Medicine, 121, S2-S23. >https://doi.org/10.1016/j.amjmed.2008.01.001
Harris, A. (2020) COVID-19—School Leadership in Crisis? Journal of Professional Capital and Community, 5, 321-326. >https://doi.org/10.1108/jpcc-06-2020-0045
Gerke, S., Minssen, T. and Cohen, G. (2020) Ethical and Legal Challenges of Artificial Intelligence-Driven Healthcare. In: Bohr, A. and Memarzadeh, K., Eds., Artificial Intelligence in Healthcare, Elsevier, 295-336. >https://doi.org/10.1016/b978-0-12-818438-7.00012-5>https://ssrn.com/abstract=3570129
Afm Recherche et Applications en Marketing. Recherche et Applications en Marketing, 32, No. 3. >https://www.jstor.org/stable/26375584
Weiner, E.B., Dankwa-Mullan, I., Nelson, W.A. and Hassanpour, S. (2025) Ethical Challenges and Evolving Strategies in the Integration of Artificial Intelligence into Clinical Practice. PLOS Digital Health, 4, e0000810. >https://doi.org/10.1371/journal.pdig.0000810
Markus, A.F., Kors, J.A. and Rijnbeek, P.R. (2021) The Role of Explainability in Creating Trustworthy Artificial Intelligence for Health Care: A Comprehensive Survey of the Terminology, Design Choices, and Evaluation Strategies. Journal of Biomedical Informatics, 113, Article 103655. >https://doi.org/10.1016/j.jbi.2020.103655
Holzinger, A., Biemann, C., Pattichis, C.S. and Kell, D.B. (2017) What Do We Need to Build Explainable AI Systems for the Medical Domain? ArXiv Prepr.
Schelenz, L., Segal, A. and Gal, K. (2020) Applying Transparency in Artificial Intelligence Based Personalization Systems. ArXiv Prepr.
Yu, L. and Li, Y. (2022) Artificial Intelligence Decision-Making Transparency and Employees’ Trust: The Parallel Multiple Mediating Effect of Effectiveness and Discomfort. Behavioral Sciences, 12, Article 127. >https://doi.org/10.3390/bs12050127
Shin, D. (2020) User Perceptions of Algorithmic Decisions in the Personalized AI System:perceptual Evaluation of Fairness, Accountability, Transparency, and Explainability. Journal of Broadcasting&Electronic Media, 64, 541-565. >https://doi.org/10.1080/08838151.2020.1843357
Platt, J., Nong, P., Carmona, G. and Kardia, S. (2024) Public Attitudes toward Notification of Use of Artificial Intelligence in Health Care. JAMA Network Open, 7, e2450102. >https://doi.org/10.1001/jamanetworkopen.2024.50102
Quinn, T.P., Jacobs, S., Senadeera, M., Le, V. and Coghlan, S. (2022) The Three Ghosts of Medical AI: Can the Black-Box Present Deliver? Artificial Intelligence in Medicine, 124, Article 102158. >https://doi.org/10.1016/j.artmed.2021.102158
Xuereb, F. and Portelli, D.J.L. (2024) The Knowledge and Perception of Patients in Malta Towards Artificial Intelligence in Medical Imaging. Journal of Medical Imaging and Radiation Sciences, 55, Article 101743. >https://doi.org/10.1016/j.jmir.2024.101743
Rudin, C. (2019) Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead. Nature Machine Intelligence, 1, 206-215. >https://doi.org/10.1038/s42256-019-0048-x
van Leersum, C.M. and Maathuis, C. (2025) Human Centred Explainable AI Decision-Making in Healthcare. Journal of Responsible Technology, 21, Article 100108. >https://doi.org/10.1016/j.jrt.2025.100108
Hoyt, G.M. (2009) Nudge: Improving Decisions about Health, Wealth, and Happiness. International Review of Economics Education, 8, 158-159. >https://doi.org/10.1016/s1477-3880(15)30073-6
Smith, J.A. and Osborn, M. (2014) Interpretative Phenomenological Analysis as a Useful Methodology for Research on the Lived Experience of Pain. British Journal of Pain, 9, 41-42. >https://doi.org/10.1177/2049463714541642