The National Commission into the Regulation of AI in Healthcare has outlined a future in which healthcare AI is introduced through staged authorisation, continuous monitoring, transparency and meaningful human oversight. For community pharmacy, these recommendations arrive at a pivotal moment.
Artificial intelligence is moving rapidly from experimental healthcare projects into everyday clinical workflows. Its ability to analyse information, structure consultations and support clinical decisions is advancing—but technical capability alone will not secure adoption.
Healthcare AI must also earn the confidence of patients, clinicians, regulators and the public.
The independent National Commission into the Regulation of AI in Healthcare, established by the Medicines and Healthcare products Regulatory Agency (MHRA), has now published recommendations intended to strengthen the regulation of healthcare AI while supporting its safe adoption.
The Commission gathered evidence from more than 12,000 patients, clinicians, healthcare leaders, industry experts and members of the public. Its recommendations indicate how the United Kingdom may approach the next stage of healthcare AI regulation.
From one-off approval to continuous assurance
Traditional medical-device regulation has generally focused on assessing a product before it enters the market. AI creates a more complex challenge.
An AI-enabled system may be updated, encounter new patient populations or perform differently when introduced into real clinical environments. Its safety and effectiveness cannot necessarily be established through a single assessment at one point in time.
The Commission has therefore recommended continuous, real-world monitoring of AI-enabled medical devices throughout their operational life.
This represents an important change in regulatory thinking. Approval should not be viewed as the end of the safety process. Developers and healthcare organisations will need to understand how an AI system performs after deployment, identify emerging risks and demonstrate that appropriate action is taken when problems occur.
For AI developers, this means building monitoring, auditability, incident reporting and change control into the product from the beginning.
Staged authorisation for healthcare AI
One of the Commission’s most notable recommendations is a system of staged authorisation for new AI models.
The Commission compares this approach to learner-driver arrangements: an AI system could be permitted to operate within carefully defined restrictions while evidence is gathered through supervised, real-world use. Greater freedom could then be granted as the system demonstrates an acceptable level of safety and performance.
This could provide a more practical route between laboratory testing and unrestricted deployment.
Healthcare environments are complex, and controlled pilots can reveal risks that may not be apparent during technical development. However, pilots must have clearly defined boundaries, named clinical responsibility, escalation procedures and measurable safety outcomes. A pilot should not simply be an informal deployment under another name.
For community pharmacy, staged authorisation could support carefully controlled testing of AI within selected pharmacies, clinical pathways or patient groups while pharmacists retain responsibility for the final decision.
Transparency must become a practical requirement
The Commission also recommends public access to safety information about specific AI-enabled medical devices, including information about adverse incidents.
Patients are increasingly prepared to accept AI as part of their care, but that acceptance depends on knowing when it is being used, what role it plays and who remains responsible for the outcome.
Transparency cannot be reduced to a statement hidden within lengthy terms and conditions. It must be understandable and meaningful.
Patients should be able to distinguish between an AI system that structures information, one that makes recommendations and one that materially influences a diagnosis or treatment decision. They should also know when a healthcare professional reviews the system’s output and how concerns can be raised.
Responsible transparency is not a barrier to adoption. It is one of the conditions that makes adoption possible.
Why this matters for community pharmacy
Community pharmacy is assuming a larger clinical role within the NHS.
Through Pharmacy First and other commissioned services, pharmacists are assessing symptoms, applying clinical pathways, determining treatment eligibility, supplying medicines and providing safety-netting advice. This creates an important opportunity for AI to help structure consultations, identify missing information and support consistent clinical documentation.
It also creates risk if AI is introduced without adequate governance.
A system used within a pharmacy consultation may influence decisions about treatment, referral or urgent escalation. Its performance must therefore be considered in the context of the complete clinical workflow—not simply as a standalone algorithm.
Several questions become essential:
- What is the system’s intended clinical purpose?
- What information does it use?
- How are recommendations generated and presented?
- Can the pharmacist challenge or override the output?
- How are omissions, errors and safety incidents identified?
- How are model or system changes controlled?
- Does performance remain consistent across different patient groups?
- What information is recorded for later audit?
- Is the patient told that AI is being used?
These are not abstract regulatory questions. They affect how AI should be designed, purchased and used inside pharmacies.
Human oversight must be meaningful
The Commission’s emphasis on human oversight is especially relevant to pharmacy.
Simply placing a healthcare professional somewhere within the process does not automatically make an AI system safe. Oversight must be meaningful.
The pharmacist must have sufficient information to evaluate the AI output, the authority to reject it and enough time to exercise independent professional judgement. The interface should make uncertainty, limitations and missing information visible rather than presenting every recommendation with the same level of confidence.
Responsibility must also be clearly allocated. Pharmacists, pharmacy owners, technology suppliers and healthcare commissioners should understand their respective roles when an AI-supported decision contributes to patient harm.
The objective should not be to replace professional judgement. It should be to make that judgement better informed, more consistent and more auditable.
Designing for the regulatory direction of travel
The Commission’s recommendations are not yet legislation. The government and the MHRA will consider them before issuing a formal response.
Nevertheless, they provide a clear indication of the likely direction of travel.
Healthcare AI developers should prepare for an environment in which evidence continues to be collected after deployment, safety information becomes more visible and regulators expect stronger controls over system updates and real-world performance.
At PharmBot AI, these principles are central to the development of AIVAe. The platform is being designed around structured clinical support, auditable decision-making and pharmacist oversight throughout the consultation.
This is not only a question of regulatory readiness. It is about creating technology that pharmacists can use responsibly and that patients can trust.
Trust will determine which systems scale
The next phase of healthcare AI will not be defined only by which system appears most intelligent.
It will be defined by which systems can demonstrate a clear purpose, appropriate clinical oversight, reliable performance, transparent limitations and continuous evidence of safety.
For community pharmacy, the opportunity is substantial. AI could help pharmacists manage increasing clinical responsibility while improving consistency, documentation and access to care. But adoption must be supported by governance that reflects the realities of frontline practice.
The Commission’s recommendations bring the UK closer to a regulatory model capable of supporting both innovation and accountability.
That balance will ultimately determine whether healthcare AI remains a collection of promising experiments—or becomes trusted clinical infrastructure.
Read the full report:
https://assets.publishing.service.gov.uk/media/6aa01a2d62ec7fe7bedf517a/Full_National_Commission_Report_-_Publication.pdf