Your trusted CER partner

A7. Analysis of the clinical data – compliance to specific Essential Requirements

A7.1. Conformity assessment with requirement on safety

(MDD ER1 / AIMDD ER1)

The information materials supplied by the manufacturer (including label, IFU, available promotional materials including accompanying documents possibly foreseen by the manufacturer), should be reviewed to ensure they are consistent with the relevant clinical data appraised in stage 2 and that all the hazards, information on risk mitigation and other clinically relevant information have been identified appropriately.

Input from the risk management and the use of standards:

• Risk management documents should determine if all identified hazards are fully covered by harmonised standards or other relevant standards or if there are gaps needed to be covered by clinical data.
• Risk management documents should determine if all identified risks relating to patient treatment, method of operation of the device or risks relating to usability have been minimised or if there are question regarding clinical risks that need to be solved.
• Harmonised standards are generally expected to be applied in full in order to confer a presumption of conformity.
• If technical developments provide a higher level of safety than current harmonised standards, then the higher level of safety should be prioritised in order to meet the Essential Requirements on reducing the risks as far as possible, that risks must be compatible with a high level of protection of health and safety, and that side effects must be acceptable (MDD ER2 and ER3 and ER6; AIMDD ER1 and ER5).

Examples:

• Electrical hazards should be covered by compliance to EN 60601-1 and applicable collateral standards regarding medical electrical equipment, so that the device will not compromise the safety and health of patients or users. Under these circumstances, residual risks regarding electrical hazards are acceptable and additional clinical data are not needed unless negative issues are detected during PMS activities.
• Harmonised standards on usability (EN 62366 and if applicable EN 60601-1-6) are expected to be applied to ensure that usability aspects are taken into consideration during the device development. However, they do not give guidance on a detailed level of design, while usability aspects are known to cause or contribute to a large portion of incidents. Therefore, clinical data may be needed to prove that the risk of use error, due to the ergonomic features of the device and the environment in which the device is intended to be used, has been reduced as far as possible.

A7.2. Conformity assessment with requirement on acceptable benefit/risk profile

(MDD ER1 / AIMDD ER1)

It is expected,

• that the clinical evaluation demonstrates that any risks which may be associated with the intended purpose are minimised and acceptable when weighed against the benefits to the patient and are compatible with a high level of protection of health and safety; and
• that the IFU correctly describe the intended purpose of the device as supported by sufficient clinical evidence; and
• that the IFU contain correct information to reduce the risk of use error, information on residual risks and their management as supported by sufficient clinical evidence (e.g. handling instructions, description of risks, warnings, precautions, contraindications, instructions for managing foreseeable unwanted situations).

1. Evaluation of the description of the intended purpose of the device
   The information materials supplied by the manufacturer (including label, IFU, available promotional materials including accompanying documents possibly foreseen by the manufacturer) should be reviewed. The evaluators should evaluate if the description provided by the manufacturer correctly identifies those medical conditions and target groups for which conformity with the relevant Essential Requirements has been demonstrated through sufficient clinical evidence. When reading the IFU, there should be no uncertainty for users as to when a given medical condition or medical indication or target population is covered by the CE marking or when it falls entirely under the user’s own responsibility (off label use).
2. Evaluation of the device’s benefits to the patient
   Positive impacts of a device on the health of an individual should be meaningful (relevant for the patient) and measurable. The nature, extent, probability and duration of benefits should be considered. Benefits may include:

• positive impact on clinical outcome (such as reduced probability of adverse outcomes, e.g. mortality, morbidity; or improvement of impaired body function),
• the patient’s quality of life (significant improvements, including by simplifying care or improving the clinical management of patients, improving body functions, providing relief from symptoms),
• outcomes related to diagnosis (such as allowing a correct diagnosis to be made, provide earlier diagnosis of diseases or specifics of diseases, or identify patients more likely to respond to a given therapy),
• positive impact from diagnostic devices on clinical outcomes, or
• public health impact (such as to the ability of a diagnostic medical device to identify a specific disease and therefore prevent its spread, to identify phases, stages, location, severity or variants of disease, predict future disease onset).

• Quantification of benefit(s) to the patients
  Defining specified endpoints is indispensable for setting up clinical investigations and properly performing the identification, appraisal, and analysis of the clinical data.

• Benefit(s) are often evaluated along a scale or according to specific endpoints or criteria (types of benefits), or by evaluating whether a pre-identified health threshold was achieved. The change in subjects’ condition or clinical management as measured on that scale, or as determined by an improvement or worsening of the endpoint, determines the magnitude of the benefit(s) in subjects. Variation in the magnitude of the benefit across a population may also be considered.
• The clinical relevance of these changes should be discussed and justified.
• Ideally, these parameters should be directly clinically relevant.
• In certain cases benefits can be assumed when validated surrogate endpoints are met (such as obtaining certain results with laboratory tests or measurements of anatomical or physiological properties).
• Based on the current state of medical knowledge, the evaluators shall justify and document the clinical relevance of endpoints used for the clinical evaluation of a device and demonstrate the validity of all surrogate endpoints (if surrogate endpoints have been used).

The probability of the patient experiencing one or more benefit(s) is another important aspect of evaluating benefits and the clinical performance of a device.

• Based on the clinical data provided and on a sound statistical approach, a reasonable prediction of the proportion of “responders” out of the target group or subgroups should be made.
• The data may show that a benefit may be experienced only by a small proportion of patients in the target population, or, on the other hand, that a benefit may occur frequently in patients throughout the target population. It is also possible that the data will show that different patient subgroups are likely to experience different benefits or different levels of the same benefit.
• If the subgroups can be identified, the device may be indicated for those subgroups only.
• In some cases, however, the subgroups may not be identifiable. Magnitude and probability of clinical benefits will have to be put together when weighing benefits against risks.
• A large benefit experienced by a small proportion of subjects may raise different considerations than does a small benefit experienced by a large proportion of subjects. For example, a large benefit, even if experienced by a small population, may be significant enough to outweigh risks, whereas a small benefit may not, unless experienced by a large population of subjects.

The duration of effect(s) (i.e. how long the benefit can be expected to last for the patient, if applicable to the device)

• The duration should be characterised (for example as a statistical distribution) on the basis of sound clinical data and appropriate statistical approaches.
• PMCF will be decisive to refine and corroborate reasonable predictions over time.
• The mode of action may play an important role: Some treatments are curative, whereas, some may need to be repeated frequently over the patient’s lifetime.
• To the extent that it is known, the duration of a treatment’s effect may directly influence how its benefit is defined. Treatments that must be repeated over time may introduce greater risk, or the benefit experienced may diminish each time the treatment is repeated.
• The evaluation of the duration of effect should take into consideration current knowledge/the state of the art and available alternatives.

• Evaluation of the clinical risks of devices
  The risk management documents are expected to identify the risks associated with the device and how such risks have been addressed. The clinical evaluation is expected to address the significance of any risks that remain after design risk mitigation strategies have been employed by the manufacturer.
  PMS reports are compiled by the manufacturer and often include details of the device’s regulatory status (countries in which the device is marketed and date of commencement of supply), regulatory actions undertaken during the reporting period (e.g. recalls, notifications), a tabulation of incidents (particularly serious adverse events/ incidents, including deaths, stratified into whether the manufacturer considers them to be device-related or not) and estimates of the incidence of incidents.
  Post-marketing data about incidents are generally more meaningful when related to usage but caution is needed. The extent of user reporting in the medical devices vigilance system may vary considerably between countries, users, and type of incident. Considerable under-reporting by users is expected. However, the analyses of data within these reports may, for some devices, provide reasonable assurance of both clinical safety and performance.
  It may be helpful to provide a table summarising device-related incidents, paying particular attention to serious adverse events/ incidents, with comments on whether observed device- related incidents are predictable on the basis of the mode of action of the device.
  To demonstrate the extent of the probable risk(s)/ harm(s), the following factors – individually and in the aggregate – should be addressed:

• Nature severity, number and rates of harmful events associated with the use of the device:
  • Device-related serious adverse events/ incidents: Those events that may have been or were attributed to the use of the device and produce an injury or illness that is life- threatening, results in permanent impairment or damage to the body, or requires medical or surgical intervention to prevent permanent harm to the body.
  • Device-related non-serious/ non-reportable harmful events: Those events that may have been or were attributed to the use of the device and that do not meet the criteria for classification as a device-related serious adverse events/ incidents.
  • Procedure-related incidents: Harm to the patient that results from use of the device but is not caused by the device itself. For example, anaesthetic-related complications associated with the implantation of a device.
• Probability of a harmful event: The proportion of the intended population that would be expected to experience a harmful event; whether an event occurs once or repeatedly may be factored into the measurement of probability.
• Duration of harmful events (i.e., how long the adverse consequences last): Some devices can cause temporary, minor harm; some devices can cause repeated but reversible harm; and other devices can cause permanent, debilitating injury. The severity of the harm should be considered along with its duration.
• Risk from false-positive or false-negative results for diagnostic medical devices :
  • if a diagnostic device gives a false-positive result, the patient might, for example, receive an unnecessary treatment and incur all the risks that accompany that treatment, or might be incorrectly diagnosed with a serious disease;
  • if a diagnostic device gives a false-negative result, the patient might not receive an effective treatment (thereby missing out on the benefits that treatment would confer), or might not be diagnosed with the correct disease or condition;
  • other risks associated with false-positives and false-negatives.
• It is also important to look at the totality of the harmful events associated with the device. The number of different types of harmful events that can potentially result from using the device and the severity of their aggregate effect has to be considered. When multiple harmful events occur at once, they have a greater aggregate effect.
• Comment specifically on any clinical data that identifies hazards not previously considered in the risk management documentation, outlining any additional mitigation required (e.g. design modification, amendment of information materials supplied by the manufacturer such as inclusion of contraindications in the IFU).

• Evaluation of acceptability of the benefit/risk profile
  • Evaluate if the clinical data on benefits and risks are acceptable for all medical conditions and target populations covered by the intended purpose when compared with the current state of the art in the corresponding medical field and whether limitations need to be considered for some populations and/or medical conditions.
  • The current knowledge/ state of the art therefore needs to be identified and defined, possibly also relevant benchmark devices and medical alternatives available to the target population. Typically, documentation of the clinical background shall include the following information:
    • clinical background
    • information on the clinical condition(s) to be treated, managed, or diagnosed
    • prevalence of the condition(s)
    • natural course of the condition(s)
    • other devices, medical alternatives available to the target population, including evidence of clinical performance and safety
    • historical treatments
    • medical options available to the target population (including conservative, surgical and medicinal)
    • existing devices, benchmark devices
  • Sufficient detail of the clinical background is needed so that the state of the art can be accurately characterised in terms of clinical performance, and clinical safety profile. The selection of clinical data that characterises the state of the art should be objective and not selective of data on the basis of being favourable for the device under evaluation. Information should be provided on alternative approaches that have been used or considered and their benefits and drawbacks. Deficiencies in current therapies should be identified from a critical and comprehensive review of relevant published literature. The literature review should demonstrate if the device addresses a significant gap in healthcare provision. Where there is no such clinical need, the design solution needs to show an improved or at least equivalent benefit/risk profile compared to existing products or therapies.
  • If or when treatment comparability versus accepted therapy is not available at the time of placing on the market, this should be clearly described in the device IFU.
  • Even if a device cannot compete with an agreed first-line treatment or the best in class, it may add to the portfolio of acceptable treatments, as even a first-line treatment will likely have contraindications or non-responders.
  • Devices, that might not be best-in-class, might provide sufficient clinical evidence for an acceptable benefit/risk-profile for specific, defined subgroups or even superior clinical performance under specific conditions (e.g. emergency outdoor conditions).
  • The position within the treatment portfolio has to be specified properly in the clinical evaluation report and other relevant documentation.
    Example: A system for deep brain stimulation has a proven effectiveness for the treatment of depression. However, the implantation of electrodes in the brain is associated with major risks. Less invasive treatment options are available to patients suffering from depression. Taking into account the available treatment portfolio, the manufacturer has limited the medical indication of the device to “therapy resistant depression”, which is reflected in the IFU and in other relevant documentation.

A7.3. Conformity assessment with requirement on performance

(MDD ER3 / AIMDD ER2)

The devices must achieve the performances intended by the manufacturer. The ability of a medical device to achieve its intended purpose as claimed by the manufacturer needs to be demonstrated, including any direct or indirect medical effects on humans as well as the clinical benefit on patients resulting from the technical or functional, including diagnostic characteristics of a device, when used as intended by the manufacturer.

Clinical performance includes any claims about clinical properties and safety of the device that the manufacturer intends to use. It is expected:

• that the devices achieve their intended performances during normal conditions of use, and
• that the intended performances are supported by sufficient clinical evidence.

Evaluation of clinical performance can vary widely between device groups, especially between therapeutic and diagnostic devices. The following list gives examples of performance data relevant particularly to diagnostic devices:

• Reproducibility of independent acquisition of images (same patient, same machine, different operator and interpreter).
• Reproducibility of independent reporting of images (same patient, same machine, same images, different interpreter/analyser).
• Diagnostic sensitivity and specificity of the test for major clinical indications; positive and negative predictive values according to varying pre-test probabilities.
• Comparisons of performance of new iterations of diagnostic software against previous software versions.
• Normal values by age and gender, covering all groups in which the diagnostic system may be used.

A7.4. Conformity assessment with requirement on acceptability of undesirable side-effects

(MDD ER6 / AIMDD ER5)

Any undesirable side-effect must constitute an acceptable risk when weighed against the performances intended.

In order to evaluate the acceptability of the side-effects of a device:

• there needs to be clinical data for the evaluation of the nature, severity and frequency of potential undesirable side-effects;
• the clinical data should contain an adequate number of observations (e.g. from clinical investigations or PMS) to guarantee the scientific validity of the conclusions relating to undesirable side-effects and the performance of the device;
• in order to evaluate if undesirable side-effects are acceptable, consideration has to be given to the state of the art, including properties of benchmark devices and medical alternatives that are currently available to the patients, and reference to objective performance criteria from applicable standards and guidance documents.

If there is lack of clinical data or an insufficient number of observations, conformity with the requirement on acceptability of undesirable side-effects is not fulfilled.

Example:

A reasonable probability (80%) of observing at least one event of an undesirable side-effect when 15 subjects are studied requires a side-effect with an actual probability of 10%. If only 15 patients have been studied, from a statistical point of view, there could be serious side-effects with an actual probability of 10% that have not had a reasonable chance to be detected. The device would only be acceptable (for any type and severity of undesirable side-effects), if that magnitude is acceptable when weighted against the performance of the device and the current state of the art.

The table below shows corresponding numbers for undesirable side-effects with an actual probability of 10%, 5% and 1%.

The threshold proposed as acceptable for any new device will depend on the severity and detectability of side effects concerned.