What do contextual factors mean

Searching will be progressively extended and refined, based on emerging findings, as the review evolves, until saturation is reached. The aim of realist review is theoretical saturation as opposed to fully comprehensive coverage of the topic area.

Selection of sources will be based on relevance to aspects of the programme theory. A combination of search strategies will be utilised. Internet search engine and electronic database searching will be carried using keywords based on the programme theories identified in the exploratory search.

Sources of grey literature, including unpublished reports, will also be investigated, as well the websites of relevant organisations, such as The Health Foundation and the Institute for Healthcare Improvement. The reviewers will make use of snowballing techniques and consultation with experts and stakeholders. Given that a wide range of documents may contain data that can contribute to a realist review, multiple types of evidence will be included. Based on the data retrieved and emerging findings, the direction of the review may shift or expand in scope.

Search results from electronic databases and other sources will be imported into reference management software Endnote and duplicates removed. The realist appraisal and extraction process differs from a traditional systematic review, as inclusion and exclusion criteria are based on the programme theory and what the literature is able to contribute to it.

In a realist review, the units of analysis are not the interventions themselves per se but the theories underpinning the interventions. Unlike data extraction forms used in traditional reviews, these will be used primarily to gather information on contextual factors, mechanisms and outcomes, along with additional data on QI implementation, intervention resources and so on, thus providing a template to interrogate the evidence.

Outcomes of interest will include both implementation process outcomes and effectiveness clinical outcomes. Extracted data will be put into evidence tables and organised into themes.

Data extraction will be carried out by at least two reviewers; interpretation of the data will be guided by the judgement and reflexivity of the review team. Any differences will be resolved through discussion with the review advisory group. Again, in contrast to the traditional review process, during this stage, the project team will revisit and if necessary revise the focus of the review based on emerging findings. The key analytic process in a realist review involves iterative testing and refinement of theoretically based explanations i.

The goal of the fourth stage is thus to test and refine the initial programme theory by drawing comparison with the primary evidence and exploring and analysing the relationships between contexts, mechanisms and outcomes. Relevant passages of included documents will be annotated and coded to identify contexts, mechanisms, outcomes and CMO configurations. The reviewers will compare and contrast the evidence, looking for recurring patterns of CMOs across the data that are able to support, contradict or inform the programme theory.

This is an iterative process, guided by the research question and primary aims of the review. Completion of the realist synthesis will allow the reviewers to modify or refine the identified CMO configurations, and use these to explain a how and why QI initiatives cause change and generate outcomes within particular contexts and b which contextual factors matter, and how, when and for whom they matter, in terms of their influence on the QI process.

It is at this point that overall conclusions are drawn and a set of tentative recommendations produced. This penultimate stage will enable the production of a final synthesis integrating review evidence with programme theory, and culminating in a revised programme theory, refined in light of the evidence and reflecting the review findings.

Findings will be translated into evidence-based, practical knowledge and recommendations that can be shared with and applied by policymakers, and QI researchers and practitioners. They will be disseminated in the form of a final report, presentations to stakeholders and peer-reviewed publications. The use of a realist approach will allow the review to describe and explain how and why QI initiatives work or fail to work in different contexts by exploring the underlying programme theories and the interactions between contextual factors, mechanisms of change and outcomes.

Synthesising current knowledge on evidence of context research-based evidence of contextual factors affecting implementation within quality improvement and making this knowledge available and accessible to stakeholders will facilitate the design and development of evidence-based, context-sensitive improvement activities that can be planned and delivered in a way that takes this latest evidence into account, mitigating for known contextual barriers and enhancing facilitators in advance wherever possible and incorporating local knowledge to enhance implementation and improvement strategies and improve transferability.

Gillam S, Siriwardena AN. Evidence-based healthcare and quality improvement. Qual Prim Care. PubMed Google Scholar. Health Foundation. Quality improvement made simple. London: Health Foundation; Google Scholar. Batalden PB, Davidoff F. Qual Saf Health Care. Overcoming challenges to improving quality. Assessing the evidence for context-sensitive effectiveness and safety of patient safety practices: developing criteria prepared under contract no.

Agency for Healthcare Research and Quality: Rockville; Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. The influence of context on quality improvement success in health care: a systematic review of the literature.

Milbank Q. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. What context features might be important determinants of the effectiveness of patient safety practice interventions? Article PubMed Google Scholar.

An exploratory analysis of the model for understanding success in quality. Health Care Manag Rev. Article Google Scholar. Dopson S, Fitzgerald LA. The active role of context. Knowledge to action?

Evidence-based health care in context. Oxford: Oxford University Press; Chapter Google Scholar. Guidance for the assessment of context and implementation in health technology assessments HTA and systematic reviews of complex interventions: the context and implementation of complex interventions CICI framework.

Accessed 3 July Achieving change in primary care-causes of the evidence to practice gap: systematic reviews of reviews. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. Context and implementation: a concept analysis towards conceptual maturity. Z Evid Fortbild Qual Gesundhwe. Fulop N, Robert G.

Context for successful quality improvement: evidence review. London: The Health Foundation; Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. May C. Towards a general theory of implementation. Implementation, context and complexity.

Appendix A provides a template, modification of which is being used by 14 AHRQ practice transformation grantees described in section II below. Vetting drafts of such a table and narrative with stakeholders, and looking for trends in data collected at various points during the study can increase the credibility of the findings. Such a template could be included as an appendix to a scientific paper reporting study findings, or preferably could be used to generate a sentence for the abstract, a succinct paragraph that summarizes the important contextual factors for inclusion in the results section of the paper, and used as the basis for interpreting the meaning and transportability of the findings in the discussion section of the paper.

For those who wish to report context, but did not begin tracking contextual data at the beginning of the study or until the study has been completed, the step-wise approach can still be accomplished by gathering relevant retrospective data. This approach, however, may provide less robust data and includes the possibility of retrospective bias, but is much better than simply ignoring contextual factors.

In this section, we describe concrete examples of current or recently completed work related to tracking and reporting contextual factors. Together, these complementary examples show how context reporting can be accomplished. The step-wise method for identifying and assessing contextual factors described above currently is being used by 14 teams of investigators in an AHRQ-supported series of projects on Transforming Primary Care Practice.

A companion paper in the supplement describes what these investigators are learning from the process of reporting context for their practice transformation projects. Formatively, Russell Glasgow, Lawrence W. Green, and Alice Ammerman facilitated a meeting of 13 health research journal editors to consider reporting requirements for external validity that consisted largely of contextual factors.

They concluded that external validity and contextual factors related to external validity and settings to which results did and did not apply should be reported more frequently.

Although all participants agreed on the importance of better reporting on these factors, they did not come to consensus on a standard set of reporting criteria for contextual factors. Several journals called for increased attention, others provided guidelines for reviewers, and others facilitated additional discussions of the issue. The Prescription for Health project, funded by the Robert Wood Johnson Foundation and AHRQ, discusses contextual issues in 27 practice-based research networks to change practice and develop community partnerships to foster health behavior change around diet, activity, tobacco, and alcohol use.

The findings have been summarized in numerous scientific publications, including supplements to the Annals of Family Medicine 42 , 43 and the American Journal of Preventive Medicine. A final example illustrating the potential contribution of reporting on contextual factors comes from the National Demonstration Project NDP for the PCMH that used a multimethod approach and flexible group randomized design that allowed assessment of multiple important contextual factors.

NDP publications provide an explicit assessment of the historical context and evolving changes in the PCMH environment 51 , 55 , 56 affecting both the internal and external validity of the study findings. The socio-political context of the PCMH movement, and the values and theory of primary care that underpin the PCMH, are described and related to the state of the PCMH movement as it evolved during the time the project was conducted.

The NDP methods changed to meet emerging participant needs to adapt to the shifting environment and as stories from the qualitative data informed statistics from the quantitative data, and vice versa. For example, the NDP developed new methods to assess financial changes when it was discovered that practice fiscal records were inadequate for the planned economic analyses. How the PCMH intervention evolved in response to forces both within and outside of project participants 52 , 55 represents a key contextual factor that often is ignored or hidden in research reports.

For example, the facilitation process was tailored to match different practice change trajectories that became apparent during the course of the project, and the support technology, communication strategies, and shared learning were updated.

Contextual data collection and analyses also pointed out a key limitation for understanding the limited effect on patient outcomes, 53 and for transporting the findings to other settings—that the NDP was an almost entirely practice-focused change intervention with almost no system-level support.

This careful, multimethod approach to paying attention and reporting of contextual factors allows for a deep understanding of what happened and why, and for a thoughtful and informed extrapolation of study findings to different times, situations, and settings. Paying attention to contextual factors during all stages of PCMH research can help investigators and implementers understand often overlooked factors that affect the reach, relevance, implementation, outcome, and generalization of PCMH interventions.

Another advantage of reporting contextual factors is that it supports the replication of effective PCMH models. Reporting relevant contextual factors can help others to make sense of what happened during the study, for what reason, and in what situations. Considering context represents an opportunity to advance health services research conceptualization and methods that are likely to reduce inconsistencies in findings and more accurately represent the effects of the implementation of PCMH models across diverse settings, people, and times Consistently assessing and reporting contextual factors should help make scientific evidence about the effectiveness of PCMH as a health delivery model more relevant and actionable—and indeed more evolvable and applicable across diverse settings, people, and times.

The approach outlined above shows how to expand the usefulness, internal, and external validity of research by: identifying relevant contextual factors; grounding an assessment process in the relevant theory and stakeholder perspectives; using a multimethod, participatory process to collect and analyze the relevant data; and then reporting contextual factors.

However, this approach must be applied with an eye on its potential limitations. First, it can be time and labor intensive. Considering and reporting context requires thought and reflection so that the most important contextual influences on the intervention are identified. It also requires collecting and analyzing indicators of concepts that are outside those typically considered necessary by researchers, reviewers, and funders focused primarily on internal validity.

Considering contextual factors may feel like adding complexity at a time when people yearn for simpler solutions, however unsuited simple approaches may be for complex phenomena such as improving primary health care. Second, many journals do not have space for or prioritize reporting context.

However, creating demand for reporting context will require sufficient examples of its real value, and of the perils of its being ignored, before it will become the norm.

Finally, it can be difficult to identify which of the myriad possible contextual factors to track in a study, to engage diverse participant and potential end-user perspectives, and to continue to pay attention to the evolution of contextual factors over time. The greater ease of specifying an immutable a priori design, of focusing on internal validity to the exclusion of external validity, and the greater appeal of decontextualized simple solutions, may make it challenging for context reporting to gain traction.

In conclusion, including contextual factors can make research more relevant to stakeholders, foster understanding, and enable wise dissemination and informed re-invention in different moments in time, settings, and situations. Paying attention to context can help research to support advancement along the continuum from information to knowledge, and from knowledge to understanding.

Understanding PCMH research in context can foster the development of shared understanding that opens the possibility of wisdom. Defining and measuring the patient-centered medical home. J Gen Intern Med ;25 6 Green LW. Fam Pract ;25 Suppl 1:i Participatory research maximises community and lay involvement.

BMJ ; Multimethod research: approaches for integrating qualitative and quantitative methods. J Gen Intern Med ;9 5 National Institutes of Health U. Office of Behavioral and Social Sciences Research. Best practices for mixed methods research in the health sciences. Public health asks of systems science: to advance our evidence-based practice, can you help us get more practice-based evidence?

Contextual planning for schools with economically disadvantaged student populations should take into consideration that students may not be able to bring their own supplies. Many students in economically disadvantaged homes do not have computers to use to do homework, and many of their parents work, so they will not be able to attend after-school functions.

When it comes to classroom management, teachers based in urban or rural schools with many economically disadvantaged homes can avoid student disciplinary problems by anticipating students' needs. But if disciplinary issues do occur, calling parents may not help because they cannot attend student-teacher conferences. Contextual factors affecting individual students include age, gender, culture and personal interests. Teachers should anticipate student needs based on these attributes.

There is no hard and fast rule. You should do it on a case by case basis, and consider it in combination with all the other contextual factors.

It should also involve inclusive dialogue across the fields of expertise that are involved in the design, development, and deployment of the AI system. Getting together different team members, who have technical, policy, compliance, and domain expertise can provide a more informed vision of the impact factor of an AI model.

This factor suggests that you should think about the nature of the data your model is trained on and uses as inputs for its outputs when it is deployed. You should consider whether the data is biological or physical eg biomedical data used for research and diagnostics , or if it is social data about demographic characteristics or measurements of human behaviour.

You should also consider whether an individual can change the outcome of a decision. This will affect the type of explanation an individual wants. However, if the data is less flexible, such as biophysical data, it will be less likely that an individual will disagree with the output of the AI system. It will often be useful to prioritise the rationale explanation, for both social data and biophysical data. Where social data is used, individuals receiving an unfavourable decision can understand the reasoning and learn from this to appropriately adapt their behaviour for future decisions.

For biophysical data, this can help people understand why a decision was made about them. However, where biophysical data is used, such as in medical diagnoses, individuals may prefer to simply know what the decision outcome means for them, and to be reassured about the safety and reliability of the decision.

In these cases it makes sense to prioritise the impact and safety and performance explanations to meet these needs. On the other hand, where the nature of the data is social, or subjective, individuals are more likely to have concerns about what data was taken into account for the decision, and the suitability or fairness of this in influencing an AI-assisted decision about them. In these circumstances, the data and fairness explanations will help address these concerns by telling people what the input data was, where it was from, and what measures you put in place to ensure that using this data to make AI-assisted decisions does not result in bias or discrimination.

What people want to know about a decision can change depending on how little or much time they have to reflect on it. The urgency factor recommends that you give thought to how urgent the AI-assisted decision is.