Open Research Online A review of research with co-design methods in health education

: Studies using co-design methods require the meaningful involvement of stakeholders in creating new knowledge and harnessing, mobilising, and transferring existing knowledge to support comprehensive and long-term solutions. In the health sector, co-design methodology is seen as a way of supporting and engaging local communities in critical decision-making about their health. However, little is known about which specific co-design methods have been adopted, used, and implemented within health education contexts. To address this gap, this paper presents a literature review of co-design methods used to design and implement health education interventions. This rapid evidence assessment (REA) was carried out by identifying 53 papers categorised into four themes: methods, stages, stakeholders, and outcomes. We examined specific co-design methods used in health education stages to support the involvement of stakeholders, second, we reviewed the outcomes of the application of these methods. Based on the review findings, the paper reflects two areas: first, the review shows that there are a wide number of co-design methods being used to support stakeholder collaboration to design health care services as products and processes. Second, there is no clear way co-design methods are evaluated for their outcomes. This review of literature contributes an evidence base to support the future development and use of co-design in health contexts by organising relevant literature into coherent themes in ways that can inform future research.


Introduction and background to the study
Our world is increasingly affected by natural disasters, prolonged conflict, economic and political crises, and disease outbreaks such as COVID19. These major societal challenges may have a stronger impact on countries with limited resources, such as in low-and middle-income countries (LMICs). Interventions designed by 'experts' who are unfamiliar with the context tend not to work. Therefore, finding solutions to these challenges requires local knowledge as well as global insights. In recent years there has been a drive towards participatory co-design methods that bring together different knowledges. Co-design is an approach that enables researchers, practitioners and communities to share responsibility around various aspects of project design and delivery, giving each group equal power (Hickey et al., 2018). Co-design of research with communities is seen as being vital for research equity and impact (e.g., ESRC 2019; NIHR, 2020). Funding calls, therefore, increasingly mandate the meaningful involvement of stakeholders in both creating new knowledge as well as harnessing, mobilising, and transferring existing knowledge to support comprehensive and long-term solutions to problems (Del Gaudio, Franzato, & de Oliveira, 2018).
In the health sector, there has been a move towards the adoption of methods that promote and enable diverse groups of stakeholders to work together to plan, research, deliver, and evaluate healthcare services (O'Brien, Fossey & Palmer, 2021). These approaches are seen as crucial in the collective response to global health challenges (Marston, Renedo & Miles, 2020). As a result, over the last decade, there has been an increase in the use of co-design methods for health sector research (Slattery, Saeri & Bragge, 2020). These methods are perceived as essential to address the needs of specific stakeholder groups and to ensure that interventions recognise and harness the different types of knowledge and experiences of each group (Fathalla, 2015). These methods are also viewed as a way of supporting and engaging stakeholders (e.g., health care professionals, patients, etc) and local communities in decision-making about their health (David, Sabiescu & Cantoni, 2013).
New ways of working and engagement through co-design are known to lead to new partnerships, improved processes and products, and offer exciting opportunities to improve learning and practice (see e.g., Redman et al. 2021). Co-design is known to support local communities in critical decision-making around access to healthcare that builds on, expands or critiques global health approaches while addressing local needs and concerns (Zhang et al., 2019). Co-design can lead to changes that address significant power imbalances in decision-making processes and representation in policy, practice and research (ibid). It leads to a better understanding of the world around us and, ultimately, an improved impact on addressing persistent health inequalities.
These benefits led to the World Humanitarian Summit calling for a "participation revolution" in 2016. However, recent evidence shows that these benefits of co-design have not yet been achieved in health sectors (Lough & O'Callaghan, 2021). On the one hand, co-design appeals to groups and organisations, but it is a challenging approach to implement and navigate and takes much longer than when using traditional research methods (Redman et al. 2021). This is partly because co-design methods are loosely defined, so it is not always clear what approaches can be defined as co-design. There can be ambiguity around the end goal and what is being designed, which co-design method is being used and what the implications for stakeholder participants are during the co-design process. A major problem is that the scope of the co-design process and what it is trying to achieve are understood differently by researchers in different fields. As a result, co-design research is implemented and reported with substantial variations in the literature (Kieslinger, Pata & Fabian, 2009). There is much to learn from examining how co-design works in diverse settings and thus respond to the substantial interest in the elements needed for co-design, including methods and outcomes. To help improve the understanding of co-design methods in healthcare contexts, this paper provides a review of co-design methods as applied within health education contexts to support the involvement of different stakeholder groups.
The purpose of health education, according to the World Health Organisation (2012), is not only to increase knowledge about personal health behaviour but also to develop forms of action to address social, economic, and environmental determinants of health. The WHO definition of health education integrates social policy and empowerment as the key principles of health education (WHO, 2012, p. 13): "Health education is not limited to the dissemination of healthrelated information but also fostering the motivation, skills and confidence necessary to take action to improve health, as well as the communication of information concerning the underlying social, economic and environmental conditions impacting on health, as well as individual risk factors and risk behaviours, and use of the health care system" Co-design in health research has been defined by Slattery, Saeri and Bragge (2020, p. 3) as: "The meaningful involvement of research users during the study planning phase of a research project, where 'meaningful involvement' is taken to refer to participation in an explicitly described, defined, and auditable, role or task necessary to the planning and conduct of health research". This definition is useful in drawing our attention to a shift in power and privilege, where stakeholders get involved and, through this involvement, may benefit from their participation by informing the design process.
Features of co-design in health education have been identified through reviews examining health education research. These reviews include a study by Mulvale et al. (2016) who identified methods used to involve patients, family and service providers in child and youth mental health service improvement research by analysing 13 publications. The authors of this study reported that the existing child and youth mental health participatory research literature aligns considerably with many elements of experience-based co-design methods -defined as a participatory action-research method for collaboratively improving health care services. Methods such as storytelling and visual media, employing youth as researcher partners, were suggested to address power imbalances with research involving this population through co-design to establish equal status among participants.
A few years later, another review by Slattery, Saeri and Bragge (2020) identified co-design approaches used in health research reported in 26 review papers. Their results suggest that reviews vary in terms of reporting the applicability of co-design and its application in different health contexts. Those reports included interaction with stakeholders in the various co-design stages using mostly focus groups, interviews, and surveys. These methods differ compared to the methods proposed by Mulvale et al. (2016), suggesting a lack of consistency in terms. The study by Slattery and colleagues (2020) provided several recommendations to improve the effectiveness of co-design, including training stakeholder participants in research skills, having regular communication between stakeholders, setting clear expectations, and assigning specific roles to everyone involved in co-design.
Two further review studies focused on co-design methods to design interventions that involved digital and mobile technologies. The first review by Bevan et al. (2020) identified the escalating use of co-design to design digital technologies to improve children and young people's mental health. The authors reviewed 25 articles and 30 digital technologies used to support the mental health of children and young people. The methods used to engage stakeholders in design activities included a range of collaborative activities. The review highlighted tensions concerning the implementation of co-design methods, for example, the flexibility needed in the planning stage leads to tensions in gaining approval by institutional review boards that often expect specifications for the technologies used as well as the cost, and scale of the intervention. The second review by Eyles et al. (2016), examined the co-design of mobile health interventions across nine relevant studies. The authors identified limitations in the way co-design is assessed and reported, including sufficiency of reporting and lack of robust assessment of efficacy.
Finally, a recent scoping review by O'Brien, Fossey and Palmer (2021) examined co-design methods used to design mental health services with culturally and linguistically diverse communities, aiming to surface specific methodological considerations for working with these populations. Through a review of nine peer-reviewed articles, the authors identified a need to improve the experience for stakeholders as they engage in co-design processes. This finding supports the need for further research into the transferability of co-design approaches when they are used with different types of communities, particularly where co-design is used for health service design and improvement.
In summary, co-design methodology is important for health care services and is fast becoming the 'go-to' methodology. However, the literature points to a number of problems in terms of planning, implementation and application of co-design methods, as well as assessing and reporting. The combined studies reported in this introduction show that the use of co-design in health education contexts is complex. The literature does not provide an accessible synthesis of the most significant characteristics of co-design. This synthesis is important if co-design methodology is to be used routinely to support the design of health services and tools. This study aims to close this gap, providing an accessible summary of existing evidence on the topic by reviewing relevant literature.
The review includes research relevant to different forms of co-design in health contexts. First, we examine specific co-design methods used in health education contexts to support the involvement of stakeholder groups and, second, we review the outcomes of the application of these methods. Our rationale for focusing on health education contexts is the lack of evidence around which co-design methods are adopted, used, and implemented in such contexts as well as a lack of understanding of which of these are more effective in specific health settings (Stonbraker et al., 2019). This review of literature contributes an evidence base to support the future development and use of co-design in health contexts by organising relevant literature into coherent themes in ways that can inform future research.
The review is framed by the following research questions (RQs): -RQ1 How are co-design methods used to support stakeholders' involvement in different stages? -RQ2. What outcomes do co-design methods lead to?
This review was undertaken as part of a large multistakeholder project that was set up to co-design, test and scale-up approaches, resources and products related to health services in LMICs and promote co-produced research and programme/intervention development. Project funding was provided by the UK Government through the UK Aid Connect programme.

Method
The method used was a rapid evidence assessment (REA), which allowed us to pursue a rigorous and systematic approach while maintaining a narrow scope of the review so it could be completed within a short period. This literature review approach makes compromises on breadth, depth, and comprehensiveness to achieve an agile approach to summarising and synthesising evidence (Barends, Rousseau & Briner, 2017). However, the review reported in this article was never intended to cover the breadth of concepts and studies included in co-design in health contexts. Instead, it had a more specific focus: to identify co-design methods and co-design outcomes in health education. We adapted the REA approach of Barends, Rousseau & Briner (2017) as follows: 1. Background and question. Specified by current existing research and research questions.

Inclusion criteria and search strategy.
This step defines the evidence to include by specifying database searches, keywords, and publications. 3. Data extraction. This step involves the extraction of relevant data using quality metrics (i.e., themes or categories). 4. Results and Synthesis. This step highlights tensions in the evidence base through examination of the themes created. A synthesis of the evidence is created. 5. Conclusions. In this step statements of the main findings are articulated.
The process, therefore, comprised a systematic sequence of scoping, searching, and screening. First, the research questions were defined, and for that purpose, a brief review of existing literature was conducted (step 1, reported in the introduction and background of this paper). Following a broad scoping search in co-design in education (i.e., not only health education) we compiled a preliminary list of relevant themes that are frequently used in studies that discuss co-design (step 2). After this, a systematic set of searches was conducted for evidence on co-design in health education. The search results were then screened according to the inclusion criteria and mapped against the identified themes (step 3) and are synthesised and reported against the research questions (steps 4-5).

Characteristics of co-design
Our scoping search began by noting relevant keywords (such as learning, education and co-design) and terms known to the authors and related to co-design that would aid our systematic search of the literature. This search had a specific focus: to identify terminology and create a preliminary list of themes that may help the authors describe and categorise literature in this field. Using this method, a list of four themes was compiled as described in the following subsections. All these themes are defined in detail in the appendices.

Definition of co-design
Terms such as co-production, co-creation, and co-design are used interchangeably (Brandsen & Honingh, 2018). Co-design draws on the tradition of participatory design from the '60s in Scandinavia where there was an increasing need to consider the views of the community in key decision-making (Bødker, 1996). In that sense, co-design is related to human-computer interaction (HCI) practices and is used to design digital, collaborative environments (Bødker & Kyng, 2018). Wang & Hannafin (2005, p.6) define participatory design as that which "involves collaboration between researchers and participants, local practices that support systematic theorising, and improvement in both theory and practice". Co-design includes a practical view and is defined as an approach that creates a shared language between stakeholders and designers to understand a new 'product' from the perspective of everyone involved in its development and use (Dodero et al., 2014). The concept of a 'product' -a legacy from the field of design -features prominently in the literature.

Theme 1 -Methods associated with co-design research
The various methods in the context of co-design have been used to design learning environments that support stakeholder collaboration (Lund & Hauge, 2011); social network technologies that support learning at scale in professional contexts (Ley et al., 2014); and workplace learning environments . Our initial search of literature pointed out that co-design methods are usually described with references to events such as workshops, show & tell, and feedback events (Alhumaidan, Lo & Selby, 2015;Page et al., 2016;Marshalsey & Sclater, 2018). These events serve as sites of exploration, experimentation, and transformation for the creation of new knowledge (Mattelmäki & Sleeswijk Visser, 2011). A variety of methods is associated with co-design, and these could be broadly categorised as design methods or social science methods. The former can involve brainstorming and reflective activities, stakeholders' requirements meeting, usability testing and evaluation, development of visualisations, and storytelling, and development of narratives and scenarios (Akpo et al., 2015;De la Harpe, Korpela & Van Zyl, 2015;Kusmin, 2019). The latter can, among others, involve interviews, observations, field notes, surveys, reflective journals, and desk research (Alvarado, 2012;Law, Yuen & Lee, 2014;Koulouris & Dimaraki, 2014;Tomberg et al., 2013).

Theme 2 -Stages in co-design research
Co-design is reported in the literature as consisting of several distinct steps or stages (Barbera, Garcia & Fuertes-Alpiste, 2017;Chen et al., 2012;Garcia, 2014;Kyza & Nicolaidou, 2017;Laitinen, Kinder & Stenvall, 2018;Treasure-Jones & Joynes, 2018). Despite the inconsistency and variety in the terminology used, the various stages reported in the literature share similarities. Specifically, the first stage usually refers to the 'scoping and understanding of the problem, the second 'the design aspects', and third 'the prototyping' and the final one 'the evaluation'.

Theme 3 -Stakeholders in co-design research
Co-design is often described as an approach that brings together stakeholders (or users, as they tend to be called) with a design team during the different stages reported above (Jacobs, Pan & Askool, 2016;Kieslinger, Pata & Fabian, 2009). Therefore, stakeholder engagement is reported in the literature as crucial to ensuring that the end outcome meets the needs of stakeholders to make sure it is accepted and adopted (Calvo, Sclater & Smith, 2016). Our initial search showed that co-design can take a variety of forms across different disciplines and sites, and different ways of involving stakeholders in the design process were reported (Adams & Lenton, 2017). The term stakeholder in the literature seems to encompass diverse and varied groups such as practitioners, policy makers, end-users, and researchers.

Theme 4 -Outcomes of co-design research
Finally, our initial search in the literature suggested that co-design is usually associated with the creation of products and/or processes, often through developing and implementing services that bring about meaningful organisational change (Holmlid et al. 2015). We thus took co-design 'outcomes' to refer to the development of processes and products. 'Processes' can imply service improvement or a change in the perception within the organisation. In terms of 'products', they can refer to resources supporting training or to create educational systems (Millard et al., 2010).
Next, we detail the inclusion criteria, search strategy and data extraction for the review using the identified themes as categories.

Inclusion criteria, search strategy and data extraction
Inclusion criteria and search strategies were focused on studies published 2009-2019 to draw on the most recent research and to consider the prominence and popularity of co-design methods over the last decade. The reason for this is the increased interest and activity related to co-design in health education research that is noted over the last decade (e.g., see Hickey et al., 2018) -a trend that we may call the participatory turn in the production of knowledge in the health sector. In a report commissioned by Nesta back in 2009, David Boyle and Michael Harris refer to co-production as a "new way of thinking about public services [that] has the potential to deliver a major shift in the way we provide health, education, policing and other services, in ways that make them much more effective, more efficient, and so more sustainable" (Boyle and Harris, 2009, p. 3). There is much to learn from examining how literature reports work associated with co-design in the decade that followed that report. Through this review, we respond to the substantial interest in the elements needed for co-design, including methods and outcomes. The review stops just before the COVID19 outbreak, which marked a significant shift and insights about co-design from responses to the pandemic. Although we recognise that the REA presented in this article is an important step, over the next few years it will be critical to carry out a more rigorous examination of the effect of co-design, including during the pandemic and in the aftermath as well as tests of when, where, and how it can be used most effectively.
The review drew on three main databases: Google Scholar, Web of Science (WoS) and Scopus, due to their controlled search vocabulary and their rich content across the domains of education and health. These databases also allowed access to relevant journals, conferences, and repositories in technology-enhanced learning. Specialised medicine databases such as PubMed and Medline were included within Scopus. The search aimed to classify key information and identify examples using the themes created. A limitation of relying on Google Scholar as one of the sources of literature was the high number of papers identified (see Fig. 1) which however were of low quality, as will be described further below.
This REA was part of a research project in educational technology and health education and the search strings and methodology were agreed upon by the research team, informed by the initial scoping search (detailed in the previous section) and by the research questions. The different characteristics of co-design as well as the baseline themes identified in the initial scoping search that were used in this review are: 1. Theme 1 -Methods associated with co-design research. The co-design methods used and reported in the studies. 2. Theme 2 -Stages in co-design research. Inclusion and references to co-design stages.

Theme 3 -Stakeholders in co-design research.
Description of the various stakeholders involved in the co-design process.

Theme 4 -Outcomes of co-design research.
A co-design framework used in the reported studies for the development of processes or products.
Theme definitions, values and key references are detailed in the appendices.
For the search, we used the following search string: "(co-design) AND (health OR health education OR nurses OR clinicians OR doctors OR clinical OR medical OR pharmaceutical OR social work OR health professionals OR health technologies OR health sciences)". The literature sourced included book chapters, journal articles and conference proceedings, all published in English. Other types of published work such as theses, brief articles, commentaries, letters, medical newsletters, book reviews and literature reviews were excluded because most tend to not be peer-reviewed. The authors critically screened the title and abstract of the extracted papers to assess whether these would be included for further analysis (see Figure  1). This screening was based on the abstract including references to: (1) a primary study (i.e., excluding reviews/ metareviews) in health settings where engagement with stakeholders took place, (2) co-design methods used in the study, and (3) references to the evaluation of research co-design. The defined search resulted in 152 unique publications across the three databases. While most identified sources met the eligibility criteria, a small, complementary collection of sources were deemed especially informative and were also included though they did not meet all criteria such as the definition of methods. However, these exceptions were only made when an article met all but one of the eligibility criteria.
The 152 extracted studies were assessed for eligibility according to the following criteria: 1. Description of primary study in health settings. 2. The described study had a purpose to address 'learning' or 'education' objectives (e.g., formal, and informal learning). 3. Description of co-design methods used in the study (e.g., synthesis and presentation of a framework) (Theme 1), with the option of including stages (Theme 2).
4. Description of a study where engagement with stakeholders took place (Theme 3). 5. Evaluation of co-design research conducted (e.g., analysis of engagement effectiveness, outcomes) (Theme 4).
The first author critically reviewed all the papers and the review resulted in 53 papers that met all the above-defined criteria. Table 1 summarises key information from the publications examined and groups the sourced literature against the four baseline themes identified earlier and used as part of our inclusion criteria. These themes are not exclusive, as a paper might be categorised against more than one theme (e.g., several methods or stakeholders involved).
The appendices indicate the clustering of all the publications following the themes. The results of this analysis are reported in the next section.

Results and Synthesis
Results are divided by each of the research questions using the baseline themes, and synthesis for both questions is then provided.

Co-design methods support stakeholders' involvement in different stages
The most frequently cited methodology was experiencebased co-design, which was mentioned in 32 of the papers sourced. Other co-design methodologies included in the sourced literature focus on supporting stakeholder participation by enabling their involvement in different . Some of the methodologies focus on specific groups of stakeholders such as patients, including user-centred design or person-centred care (Tee & Özçetin, 2016). A small number of papers (n=6) reported that they used a specific health education co-design methodology (i.e., designed specifically for health purposes). These articles described the use of co-design to involve stakeholders in the development and delivery of various health education services. For example, McGregor et al. (2018) describe how co-design is used by health workers to develop an 'eHealth Capabilities Framework' to produce a Massive Open Online Course (MOOC) relevant to health degree graduates. El Mawas and Cahier (2013) use the 'Architecture for Representations, Games, Interactions, and Learning among Experts (ARGILE)' co-design approach to design, using reflective activities and involving health workers, training for Emergency Medical Services (EMS).
The use of different terminology to describe (similar) methodological frameworks make cross-study comparisons challenging. Following our categorisation (see section 'Characteristics of co-design' and appendices), methods are then divided into events, design methods and social science methods. In the sourced literature reviewed, co-design methods are classified as 'events' (n=26) that predominately tend to refer to workshops (see Figure 2). We recognise that in academic work there is a tendency to use the term 'workshop' to describe a wide range of events and this might be the reason why this term is frequently used in the description of 'events'. Other co-design methods reported in the literature as 'events', include feedback events (i.e. participants are brought together to form a consensus on a 'touchpoint priority') (Larkin, Boden & Newton, 2015), a training course (Yokota et al., 2018) or a dissemination event (Ward et al., 2019).
Design methods identified in the reviewed papers (n=48) include brainstorming (e.g., discussion to produce ideas or solve a problem); reflective activities (French, Teal & Raman, 2016); open-ended, participatory exercises; hands-on activities using creative materials with varied content and formats which were used which working across languages to cross-examine diverse understandings (Ali, 2018). Other design methods described in the papers include the development of cards, visualisations, storytelling, storyboarding, narratives, personas, and scenarios. For example, Walker and Schumacher (2017) propose the development of personas and scenarios with ideation sketches and design drawings, while Cleland et al. (2018) use artefacts to visualise interface components that are later converted into detailed digital wireframes. Finally, several 'traditional' social science methods were also reported in the literature (n=40), including interviews, focus groups and observations (e.g., Christensen et al. 2017;Mulvale et al., 2019;Yin et al., 2017). The literature reviewed points to an area of co-design that is about creatively co-designing activities between researchers and participants (i.e. design methods). We argue that this has implications for researchers who may aspire to use co-design approaches to research whereas they are often offered no opportunities to develop capacity on creative methods. The various training programmes offered (e.g. doctoral training) tend to focus on developing capacity with traditional research methods. Another point to raise here is that being involved in creative methods may require more time for researchers, not only in terms of implementation but also analysis of the data generated through these methods. This may explain why part of the discourse associated with co-design is often related to being a 'time-consuming' process.
About one-third of the papers sourced (n=17) provide descriptions of specific co-design stages (see Figure 2). The terminology used in the literature is not consistent and different authors use a variety of terms to describe distinct stages of the co-design process (Laitinen, Kinder & Stenvall, 2018). These stages are however often similar across studies. We summarise the various co-design stages reported in the literature as follows: 1. Framing the problem. This stage involves the definition of the co-design 'problem'. In this stage, several qualitative (interviews, desk research and literature reviews) and quantitative methods (questionnaires and surveys) are used to understand the problem that the co-design process aims to address (e.g. Hackett, Mulvale & Miatello, 2018;Latulippe, 2019). A distinct co-design activity included in this stage is a stakeholder requirement meeting, where researchers meet with end-users to gather 'product' requirements or conduct needs' assessments, which shows co-design as a problem-solving approach (Walker & Schumacher, 2017).

Design criteria and plan prototype. This stage
includes the co-design process and prototype of 'a solution' to the identified problem. It utilises a combination of social science and design methods. In the literature reviewed, these stages often are presented as following a linear progression, but in practice, this may not be the case -most of the time continuous cycles of reflection and community feedback of researchers and participating stakeholders are involved across and in between stages. The reported stages often fail to make connections to all stages of research, such as dissemination and analysis. Finally, the reviewed literature shows that a range of stakeholders has been involved in co-design. Thirty-two papers (n=32) refer to a range of stakeholders who often are public health workers: health providers, clinical staff and managers (see e.g. Cranwell (n=6) reported engagement with NGOs, carers, district nurses, community pharmacists and social care workers (e.g. Vella et al., 2015) and only 2 papers with the government itself (e.g. Yokota et al., 2018). Our review suggests that stakeholder involvement in the co-design process takes two forms: the first relates to stakeholders being the 'subjects' of the research, so they are involved in shaping and delivering the research itself; and the second relates to service design and delivery where stakeholders who use the services are involved in changing and/or designing the (new) service.

RQ1: How are co-design methods used to support stakeholders' involvement in different stages?
The analysis demonstrates that the most popular methodology was experience-based co-design. The most commonly used co-design method is the workshop, which often includes design methods such as brainstorming and reflective activities. In health educational contexts, workshops methods tend to be based on the development of artefacts (e.g. cards) and use of visualisation techniques (e.g. films), narration techniques (e.g. storytelling or storyboarding), as well as other user-centred design approaches, such as the development of narratives, personas, and scenarios. Interviews, focus groups and observations were also widely used. Social science methods tend to be applied at an early stage in the co-design process (e.g. while framing the problem), while the design methods are more commonly applied in stages 2 and 3 (i.e. design criteria, ideas information and plan prototype). Both social methods and design methods appear in the trial experiment output (final stage). Public health workers are included most frequently in health co-design processes. The community (i.e. the public) is represented less frequently, but, when included, tends to be influential in decision-making processes during co-design. Critically, health worker organisations (e.g. NGOs, reproductive health rights, advocacy groups, women organisations, faith-based organisations, or refugee organisations) and public organisations are not well represented.

The outcomes co-design methods lead to
Seventeen (n=17) of the papers reviewed focused on process enhancement as the outcome. For example, Larkin, Boden and Newton (2015) describe how they brought together service users, parents and staff to create health service improvements. Several studies (e.g., Farmer et al., 2018; Aidemark & Askenäs, 2019) involved consumers to help with the planning, delivery, implementation and evaluation of services and strategies. Jessup et al. (2018) use co-design to address the health literacy needs of a hospital population. Co-design has also been used to enhance mental health or employment services for youth (Matthews, Cowman & Denieffe, 2017). Other studies reported using co-design as a health improvement process, to upgrade the quality of interventions within their communities (e.g., Villalba et al., 2018;Walker & Schumacher, 2017). Scharoun et al. (2018) used co-design methods to change children's perceptions of healthy habits and behaviours while Rodriguez, Beaton & Freeman (2019) to change the behaviour and social interaction of homeless young people. Finally, co-design has been used in developing countries to create service innovations to promote social innovation around public health services (Herselman & Botha, 2017).
Two-thirds of the papers sourced (n=36) used co-design to develop products, such as resources, tools, or software. For example, Tee and Özçetin (2016) describe how resources for a professional development intervention to train student nurses in mental health were co-designed. Co-design has also been used to create a package of resources for a 'Dignity in care' nursing curriculum (Munoz et al., 2017) and in health promotion research and exhibition design practice (Bønnelycke, Thiel Sandholdt & Pernille Jespersen, 2019). Co-designed support tools include tools to help carers to identify the needs of their clients (Tremblay et al., 2019); tools to support the health needs of families (Chisholm, Holttum & Springham, 2018); tools to help health and social care practitioners engage in difficult conversations (Whitham et al., 2019); and tools to support stroke diagnosis in patients (Gonzales & Riek, 2013). For most of these studies, short-term research funding was provided so this may not effectively support the development of long-term partnerships or collaborative infrastructure to allow for co-design processes to become embedded in communities, organisational culture and become sustainable.
Several articles focused on the production of a software product as the outcome of co-design. These products include platforms systems (most frequently reported), including a 'computer-assisted medical decision support system' (Gil et al., 2016), a 'telerehabilitation system' (How et al., 2017), a 'medical assistive and transactional technologies system' (Vella et al., 2015), a 'portable health clinic system' (Yokota et al., 2018) or a 'health analytics platform ' (Cleland et al., 2018). Mobile and tablet apps were also produced through co-design, including mobile health (mHealth) and educational apps (e.g., Treasure-Jones, Dent-Spargo & Dharmaratne, 2018).

RQ2: What outcomes do co-design methods lead to?
Most studies using co-design methods focus on generating products, while 1 out of 3 papers reports on the process. The data analysis provided examples that illustrate processes developed through co-design. Several studies explored the stakeholders' views on planned improvements or their opinions about the implementation and evaluation of health services and the quality and value of this service. The development of products may include the production of resources for professional development to train professionals or students, the development of support tools to identify professional needs or engage with the community or software products such as platforms systems, mobile apps, web apps, online courses, and tablet apps.

Synthesis
As a final step in the review process, we consider the research questions and synthesise findings with an aim to distinguish co-design methods used to develop 'products' that 'deliver' health information solutions as well as services from co-design methods used to develop processes. The outcome of this process is represented in Figure 3. To achieve this, we only considered the 17 papers identified in the review that clearly describe co-design stages and we map the methods used in the reported studies against reported outcomes: process (n=10) and product (n=7). As Figure 3 shows, most methods are used consistently to generate both outcomes. Some methods used to introduce, deliver or enhance processes include desk research and literature reviews. An example is provided by Perrott (2014) who explored the concept and meaning of service co-design as it applies to the design, development, and market testing of health services. Similarly, the development of narratives, personas, and scenarios in the design stages only appear in studies leading to processes as an outcome. Examples include the work of Walker and Schumacher (2017) who explore quality enhancement to meet the specific needs of residential aged care by nurses and Ali (2018) to provide a health literacy intervention aimed at supporting informed reproductive choice among members of the UK communities practising customary consanguineous marriage. Finally, several authors include training, helpdesk, and feedback events to trial the solution, as described by Larkin, Boden and Newton (2015) who explored the perspectives of serviceusers, parents, and staff about hospitalization in early psychosis to provide service improvements. Figure 3 further shows that across both outcomes, interviews and questionnaires are predominantly used in the initial stages ("framing the problem") to frame both the problem and identify the stakeholders. They are then used at the final stage ("trial experiment output") to assess the satisfaction with the outcome. As reported earlier in the paper, in the second stage "design criteria and plan prototype", workshops are commonly employed. Workshops are often used in combination with brainstorming and reflective activities, curriculum design, visualisations, storytelling, and storyboarding. Finally, usability testing and evaluation methods tend to be incorporated into the later stages ("trial experiment output") to validate the outcome. What Figure 3 shows is that co-design process does not always get to be associated with stages including analysis of evidence gathered and dissemination. It is important to engage stakeholders in stages beyond the three shown in Figure 3. Failing to do this may imply that issues of power are not fully taken into consideration and addressed. Furthermore, one may assume that enhancing, embedding, or assessing new processes in organisations and service provision might require more time and more complex mix of methods compared to delivering a 'product'. Figure 3 does not reveal how much time was required in each of these studies to carry out the various stages and implement the co-design methods. Finally, it may also be the case that papers published and identified in this review tend to feature delivery of 'products' as it may be more straightforward to report on this, especially in the confines of a scholarly manuscript.

Conclusions
This review has identified a number of areas that need to be improved if co-design is to continue to be used to support health education contexts. First, the review has shown that there are a wide number of co-design methods being used to support stakeholder collaboration to design health care services as products and processes. Second, there is no clear way co-design methods are evaluated for their outcomes. Therefore, it is important to consider how co-design methods are evaluated for their validity and quality. Products and processes rarely evolve as the direct outcome of a single co-design event and it could be argued that, through co-design, validity is being improved (Dennerlein et al. 2020). Each outcome brings together multiple types of knowledge representing different stakeholder perspectives. However, all these diverse types of knowledge need to be brought together and taken into consideration to increase the validity of each co-design. Bringing together diverse knowledge is complex and this is the main criticism of co-design methods (Vindrola et al., 2019;Zamenopoulos & Alexiou, 2018). Nevertheless, it is important to find ways to bring together different knowledge: in challenging times, when societies are undergoing unprecedented changes, the broadest possible range of knowledge and insights is needed.
The authors acknowledge several limitations of this research. The use of 'co-design' as a key search term, since not all the papers that use this term in the title, abstract or as a keyword has systematically applied co-design processes. Additionally, we acknowledge the risk of selection bias in REA compared with systematic literature reviews, because of the way the systematic review process is streamlined (Barends, Rousseau & Briner, 2017). The process has not followed an inter-rater reliability process to add validity and reliability, and finally the review does not include studies developed during the COVID19 outbreak, which marked a significant shift and insights about co-design from responses to the pandemic.
The review identified a distinction in the conceptualisation of different co-design approaches and the resultant use of methods to develop products and processes. Some co-design methods are conceptualised to support the involvement of different stakeholders in designing and implementing health interventions. These interventions can develop products (tools, resources, etc), including digital technologies (Bevan et al., 2020) as an outcome, where a key aspect is to involve stakeholders in the different stages as reported by Slattery, Saeri and Bragge (2020). While other co-design methods are used to engender change in perceptions or processes. These interventions tend to result in changes in processes within health education contexts that can support the community (Mulvale et al., 2016;O'Brien, Fossey & Palmer, 2021).
As seen in RQ1 results in many of the papers reviewed the methodological framework presented was not adequately described, a limitation in co-design research previously reported (Eyles et al., 2016;Slattery, Saeri and Bragge, 2020). The studies sourced through this literature review report co-design methods, however, there are variations in the detail of their descriptions and terminology is not used consistently. For example, not all studies adequately synthesise and present a methodological framework (e.g., identify different stages of co-design). There are also differences in the understanding of the purpose of the method and how it is applied. Some studies use terms interchangeably (e.g., focus groups are also reported as a reflective activity) creating overlaps across the methods identified. This could be an indication that despite claims to be using co-design methods, the implementation of co-design in these studies may lack conceptual robustness and rigour. This may further have implications for crossstudy examination and will also affect the capacity to assess the effectiveness of co-design research in ways that capture the value to stakeholders involved. That said, we recognise that co-design research is highly contextdependent (Redman et al., 2021) and methods that may work well in one situation and at one time may be impossible in another. However, it seems that co-design methods cover a range of approaches and there are gaps in terms of how similar methodological approaches are being reported in the literature. Whether and how co-design can occur will also be determined by research culture and the development of researchers' capacity to understand and engage appropriately with such complex methods.
Co-delivery is not captured in the defined stages which might be because co-design as a term is often used in the initial stage of planning an intervention, and as such literature reviewed on co-design rarely reports 'co-delivery'. However, a basic tenet of co-design is that those stakeholders who are the intended 'beneficiaries' of what we are planning and delivering should be involved in both that planning and delivery (Millard, Howard, Gilbert & 2010). Co-design occurs when, for example, service providers in healthcare and people who use services (e.g., patients), work together on an equal footing to create interventions and projects. This forms a shift in power and concepts of 'expertise' -namely whose expertise and knowledge do we value, the fact that literature does not capture this aspect might suggest that the co-delivery aspect of co-design is not happening.
The definition of co-design in health research used in this review (Slattery, Saeri & Bragge, 2020) refers to powersharing (Hickey et al., 2018) since requires trust, genuine power-sharing, and respect for the different expertise brought by stakeholders (Redman et al., 2021). These should also be part of the outcomes that co-design leads to and something missing in the results from RQ2. The literature reviewed does not touch upon these issues that are critical in a co-design process and mainly discusses tangible outputs and measurable outcomes because these are valued by the systems (e.g. funding bodies). Powersharing is essential, especially when working with less powerful stakeholders which might be the case in LMICs. Agencies, funders, and donors should place value on these outcomes and establish mechanisms to support reflecting on processes embedded by researchers to enable powersharing, communication, and trust to be developed as part of the co-design process.
Our review illustrates that co-design is not a term that is consistently understood. Although co-design is increasingly becoming popular and marks a major shift in how knowledge is produced, a gap between theory and practice is persistent. In that sense, this research is timely to pinpoint a range of co-design methods used in health education, identifying methods that have been used and their outcomes. In conclusion, implementation of effective health education, careful consideration should be given to the selection of co-design methods. Our review shows that co-design methodologies include methods commonly used in educational research. However, it is the use of these methods combined with design and creative methods that appear to add value to various stages of the research process (e.g. see Cranwell, McCann & Polacsek, 2015;Latulippe et al., 2019;Walker & Schumacher, 2017).
Finally, the different types of outcomes signal a call for future research to apply co-design methods that not only focus on the production of products and resources as material artefacts, but also consider process outcomes. There is also a need to include stakeholders at all stages of co-design in health education interventions since their input tends to be focused around the initial and final stages, but not in the critical stages of analysis and co-delivery. In that sense, as reported by RQ1 results, the communities and public organisations need to be better represented (e.g. see Altuna & Jun, 2014;Cleland et al., 2018;Mulvale et al., 2019;Yokota et al., 2018). This opens a line of research into how stakeholders can participate equitably in research and questions the roles different organisations and actors should play to incorporate insights and ideas emerging from co-designed research.