User-driven innovation and technology-use in public health and social care : A systematic review of existing evidence

This systematic review revealed and discussed empirical evidence generated from 21 international user-driven innovation studies in the public health and social care. We used PRISMA guideline to ensure a transparent and replicable research process. With the guide of relevant theoretical models, we identified the distinct characteristics of user-driven innovation in current public social care sectors, in respect of its strategic innovation process and user-oriented, empower-based objectives. We categorized different endand intermediate user groups and discussed how them directly and indirectly engaged in various innovation phases via the support of different methods, and how their participation helped to fulfill users? roles for exploration, experiments, test, and innovation. We also took a closer look at technological options addressed via included studies and in particular discussed how technologies interact with users in the innovation process.


Introduction
For decades, public health and social care in most industrialized societies have encountered significant challenges in delivering better and high-quality services.Most countries, for example, experience substantial demographic changes and the resultant projected increase in need of eldercare combined with a concurrent decline in the number of professional careers (Couture et al., 2016;Huber et al., 2017).Also, health agencies today have to address the sophisticated threats, including morbidity and mortality associated with chronic, infectious and lifestyle diseases, in the context of flat or constrained funding and new and changing healthcare legislation (Lister et al., 2017;Tursunbayeva et al., 2017).Nevertheless, due to the high tax level, citizens urge to receive public services shaped for their needs and of the best quality (Granier and Kudo, 2016).At the same time, technologies open for new services that lead to social, political, economic, and legal transformation.Such transformation accelerates citizens' expectation on states in harnessing technology advances for value creation and social inclusion (Chan and Holosko, 2016;Keys, 2016;Huber et al., 2017).
Innovation is high on the agendas in most developed economies to meet undergoing challenges in public health and social care.In classic innovation theory, technology is an essential driver for innovation (Schumpeter, 1934).It has also been the same situation in the public sector where the introduction of technologies lead to the change in services (Marsan and Paré, 2013;Pang et al., 2014;Huber et al., 2017).However, the development of technologies in public domain conforms to an expert-driven and top-down process where users of those technologies, including service recipients and employees, are not actively involved (e.g., Hill and Shaw, 2011;Andersen and Jansen, 2012;Moen, 2012).This brought common problems such as dehumanized and undemocratic technical services and the loss of valuable views, resources, and knowledge from users in the innovation process (Hill and Shaw, 2011;Baker et al., 2014).Furthermore, due to the mass implication of new technologies, primarily increasingly digitized production/service process, citizens gradually grew their demand for a better and open system where they can be part of service design and delivery (Von Hippel, 2009;Sørensen and Torfing, 2012;Andersen, 2013).All these present issues make the public health and social care sector to further explore an innovative approach to work with different user groups, exchange their views and knowledge, and eventually facilitate the value co-creation process in service design and delivery (Chan and Holosko, 2016;Palumbo, 2016).
User-driven innovation (UDI), in this light, has been gaining attention over the years, especially in the welfare sector of Northern Europe (e.g., Pässilä et al., 2013;Hvenegaard Rasmussen, 2016;Puig-Pey et al., 2017).It refers to a process of "tapping users' knowledge" to develop new products, services, and concepts"(Von Hippel, 2009, p. 30).Users, instead of technologies, become the primary driver for innovation.UDI concerns both an understanding of users needs and a systematic involvement of users at different stages of the innovation process.Its strengths lie in the fact that both service recipients and professionals working in the public sector perform as crucial players in the provision of solutions for defined problems in the field.Those individuals thus will have better insight to innovate and spur development of precisely what they want (Andersen and Jansen, 2012;Wihlman et al., 2014;Scupola and Zanfei, 2016).
Since user knowledge is gradually seeing as a crucial source of innovation in the public sector, there is a rasing need to discuss questions such as what ultimate goals of UDI in public health and social care, how to support user participation in the innovation process, and how to gather accurate user knowledge in the practice for UDI.However, there has been relatively little research discussed those questions (Andersen, 2013;Kallio et al., 2013;Szymańska, 2017).Further, some previous studies in the public sector have argued that although users, instead of technologies, became the primary drive in UDI, we cannot neglect the fact that technology is still the force of all kinds of innovation in today's digitalized world.It can, for example, help gather, store and sort out user knowledge for innovation, supporting the institution access user knowledge and making the access cost inexpensive (Hill and Shaw, 2011;Barlott et al., 2016).However, even fewer studies investigated if technologies can facilitate UDI by providing easy and inexpensive access to user knowledge.To address current literature gap, we conducted a systematic review of existing international evidence related to UDI in public health and social care.Our study aims to support an in-depth understanding towards research topic by providing answers to following three essential questions: 1. What are distinct characteristics of UDI in public health and social care, in respect of its objective and process?
2. Who are users of public health and social care, and how different groups of users can be engaged in UDI process?
3. What roles of technologies play across UDI process, and what their relationship with users?

Theoretical Framework
Previous studies are valuable resources shaping a fundamental view towards the study topic.Some of these useful theoretical perspectives on innovation, UDI, the process of UDI, the role of users, and interaction between technology and user, are briefly presented in the following paragraphs.

Innovation
Early in 1934, Joseph Alois Schumpeter (1934) defined innovation as "combinations of resources" that drive the economic development substantially (p.66).Technology plays a significant role in innovation process -leading to the launch of a new product, method, market, sources, or industry structure (Joseph Alois Schumpeter, 1934).Over decades, Schumpeter's classic innovation theory has inspired practitioners and scholars to understand the meaning and the process of innovation in private and market-based sectors.Now innovation is often viewed as the new and improved application of the product, process, marketing and organizations (OECD 2005).Indicative of growing cross-sector exchange, the proliferation of innovation has been developed in public domain (Phills et al., 2008).However, it is also subject to varying definitions.In this article, we operate on the definition that innovation is the creation and implementation of new products, services, process, and methods, which helps to meet new requirements and unarticulated needs in public health and social care (Albury, 2005).

User-driven Innovation
The user-driven approach is an interdisciplinary concept, which derived from fields such as architecture and design, IT and engineering, business management and social sciences.In architecture and design, for example, UDI refers as "systematic approach to develop new products and services, building on the investigation of adoption of users' life, identity, praxis, and needs" (Christiansson et al., 2008, p. 249).Shaped within multiple disciplines, in this article, we use Von Hippel's definition on UDI that mentioned in the Introduction.UDI is a process of drawing on users' knowledge" to develop innovation (Von Hippel, 2009).It often links with terms such as co-design (Dekelver et al., 2011;All et al., 2013;Habicht and Thallmaier, 2017) and co-production (Pestoff, 2012;Durose et al., 2017).UDI is open in character and focuses on identifying existing and potential users, systematic searching and understanding users' explicit and implicit needs, knowledge, and ideas, and intensive involving users in co-creation and innovation process (Tuomi, 2002;Wise and Høgenhaven, 2008;Von Hippel, 2009).Users in this definition include intermediate users (e.g., user organizations) and consumer users (individual end-users or user communities), who are distinguished from suppliers (e.g., producers or manufacturers) (Trott et al., 2013).

UDI Process and Users' Roles
The Innovation Wheel (Fig. 1) is a model for UDI described by Wise and Høgenhaven (2008), which has been applied to describe an organization's innovation process and the involvement of users throughout the process (e.g., Røtnes and Staalesen, 2010;Pugh, 2014).As Fig. 1

I m p l e m e n t a t i o n T e s t P r o t o t y p in g
Fig. 1.The Innovation Wheel -cited from Wise and Høgenhaven (2008, p. 17) To support The Innovation Wheel, Wise and Høgehaven (2008) outline another framework (Fig. 2) which was modified in this study to help to understand the users' four different situations, namely observation, experiments, tests, and innovation, across WHAT and HOW phases in the innovation.These four categories suggest users' direct/indirect involvement in innovation steps, together with their various roles and responsibilities.Innovation teams often applied different methods and strategies to fulfill specific functions of user in particular innovation stages (Wise and Høgenhaven, 2008).
Above two models are beneficial to our review study especially in a situation where there is rare literature offers a systematic theoretical view to understanding UDI process and users' roles inside (e.g., Røtnes and Staalesen, 2010;Paiva et al., 2016;Szymańska, 2017).Both models are valuable frameworks developed through many scientific case studies upon UDI.Also, they have been continuously applied and further developed via other innovation practitioners and scholars who dedicate to identify new and better ways to facilitate UDI (e.g., Røtnes and Staalesen, 2010;Pugh, 2014).In this study, therefore, we use these two models as conceptual tools to help us extracting, categorizing, comparing and summarizing existing evidence regarding UDI process, users' roles, and methods applied to encourage user participation.It worth to note that these two models lack a particular focus on technologies that can be the force in supporting all kinds of innovation (Burgelman et al., 1996;Lundvall and Borrás, 2005) and the tool of helping tapping user knowledge (Bhatt, 2001;Wang et al., 2017).In next paragraphs, thus, we placed attention on previous studies that address the interaction between technology and users.

Interaction between Technologies and Users
The need to create and deliver innovation at an ever-increasing rate is requiring organization today to apply a more deliberate and systematic approach to manage and utilize collective knowledge stemming from users (Jenny and Rod, 2002;Wang et al., 2017).In previous studies on technology and knowledge management, we found that efficient technology appliances can help users with the facilitation of knowledge creation and diffusion (Jenny and Rod, 2002;Marina, 2007;Wang et al., 2017).For example, information and communication technologies (ICTs), particularly social media, may accelerate the speed of knowledge transfer and creation by affording new types of behaviors that were not possible with the previous form of computer-mediated communication (Lewis et al., 2010;David et al., 2014;Hussain, 2017).Favorite technological tools such as employee competence databases, online search systems, expert networks, workflow software, decision support systems, data warehouse and so the forth, all can enable organization to gather, store, response, sort out and utilize user knowledge more efficiently and economically (Sher and Lee, 2004;Park et al., 2015).
While technologies are used for knowledge management, their adoption in practice sometimes meets resistance from users.Many existing studies focus on the interaction between human and technology, which aims to make technologies more usable and useful to provide people with experiences fitting their specific background knowledge and objectives.For example, technology acceptance theory identifies two main factors that affect users' acceptance of technological appliance, namely perceived usefulness and perceived ease of use (Davis, 1985).Also, there are some key determinants of perceived usefulness and usage intention constructs, which can be divided into two groups as social influence processes (e.g., subjective norm, voluntariness, and imagination) and cognitive instrumental processes (job relevance, output quality and result demonstrability) (Venkatesh and Davis, 2000).
From performance-based perspectives, on the other hand, the utility and usability of technical systems can be assessed by effectiveness, efficiency (Bevan, 1995) and user satisfaction (Dillon, 2002).More recently, increasing studies highlight that user emotion and intrinsic motivation, influenced by personal experience with technology, preferred working style, and the aesthetics of system design, can be more crucial for users to adopt a particular technological system (Venkatesh, 2000;Beaudry and Pinsonneault, 2010;Cohen, 2014).Further, from an engagement aspect, factors such as attributes of challenges, playfulness, endurability, aesthetic and sensory appeal, perceived user control, etc., might also influence people's interactive relationship or experience with the use of technologies (O'Brien and Toms, 2008;Kim et al., 2013).

Methodology
We used the systematic review as the method in this research.In this section, we elaborate on the research strategies that were used to select studies and the detailed review process across research stages.

A Systematic Review
A systematic review of previous research can bring together the results of existing evidence and provide a reliable answer "how" and "why" of a particular phenomenon (Booth et al., 2016).This study used a systematic review as a methodology because this form of review differs from traditional narrative review by adopting a replicable, scientific and transparent process (Thomas and Harden, 2008).It helps us to minimize bias through exhaustive literature searches of both quantitative and qualitative studies and by offering an audit trail of the reviews decision, procedures, and conclusions (Tranfield et al., 2003).During the review, we followed Proffered Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines developed by Moher et al. (2009).This guideline has been widely used in various systematic review studies related to public services across countries (e.g., Lopez-Hartmann et al., 2012;Seys et al., 2013;Elia et al., 2016).It helped us to ensure a transparent and replicable research process.

Search Strategies
This review searched two international electronic databases including ISI Web of Science and Scopus.To focus on very current studies, search centered in January 2013-December 2017 texts.
Only articles published in peer-reviewed academic journals in English were included.The search strings we applied include "user-driven" OR "co-design" OR "co-production" AND "innovat*" in the title/abstract/keywords.With ISI Web of Science, we further refined results by choosing exiting categories including Health Care Sciences Services, Health Policy Services, Public Environmental Occupational Health, Social Sciences Interdisciplinary, Social Work, Sociology, and Rehabilitation.Similarly, for Scopus, we refined results by selecting categories such as Social Sciences, Medicine and Health Professions.

Sample Selection
The literature search was conducted from 1 st to 12 th March 2018.Initially, the database yield 197 hits based on our search keywords (45 from Web of Science and 152 from Scopus).Scopus included more studies in our case because it has a relatively less extensive category system that failed to help us further refine results as Web of Science did by narrowing down research subject areas.All outputs from both databases were exported in Endnote X8 software for reference management.We deleted manually 25 duplicates and led to 172 records for further reviewing.
To examine the most rigorous studies and to adequately address the research questions, we applied following inclusion and exclusion criteria.As these criteria suggest, we only want to select studies that are empirical, and directly addressed the concept of UDI by offering a precise definition of it.Since this research target exclusively on public health and social care, we want to particular include studies within public fields of healthcare, eldercare, childcare, community services, etc.Also, to answer research questions, we also need to select studies that have a clear description towards UDI objectives and process, together with roles of users and technologies in UDI.

Table 1. Inclusion and Exclusion Criteria
Inclusion Criteria Exclusion Criteria 1) The empirical study directly addresses UDI in public health and social care, which may include fields of healthcare, community services, work welfare, eldercare, childcare, etc.; 2) It has a description towards innovation concept, process, and outcomes; 3) It has a precise categorization of users in innovation process; 4) It has a clear explanation towards the roles of technology in innovation process; 5) It is theory-guided.
1) The study is not empirical; 2) It is irrelevant to public health and social care; 3) It lacks detailed description towards innovation concepts, targets, and process; 4) It has no explanation towards the roles of users in innovation process; 5) It has no focus or lack of detailed information towards technologies; 6) Lack of full-text access to the article.
After screening by title, keywords, and abstracts, we deleted 105 studies since 77 studies are irrelevant to public health and social care while 28 hits are not empirical studies.Further, 67 studies went through a full-text assessment, and 46 were removed.In total 21 studies were selected for this review as they fully meet all inclusion criteria.Each study was then independently reviewed using our inclusion criteria by both authors, to ensure their election eligibility.The stage of selection is illustrated in Fig. 4, using a PRISMA flow chart.
Records identified through database searching (n=197)

Critical Appraisal of Study Quality
In order to ensure quality of included studies, we rated each study using the Critical Appraisal Skills Program (CASP, 2017) checklist (Appendix A), which was slightly modified by adding a "not clear" (1) option for each questions and to the standard "yes" (2) or "no" (0) .There are ten questions on CASP checklist, so the final score of a study can be between 0 and 20, where 20 was most applicable while 0 was least applicable.The first author assessed all the eligible studies, while the second author independently assessed a random sample to appraise inter-rater consistency and resolve any ambiguities.

Qualitative Synthesis
The systematic review helped us to collect and assess both qualitative and qualitative evidence related to the investigated topic.The qualitative synthesis was further used to interpret the meaning of the received data.The strength of qualitative synthesis is to provide in-depth focus and contextualized details towards phenomenon under the review (Bearman and Dawson, 2013).
We first extracted and coded information from all eligible studies with the support of a structured form.It contains the following items: the study's publication features; description of technologies applied; the categorization and description of users; the innovation objectives, process, and outcomes; research questions; theoretical frameworks; study methodology; and main findings and conclusion.Followed, with the support of theoretical underpinnings, we categorized and summarized the extracted information into key 'themes' to answer research questions., 3, 5, 9, 10, 12, 14, 15, 16, 20).Also, there are four papers (S4, 11, 13, 18) did not specify the timeframe of their data collection.We observed that, on average, almost one or two years typically passes during the period of data collection and the publication of results, although research on innovation should, represent a rapidly changing landscape.It indicates that the conventional academic literature may be lagging behind as a source of relevant information on UDI in studied sectors.
Empirical materials from chosen studies were collected from 16 countries.There are 20 studies targeting high-income countries: thirteen from Western Europe (S1, 2, 3, 4, 5, 6, 8, 13, 15, 17, 18, 19, 21), seven from Scandinavia countries (S2, 3, 7, 9, 10, 14, 21), and two from Canada and USA (S12, 21).There is only one study addressing lower-middle-income state, India in particular (S16).No studies meeting our inclusion criteria took place in low-income countries, despite there being a high need for bottom-up technical innovation as the leading force in sustainable development and establishment of universal health and social services for citizens in those regions (UN, 2016).
Regarding the methodology of selected studies, only one adopted quantitative approach (S3) and one with mixed methods (S21).The rest of studies were qualitative, using mainly a single or multiple case study approach.In addition, there were four articles applied comparative studies across different countries (S2, 3, 13, 21).As we notice, a qualitative bias may prevail among studies regarding UDI in public health and social care sector.It might because qualitative have an in-depth and detailed focus on user-innovation processes including different actors (Konsti-Laakso, 2017).
According to CASP assessment, none of the qualifying studies received a maximum score of 20 on the quality evaluation scale, although all of them were ranked as being of good quality (scoring 14+) and 14 as high quality (scoring 16+).

UDI objectives
Specific motivation for each UDI varied across projects and fields in public health and social care.However, in general, there are three common objectives of involving users to generate bottom-up innovation, including creating a user-centric service/product, to democratize service-designing process and support a partnership, and to empower service recipients.In details, among studies in healthcare, the most mentioned objective of UDI is to create a user-centric service, expressed in terms such as "tailoring service", "customized treatment", and "people-oriented" healthcare (S 5, 7, 11).This target is quite closely followed by another two objectives -changing the "culture of care" by bringing a relatively equal cohort between patients and care provider (S5, 17) and increasing services' effectiveness through better-addressing patients' needs (S7, 11, 17).
In eldercare, the principal purpose of involving different user groups in the innovation process is to develop user-centric, value-added service/products, and to support a partnership between the elderly and healthcare professionals (S1, 2, 10, 13, 15).By achieving those objectives, the innovation can further empower the elderly by enabling their "security" "independence" "mobility" "self-management" and "social interaction", which in general increase the elderly's quality of life, expressed in terms such as aging well and active aging (S1, 10, 13).Likewise, for childcare, the use of UDI is to create service that addresses children's particular needs and eventually improve children's quality of life and potentials for development (S3, 14).
Within three studies related to community service, we found that empowering citizens (S8, 20) and supporting citizen engagement (S9) were primary reasons for adopting UDI.Additionally, developing the ailment among different stakeholders in the community for a better social, culture and economic return was also an essential aim (S8, 20).

UDI process
With the guide of the Innovation Wheel (Fig. 1), we categorized UDI process in public health and social care into eight progressive stages: opportunity identification, data collection, pattern recognition (data analysis), concept ideas, conceptualization, prototypes, test, and implication.Not all included studies described this eight-step innovation process in detail, nor did an innovation go through those stages consecutively.
First, a UDI often starts with opportunity identification.Such step aims to identify an exciting field where organizations might have a chance to offer a better or improved service in the future.Illustrated in S7, the innovation opportunity and objectives were initiated collectively by Department for Health Science and Technology at Aalborg University and Vendsyssel Hospital, Denmark, who decided to develop a new application, "The Heart Game", to support heart patients' participation in rehabilitation activities.According to S7, only 13 percent of heart patients engage in rehabilitation activities in Denmark.The reasons for low participation are manifold, including long transportation times, lack of information about activities, emotional instability, lack of motivation among the patients, and the time of day due to patients having a job.To better address existing problems, relevant organizations decided to apply UDI to develop a more individualized rehabilitation program (Dithmer et al., 2016).
After identifying the innovation direction, steps of data collection and analysis (pattern recognition) have to be taken to generate a better understanding towards targeted users' demands.Data collection and pattern recognition usually happen consecutively, and they are two essential stages across UDI since all included studies have addressed them particularly.Different approaches usually applied to ensure the richness and quality of collected data.S15, for instance, demonstrated a UDI project that used older people as co-innovators in the design, development, and implementation of an intelligent activity monitoring system.Abundant data were collected through multiple methods including in-depth interviews, telephone interviews, focus groups, and workshops.This led to an intensive analysis of older people's technological, psychological and social needs and preferences, as well as their requirements for an activity monitoring system for use in homes and residential care setting (Pratesi et al., 2013).
As the outcome of pattern recognition, organizations or innovation teams usually can better present innovation concepts in detail and evaluate the possible solution regarding how well they can be adapted to users' requirements.The Innovation Wheel (Fig. 1) recognize these as two stages, concept idea and conceptualization, which were perceived as difficult to separate from each other in our included papers.The UDI project in S11, for example, concept ideas and conceptualization were completed together with the participation of 176 elderly and 105 experts in the selected municipality.The outcome of conceptualization led to defining six important themes for further developing interactive programs for eldercare.Likewise, S14 presented a UDI program that supports parental co-development of technology for children with special needs.This project included nine parents, eleven children, three researchers, and four student teaching assistants in data collection when 79 ideas related to technology development were generated.After pattern recognition, five themes related to created concepts were identified and presented to fuel the further growth of technologies.
In the prototype stage, the first concrete example of a new product/service/idea is created.This stage was emphasized via all qualifying studies as it helps to visualize the original idea and improve the innovative concepts.Prototypes addressed via qualifying studies are most physical products or models, as the Cancer Evaluation and Education game presented by S4, new home-based telecare solutions in S13, and an ICT-based decision support tool in S20.Few articles presented non-physical prototypes, as the form of descriptions or experiments.S17, for example, discussed how an interactive health customer website could facilitate user-driven selfcare solution.The prototypes, as a description of the new solution, were pitched from different online scenarios at this site.Such user-generated solutions were further discussed and expanded by various other end-user groups (Rubinelli et al., 2013).
Followed by the prototype, the next stage of UDI is testing.Not all studies (e.g., S5, 6, 9, 11, 14, 17, 20) have detailed information towards this stage although it is crucial to provide an opportunity for users to react and further provide input, and for the further adjustment to the innovation.Presented via S2, a Norwegian UDI conducted its testing via involving 31 municipalities (out of 426 nationally) to generate feedback and further help for the improvement of methodologies and practical tools and service models (Bugge et al., 2017).It is important to mention that the innovation process is not never linear, and often there are loops in which the project circled.In S2's scenario, the new problems or needs from users were addressed after the test, which created a deep collaborated loop within prototype and test.
In the final stage of implementing, new products or services are launched in the market, or the new concepts are adopted.In S12, an innovative product, the Lung Cancer Care Toolkit, was launched nationwide after the one-year development process.During the implementation, a social marketing approach was applied to accelerate dissemination to target end-users (Luck et al., 2015).There is often another loop between implementation and test, where the innovation process may jump back and forth.As S15 stated, during the first year of implementation the new monitoring system for the elderly care, the advisory user group continually reported their feedback towards the system and helped the technical team to improve how the system affects the everyday lives of end-users in different settings (Pratesi et al., 2013).

Categorization of Users and Their Roles in UDI
Guided by previous literature, we categorized user groups in public health and social care into two main types: end-and intermediate users (Von Hippel, 2009;Szkuta et al., 2014).End-users referred to those who use innovation directly (Von Hippel, 2009).In our selected studies, they are groups of health and social care recipients (e.g., citizens, the elderly, patients, as well as children and their families); health and social care professionals (e.g., clinicians, nurses, social workers, researchers, and others); and relevant organizations' administrators.Intermediate users referred to users' organizations (Bogers et al., 2010).They included public bodies (e.g., health authority and institutions, municipalities, research agencies, and universities); private sectors; and NGOs.All selected studies have presented both end-and intermediate users in their innovation.However, recipients and professionals of health and social care are two primary end-user groups while public organizations are dominated intermediate users in our reviewed studies.Also, both endand intermediate users were involved intensively in eight innovation processes.However, in the first stage of the process, opportunity identification, intermediate users play the central role in identifying an innovation direction.In stages of concept ideas, conceptualization, or prototype, there are some cases (e.g., S7, 11, 17) that only involved the participation of technical developers; users were excluded.In the project of S7, for example, application designers analyzed alone the qualitative data from interviews with end-users and developed an initial architecture of prototype for addressing identified needs.
With applying the framework for Mapping of Users' roles (Fig. 2), we grouped different end-users' activities and responsibilities across UDI into four generic categories: exploration, experiments, tests, and innovation.Different groups of end-users may be involved in different stages of UDI and play various roles in the innovation process.At the early stage of UDI, especially in data collection, some end-users can play an indirect role in contributing knowledge towards their unacknowledged needs (exploration).In this scenario, the innovation team often apply a range of ethnographic methods, such as shadowing, participation observation, focus group, workshops, personal interviews, and user dairies, to obtain information and perspectives towards users' unacknowledged needs (e.g., S1, 7, 13, 14, 17, 21).In S1' UDI, for instance, three ethnographic methods were adopted including shadowing technique, focus group, and world café.The shadowing technique allowed discovering the latent needs of the users (the elderly), by following them during their daily activities and doing observations.Focus group was a method to debate about open topics, such as nutrition, mobility, and interpersonal communication; it also helped to investigate the users' perception of existing technological options.The World Café was a gathering that involved the users in discussing prepared topics, such as the most valuable elements that relevant to the older adults' health aging and the innovative ways to obtain them (Angelini et al., 2016).
In many cases, end-users can also directly involved in innovation process together with the project team (experiments) throughout the WHAT phase of UDI; however, they act as participator rather innovators.Those individuals' direct participation aims to help innovation team to fully understand their unexpressed requirement through a range of participatory approaches, such as co-design workshops and participatory research/design (e.g., S6, 8, 9, 11, 14, 15, 16).In S15's program, end-users (old adults) have been invited as co-designers who has a partnership with technical designers.The co-designers' role is both to advise the technical team about the research methods, and to participate in collaborative design workshops where their input and ideas will assist to "design" assistive technologies for themselves (Pratesi et al., 2013).
When coming to the WHEN phase of UDI, end-users are mostly involved in testing step.Common methods applied to involving end-users in testing include interview, focus group, questionnaires, clinical trial, usability testing, and user acceptance test (e.g., S2, 3, 4, 12, 14, 16, 19, 20, 12).Usability testing is a most common technique used in UDI to evaluate a prototype by testing it on users.It is a crucial usability practice because it offers direct input from real users who use the system (Bugge et al., 2017).Usability testing focuses on measuring a human-made product's capacity to meet its intended purpose.In contrast with other inspection methods where experts adopt different tools to evaluate a user interface, usability testing empowers users to modify and improve new product/service/concepts to ensure a successful diffusion in the future (Devlin et al., 2016;Timmings et al., 2016).
There are some UDI programs addressed via selected articles involved particular groups of the end-users as co-innovator who have specific control over whole innovation process.Those users can be part of expert teams, who are highly qualified in their area of expertise and possess specific knowledge that will contribute most to the innovation.The lead-user method, such as seminars, workshops and dialogue conferences, is most often applied to support the involvement of this group of users in UDI (e.g., S2, 8, 9, 12, 17, 18).Beside of experts, advanced end-users can also be co-innovator in UDI.They refer to those who often stay enthusiasm when it comes to specific products/service/concepts, and they are more knowledgeable than the average users.The living lab is an emerging approach used to support advanced users as co-innovator into UDI process (e.g., S1, 8).It offers a real-life setting and platform for learning and experimenting, facilitates the management of innovation process, and actively encourage continuous and meaningful interaction among user groups (Angelini et al., 2016).

Technologies' Roles with Users in UDI
All selected studies have a particular focus on the technology-use in UDI.However, the purpose of these technological appliances varied across different sector and projects.On the one side, many studies focus on how to adopt a user-driven approach to develop new technologies.In eldercare, for example, a dominative discussion was draw upon the engagement of the elderly into co-development of assistive technologies, such as monitoring, electronic sensors, and computerbased cognitive stimulation, which are likely to make significant contribution to the care of end-users in institutions and at home (e.g., S2, 10, 13, 15, 19).By involving end-users into co-production, those new assistive technologies can better meet individuals' special requirements on safety, security, mobility, and social contact.In healthcare, likewise, many studies examine the use of co-design approach for developing new health technologies, such as clinical information system and ICT-enhanced treatment, to fulfill end-users' ever-changing demands (S5, 7, 11, 16, 18).
On the other side, some studies locate their attention on the issue how to use different technological options to facilitate UDI (S4,8,14,16,17,20).According to those studies, ICTs, such as the internet (user websites), social media, and mobile applications, are most popular options to support UDI.Those tools can help with reaching out new users, encouraging their participation and communication, and empowering individuals in knowledge contribution for self-care, rehabilitation, or other social, cultural and economic development.
Users' relationship with technology is heterogeneous, depending on characteristics of user groups, type and features of technology, and many other tech-social factors.Therefore, at the initial stages of UDI, users' technology acceptance, preference and expectation are often examined (S1, 2, 3, 5, 8, 10, 12, 14, 19).As selected studies argue, the perceived usefulness of technologies is not enough to make such technical options acceptable.However, personal experience with technologies (familiarity) can greatly decide users' preference and acceptance (S1, 3, 5, 7, 12).In both projects of S3 and S5, the choice of the technical solution was made by collective decision among different user groups.Through workshops and meetings, users considered mobile technology serve as a most appropriate platform for support designed health service.Such platform leads to a reduction in traveling and operation cost.More importantly, it already owns large scale of installed base -most end-users have familiar with such technology and use it in daily life.This user-friendly, familiar mobile phone network can offer a technical platform making it easier for the users to participate in the innovation development and implementation (Calvo-Lerma et al., 2017; de Souza et al., 2017).
Further, if the innovation team wants to reduce impact of familiarity on users' adoption towards technology, it should consider applying invisible technological options (technologies that are transparent to the users' eyes) or options that easy to learn and operate reduce users' fear of novel solution (Pratesi et al., 2013;Angelini et al., 2016;Dithmer et al., 2016).Few studies also argued that some intrinsic motivation, such as playfulness, pecuniary interests, as well as aesthetic and sensory appeal, could also attract users to participate in the innovation process.In this sense, a widely adopted technology in the society that encourages that intrinsic motivation may be a promising way to support user engagement (S4, 7, 8, 12, 17).By using user websites and social media, S17 demonstrated how creating an online entertainment environment can facilitate user engagement, interaction, and knowledge sharing and learning (Rubinelli et al., 2013).Similarly, in the S7 project, mobile technology and gamification work tother as useful tools to motivate endusers in rehabilitation.Digital games apparently offer a promise for stimulating and entrainment, while mobile platform creates a connection between players and offer the chance for social and dynamic experience while also allowing these individuals to access the game independent of time and place (Dithmer et al., 2016).
No double, in some cases, the groups of end-users might not be cognizant of or automatic users of much of the more basic technologies that are fundamental for innovation.Therefore, suggested via included papers (S2, 8, 18), involving those people directly as a participator in WHAT phase of UDI are crucial.Such direct participation helps to explore appropriate technological options and methodologies to users; simultaneously; it can empower end-users by enhancing their technical knowledge and innovation competencies (Bugge et al., 2017;Gasco, 2017).In another word, UDI in health and social care with a technology end goal must apply deliberate methods for experiencing the potentiality of technologies for more significant innovation by co-designers themselves to made possible (Savory and Fortune, 2015).

Discussion
The most distinct characteristic of UDI in public health and social care is to involve wide ranges of the end-and intermediate users into both WHAT and HOW innovation stages to meet specific goals in the fields.Unlike the private sectors where the concept of UDI was often discussed within the last decade, UDI in the public sector is still a emerging theme which has been brought into focus very recently (Andersen and Jansen, 2012;Lassen et al., 2015;Dithmer et al., 2016;Hennala and Melkas, 2016).In general, UDI in public and private sectors pursue very different objectives.Where the private companies usually apply user-driven approaches to create valueadded products/services for maximizing the profit and enhance the competitive position in the market, the public institutions are engaged in UDI aiming at increased welfare and democracy (Waugh et al., 2013;Savory and Fortune, 2015;Paiva et al., 2016;Bugge et al., 2017).Also, the health and social care sector may different from other public sectors, with regard to a wider range of user groups with different interests and a more complex dimensions of user involvement (Rubinelli et al., 2013;Bugge et al., 2017;Gasco, 2017;Pihlainen et al., 2017).
Generally speaking, all included studies demonstrate a strong believe that UDI in public health and social care is far more sustainable than innovations generated by a single institution.By focusing on the end-users, health and care recipients and professionals, UDI first unfolds welfare principles, such as empowerment and client self-determination (Humphreys, 2015;Lassen et al., 2015;Angelini et al., 2016); it also facilitates changes of culture of care, and organizational structure where top-down innovation process was dominate (Sugarhood et al., 2014;Wihlman et al., 2014;Luckock et al., 2017;Valaitis et al., 2017).Through involving different intermediate users, UDI brings closer cooperation among different health and social care stakeholders and facilitate the change of existing system for a better solution to addressed revealed challenges (Lindgren, 2013;Purkayastha et al., 2015;Savory and Fortune, 2015;Bugge et al., 2017).However, despite benefits of UDI, many user-driven projects presented via include articles encountered challenges during the operation, and the management of which was not always successful.We will further explain this in the following two section.

Challenges of Continuous and Adequate Engaging Users
Our findings indeed suggest that end-users can play various roles across UDI process, and there are comment methods available to apply to support user participation.However, we also noticed that challenge exist concerns the need for the continuous and adequate engagement of different groups of end-users in public health and social care.Many UDI projects in our review only involved end-users in specific innovation steps, and there is still rare case that users are seen as co-innovators from beginning to the end and take significant control of whole innovation process.Moreover, several dominative barriers can hinder user participation in UDI: namely differences in willing of involvement, inefficient communication, and constrained resource.
We know that the willingness to innovate and learning are crucial to stimulate individuals to innovate (e.g., Savory and Fortune, 2015;Bugge et al., 2017).However, at beginning of many UDI projects, it is often to have some end-users were not necessarily enthusiastic, not with the same level of intensity at least, to involve the innovation process, and to learn and share relevant knowledge or experience (Pratesi et al., 2013;Pihlainen et al., 2017).To address such situation, early relationship-building with the members of innovation team beyond the innovation context may be necessary for identifying most enthusiastic participators, and cultivating a genuine partnership based on understanding, respect, and trust (Pratesi et al., 2013).Also, a more creative, flexible and interactive work approach needs to be adopted during the innovation process, and the presentation of complex information should be easy to understand.It might be beneficial to emphasizing the importance of participation as a democratic form of inquiry and grounding the outputs in the experiential aspects of users themselves (Pihlainen et al., 2017).Above two solutions, however, are time-consuming which might not be realist within a planned UDI within the constrained timeframe.
When there are many end-user groups involved in UDI process, inefficient communication between different parties can quickly rise because of diversity in individuals' interests, knowledge background, and purposes of participation (Purkayastha et al., 2015;Gasco, 2017).Ensuring that channels of communication are opened up and transparent across all partners, therefore, is essential to avoid significant conflicts and misunderstanding (Gasco, 2017).Further, reducing geographical distance could be promising in supporting the open discussion on the innovation progress and share ideas in real time (Purkayastha et al., 2015).Also, exploring new and interdisciplinary methodological tools to address communication issues is necessary (Savory and Fortune, 2015).
Finally, facing constrained innovation resource might be the most critical challenging of UDI.Budget and time constraints all limit the number of users can be engaged and the degree of their participation (Pratesi et al., 2013;Sugarhood et al., 2014;Angelini et al., 2016).In addition, further efficiently tailoring user-driven solution requires sustained assessment, support, and review over a period.However, in reality, lack of a financial and human resource often leave this continued work unclear (Sugarhood et al., 2014).Argued via Timmings et al., (2016), given the complexity of UDI, preparatory work to avoid costly implementation errors should be considered including assessing barriers and facilitators to change, developing a detailed budget plan, and assessing and establishing organizational readiness for change.Resources should be adequately assessed and evaluated from the outset and built into the development of the innovation proposal.During the UDI process, it is also vital to assess resource in real time to relocate resource if challenges arise (Pratesi et al., 2013).

Challenges of Creating Synergy between Users and Technologies
Included studies suggested that users' relationship with technology in UDI, is heterogeneous, depending on users' characteristics, the types and features of technology, and availability of technology facilitator.Technologies can be the outcome of a UDI process or a supportive tool for innovation.No matter of which is the case, users' perception towards specific technological options more or less affect their motivation, the kind of, and the degree of participation in UDI process.Such finding rises our concern about the issue upon how to create synergy between users and technologies across UDI process.Firstly, although it is essential that user insights and knowledge generated find their way into the development process, it is still challenging to adequate translation and transformation of users information and requirements into more technical requirements (and vice versa) (Joseph et al., 2010;Durose et al., 2017).
For example, when health professionals need to translate their medical knowledge into formal computer-interpretable formats, they might face the challenge of switching their role from clinician to a knowledge engineer, because such process requires learning the relevant tools, in addition to time resources.Further, the laws and regulations in health and social care often restrict the members of the care providing organizations in how they can use information and system (Lindgren, 2013).Some selected studies suggest using deliberate methods to support experiencing the potentiality of technologies by participated end-users for knowledge translation and transformation.For instance, through the live lab approach, end-users were free to experience different technical systems in a real-life setting, and they eventually turned to knowledge engineer who can develop new ideas via the support of technologies they have tried (Gasco, 2017).Likewise, by inviting end-users to join different technology clubs where workshops, seminars, and product presentations were frequently organized across a period, those people were encouraged to try the different technological options, to learn relevant knowledge, and eventually they could contribute their ideas for further technology improvement (Pihlainen et al., 2017).
Although above studies to some extent presented and discussed methodologies for creating synergy between users and technologies in UDI process, there are still problems remain.First, those methods are primarily on certain technologies and new applications, instead on the way users interact with different access network in their natural context.In addition, we lack enough evidence to see if those methods in UDI can fulfill the promoting objectives of projects and further lead to radical and sustainable innovation in public health and social care.As we can see, indeed, the living lab approach, interactive workshops and other methods, might support knowledge transfer and empower end-users to innovate.However, there is no study conclude those methods are better than other methods or innovation system in leading a better innovation outcome and more sustainable impact.

Conclusion
In summary, through a systematic review of existing evidence, we illustrated how current UDI in public health and social sectors operates with distinct characteristics in respect of its strategic innovation process and user-oriented, empower-based objectives.Both end-and intermediate users in directly and indirectly engage in different innovation stages and fulfill their roles for exploration, experiments, test and innovation.Exploring users with ethnographic tools and participatory methodologies are very popular at WHAT stages to in-depth understand what problem user face and therefore what solution can be offered.During HOW phases, however, living lab and lead users approaches often operate to encourage users as innovator or co-innovator for the innovation conceptualization and prototype, while various user testing tools are available for helping users to experience and evaluate designed prototypes.However, users as completely innovators seem less popular than users as co-innovators who only contributed partially to innovation teams.It may because the former is a relatively new field regarding its application in innovation processes and organizations in public health and social still face challenging to learn how to achieve the best results when the users have the most powers in the innovation process.

Further Implementation
With an understanding of how diverse the role of users is, it is likely that users will participate in UDI in many different ways.However, in some cases in practice, there is still a discrepancy between theory and practice.We discussed two crucial challenges among UDI in public health and social care: the need for continuous engagement with users cross each step of the innovation process, and the demand for creating mechanisms to integrate users and technologies to support a better innovation outcome.For addressing those challenges, we have to continuously explore and test new interdisciplinary methodologies or tools to support user participation in a natu-ral setting, to encourage a smooth interaction and communication, and to balance the power relationship between human and technologies.Further, we have to continuously and critically evaluate UDI outcome in respect of their promising objectives with a large scale of research evidence.By focusing on how UDI took place in public health and social care is not enough; we need to further systematically evaluate if UDI can achieve better results than other innovation types, can help to solve the confronted challenges in the field, and eventually change organizational culture towards innovation or the innovation system in policy network.

Limitation and Further Research
Since UDI is a somewhat new concept in public health and social care, which leads to this article become one of the first systematic reviews on this topic in which has a limited number of empirical evidence available.It decides the nature of this review to be more explorative and be descriptive in general and lack of an in-depth critical reflection due to constrained materials.Further, although we developed a research protocol in advance and used PRISMAS to guide the whole review process, there might be still bias in the selection of publications included and inaccuracy in data extraction.If we want to fully understand of UDI in the public health and social care, it is necessary to develop a theoretical and methodological framework, which is based on the concept of innovation but re-defined in a way that takes into account that the public health and social care is operating on other conditions than private companies.At the same time, the framework must take into account that users of present public services are a variety of different actors, who play different roles and participate in the process of innovation in very different ways.Also, more evidence needed to conduct a proper evaluation of the effect of UDI in the studied sector, as well as an assessment towards various methods applied to support user participation.

Table 2
reports the essential publication characteristics of selected studies (n=21).All eligible studies were published between January 2013 and December 2017.Data collection for those studies was undertaken during 2006 and 2016.The length of data collection period of selected studies was varied: seven studies less than two years(S1, 6, 7, 8, 17, 19, 21); while ten within two to five years (S2

Table 2 .
Basic Publish Characteristics of Selected Studies The Distinct Characteristics of UDI in Public Health and Social CareWe identified the distinct characteristics of UDI in public health and social care by addressing two important elements of innovation: objectives and processes.Through comparing, summarizing and discussing those two elements among different UDI, we can, in general, understand where the UDI in public health and social care is going, how it distinguishes itself from other types of innovation in the studied sector, and what kinds of action and effort needed for goals achievement.