Evaluating the impact of a service-oriented framework for healthcare interoperability

Transcription

1 Evaluating the impact of a service-oriented framework for healthcare interoperability Stylianos DASKALAKIS and John MANTAS Health Informatics Laboratory, Faculty of Nursing, National and Kapodistrian University of Athens, 123 Papadiamantopoulou street, Goudi , Athens, Greece. {stdask,jmantas}@nurs.uoa.gr

2 Contents Introduction Research model and method Data Analysis Results Discussion Conclusions Further work

3 Contents Introduction Research model and method Data Analysis Results Discussion Further work - Conclusions

4 Introduction A service-oriented architectural (SOA) framework was implemented at the Health Informatics Laboratory, Faculty of Nursing, University of Athens, Greece [1] The SOA prototype framework: Aims to investigate the adaptation of service oriented architectures (SOA) for achieving healthcare interoperability [2-4] Attempts to formulate a virtual patient record (VPR) paradigm, operating over a serviceoriented framework Exploits the use of the business process execution language (BPEL) standard for the design and implementation of business and technical processes that coordinate software services in order to achieve interoperability amongst a diversity of applications [2,5]

5 Introduction A subsequent step is conducting an evaluation Investigation of strategies for evaluation With respect to interoperability, a variety of aspects need to be considered, either technical or non-technical Associated with a variety of concerns, perceptions and approaches whereas also involves several standards, methods, stakeholders and roles [1]. Decided not to concentrate on pure technical characteristics, aiming to assess perceptions and the trend towards the proposed technology Conducted an empirical assessment based on a hypothetical adoption of the prototype

6 Contents Introduction Research model and method Data Analysis Results Discussion Further work - Conclusions

7 Research model & method The study attempts to utilize the initial proposition of the DeLone and McLean model of Information Systems Success [11], as modeled by Iivari[12] DeLone and McLean define six dimensions namely system quality, information quality, use, user satisfaction, individual impact and organisational impact [11]. By definition, the dimensions of system and information quality both affect the dimensions of use and user satisfaction respectively. Furthermore, use and user satisfaction affect each other while both also affect the dimension of individual impact. Finally, individual impact affects organisational impact [11-14]. The study attempts to empirically encapsulate SOA and VPR characteristics within the six dimensions of the DeLone and McLean model The study refers to a hypothetical adoption, due to the prototype nature of the SOA framework The aim of the study is to observe and discuss on the trend and the perception that such a prototype might create amongst participants with clinical and/or informatics background

8 Research model & method Research method and data modeling approach were adopted from Iivari [12] : Questionnaire along with number of measures per dimension and scaling, where applicable Extended to include the dimension of organisational impact of IS Success model Second order model, regarding the dimensions of system and information quality Relevant Hypotheses were formulated Two scenarios, in order to reflect the bi-directional relation between the dimensions of use and user satisfaction

9 Research model & method Standarised characteristics per dimension adopted from previous studies [12,15], combined with empirical propositions : System Quality (24 measures) Flexibility, Usefulness, Interoperability, Maintainability, Scalability and Reusability Attempt to reflect possible SOA characteristics and architectural advantages [1,2,5] Information Quality (24 measures) Completeness, Relevancy, Consistency, Data Unification, Semantically mapping capabilities and Data Standardization Attempt to reflect possible VPR characteristics related with the prototype [1] Rest of dimensions include: Use (2 measures), User Satisfaction (6 measures), Individual impact (6 measures) and Organisational impact (6 measures)

10 Research model & method Study Sample Sample size: 62 participants Total sample relevant with healthcare domain / diverse professions Overall, 35.5 % of the sample employed at healthcare organisations Diverse backgrounds: Mainly past and present postgraduate students at the Faculty of Nursing (Health informatics and Healthcare services management) Minor set: either from nursing departments of healthcare organisations or the healthcare IT industry, who agreed to participate in the study on an individual basis Actual evaluation A short introduction on design issues and a live demonstration of the prototype were presented to the evaluators Afterwards, participants were asked to complete an anonymous questionnaire Evaluation was based on hypothetical adoption due to the prototype nature of the framework

11 Contents Introduction Research model and method Data Analysis Results Discussion Further work - Conclusions

12 Data analysis The data gathered was analyzed using partial least squares (PLS) Emphasis on: Measurement model Item loadings Construct reliability Convergent validity Discriminant validity Structural model [12,13,17-20] The software package smartpls version 2.0 M3 beta was used [16]

13 Data analysis Hypotheses were formulated, according to the theoretical basis of DeLone and McLean H1: System Quality positively affects User Satisfaction H2: Information Quality positively affects User Satisfaction H3: System Quality positively affects Use H4: Information Quality positively affects Use H5: Use positively affects User Satisfaction H6 : User Satisfaction positively affects Use H7: User Satisfaction positively affects Individual Impact H8: Use positively affects Individual Impact H9: Individual Impact positively affects Organisational Impact

14 Contents Introduction Research model and method Data Analysis Results Discussion Further work - Conclusions

15 Results Measurement model Item loadings All values considered reliable, since they satisfied the relative threshold values Construct reliability Internal consistency Cronbach s alpha: All values were considered reliable, except the value of Use Composite Reliability: All values were considered reliable Convergent Validity All constructs generated a satisfactory value Exceptions were system and information quality Discriminant Validity Inadequacies observed Structural Model Bootstrapping (500 resamples) [12,13,17,18,20] The examination of the t-values was based on a two-tail test with 3 statistically significant levels - p<0.05 (*), p<0.01 (**) and p<0.001 (***)

16 Results flexibility interoperability 0,204*** 0,204*** 0,181*** usefulness 0,182*** 0,193*** 0,193*** R ,321* R 2 0,559 0,598 0,226*** System Quality 0,215 Use maintainability 0,226*** 0,218*** 0,218*** 0,220*** scalability 0,221*** 0,225 0,309 0, reusability 0,792*** 0, Individual Impact 0,793*** Organisational Impact completeness R 2 0, R unification 0,225*** 0,225*** 0,205*** 0,205*** 0,456*** 0,260 0,792*** 0,216*** standardisation 0,216*** 0,174*** 0,174*** semantically mapping 0,222*** 0,222*** 0,203*** relevance 0,203*** R Information Quality 0,431* 0,546*** User Satisfaction R 2 0,700 0, *** consistency Results of the Structural Model (Source: Adopted from Daskalakis [22], p. 140)

17 Contents Introduction Research model and method Data Analysis Results Discussion Further work - Conclusions

18 Discussion The significant relationships may emphasize that participants focused more on the quality of information provided Potentially, the dimension of information quality was more appreciated because the majority of the sample did not have strong IT background Sample synthesis appeared comfortable with high-level concepts related with information quality The live demonstration of the VPR capabilities assisted the participants to comprehend the characteristics of information quality due to interaction via a homogeneous Web-based environment Adequate comprehension of information quality consequently leads to User Satisfaction Continuously, perceived User Satisfaction positively affects Individual Impact which in turn affects Organisational Impact

19 Discussion System quality appeared with a minor contribution in terms of significance of the paths in the structural model Attributes were mainly related with back-end functionality and concepts Such concepts seemed more advanced to be appreciated by non-technical people through a high-level overview Overall the hypothetical conditions of the SOA framework prototype demonstration and the lack of real applications consequently produce hypothetical opinions and perceptions from the participants side

20 Contents Introduction Research model and method Data Analysis Results Discussion Further work - Conclusions

21 Further work - Conclusions Limitations of the study Hypothetical nature of the prototype under evaluation, thus not realistic conditions Empirical encapsulation of SOA and VPR attributes within the dimensions of the DeLone and McLean model Sample synthesis originates from a variety of healthcare professionals The perception of each distinct stakeholder group is not clearly depicted Further work Future adoption of the prototype in real healthcare settings, in order to investigate its impact in pragmatic conditions Further empirical evaluation of alternative and/or additional attributes on possible SOA and VPR characteristics Thorough empirical assessments on several distinct stakeholder groups, in order to capture specific perceptions and beliefs

22 Further work - Conclusions A service-oriented interoperability prototype with VPR capabilities has been developed and empirically evaluated, based on a hypothetical adoption The evaluation strategy was based on the DeLone and McLean model of information systems success, as modeled by Iivari[12], by encapsulating specific service-oriented and VPR characteristics in a subjectivist manner An empirical study was conducted amongst sixty two participants in order to observe their perceptions regarding the prototype framework The current work aims to move towards an evaluation strategy for SOA implementations with VPR functionality in the healthcare domain

23 References [1] S. Daskalakis and J. Mantas, A SOA framework for healthcare interoperability using BPEL: Design and implementation, Proceedings of the 4th International Conference on Information and Communication Technologies in Healthcare, Samos, Greece, (2006), [2] M.P. Papazoglou, Service-oriented computing: Concepts, characteristics and directions, Proceedings of the 4th International Conference on Web Information Systems Engineering (WISE 03), Rome, Italy, (2003), [3] D.W. Forslund, J.E. George, S. Koenig, T. Staab, M. Kratz and R. Carter, The importance of distributed, component-based healthcare information systems: The role of a service-based architecture, Proceedings of the 14th IEEE Symposium on Computer-Based Medical Systems, Bethesda, MD, USA, (2001), [4] W. Wang, M. Wang and S. Zhu, Healthcare information system integration: A service oriented approach, Proceedings of the 2005 International Conference on Services Systems and Services Management (ICSSSM '05), Chongqing, China, (2005), [5] Z. Zou and Z. Duan, Building business processes or assembling service components: Reuse services with BPEL4WS and SCA, Proceedings of the 4th European Conference on Web Services (ECOWS'06), Zurich, Switzerland, (2006), [6] F. Malamateniou and G. Vassilacopoulos, Developing a virtual patient record using XML and web-based workflow technologies, International Journal of Medical Informatics 70 (2003), [7] A. Berler, S. Pavlopoulos, G. Karkalis, E. Sakka, G. Konnis and D. Koutsouris, Implementation of a novel virtual patient record architecture, Proceedings of the 24th Annual International Conference of the IEEE in Medicine & Biology, Houston, TX USA, (2002). [8] S. Chu, From component-based to service oriented software architecture for healthcare, Journal on Information Technology in Healthcare 4 (2006), 5 14 [9] R. Lenz, M. Beyer and K.A. Kuhn, Semantic integration in healthcare networks, International Journal of Medical Informatics 76 (2007), [10] E. Ammenwerth, J. Brender, P. Nykänen, H-U. Prokosch, M. Rigby and J. Talmon, Visions and strategies to improve evaluation of health information systems: Reflections and lessons based on the HIS-EVAL workshop in Innsbruck, International Journal of Medical Informatics 73 (2004), [11] W.H. DeLone and E.R. McLean, Information systems success: The quest for the dependent variable, Information Systems Research 3 (1992), [12] J. Iivari, An empirical test of the DeLone-McLean model of information system success, The data base for Advances in Information Systems 36 (2005), [13] J.L. Roldán and A. Leal, A validation test of an adaptation of the DeLone and McLean s model in the Spanish EIS Field, In Critical reflections on information systems: a systemic approach. J. J. Cano. IGI Publishing, [14] A. Molla and P.S. Licker, E-commerce systems success: An attempt to extend and respecify the DeLone and McLean model of IS Success, Journal of Electronic Commerce Research 2 (2001), [15] M.M. Yusof, R.J. Paul and L.K. Stergioulas, Towards a framework for health information systems evaluation, Proceedings of the 39th Annual Hawaii International Conference on System Sciences, Big Island, Hawaii, USA, (2006), 95a. [16] C.M. Ringle, S. Wende and A. Will, SmartPLS 2.0 (M3) beta, Available from: University of Hamburg, [17] D. Gefen, D.W. Straub and M. Boudreau, Structural equation modeling techniques and regression: Guidelines for research practice, Communications of AIS 4 (2000). [18] S. Chea and M.M. Luo, Cognition, emotion, satisfaction and post-adoption behaviors of e-service customers, Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS 07), Big Island, Hawaii, USA, (2007), 154b. [19] M.M. Shepherd, D.B. Tesch and J.S.C. Hsu, Environmental traits that support a learning organization: The impact on information system development projects, Comparative Technology Transfer and Society 4 (2006), [20] O. Yigitbasioglu, Information sharing with key suppliers: A contingency approach, Proceedings of the 10th European Conference on Accounting Information Systems, Lisbon, Portugal, (2007). [21] C. Fornell and D.F. Larcker, Evaluating structural equation models with unobservable variables and measurement error, Journal of Marketing Research 18 (1981), [22] Application integration and interoperability of healthcare information systems. S. Daskalakis. PhD Dissertation, National and Kapodistrian University of Athens, 2007.

24 Thank you!