The 10 th International Scientific Conference elearning and software for Education Bucharest, April 24-25, / X

Transcription

1 The 10 th International Scientific Conference elearning and software for Education Bucharest, April 24-25, / X An Online Cloud Based Mobile Enabled 3D Human Body E-Learning Solution Vasile-Alexandru Butean, Alin Moldoveanu, Elena Ovreiu, Anca Morar, Alexandru Egner Faculty of Automatic Control and Computer Science, Politehnica University of Bucharest,Splaiul Independenţei nr. 313, sector 6, Bucharest, Romania Abstract: Nowadays, 3D digital atlases of human body are extensively used in medical education. A 3D representation of the anatomical structure is quite difficult requiring many hardware resources. Therefore, a 3D full human body rendering on a Smartphone or tablet is more difficult because of the limitations, both software and hardware. In this paper we propose a system that allows full 3D human body model to be available directly on a mobile device. Using last-minute 3D rendering technologies, this study shows that benefiting from cloud computing modern expansion and processing power, it is possible to use the mobile device only as a display and the power of cloud as a source from unlimited resources for processing, storage and video rendering. This kind of system involves a single model available for everyone and has a great capability. More exactly, when the model is updated, everyone can see those updates in real time (for instance, if a doctor from a part of a world adds a scenario, a description of a small part of the model or a new version of an existing piece (heart, lungs, etc.) in a few minutes a student from the other side of the world can see on his own model those changes. A 3D human body model is itself a challenging research topic and in this paper it is shown that such a model can be updated in real time and be available on smart modern mobile devices with no more than several screen touches. Keywords: 3d human body; healthcare tool; cloud computing; mobile enabled; cross-platform; e- learning; I. Introduction Medicine is a very important aspect of our lives being in the same time a science that requires a lot of time, patience and resources. The amount of information held inside of a human body is huge, and that information must be taken out from books and atlases, understood and applied by students and doctors. It is quite difficult for students to learn the 3D structure of the human body from 2D images provided by the traditional medical atlases. Moreover, the use of cadavers in universities as a learning tool is becoming forbidden. With the improvement of the hardware capabilities, 3D computer graphic techniques and the development of the medical acquisition techniques from the last years, the traditional medical atlases have been successfully replaced by the 3D digital ones and 3D reconstruction of the human body can provide an educational tool, replacing the use of cadavers. 3D digital atlases provide a better visualization and a more intuitive understanding of the human anatomic organs for students and not only. The students can interact with the 3D anatomy models, focus some areas of interest, and explore the human anatomy. The relationship between the anatomic structures is provided. These facilities definitely allow a better and easier understanding and learning of the human anatomy.

2 From a technical point of view, an important aspect of medicine is the 3D modeling of the human body. Initially, the digital atlases were limited (in terms of displaying and visualizing the 3D synthetic models of the human organs), today, the atlases are not limited to the surface visualization, but the internal anatomy visualization is required. The accurate representation of the human anatomy is a challenge even today with the all available technology. Such a model is a very complex, and requires many hardware resources. Moreover, it is quite difficult to render, manipulate or handle a 3D model of the human anatomy. Nowadays, mobile devices are dominating the world of technology becoming an important tool in the learning process. Running a full 3D human body directly from a Smartphone or tablet, for learning purposes, hospital practice or even for direct guidance into medical procedures would become a trivial method but it is still quite difficult because of the limitations, both software and hardware. In this paper we have proposed a platform that can render a complex 3D human body using the power of cloud computing environments. The model itself is using adaptive technologies to display dynamic 3d content on any kind of modern device from tablets to smartphones and notebooks. This paper covers the following key topics: in the Related Work section we list some medical educational tools related to our, in the next section, the system design and the implementation details are presented. Section 4 contains the results produced by our system. The conclusions and future work are discussed in the last Section II. Related Work E-learning has become more and more important in the last years in medical education. Traditional human anatomy atlases were replaced by the 3D digital ones which offer a better 3D visualization and a more intuitive understanding of the human body. Educational tools represented a subject of research in the last years such as [1] where a digital atlas of the human body was generated in order to help students easily learn about the human anatomy. The system provides real time sectional images of any part of the human body. A platform that provides interactive anatomy models for learning and teaching is proposed in [2]. The platform based on a Virtual Reality system allows adding digital supplementary learning materials (animation, audio, video, etc). Furthermore, the platform provides a multi-touch mobile learning interface, allowing the access of the models from the mobile devices. Recently, in [3] a new educational tool - a virtual 3D atlas of human body was proposed. The application provides, besides an interactive, animated 3D atlas, an immersive facility which allows a better visualization and therefore a better understanding for students. Moreover, the system provides dynamic graphical data, such as animation of the pulmonary alveoli. The system proposed in [4] is an interactive atlas focused on the representation in detail of the human heart and of the artery detection. The system provides information about each morphological structure present within the human heart. The project BioDigital Human [5] provides an interactive 3D model of the human body for educational purpose. The system allows exploring not only the anatomy, but also visualizing the effects that a disease has on an anatomic organ. In this way, the students can learn more easily. Visible Body project [6] is a complex platform that offers a detailed human anatomy atlas together with the functionalities of the human organs, the detailed description of the heart and circulatory system, brain and nervous system anatomy and the muscle system. The platform can be accessed from a PC, Mac, and mobile devices as well (ipad, Android, etc.). The system can be used by both medical students and young physicists for learning and for teachers during the classes. There are a lot of 3D digital atlases which present detailed description of human anatomy dedicated to students learning [7], [8] but because of their complexity, most of them can be accessed only from the desktop devices and not form mobile ones. In this paper we present a platform that is an accurate 3D body replica that can be customized in detail, and a complex mechanism for recording every cause, effect or treatment for every known and unknown illness scenario. Our solution fosters win-win situations such as the one in which the physician s job is simplified by accessing other specialist opinions, not only accessing that

3 information but also showing exactly those changes or explanations on a virtual body, or the one that allows a student to learn in both standard and modern way by seeing in the same time his version of the human body but with changes and explanations from students around the world. Our solution is built from multiple modules. Each module has a different functionality but uses the same intuitive user interface and adapts to the user s device not only as simple as a native platform application but also with many other features like frames per second calculations, processing power estimation, mobile data traffic control, data size, etc. What makes everything special is that our solution allows people to merge their medical knowledge and experiences as easy as possible from any device, into a system that can start from nothing and can become a large database of medical information mapped into a dynamic visual 3d human body. III. System Design and Implementation 3.1 Application scenarios Students, medical staff, people from all around the world can view a cross platform 3D human body model, anyone can learn, study or look through a model that can be populated and seen in many different ways: muscles, bones, blood vessels, body parts, etc. The most important aspect is that a user can visualize on the model how a disease can affect human body parts, can read comments and observations about a specific body part and/or disease. Physicians can review the body parts added in the system, add comments, write feedback. Based on those reviews some body parts are accepted and then can be available for all users. The same mechanism is applied regarding diseases. If a doctor is adding right now a new observation or changing a parameter for a disease, in real time all the users that are watching that disease will start to see those changes. Any user can add a new disease, modify or extend an existing one by changing various parameters in the system. Even if we present it as the last feature, we think that generating statistics and exporting various data sets is extremely important, because in a matter of seconds everyone can have a great view from above regarding a lot of interesting things like: how many diseases from the system are affecting the brain, what is the probability to have a diseases at a specific age, etc. 3.2 Medical accuracy From the early stages of the project we have always received help and advices directly from experienced physicians. During every development stage doctors where permanently involved into various activities like: disease description and parameters, developing data sets for disease testing, accuracy of the 3D body parts and the way that those organs are positioned inside the model, etc. We believe that without help from medical experts it is impossible to work on such a healthcare tool, because every assumption, later, can lead to a lot of problems and medical errors. 3.3 Data validation We believe that trust and accuracy are two of the most important aspects of every tool that has something to do with medicine, that is why the information that is received can pass into the databases of the application only after specialists approval, and above all, it must pass through more than one inspection. Using this kind of user logic we are preventing problems, abnormalities, wrong information to reach to our model. We are building an app that receives only verified data (parameter measurements, affected body parts, treatments side effects) from each user. A doctor can consult pieces of data provided by other people (students or other doctors) and can agree or disagree with their findings. Only after several doctors and medical specialists inspect the data provided and agree over its accuracy, other users can view it.

4 3.4 Users Having in mind that our project is based on human contribution and that we want to use the power of many, our system allows many user types. Each user type has different rights and can see a partially different 3D model. Users with a lower access level can see a robust version with verified body parts, complete information and description of diseases while users with higher access rights can see a more detailed model with latest body parts (tested and untested) together with currently developed and implemented diseases and scenarios. In this way we make sure that the people with less healthcare experience can have less problems regarding user experience and can receive only reliable information. We are using the knowledge and skills of the experts to help us get feedback and to help and guide other users right from the beginning of the process and during all its phases: adding 3d models, configuring diseases, modifying existing parameters or adding new ones, etc. 3.5 The Technology Since the beginning of the project, because we were trying to solve the problem of having a multi-platform compatible model that allows us to create mobile ready content, the most challenging part of working at this platform was finding and using technologies that can support and provide enough technical features with good documentation and less unresolved bugs. Our project was developed using the following technologies and development environments: Technologies.NET Framework ;ASP.NET - MVC 4; C#; ADO.NET Entity Framework; SQL Azure; Microsoft SQL; JQuery, JqueryUI; JavaScript; HammerJS (TouchScreen functions); D3.js ThreeJS (WebGL JavaScript Framework); Development and Design environments MS Visual Studio 2012 MS Windows Azure SDK MS SQL Server Management D StudioMax Blender Maya Cinema4D Adobe Illustrator (UI) 3.6 Cloud computing environment We achieved all functionalities by using the latest technologies for advanced browser based 3D modeling (for precise view of the components of the human body) combined with the outstanding power of Windows Azure cloud computing services to ensure prompt and efficient interaction through the application. The most important service that we have used was Blob Storage. After exporting and adding the parts of the body to the app (3D models), each was converted into a blob, so we got a lot of blobs, each one containing a small piece of the human body. This way we could benefit from CDN (Content Delivery Network) that allows us to send data from multiple locations or from the closest and most reliable location to the user. Since in the cloud we have unlimited processing power we decided to use that feature not only to convert the 3d object into a blob and vice-versa but also to do a small preprocessing phase by converting the classic 3d object to WebGL 3d object server side so that a lot of the processing power was moved from client side to server side, which by the way is the thing that allows us to render the models on low performance devices. Another important aspect that makes our solution unique was achieved using object oriented databases because in this way we can benefit from the elasticity of cloud based databases to map all the 3D features (color, size, neighbors, status, etc) not inside 3D objects itself but directly into a database and from there to the app, so whenever someone tries to edit a 3D object feature we are not editing the object itself, but we are changing only the info in the database.

5 3.7 Mobile enabled, cross-platform After finishing the technological research we figured out that there are 2 ways of approaching the problem: a) Design and maintain multiple similar systems with common databases and data providers (for 3d object files) but with multiple native apps, an app for each platform b) Design and build browser based system that can be compatible with multiple platforms but that has known problems caused by using various libraries and solutions that have not reached technological maturity Most of the similar platforms are using the advantages of option a) because of its stability, but now that the technology is evolving we decided to choose the b) option. Since browser based 3d object rendering using WebGL does not have full hardware support, a decision was taken to find a way to render different quality levels so that a low performance device can render the same 3D model but with reduced visual quality using not only WebGL but also SVG HTML Canvas and even JPG for some devices. Using the benefits of Three.js we implemented a FPS filter and we used that filter to see if a device can render a test model, calculate the FPS, check compatibility, check existing hardware support, etc. Our platform has 6 different rendering quality levels, 3 for WegbGL compatibility, 1 for SVG, 1 for HTML Canvas and 1 for JPG screen-shots, so the same 3D body part can be converted into any one of those levels to fit the user s device. If a device is asking for a 3D resource we are testing it is abilities starting from high quality to low quality and calculate an overall score for each level. Considering his score and his internet connection speed we deliver an already preprocessed 3D model to the device. Since one device does not run only our app, we are not testing this scenario only in the beginning because we might be wrong and after the initial test the user can open simultaneously another 3 rd party app and if we keep delivering high quality data without testing from time to time, we might force the browser to crash and risk to loose not only our temporary data but also other browsing data or history. Making our platform mobile ready was the toughest part but it also made our project to be innovative and adaptive to modern users. IV. Results 4.1 3D Human Body

6 As a result of our work we got a working browser based 3D human body model that can run on mobile devices and can benefit from hardware acceleration (when device is compatible), or can use software rendering with various techniques. 4.2 Menus and user actions Our platform has touch-enabled user menus and buttons. The user menu was developed in such a manner that can save processing time, prevent 3D model flickering and page refreshing. From within the menu the user can select what body parts he wants to see and can interact with the model using classical 3D buttons like move, zoom, reset perspective, etc. 4.3 Performance When we are talking about mobile word, alongside with compatibility, performance is a very important aspect. Having this in mind, we make a lot of precise measurements (some of them available to the user and some not) in order be able to determine which is the right level of details a device can render. Using FPS calculus we are deciding if a device can benefit from hardware acceleration or not. Using FPS history we can easily figure out what are the body parts that demand more resources. Knowing the exact size of a sample 3D object we are can predict how fast is the data connection in order to know when to update the model or when to stop until the previous model is ready for the view.

7 4.4 Handling Diseases As seen in the example above, users can browse through existing diseases and can see in real time how a disease can affect a human body part. In the beginning the lung was healthy, after the user clicked the Lung Cancer option, the lung became brown, in the last stage became black and his left lobe started to shrink. The system is still in alpha version, but after our team of doctors will add more parameters to the system the number of available diseases and visual effects will increase. 4.4 Statistics and exports Proneness to diseases is a very important export that is based on existing datasets type from the database that can show us in real time how the diseases in the system are affecting the body parts, which is the most affected body part, etc. How most common diseases affects the population based on age is a statistical measurement based on what age (range of the most likely to appear age)has been introduced alongside with each disease.

8 V. Conclusions In conclusion, we have proposed a browser based mobile ready e-learning platform which provides an interactive 3D human body atlas dedicated to both students and young physicians. The atlas contains a detailed and accurate representation of the human anatomy and contains a unique interesting feature: real time modification of body parts parameters directly related to an associated disease. For instance the lung cancer" (see the example in the results part). The latest technologies D3.js, Three.js, SQL Azure, jquery, jqueryui, Hammer.JS, etc, are used to implement the platform permit an easy visualization of and interaction with 3D human body model. The cloud computing environment (Windows Azure) allows our platform to run smoothly even on low performance devices such as mobile devices. Running the platform on mobile devices is a very convenient feature of the proposed system, in this way the platform is accessible to the students even if they are in a hospital during practical classes or during the courses. The digital atlas could also be very useful for teacher during the classes or doctors that want to keep a well detailed research on how a particular disease can spread and affect organs. As a future work, for the devices that can benefit from the hardware acceleration, the authors are thinking to add more complex 3D parts so that the users will be able to see not only the outer part but also what is inside a human body part (for example how a heart works and what happens every time it beats). Another great feature that will be ready soon, is one for low performance devices that will allow our platform to convert user actions into predefined scenarios and instead of showing 3D models direct to view, the platform will output real-time generated video content according to every user action. Acknowledgements We would like to thank our students from the team of programmers who helped us to implement our project idea. Hurtupan Bogdan, Cotofana Cristian, Benea Naomi, Bogdan Diaconescu. Reference Text and Citations [1] Kumar, T.S., Pankajan, V. S. 3D Atlas of Human Anatomy for Educational Purposes. In Proceedings of the 4th National Conference, INDIA, 2010 [2] Gopalakrishnakone, P., Jianfeng, L., Sun, G. P., Abeykoon, Fernando, A., O., N., Cheok, A. D., A Multimodal Virtual Anatomy Learning Tool for Medical Education, In Entertainment for Education, Digital Techniques and Systems 2010; LNCS 6249, pp , Springer Berlin Heidelberg [3] mrol, A., Górski, F., Grajewski D., Zawadzki P., Virtual 3D Atlas of a Human Body Development of an Educational Medical Software Application, International Conference on Virtual and Augmented Reality in Education 2013 [4] 2D Atlas and Coronary Detection of Human Heart for Educational Purposes, Kumar, T., Pankajan, V., Dora, B., Tarun, E., Varun, E., Proceedings of ICETECT 2011 [5] [6] [7] [8] Zygote Project-