UTM Computing Proceedings Innovations in Computing Technology and Applications Volume 2 Year: 2017 ISBN: 978-967-0194-95-0 1 BoBoiBoy Interactive Holographic Action Card Game Application Chan Vei Siang 1 and Farhan Mohamed 2 1,2 Faculty of Computing, Universiti Teknologi Malaysia (UTM), 81310, Johor Bharu, Johor, Malaysia {chanveisiang@hotmail.my, farhan@utm.my} Abstract. Augmented reality (AR) is a technology that integrate the virtual objects, such as graphics, sounds, with the real world environment to improve the users sensory perception toward the world they are viewing and experiencing. As for the hologram, this technology allows the virtual objects appears realistically in the real world environment, by letting the user to look at different perspective of these holograms when viewing from different angles. However, these technological devices are expensive and far reach from general consumers, such as Google Glass. The current technologies also make human-computer interaction difficult and makes holographic effect less realistic and unnatural. Therefore, a method of implementation of AR tracking and holographic display in the action card game is introduced. Keywords: Feature-based augmented reality (AR), holographic projection, holographic pyramid, action card game, human-computer interaction. 1 Introduction Nowadays, many researchers and computer engineers are trying to bring the computer graphics out from the television screen and displaying them in our real world environment. Such technologies include the augmented reality (AR) as well as the hologram, which changed the way people view and interact with the world. With the help of AR display devices, such as smartphones and smart glasess, user can receive additional information, which is in the form of informative graphics based on his or her field of view through the devices. AR is now significance in many area, such as engineering design, entertainment, gaming, and other industries. On the other hand, the use of holographic pyramid prism can produce the holographic results that can display 3D objects in the real world environment, by letting the user to look at different perspective of these holograms when viewing from different angles. Although technically holographic pyramid is not considered a hologram, but it can still create a holographic effect and illusion that can trick the end-user s eyes. However, there are also some issues and problems found in the current scenario. The current AR application is displayed in a small screen of smartphone where not many information can be displayed on the screen and the user is difficult to view the content on the screen [1]. Users also need to always hold their smartphones or tablets and target their devices to an image target, in order to view the virtual objects present in the real world environment [1]. Besides that, these AR technologies are only available for only one user to view the AR object at a time, socially separated user experience [2]. Furthermore, most of the current application of holographic pyramid is static display where user cannot interact
2 with the holographic display. Although there is some effort being made to improve the interactivity of holographic display, such as the addition of tracking sensor that allow the user to interact with the 3D objects using hand gestures [2][3]. This also created an issue where each holographic display device can accommodates one 3D objects only. In this project, a method to implementat the AR tracking technique and holographic pyramid in the action card game is introduced. The objectives of this project are: 1) to analyse the requirements of AR and holographic effect in the interaction on action card; 2) to design and develop a prototype that can shows the interaction on action card by using AR and holographic pyramid, as well as human-centric design; 3) to implement the AR technology and holographic pyramid into the action card application and then integrate the application into the computer. 2 Related Work 2.1 Choki Choki s BoBoiBoy AR In 2016, Anantarupa Studios cooperated with the Mayora Group a large Indonesian consumer products company, had created the Choki Choki s AR BoBoiBoy action card game application. By using this application, the characters in BoBoiBoy the Movie are superimposed in the real world environment with the help of handheld devices and AR technology. Besides that, the player can also initiate the battle between BoBoiBoy & Friend card and Enemy card, when the characters are facing toward each other. Figure 1 shows the BoBoiBoy AR characters (left) and the battle between AR characters (right) that are displayed in the smartphone screen. Figure 1. The BoBoiBoy AR characters (left) and the battle between AR characters (right) that are displayed in the smartphone screen [4] 2.2 Holus Tabletop Holographic Display Holus is a microwave-sized device with a projector inside the lid that beams four images of the same object onto the walls of Plexiglas prism and create an optical illusion to the user that the images appear to form in thin air, and the users can control the virtual objects with smartphone or tablet connected via Bluetooth or Wi-Fi [5]. The aim of this product is to let digital content can be easily shared in a group of people [6], without the use of smart glasses and wearable devices. The Holus is created based on the Pepper s Ghost illusion. Figure 2 shows a group of users share the same digital content by using the Holus.
3 Figure 2. A group of users share the same digital content by using the Holus [6] 3 Project Methodology Figure 3 illustrates the project methodology. In order to implement AR tracking technique and holographic display in the action card game, AR marker need to be detected in real-time and then the generated AR 3D object need to be projected to the real world environment through the use of holographic pyramid. Figure 3. Project methodology Several phases of project development to achieve the stated requirement are planned as follow:
4 Phase 1. Preliminary investigation and information collection of AR for holographic action card game: Research was conducted on exploring the existing AR and holographic display technology, as well as the technique to implement them, in order to achieve a highly interactivity and enhancing experience when playing the action card game. Phase 2. Design the AR tracking with holographic display technique: The AR tracking with holographic display technique is designed. The AR tracking technique uses the feature-based tracking, while the holographic display requires configuration, such as render texture facet process, as well as the design of holographic pyramid for display the 3D objects. Phase 3. Develop the action card game prototype: The frameworks of action card game are setting up, which includes the design of game concept and gameplay. The game prototype is developed in this phase by using suitable tools and game engine, such as Unity 3D. Phase 4. Integrate the AR tracking with holographic display technique for the action card game prototype: The AR tracking with holographic display technique is integrated into the action card game prototype. The holographic action card game is developed to support real-time 3D object generation based on the AR marker in the real-world environment and allow the interaction between the virtual objects. Phase 5. Testing and evaluation. This phase will test and evaluate the AR tracking with holographic display technique in the action card game. This will include the black-box testing as well as the usability testing. 4 Implementation Figure 4 shows the framework to integrate action card game with holographic display. The Vuforia SDK enable AR to detect and track the action card, while the application represents the game application developed by the Unity 3D. Figure 4. The integration of action card game with holographic display framework
5 The final outcome of the game application is to display the holographic characters in the holographic pyramid. 4.1 Implementing AR Tracking with Holographic Display The tracking technique used for this project is the feature-based tracking. The action card represents the image target that will be detected by the Vuforia. From the Vuforia Configuration found in the ARCamera, the Max Simultaneous Tracked Image field is set to 2, which means that the Vuforia can only detected and tracked maximum two image targets at the same time when there are two AR action cards presented in the webcam s field of view. Besides that, the AR Camera has fixed position and orientation, while the target objects can move around the virtual world according to the position of the AR action card, by set the World Center Mode to CAMERA in the the Vuforia Behaviour component under the ARCamera. In order to display the holographic 3D object in the real world environment, the texture facet rendering in the Unity 3D by placing four cameras in the Unity 3D scene to render the four viewports, which represent front view, back view, left view and right view, of the virtual object to be projected onto each side of the pyramid [7], as shown in Figure 5. Ultimately, all the cameras viewport are feed onto a TextureFacetCanvas that act as the final output to be projected on the monitor screen as shown in Figure 6 and the hardware configuration is as shown in Figure 7. Figure 5. The four-sided camera setup in the Unity scene (clockwise from top: FrontCam, LeftCam, BackCam, RightCam )
6 Figure 6. The TextureFacetCanvas displayed in the monitor screen (clockwise from top: render textures from FrontCam, RightCam, BackCam, LeftCam ) Figure 7. The hardware configuration of holographic pyramid 4.2 Developing Action Card Game The development of the holographic action card game is done in Unity 3D. There are three important processes involves when developing the game prototype, which are the preparation of BoBoiBoy character 3D models, create prefabs of the character and projectile, as well as scripting the characters behaviour and game logic. Figure 8 shows the concept and gameplay of this project.
7 Figure 8. The steps to generate the holographic display: 1) The AR action card before placing into the holographic pyramid; 2) The AR action card being placed into the holographic pyramid; 3) the 3D object displayed in holographic pyramid. 5 Result of the Project The result of this project is a standalone PC game application that integrating action card game with augmented reality (AR) tracking and holographic display. The main content of the action card game used the BoBoiBoy animation, such as the characters and the fighting style of each character. The user can interact with the game by placing the specially designed action card onto the action card platform, which is within the viewing range of the web camera to track the image target. Once the image target is detected, the 3D model of the action card character is displayed in the holographic pyramid. Besides display the characters, the user can also perform some action to invoke the interaction between the action card characters. There are two type of action card, which are the Allies and the Enemies. If there are two action cards tracked by the application where each card has type Allies and Enemies respectively, the 3D characters fought against with each other. If the two action cards tracked are of the same type, the 3D characters are also playing the running animation. When the characters fought against each other, the health bar of character who reaches 0 first is lost, and the battle ends. The lose character is destroyed and removed from the scene which leaves the win character and a text panel that display the winning character s name that wins the game in the scene. If the user to remove all action cards from the card platform, the scene is restarted as well as the game. Figure 9 shows the results of the project.
8 Figure 9. The result of the project: 1) The start scene; 2) The 3D character displayed in the scene; 3) The characters fought against each other in the scene; 4) The characters of same card type playing the running animation in the scene; 5) The battle result scene. 6 Conclusion The BoBoiBoy interactive holographic action card game application is one of the starting point to revolutionise the action card game, as well as the holographic application with user interaction using AR tracking technique. The project achieved all the stated objectives in Section 1. However, there are still have some limitation observed during the development of this project, such as the use of small monitor that results in small pyramid prism and hence, reduce the size of the holographic characters generated on the pyramid prism as well as reduced the space that can display necessary information, for example graphical user interface (GUI). Besides that, the small viewing range of webcam also sacrificed the performance of the system, such as can only detect the image target in a small area. References 9. Nazri, N. I. A. M., & Rambli, D. R. A. (2014). Current limitations and opportunities in mobile augmented reality applications. In Computer and Information Sciences (ICCOINS), 2014 International Conference, pp. 1-4. Available from: IEEE. [11 June 2017]. 10. Sutter, J. D. (2010). New phone apps seek to 'augment' reality. CNN. Available from: http://edition.cnn.com/2009/tech/10/24/tech.augmented.reality.apps/index.html?_s=pm:tech >. [7 June 2017]. 11. Bovier, F., Caggianese, G., De Pietro, G., Gallo, L., & Neroni, P. (2016). An Interactive 3D Holographic Pyramid for Museum Exhibition. In Signal-Image Technology & Internet-Based Systems (SITIS), 2016 12th International Conference, pp. 428 434. Available from: IEEE. [11 June 2017]. 12. Anantarupa Studios (2016). Choki-Choki AR Boboiboy v19.0 [Mobile Application Software]. Available from: <https://play.google.com/store/apps/details?id=com.anantarupastudios.chokichokiarboboiboy &hl=en>. [10 June 2017]. 13. Metz, R. (2015). Virtual Creatures in a Box, Controlled by You. Available from:
<https://www.technologyreview.com/s/535456/virtual-creatures-in-a-box-controlled-by-you/>. [11 June 2017]. 14. Holus (2017). The World s First Interactive, Tabletop Holographic Display. Available from: <https://hplustech.com/>. [11 June 2017]. 15. Muhammad, A. B. A., Ismail, N. A. F., & Sunar, M. S. (2016). Reflective Prism Display Using Pepper s Ghost Technique Software Toolkit Plugin for Unity 3D. Jurnal Teknologi (Sciences & Engineering). 78(12 3), pp. 189-196. 9