Entertainment Computing (EC) Topics WS 2018/19

Transcription

1 Entertainment Computing (EC) Topics WS 2018/19 Praktikum mit Bachelor-Arbeit Praktikum 1 / 2, Master Medieninformatik Prof. Helmut Hlavacs helmut.hlavacs@univie.ac.at EC1 Invisible Sound go Product Take over the App Invisible Sound (composing app for blind people), port to newest Unity, and bring it as a product into the App Stores. EC2 Blind Adventure go Product Take over Blind Adventure (game engine for blind people), port to newest Unity, implement a handful of new features, bring into App stores as product. EC3 Port Internet Hero to Unity On the basis of existing ports and the game Internet Hero, port a part of this game from Flash to Unity. EC4 House Pack Create an asset pack that can be used e.g. in Unity etc. The assets should be houses e.g. from small or large cities. EC5 Human Pack Create an asset pack with humans. Humans should incorporate men, women, children. They should at least be able to walk around in an area and show intelligent behavior, like avoiding each other or obstacles, and cross streets at traffic lights. EC6 Being hurt in VR Create a VR experience where your virtual body is inflicted injuries. Make a study how people react to this. EC7 Yourself in 360 videos Take existing 360 videos and bring a realistic body model of the watcher into the videos. Test with subjects whether this increases the sense of presence. EC8 Sentiment detection with AI Detect sentiments of speech. Let test subjects speak a text with different emotional loadings. Create a spectrogram and feed them into NNs or CNNs. Try to detect the emotional loadings from a given audio recording. EC9 Danger Detection for Car Drivers

2 The EC group has developed a car driving simulation, for driving through a city forever. Use this simulation to populate the streets with cars, people, cyclists etc. Implement a system that detects them by Deep Learning. Simulate a system that has cameras around the cars covering 360 deg angle. Detect possible obstacles and show them, warn if a collision can happen. EC10 Indoor Navigation with Feature Point Maps We can create 3D models out of laser scans. Create a map of feature points (SIRF, SURF, ORB) from this model, simulate an app that uses these features to detect its position and orientation indoor (where GPS does not work). Use this system also in artificial, but photorealistic indoor scenarios. EC11 Body Pose tracking Use Deep Learning to estimate the pose of a body. Start with a black/white map, then try on RGB with a green screen, and finally with known background. EC12 Hand Pose Tracking by CNNs or ARCore/ARKit We have worked on hand pose detection using only RGB cameras. Building upon this base, use the material and Deep Learning (CNNs or ARCore/ARKit) to identify hand poses. EC13 Video Database based on Image Features Use image feature points (SIRF, SURF, ORB) to create a large video database. Given footage of such a video, identify the video it belongs to. Can use for upload filters as planned by the EU, instead of having to pay Google for it. EC14 Video Resolution Upscaling using Deep Learning Use High Resolution video from anywhere, downscale it, then train a Deep Learning model to upscale the resolution back to its original resolution. Use it on different video content. Validate with validation set of videos. EC15 Video Inpainting using Deep Learning Implement the video inpainting method as proposed by NVIDIA. EC16 Drone/Crane Pendulum Detection and Compensation Imaging you have a drone/crane transporting something on a rope. Due to motion of the drone/crane the rope acts like a pendulum. In a simulation, detect the pendulum motion and compensate it. EC17 Vulkan Imgui Interface Designer Create an interface designer for Imgui that produces valid Vulkan code. EC18 Diminishing Reality with ARCore/ARKit Make a 3D model with ARCore/ARKit. Detect objects and let them disappear using the model as background. EC19 Bipedal walking with Deep Learning Use Bullet 3D to create a physical model of a bipedal walker. Use Deep Learning / Reinforcement Learning to train a controller for walking / running etc. EC20 Drone Swarm Simulation

3 Create a simulator for simulating drone swarms, for e.g. formations, search patterns, battery charging, detection of hazards or intruders, video streaming, etc. EC21 Photorealistic Street Scene Use Unity 3D to create a photorealistic street scene. EC22 Graphical Vulkan Editor Create a graphical editor for defining Vulkan based renderers. The output should be valid Vulkan code, e.g. for pipeline state objects, render passes, synchronization, threads, command buffers etc. EC23 - Distributed Autonomous Corporation Simulation A distributed autonomous corporation is a company that is led by an AI, and without human personnel. There is only the AI and shareholder. The task is to find a business model for such a company, and create an AI that could actually achieve income. EC24 Deutschkurse für das AMS OÖ Das BFI OÖ, ist an einer Kooperation im Rahmen einer Bachelorarbeit interessiert. Das BFI OÖ führt Deutschkurse nicht nur im Auftrag des AMS sondern auch für den ÖIF durch. Der Task ist eine Bachelorarbeit, die sich damit beschäftigt, wie man und welche digitale/n Lehrmittel unterstützend bestmöglich in Deutschkursen eingesetzt werden könnten um einen individualisierten Unterricht gewährleisten zu können, wäre spannend und hätte bei Erfolg eine große Breitenwirkung. EC25 Piano/Guitar/Drums Transcript using NNs Starting from a piano/guitar, make audio recordings of keys and key combinations, create a spectrogram, then use NNs to identify the keys. EC26 Teacher Training using 360 deg Video Create 360 deg videos from typical class room situations. These could be positive or negative. Define Cut Poses for the actors, i.e. body and head poses that can be used to make cuts (e.g. define a start pose and an end pose). Teaching students could then use this to train for how to react in such situations. An operator would change from one scene to the next scene, cutting would be done along the cut poses. EC27 VR Learning Experience for Physics Create simple VR scenes for learning physics to children. The children would manipulate objects that behave in physically credible way. EC28 Face Animation Use photogrammetry (e.g. AgiSoft) to reconstruct faces. Use Blender to cut open eyes and mouth, and insert teeth. Define artificial muscles that move the face mesh to animate the faces. Try this for serveral faces. Align the faces and define common attach points of the muscles. Implement face animation in Unity 3D. EC29 Parkinson Project Like face animation, but for a specific person suffering from Parkinson. Create an avatar of this person that can be used to represent the person remotely.

4 EC30 - Conference Camera Use a 360 degree camera (e.g. and some microphones in a conference setting. Determine the direction of the speaker using the microphones. Find a face using e.g. OpenCV, and select the part of the video showing the face. Zoom and pan to the speaker softly. Send the video to a remote partner. EC31 - Positionierung einer Puppe in der VR Erzeugen sie lebensgroße Puppen von Menschen. Diese sollen auf dem Boden liegen, und von einem Tracking System erfasst werden. Dabei sollen Poition von Kopf, Torso, Ellenbogen, Hand, Hüfte, Knie und Füße erfasst werden. Das kann mit optischen Markern, IR, oder per Deep Learning gemacht werden. EC32 - Tactile Board to visualize Side Scroller Games to Blind Gamers Create a tactile board for blind gamers so that they can feel the 2D arena and feel e.g. their own avatars, friends, and enemies. Create also a simple game that shows how it works. This might include hardware like Arduinos and vibration motors. EC33 - Minecraft for disabled players Create your own MineCraft Mod. Add ways to enable payers with disabilities to steer the game, e.g. using eye trackers. We could have access to high class eye trackers from a Viennese company for this. Alternatively we can use off-the-shelf eye trackers. EC34 - Joystick for fmri Create a fully functional joypad that can be used inside an fmri scanner. Due to the high magnetic fields, the joypad must be 100% mechanic inside the scanner. EC35 - Directed Drone Antenna Put a directed antenna onto a drone, assume direct line of sight, but long distances >10km Try to connect using WLAN, what data rate do you achieve? May also try other networks, like Bluetooth, LightBridge,? EC36 A public speaking task in VR Create a photorealistic scene where you have to give a speech in front of an audience. Use Unreal or Unity. EC37 Virtual Driving School Use Unity or Unreal to create a driving experience in a car. Create lessons, import street signs, create a simple steering wheel based on a tablet computer. Attach this to our city generator project. EC38 3D head reconstruction Create an app that uses VRCore/VRkit to reconstruct heads using the camera only. Create face animations for the heads. EC39 - Embodiment for Paralyzed Use VR to create the impression for paralyzed people that they go, run, jump, dance. EC40 Children at the Doctor

5 Create a colorful VR experience run on mobile phones and e.g. Samsung Gear VR. This should be used by doctors who examine children or administer painful treatments. Children should be distracted from the pain. EC41 - Calibrate Colors in a 3D- Model We scan an interior rooms using laser scanners and photos. The results are 3D-Models from large rooms. The colors depend on the current light situation, and might shift e.g. to blue. The task is to use the available information (3D-model, color palette information, 360 degree photos) to undo this color shift, in order to get the original albedo (surface reflectance) for the colors. EC42 - Audio cues and AI for mixed play of Blind and Unchallenged Players Including blind players into action games requires that each blind gamer gets her own audio cues. Find a way to use off-the-shelf playing equipment, e.g. a 5+1 sound card, to provide individual audio cues to each blind player, such that they can identify the cues meant for them. Design a simple multiplayer game that can be played together, and experiment with the cues.