Mahdi Amiri. February Sharif University of Technology

Transcription

1 Course Presentation Multimedia Systems Introduction Mahdi Amiri February 2014 Sharif University of Technology

2 Multimedia System Previous session Review course website resources Announcements, Password, Handling large files Page 1

3 Multimedia System Text sms address to me. Drawing Audio Media Let me see the bar chart for this data table. Can t chat, I have to call you. Still image Video See to believe. Wonder how it works. Also inside above categories: Graphs, Maps, Animations, 3D Objects or Models, Diagrams, etc. Page 2

4 Multimedia System Ascii art Early printers often lacked graphics ability. Ultrasonography Media: Visual Representation Visualizing subcutaneous body structures. Spectrogram Audio signal analysis. Magnetic Resonance Imaging (MRI) Page 3

5 Multimedia System Haptics Media: Sense of Touch The sense of touch. It is a form of nonverbal communication. Hugging, Shaking hands, Vibration, Tickling. Haptics are enabled by actuators that apply forces to the skin for touch feedback, and controllers. Common Haptic Feedback: Cause the handlebars to vibrate during a collision with another vehicle in racing video games. Now are gaining widespread acceptance as a key part of Virtual reality systems. Page 4

6 Multimedia System Media: Sense of Touch Example Haptics: Tactus Technology Page 5

7 Multimedia System Media: Sense of Touch Why This Haptic Steering Wheel Could Save Your Life Source: Seth Porges, :15:04 UTC The car world's best hope for fighting in-car distractions can be summed up with one word: Haptics. Haptics is the study of touch, and haptic feedback is when a device directs a sensation at you. You've felt this form of feedback every time your smartphone has vibrated, or your Xbox controller has rumbled along with the explosions on the screen. Here's how it involves cars. Since touchscreens don't have the feel of actual buttons, they require users to pay extra attention to making sure they've pressed the right key. This is a particular problem for in-car touchscreens, since every second a driver's eyes are off the road increases their chance of a collision. With haptic touchscreens, such as a prototype developed by California-based Tactus Technology, the flat screen pops up into actual buttons when they're needed. Users would technically spend less time with their eyes off the road. In other words, they'd be safer. This is probably where the emphasis on haptic touchscreens as a safety measure sort of misses the point. The safest in-car systems will keep your eyes on the road and your hands on the wheel. With haptic touchscreens, you still need to move your hands, and you still need to at least glance their way. This is why the prototype steering wheel that was jointly developed by researchers at AT&T Research Labs and Carnegie Mellon University is so interesting. The steering wheel is embedded with a series of tiny motors, similar to the ones that cause you cellphone to vibrate. As drivers get closer and closer to a turn, the motors cause the steering wheel to vibrate with increasing frequency in a clockwise or counterclockwise pattern, depending on if it's a left or right turn. According to Kevin Li, the AT&T researcher who developed the wheel, these vibration patterns trigger a human perception trick that causes the brain to sense a number of discrete vibrations as a continuous line of motion. So you don't just feel a vibrating steering wheel, but actually sense a clockwise or counterclockwise motion, which causes you to instinctively turn the steering wheel along with it. Li says his research has shown this has a noticeable impact on how much certain drivers keep their eyes on the road. Interestingly, this seemed to overload the senses of older drivers, Li says. But for younger drivers, eyes off the road as a proportion of driving time decreased by about 10%. But sensing upcoming turns isn't all haptics can do. The same sensory trick that causes discrete vibrations to feel like a continuous line can also be used to convey other, more complex types of information to drivers as well. For example, Li says that he is playing around with a haptic seat that lets drivers "feel" a car pass through a blind spot. This sensory trick would allow the vibrations to tell drivers where a car is, and how it is moving in relation to them. The way I see it, this is basically a sort of spidey sense, where you instinctively feel the presence of on-the-road danger. And you do it without moving your eyes, or your hands. Turn when the steering wheel tells you (AT&T Labs Research) Page 6

8 Multimedia System Methods Media: Olfaction, The Sense of Smell All computerized scent output systems work in the same way: the computer, be it a full-featured desktop machine or a simple embedded chip, sends a signal out through a serial or parallel port to a relay, which turns on the output device itself for a designated period of time. Scentography is the technique of creating and storing odor by artificially recreating a smell using chemical and electronic means. Page 7 ismell Personal Scent Synthesizer, one of the "25 Worst Tech Products of All Time" by PC World Magazine. Nasal Ranger, an olfactometer, in use. An olfactometer is an instrument used to detect and measure ambient odor dilution. Olfactometers are used in conjunction with human subjects in laboratory settings, most often in market research, to quantify and qualify human olfaction (en.wikipedia.org/wiki/olfactometer).

9 Multimedia System Media: Olfaction, The Sense of Smell Madeleine, a prototype of a desktop device to record aromas (Jul. 2013). Click on image to play video. Click on image to play video. Amy Radcliffe en.wikipedia.org/wiki/scentography Page 8

10 Multimedia System Multimedia Computer controlled integration of different media Text Audio Graphic Drawing Still image Video Haptics Animation Every other type of data representation forms Page 9

11 Multimedia System Where multimedia used to be digitally. Processed Generated Represented Stored Transmitted Retrieved Delivered System Page 10

12 Multimedia System HyperText A text which contains links to other texts Ted Nelson, 1965 Non-linear traversal HyperMedia Can include other media too Text, HyperText,, HyperMedia Page 11

13 Multimedia Systems WWW PowerPoint Adobe Acrobat isilo HyperMedia Examples Text compression, Hyperlinks, Images, Tables, Formatted text epub Free and open e-book standard Tim Berners-Lee first proposal for www: 1989 Page 12

14 Multimedia Systems Ted Nelson Page 13 HTML No version management, no rights management. Project Xanadu A word processor capable of storing multiple versions, and displaying the differences between these versions. Find more at: ZigZag Data Model, Database and visualization system See the movie at HyperMedia: True generalization! Computer documents are not just to SIMULATE paper. They should allow us to improve on paper.

15 Multimedia Systems Computer controlled Integrated Seamless Characteristics Data are represented digitally Interactive interface End-user is able to control the elements of media Non-linear Page 14

16 Multimedia Systems Digital representation Analog to digital conversion, sampling. Large data requirements Storage, compression. Distributed networks Bandwidth, QoS. Challenges Maintaining temporal relationship between data Sequencing (playing frames in correct order/time frame in video). Synchronization (inter-media scheduling, e.g. Lip synchronization). Page 15

17 Multimedia Systems Hardware Very high processing power. Desirable Features Efficient and high performance I/O. Large storage and memory units. Network support. Software Multimedia capable file system. Special operating system. User friendly software tools: Easy to use, Easy to understand. Network support. Page 16

18 Multimedia Systems Capture device Video camera, Audio microphone, Keyboard, Mouse, Tablet, OCR, different sensors. Storage device Components Hard-disk, CD-ROMs, DVD-ROM. Communication Network LAN, Intranets, Internet. Computer System Multimedia desktop machine. Display Device Speaker, Monitor, Printer. Page 17

19 Multimedia Systems World Wide Web. Video conferencing. Applications Interactive TV, Video-on-demand. E-learning, Hypermedia courseware. Home shopping. Games, Virtual reality. Digital video editing and production systems. Collaborative software, Groupware. Edutainment, Infotainment. Page 18

20 Multimedia Systems Format Size Raw text, Rich text format. Multimedia Data Bitmap graphics (Image raster), Vector graphics. Time independent (text), Time dependent (video). Typ. PAL digital video: 720 by 576 pixels for each one of 3 color components, 25 frames per sec. ~ 30 MB per sec., ~ 104GB per hour. Standards JPEG MPEG-2, MPEG-4 largest capacity of a commercially available HDD at the end of 2010 Page 19

21 Multimedia Systems Graphics Example Raster graphics Are composed of pixels..gif or.jpg Data file contains: an array of pixels of various colors. Pro.: Small file size for lower resolutions. Con.: Will become "blocky" if scaled up. Vector graphics Are composed of paths..eps file or Adobe Illustrator (.ai) file. Data file contains: the points where the paths start and end, how much the paths curve, and the colors that either border or fill the paths. Pro.: The images can be scaled to be very large without losing quality (logos are typically created in vector forma). Con.: Not the best format for photographs. vector vs. raster (bitmap) graphics Page 20

22 Multimedia Systems Introduction Thank You Next Session: Review of Signals and Systems FIND OUT MORE AT Page 21