Comm. 502: Communication Theory. Lecture 6. - Introduction to Source Coding
|
|
- Anissa Quinn
- 6 years ago
- Views:
Transcription
1 Comm. 50: Communication Theory Lecture 6 - Introduction to Source Coding
2 Digital Communication Systems Source of Information User of Information Source Encoder Source Decoder Channel Encoder Channel Decoder Modulator De-Modulator Channel Communication systems are designed to transmit the information generated by a source to some destination.
3 Types of Information Sources Types of Information Sources Analog Discrete Analog : The output is analog signals (examples: TV and Radio broadcasting). Discrete : The output is discrete (a sequence of letters or symbols (example: Computers, storage devices, ).
4 Source Encoder Whether a source is analog or discrete, a digital communication system is designed to transmit information in digital form. Consequently, the output of the source must be converted to a format that can be transmitted digitally. This conversion of the source output to a digital form is generally performed by the source encoder, whose output is a sequence of binary digits. Ex: ASCII code which converts characters to binary bits.
5 Why Source Coding is Important It enables us to determine: - The amount of information from a given source. - The minimum storage and bandwidth needed to transfer data from a given source. - The limit on the transmission rate of information over a noisy channel? - Data compression.
6 Fixed Length Codes The standard character codes are of fixed length such as 5, 6, or 7 bits. The length is usually chosen so that there are enough binary characters to assign a unique binary sequence to each input alphabet character. Fixed length codes have the property that character boundaries are separated by a fixed bit count. Example: This allows the conversion of a serial data stream to a parallel data stream by a simple bit counter.
7 Variable Length Codes Data compression codes are often variable length codes. We expect that the length of a binary sequence assigned to each alphabet symbol is inversely related to the probability of that symbol. A significant amount of data compression can be realized when there is a wide differences in probabilities of the symbols. To achieve this compression, there must also be a sufficiently large number of symbols.
8 Discrete Information Source Information Source Assume S Pr S S k pk Information Source Generates a group of symbols from a given alphabet S Each symbol has a probability: p k o, S 1,..., SK 1 - Symbols are independent S Information Source s, s,..., s 0 1 K 1 Pr s p,k 0, 1,...,K 1 k k K-1 k 0 p k 1
9 If =1: Measure of Information - The occurrence of the event does not correspond to any gain of information (i.e there is no uncertainty). In this case, there is no need for communications because the receiver knows everything. As p k p k decreases, The uncertainty increases The reception of s k corresponds to some gain in information. BUT HOW MUCH?
10 Measure of Information (Cont.) Information is measured by: (1) Self information () Entropy We use the probability theory to quantify and measure information.
11 (1) Self Information The amount of information in bits about a symbol is closely related to its probability of occurrence. A function which measures the amount of information after observing the symbol sk is the self-information: I ( s k ) 1 I( sk ) log log p p k k [bits] 0 1 P S k
12 Properties of Self Information Properties of I(s): I( s k ) 1) I(s) 0 (a real nonnegative measure). 0 1 P S k ) I( S k ) I( S ) i if P k P i 3) I(s) is a continuous function of p.
13 () Entropy Entropy is the average amount of information of a finite discrete source, More precisely, it is the average number of bits per symbol required to describe that source. For a source containing N independent symbols, its entropy is defined as H Since Then E( I( S N i1 H H i1 Unit of entropy: bits/symbol (Infor. bits/symb) p i I( S i i )) I( S i ) log N i1 N ) p i log ( p i ) ( p p i log ( p i i ) )
14 H is a Properties of Entropy positive quantity H 0 The unit bit is a measure of information content and is not to be confused with the term bit meaning binary digit. If all a priori probabilities are equally likely ( P i 1/ N for all N symbols) then the entropy is maximum and given by: H log N Then 0 H log N
15 0 H log N If p i H all a p N) N (1/ N) p Proof 1/ N for all symbols N i1 (1/ log log priori probabilities i log N i1 (1/ N)[ N ( i log log ) (1/ N) (1/ N)] are equally likely
16 Example A source puts out one of five possible messages during each message interval. The probs. of these messages are p 1 = 1 ; p = 1 ; 4 p 3 = 1 : p 4 = 1 1, p 5 = What is the information content of these messages? I (m 1 ) = - log = 1 bit I (m ) = - log = bits I (m 3 ) = - log = 3 bits I (m 4 ) = - log = 4 bits I (m 5 ) = - log 1 16 = 4 bits
17 Entropy Example Find and plot the entropy of the binary code in which the probability of occurrence for the symbol 1 is p and for the symbol 0 is 1-p H i1 P p log i log p P i 1 plog 1 p 1 H p v logv 0 as v 0 0 1/ 1 p p H log log 1bit/symbol p H log log bits/symbol H 0 bit/symbol; p 1 H 0 bit/symbol
18 Average Information Content in English Language Calculate the average information in bits/character in English assuming each letter is equally likely Since characters do not appear with the same frequency in English, use the probabilities P=0.70 for a,e,o,t P=0. for h,i,n,r,s P=0.08 for c,d,f,l,m,p,u,y P=0.0 for b,g,j,k,q,v,w,x,z H log 6 6 i1 4.7 bits / char Solve this problem
19 Objective Source Coding Efficient representation of data generated by an information source What Does the word EFFIECIENT Mean? Efficient Source Coding means: - Minimum average number of bits per source symbol
20 How could we be EFFICIENT in Source Coding? By using knowledge of the statistics of the source Clearly: Frequent source symbols should be assigned SHORT CODEWORDS Rare source symbols should be assigned LONGER CODEWORDS Example Morse Code E is represented by:. Q is represented by:.
21 Morse Code Morse code is a method of transmitting text information as a series of on-off tones, lights, or clicks that can be directly understood by a skilled listener or observer without special equipment. The letters is transmitted as standardized sequences of short and long signals called "dots" and "dashes. The duration of a dash is three times the duration of a dot. Each dot or dash is followed by a short silence, equal to the dot duration.
22 Morse Code The letters of a word are separated by a space equal to three dots (one dash), and the words are separated by a space equal to seven dots. The dot duration is the basic unit of time measurement in code transmission. For efficiency, the length of each character in Morse is approximately inversely proportional to its frequency of occurrence in English. Thus, the most common letter in English, the letter "E," has the shortest code, a single dot.
23 Morse Code
24 Average Code Length Information Source s k Source Encoder c k Source has K symbols Each symbol s k has probability p k Each symbol s k is represented by a codeword c k of length v k bits Average codeword length Variance of the code length: L K k1 K k1 p k v k p k ( v k L)
25 Example: Average Codeword Length L L = 0.5() +0.30() + 0.1(3) +0.15(3) () =.7 bits It does not mean that we have to find a way to transmit a noninteger number of bits. Rather, it means that on the average the length of the code is.7 bits. K k1 p k v k Symbol p (S) Code A B C D E Calculate the variance of the code length
26 Code Efficiency L represents the average number of bits per source symbol used in the source encoding process. L min If denotes the minimum possible codeword length, the coding efficiency of the source encoder is defined as L min An efficient code means η1 What is L min? L
27 Shannon s First Theorem: The Source Coding Theorem L HS L min = H(S) The outputs of an information source cannot be represented by a source code whose average length is less than the source entropy
28 Compression Ratio We define the compression ratio as: CR Number of bits of the fixed code that represents the symbols Average code length of the variable length code We define the code efficiency as: Code efficiency Entropy Average code length H( S) L L L min It measures how much the code achieves from the possible compression ratio.
29 Source Symbols k Symbol Probability p k Example Symbol Code word Code I Code word Length Symbol Code word Code II Code word Length c k v k c k v k s 0 1/ s 1 1/ s 1/ s 3 1/ Source Entropy: H i1 H(S) =1/log ()+1/4log (4)+ 1/8log (8)+1/log (8)= 1.75 bits/symbol= Lmin CR 1 CR Code I L N p i log (1/ p i L ) K p k v k Code II k L H ( S) bits L
Entropy, Coding and Data Compression
Entropy, Coding and Data Compression Data vs. Information yes, not, yes, yes, not not In ASCII, each item is 3 8 = 24 bits of data But if the only possible answers are yes and not, there is only one bit
More informationIntroduction to Source Coding
Comm. 52: Communication Theory Lecture 7 Introduction to Source Coding - Requirements of source codes - Huffman Code Length Fixed Length Variable Length Source Code Properties Uniquely Decodable allow
More informationLecture5: Lossless Compression Techniques
Fixed to fixed mapping: we encoded source symbols of fixed length into fixed length code sequences Fixed to variable mapping: we encoded source symbols of fixed length into variable length code sequences
More informationCommunication Theory II
Communication Theory II Lecture 13: Information Theory (cont d) Ahmed Elnakib, PhD Assistant Professor, Mansoura University, Egypt March 22 th, 2015 1 o Source Code Generation Lecture Outlines Source Coding
More informationInformation Theory and Huffman Coding
Information Theory and Huffman Coding Consider a typical Digital Communication System: A/D Conversion Sampling and Quantization D/A Conversion Source Encoder Source Decoder bit stream bit stream Channel
More informationCoding for Efficiency
Let s suppose that, over some channel, we want to transmit text containing only 4 symbols, a, b, c, and d. Further, let s suppose they have a probability of occurrence in any block of text we send as follows
More information# 12 ECE 253a Digital Image Processing Pamela Cosman 11/4/11. Introductory material for image compression
# 2 ECE 253a Digital Image Processing Pamela Cosman /4/ Introductory material for image compression Motivation: Low-resolution color image: 52 52 pixels/color, 24 bits/pixel 3/4 MB 3 2 pixels, 24 bits/pixel
More informationLECTURE VI: LOSSLESS COMPRESSION ALGORITHMS DR. OUIEM BCHIR
1 LECTURE VI: LOSSLESS COMPRESSION ALGORITHMS DR. OUIEM BCHIR 2 STORAGE SPACE Uncompressed graphics, audio, and video data require substantial storage capacity. Storing uncompressed video is not possible
More informationSOME EXAMPLES FROM INFORMATION THEORY (AFTER C. SHANNON).
SOME EXAMPLES FROM INFORMATION THEORY (AFTER C. SHANNON). 1. Some easy problems. 1.1. Guessing a number. Someone chose a number x between 1 and N. You are allowed to ask questions: Is this number larger
More informationModule 8: Video Coding Basics Lecture 40: Need for video coding, Elements of information theory, Lossless coding. The Lecture Contains:
The Lecture Contains: The Need for Video Coding Elements of a Video Coding System Elements of Information Theory Symbol Encoding Run-Length Encoding Entropy Encoding file:///d /...Ganesh%20Rana)/MY%20COURSE_Ganesh%20Rana/Prof.%20Sumana%20Gupta/FINAL%20DVSP/lecture%2040/40_1.htm[12/31/2015
More informationCommunication Theory II
Communication Theory II Lecture 14: Information Theory (cont d) Ahmed Elnakib, PhD Assistant Professor, Mansoura University, Egypt March 25 th, 2015 1 Previous Lecture: Source Code Generation: Lossless
More informationCSCD 433 Network Programming Fall Lecture 5 Physical Layer Continued
CSCD 433 Network Programming Fall 2016 Lecture 5 Physical Layer Continued 1 Topics Definitions Analog Transmission of Digital Data Digital Transmission of Analog Data Multiplexing 2 Different Types of
More informationInformation Theory and Communication Optimal Codes
Information Theory and Communication Optimal Codes Ritwik Banerjee rbanerjee@cs.stonybrook.edu c Ritwik Banerjee Information Theory and Communication 1/1 Roadmap Examples and Types of Codes Kraft Inequality
More informationComputing and Communications 2. Information Theory -Channel Capacity
1896 1920 1987 2006 Computing and Communications 2. Information Theory -Channel Capacity Ying Cui Department of Electronic Engineering Shanghai Jiao Tong University, China 2017, Autumn 1 Outline Communication
More informationMultimedia Systems Entropy Coding Mahdi Amiri February 2011 Sharif University of Technology
Course Presentation Multimedia Systems Entropy Coding Mahdi Amiri February 2011 Sharif University of Technology Data Compression Motivation Data storage and transmission cost money Use fewest number of
More informationCSCD 433 Network Programming Fall Lecture 5 Physical Layer Continued
CSCD 433 Network Programming Fall 2016 Lecture 5 Physical Layer Continued 1 Topics Definitions Analog Transmission of Digital Data Digital Transmission of Analog Data Multiplexing 2 Different Types of
More informationA Brief Introduction to Information Theory and Lossless Coding
A Brief Introduction to Information Theory and Lossless Coding 1 INTRODUCTION This document is intended as a guide to students studying 4C8 who have had no prior exposure to information theory. All of
More informationMATHEMATICS IN COMMUNICATIONS: INTRODUCTION TO CODING. A Public Lecture to the Uganda Mathematics Society
Abstract MATHEMATICS IN COMMUNICATIONS: INTRODUCTION TO CODING A Public Lecture to the Uganda Mathematics Society F F Tusubira, PhD, MUIPE, MIEE, REng, CEng Mathematical theory and techniques play a vital
More informationELEC3028 (EL334) Digital Transmission
ELEC3028 (EL334) Digital Transmission Half of the unit: Information Theory MODEM (modulator and demodulator) Professor Sheng Chen: Building 53, Room 4005 E-mail: sqc@ecs.soton.ac.uk Lecture notes from:
More informationCT111 Introduction to Communication Systems Lecture 9: Digital Communications
CT111 Introduction to Communication Systems Lecture 9: Digital Communications Yash M. Vasavada Associate Professor, DA-IICT, Gandhinagar 31st January 2018 Yash M. Vasavada (DA-IICT) CT111: Intro to Comm.
More informationModule 3 Greedy Strategy
Module 3 Greedy Strategy Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu Introduction to Greedy Technique Main
More informationCommunications I (ELCN 306)
Communications I (ELCN 306) c Samy S. Soliman Electronics and Electrical Communications Engineering Department Cairo University, Egypt Email: samy.soliman@cu.edu.eg Website: http://scholar.cu.edu.eg/samysoliman
More informationBSc (Hons) Computer Science with Network Security, BEng (Hons) Electronic Engineering. Cohorts: BCNS/17A/FT & BEE/16B/FT
BSc (Hons) Computer Science with Network Security, BEng (Hons) Electronic Engineering Cohorts: BCNS/17A/FT & BEE/16B/FT Examinations for 2016-2017 Semester 2 & 2017 Semester 1 Resit Examinations for BEE/12/FT
More informationDigital Communication Systems ECS 452
Digital Communication Systems ECS 452 Asst. Prof. Dr. Prapun Suksompong prapun@siit.tu.ac.th 2. Source Coding 1 Office Hours: BKD, 6th floor of Sirindhralai building Monday 10:00-10:40 Tuesday 12:00-12:40
More informationCOMM901 Source Coding and Compression Winter Semester 2013/2014. Midterm Exam
German University in Cairo - GUC Faculty of Information Engineering & Technology - IET Department of Communication Engineering Dr.-Ing. Heiko Schwarz COMM901 Source Coding and Compression Winter Semester
More informationModule 3 Greedy Strategy
Module 3 Greedy Strategy Dr. Natarajan Meghanathan Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu Introduction to Greedy Technique Main
More informationChannel Concepts CS 571 Fall Kenneth L. Calvert
Channel Concepts CS 571 Fall 2006 2006 Kenneth L. Calvert What is a Channel? Channel: a means of transmitting information A means of communication or expression Webster s NCD Aside: What is information...?
More informationChapter 1 INTRODUCTION TO SOURCE CODING AND CHANNEL CODING. Whether a source is analog or digital, a digital communication
1 Chapter 1 INTRODUCTION TO SOURCE CODING AND CHANNEL CODING 1.1 SOURCE CODING Whether a source is analog or digital, a digital communication system is designed to transmit information in digital form.
More informationCompression. Encryption. Decryption. Decompression. Presentation of Information to client site
DOCUMENT Anup Basu Audio Image Video Data Graphics Objectives Compression Encryption Network Communications Decryption Decompression Client site Presentation of Information to client site Multimedia -
More informationHuffman Coding - A Greedy Algorithm. Slides based on Kevin Wayne / Pearson-Addison Wesley
- A Greedy Algorithm Slides based on Kevin Wayne / Pearson-Addison Wesley Greedy Algorithms Greedy Algorithms Build up solutions in small steps Make local decisions Previous decisions are never reconsidered
More informationS Coding Methods (5 cr) P. Prerequisites. Literature (1) Contents
S-72.3410 Introduction 1 S-72.3410 Introduction 3 S-72.3410 Coding Methods (5 cr) P Lectures: Mondays 9 12, room E110, and Wednesdays 9 12, hall S4 (on January 30th this lecture will be held in E111!)
More informationTime division multiplexing The block diagram for TDM is illustrated as shown in the figure
CHAPTER 2 Syllabus: 1) Pulse amplitude modulation 2) TDM 3) Wave form coding techniques 4) PCM 5) Quantization noise and SNR 6) Robust quantization Pulse amplitude modulation In pulse amplitude modulation,
More informationBasic Communications Theory Chapter 2
TEMPEST Engineering and Hardware Design Dr. Bruce C. Gabrielson, NCE 1998 Basic Communications Theory Chapter 2 Communicating Information Communications occurs when information is transmitted or sent between
More information4. Which of the following channel matrices respresent a symmetric channel? [01M02] 5. The capacity of the channel with the channel Matrix
Send SMS s : ONJntuSpeed To 9870807070 To Recieve Jntu Updates Daily On Your Mobile For Free www.strikingsoon.comjntu ONLINE EXMINTIONS [Mid 2 - dc] http://jntuk.strikingsoon.com 1. Two binary random
More informationCOPYRIGHTED MATERIAL. Introduction. 1.1 Communication Systems
1 Introduction The reliable transmission of information over noisy channels is one of the basic requirements of digital information and communication systems. Here, transmission is understood both as transmission
More informationECEn 665: Antennas and Propagation for Wireless Communications 131. s(t) = A c [1 + αm(t)] cos (ω c t) (9.27)
ECEn 665: Antennas and Propagation for Wireless Communications 131 9. Modulation Modulation is a way to vary the amplitude and phase of a sinusoidal carrier waveform in order to transmit information. When
More informationChapter 3 Data and Signals
Chapter 3 Data and Signals 3.2 To be transmitted, data must be transformed to electromagnetic signals. 3-1 ANALOG AND DIGITAL Data can be analog or digital. The term analog data refers to information that
More informationDigital Communication Systems ECS 452
Digital Communication Systems ECS 452 Asst. Prof. Dr. Prapun Suksompong prapun@siit.tu.ac.th Source Coding 1 Office Hours: BKD 3601-7 Monday 14:00-16:00 Wednesday 14:40-16:00 Noise & Interference Elements
More informationCS101 Lecture 01: Introduction. What You ll Learn Today
CS101 Lecture 01: Introduction Aaron Stevens (azs@bu.edu) 16 January 2013 What You ll Learn Today What is computer science? What are data and information? What is a computer? What are hardware and software?
More informationCHAPTER 4 SIGNAL SPACE. Xijun Wang
CHAPTER 4 SIGNAL SPACE Xijun Wang WEEKLY READING 1. Goldsmith, Wireless Communications, Chapters 5 2. Gallager, Principles of Digital Communication, Chapter 5 2 DIGITAL MODULATION AND DEMODULATION n Digital
More informationUNIT-1. Basic signal processing operations in digital communication
UNIT-1 Lecture-1 Basic signal processing operations in digital communication The three basic elements of every communication systems are Transmitter, Receiver and Channel. The Overall purpose of this system
More informationOverview. Lecture 3. Terminology. Terminology. Background. Background. Transmission basics. Transmission basics. Two signal types
Lecture 3 Transmission basics Chapter 3, pages 75-96 Dave Novak School of Business University of Vermont Overview Transmission basics Terminology Signal Channel Electromagnetic spectrum Two signal types
More informationDIGITAL COMMUNICATION
DEPARTMENT OF ELECTRICAL &ELECTRONICS ENGINEERING DIGITAL COMMUNICATION Spring 00 Yrd. Doç. Dr. Burak Kelleci OUTLINE Quantization Pulse-Code Modulation THE QUANTIZATION PROCESS A continuous signal has
More informationEENG 444 / ENAS 944 Digital Communication Systems
EENG 444 / ENAS 944 Digital Communication Systems Introduction!! Wenjun Hu Communication Systems What s the first thing that comes to your mind? Communication Systems What s the first thing that comes
More informationUNIT III -- DATA AND PULSE COMMUNICATION PART-A 1. State the sampling theorem for band-limited signals of finite energy. If a finite energy signal g(t) contains no frequency higher than W Hz, it is completely
More informationTSKS01 Digital Communication Lecture 1
TSKS01 Digital Communication Lecture 1 Introduction, Repetition, Channels as Filters, Complex-baseband representation Emil Björnson Department of Electrical Engineering (ISY) Division of Communication
More informationChannel Coding/Decoding. Hamming Method
Channel Coding/Decoding Hamming Method INFORMATION TRANSFER ACROSS CHANNELS Sent Received messages symbols messages source encoder Source coding Channel coding Channel Channel Source decoder decoding decoding
More informationHistory of Communication
1 History of Communication Required reading: Forouzan Ch. 1 Garcia 1.1 and 1.2 CSE 3213, Fall 2015 Instructor: N. Vlajic History of Telecommunications 2 Papyrus 3000 BC http://www.prologprintmedia.co.uk/news-whats-next-in-the-evolution-of-communication
More informationChapter Two. Fundamentals of Data and Signals. Data Communications and Computer Networks: A Business User's Approach Seventh Edition
Chapter Two Fundamentals of Data and Signals Data Communications and Computer Networks: A Business User's Approach Seventh Edition After reading this chapter, you should be able to: Distinguish between
More informationEE303: Communication Systems
EE303: Communication Systems Professor A. Manikas Chair of Communications and Array Processing Imperial College London An Overview of Fundamentals: Channels, Criteria and Limits Prof. A. Manikas (Imperial
More informationCh 5 Hardware Components for Automation
Ch 5 Hardware Components for Automation Sections: 1. Sensors 2. Actuators 3. Analog-to-Digital Conversion 4. Digital-to-Analog Conversion 5. Input/Output Devices for Discrete Data Computer-Process Interface
More informationModulation and Coding Tradeoffs
0 Modulation and Coding Tradeoffs Contents 1 1. Design Goals 2. Error Probability Plane 3. Nyquist Minimum Bandwidth 4. Shannon Hartley Capacity Theorem 5. Bandwidth Efficiency Plane 6. Modulation and
More informationEIE 441 Advanced Digital communications
EIE 441 Advanced Digital communications MACHED FILER 1. Consider the signal s ( ) shown in Fig. 1. 1 t (a) Determine the impulse response of a filter matched to this signal and sketch it as a function
More informationComm 502: Communication Theory
Comm 50: Communication Theory Prof. Dean of the faculty of IET The German University in Cairo 1 COMM 50: Communication Theory Instructor: Ahmed El-Mahdy Office : C3.319 Lecture Time: Sat. nd Slot Office
More informationMAS160: Signals, Systems & Information for Media Technology. Problem Set 4. DUE: October 20, 2003
MAS160: Signals, Systems & Information for Media Technology Problem Set 4 DUE: October 20, 2003 Instructors: V. Michael Bove, Jr. and Rosalind Picard T.A. Jim McBride Problem 1: Simple Psychoacoustic Masking
More informationError Control Coding. Aaron Gulliver Dept. of Electrical and Computer Engineering University of Victoria
Error Control Coding Aaron Gulliver Dept. of Electrical and Computer Engineering University of Victoria Topics Introduction The Channel Coding Problem Linear Block Codes Cyclic Codes BCH and Reed-Solomon
More informationDIGITAL COMMINICATIONS
Code No: R346 R Set No: III B.Tech. I Semester Regular and Supplementary Examinations, December - 23 DIGITAL COMMINICATIONS (Electronics and Communication Engineering) Time: 3 Hours Max Marks: 75 Answer
More informationFundamentals of Digital Communication
Fundamentals of Digital Communication Network Infrastructures A.A. 2017/18 Digital communication system Analog Digital Input Signal Analog/ Digital Low Pass Filter Sampler Quantizer Source Encoder Channel
More informationSolutions to Information Theory Exercise Problems 5 8
Solutions to Information Theory Exercise roblems 5 8 Exercise 5 a) n error-correcting 7/4) Hamming code combines four data bits b 3, b 5, b 6, b 7 with three error-correcting bits: b 1 = b 3 b 5 b 7, b
More informationSIGNALS AND SYSTEMS LABORATORY 13: Digital Communication
SIGNALS AND SYSTEMS LABORATORY 13: Digital Communication INTRODUCTION Digital Communication refers to the transmission of binary, or digital, information over analog channels. In this laboratory you will
More informationGENERIC CODE DESIGN ALGORITHMS FOR REVERSIBLE VARIABLE-LENGTH CODES FROM THE HUFFMAN CODE
GENERIC CODE DESIGN ALGORITHMS FOR REVERSIBLE VARIABLE-LENGTH CODES FROM THE HUFFMAN CODE Wook-Hyun Jeong and Yo-Sung Ho Kwangju Institute of Science and Technology (K-JIST) Oryong-dong, Buk-gu, Kwangju,
More information6.450: Principles of Digital Communication 1
6.450: Principles of Digital Communication 1 Digital Communication: Enormous and normally rapidly growing industry, roughly comparable in size to the computer industry. Objective: Study those aspects of
More informationProblem Sheet 1 Probability, random processes, and noise
Problem Sheet 1 Probability, random processes, and noise 1. If F X (x) is the distribution function of a random variable X and x 1 x 2, show that F X (x 1 ) F X (x 2 ). 2. Use the definition of the cumulative
More informationIntroduction to Communications Part Two: Physical Layer Ch3: Data & Signals
Introduction to Communications Part Two: Physical Layer Ch3: Data & Signals Kuang Chiu Huang TCM NCKU Spring/2008 Goals of This Class Through the lecture of fundamental information for data and signals,
More informationPROJECT 5: DESIGNING A VOICE MODEM. Instructor: Amir Asif
PROJECT 5: DESIGNING A VOICE MODEM Instructor: Amir Asif CSE4214: Digital Communications (Fall 2012) Computer Science and Engineering, York University 1. PURPOSE In this laboratory project, you will design
More informationChapter-1: Introduction
Chapter-1: Introduction The purpose of a Communication System is to transport an information bearing signal from a source to a user destination via a communication channel. MODEL OF A COMMUNICATION SYSTEM
More informationNoisy Index Coding with Quadrature Amplitude Modulation (QAM)
Noisy Index Coding with Quadrature Amplitude Modulation (QAM) Anjana A. Mahesh and B Sundar Rajan, arxiv:1510.08803v1 [cs.it] 29 Oct 2015 Abstract This paper discusses noisy index coding problem over Gaussian
More informationThe ternary alphabet is used by alternate mark inversion modulation; successive ones in data are represented by alternating ±1.
Alphabets EE 387, Notes 2, Handout #3 Definition: An alphabet is a discrete (usually finite) set of symbols. Examples: B = {0,1} is the binary alphabet T = { 1,0,+1} is the ternary alphabet X = {00,01,...,FF}
More informationExercises to Chapter 2 solutions
Exercises to Chapter 2 solutions 1 Exercises to Chapter 2 solutions E2.1 The Manchester code was first used in Manchester Mark 1 computer at the University of Manchester in 1949 and is still used in low-speed
More informationContents. Telecom Service Chae Y. Lee. Data Signal Transmission Transmission Impairments Channel Capacity
Data Transmission Contents Data Signal Transmission Transmission Impairments Channel Capacity 2 Data/Signal/Transmission Data: entities that convey meaning or information Signal: electric or electromagnetic
More informationDEPARTMENT OF INFORMATION TECHNOLOGY QUESTION BANK. Subject Name: Information Coding Techniques UNIT I INFORMATION ENTROPY FUNDAMENTALS
DEPARTMENT OF INFORMATION TECHNOLOGY QUESTION BANK Subject Name: Year /Sem: II / IV UNIT I INFORMATION ENTROPY FUNDAMENTALS PART A (2 MARKS) 1. What is uncertainty? 2. What is prefix coding? 3. State the
More informationIntroduction to Error Control Coding
Introduction to Error Control Coding 1 Content 1. What Error Control Coding Is For 2. How Coding Can Be Achieved 3. Types of Coding 4. Types of Errors & Channels 5. Types of Codes 6. Types of Error Control
More informationDCSP-3: Minimal Length Coding. Jianfeng Feng
DCSP-3: Minimal Length Coding Jianfeng Feng Department of Computer Science Warwick Univ., UK Jianfeng.feng@warwick.ac.uk http://www.dcs.warwick.ac.uk/~feng/dcsp.html Automatic Image Caption (better than
More informationHUFFMAN CODING. Catherine Bénéteau and Patrick J. Van Fleet. SACNAS 2009 Mini Course. University of South Florida and University of St.
Catherine Bénéteau and Patrick J. Van Fleet University of South Florida and University of St. Thomas SACNAS 2009 Mini Course WEDNESDAY, 14 OCTOBER, 2009 (1:40-3:00) LECTURE 2 SACNAS 2009 1 / 10 All lecture
More informationReview of Lecture 2. Data and Signals - Theoretical Concepts. Review of Lecture 2. Review of Lecture 2. Review of Lecture 2. Review of Lecture 2
Data and Signals - Theoretical Concepts! What are the major functions of the network access layer? Reference: Chapter 3 - Stallings Chapter 3 - Forouzan Study Guide 3 1 2! What are the major functions
More information6.004 Computation Structures Spring 2009
MIT OpenCourseWare http://ocw.mit.edu 6.004 Computation Structures Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. Welcome to 6.004! Course
More informationPrinciples of Communications ECS 332
Principles of Communications ECS 332 Asst. Prof. Dr. Prapun Suksompong prapun@siit.tu.ac.th 5. Angle Modulation Office Hours: BKD, 6th floor of Sirindhralai building Wednesday 4:3-5:3 Friday 4:3-5:3 Example
More informationMAS.160 / MAS.510 / MAS.511 Signals, Systems and Information for Media Technology Fall 2007
MIT OpenCourseWare http://ocw.mit.edu MAS.160 / MAS.510 / MAS.511 Signals, Systems and Information for Media Technology Fall 2007 For information about citing these materials or our Terms of Use, visit:
More informationLecture 1 Introduction
Lecture 1 Introduction I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw September 22, 2015 1 / 46 I-Hsiang Wang IT Lecture 1 Information Theory Information
More informationECE Advanced Communication Theory, Spring 2007 Midterm Exam Monday, April 23rd, 6:00-9:00pm, ELAB 325
C 745 - Advanced Communication Theory, Spring 2007 Midterm xam Monday, April 23rd, 600-900pm, LAB 325 Overview The exam consists of five problems for 150 points. The points for each part of each problem
More informationDigital Design Laboratory Lecture 7. A/D and D/A
ECE 280 / CSE 280 Digital Design Laboratory Lecture 7 A/D and D/A Analog/Digital Conversion A/D conversion is the process of sampling a continuous signal Two significant implications 1. The information
More informationThe idea of similarity is through the Hamming
Hamming distance A good channel code is designed so that, if a few bit errors occur in transmission, the output can still be identified as the correct input. This is possible because although incorrect,
More information2. TELECOMMUNICATIONS BASICS
2. TELECOMMUNICATIONS BASICS The purpose of any telecommunications system is to transfer information from the sender to the receiver by a means of a communication channel. The information is carried by
More informationC06a: Digital Modulation
CISC 7332X T6 C06a: Digital Modulation Hui Chen Department of Computer & Information Science CUNY Brooklyn College 10/2/2018 CUNY Brooklyn College 1 Outline Digital modulation Baseband transmission Line
More informationMulti-user Two-way Deterministic Modulo 2 Adder Channels When Adaptation Is Useless
Forty-Ninth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 28-30, 2011 Multi-user Two-way Deterministic Modulo 2 Adder Channels When Adaptation Is Useless Zhiyu Cheng, Natasha
More information2.1. General Purpose Run Length Encoding Relative Encoding Tokanization or Pattern Substitution
2.1. General Purpose There are many popular general purpose lossless compression techniques, that can be applied to any type of data. 2.1.1. Run Length Encoding Run Length Encoding is a compression technique
More informationDigital Modulation Techniques
Digital Modulation Techniques Digital-to-Analog signals is the next conversion we will discuss in this chapter. These techniques are also called as Digital Modulation techniques. Digital Modulation provides
More informationIMPERIAL COLLEGE of SCIENCE, TECHNOLOGY and MEDICINE, DEPARTMENT of ELECTRICAL and ELECTRONIC ENGINEERING.
IMPERIAL COLLEGE of SCIENCE, TECHNOLOGY and MEDICINE, DEPARTMENT of ELECTRICAL and ELECTRONIC ENGINEERING. COMPACT LECTURE NOTES on COMMUNICATION THEORY. Prof. Athanassios Manikas, version Spring 22 Digital
More informationPulse Code Modulation
Pulse Code Modulation Modulation is the process of varying one or more parameters of a carrier signal in accordance with the instantaneous values of the message signal. The message signal is the signal
More informationReceiver Design for Noncoherent Digital Network Coding
Receiver Design for Noncoherent Digital Network Coding Terry Ferrett 1 Matthew Valenti 1 Don Torrieri 2 1 West Virginia University 2 U.S. Army Research Laboratory November 3rd, 2010 1 / 25 Outline 1 Introduction
More informationFundamentals of Data and Signals
Fundamentals of Data and Signals Chapter 2 Learning Objectives After reading this chapter, you should be able to: Distinguish between data and signals and cite the advantages of digital data and signals
More informationLecture 9b Convolutional Coding/Decoding and Trellis Code modulation
Lecture 9b Convolutional Coding/Decoding and Trellis Code modulation Convolutional Coder Basics Coder State Diagram Encoder Trellis Coder Tree Viterbi Decoding For Simplicity assume Binary Sym.Channel
More informationECE 556 BASICS OF DIGITAL SPEECH PROCESSING. Assıst.Prof.Dr. Selma ÖZAYDIN Spring Term-2017 Lecture 2
ECE 556 BASICS OF DIGITAL SPEECH PROCESSING Assıst.Prof.Dr. Selma ÖZAYDIN Spring Term-2017 Lecture 2 Analog Sound to Digital Sound Characteristics of Sound Amplitude Wavelength (w) Frequency ( ) Timbre
More informationThe information carrying capacity of a channel
Chapter 8 The information carrying capacity of a channel 8.1 Signals look like noise! One of the most important practical questions which arises when we are designing and using an information transmission
More informationAdvanced Digital Signal Processing Part 2: Digital Processing of Continuous-Time Signals
Advanced Digital Signal Processing Part 2: Digital Processing of Continuous-Time Signals Gerhard Schmidt Christian-Albrechts-Universität zu Kiel Faculty of Engineering Institute of Electrical Engineering
More informationDigital Communication Prof. Bikash Kumar Dey Department of Electrical Engineering Indian Institute of Technology, Bombay
Digital Communication Prof. Bikash Kumar Dey Department of Electrical Engineering Indian Institute of Technology, Bombay Lecture - 03 Quantization, PCM and Delta Modulation Hello everyone, today we will
More informationInfo theory and big data
Info theory and big data Typical or not typical, that is the question Han Vinck University Duisburg Essen, Germany September 2016 A.J. Han Vinck, Yerevan, September 2016 Content: big data issues A definition:
More informationChapter 3 LEAST SIGNIFICANT BIT STEGANOGRAPHY TECHNIQUE FOR HIDING COMPRESSED ENCRYPTED DATA USING VARIOUS FILE FORMATS
44 Chapter 3 LEAST SIGNIFICANT BIT STEGANOGRAPHY TECHNIQUE FOR HIDING COMPRESSED ENCRYPTED DATA USING VARIOUS FILE FORMATS 45 CHAPTER 3 Chapter 3: LEAST SIGNIFICANT BIT STEGANOGRAPHY TECHNIQUE FOR HIDING
More informationChapter 2: Fundamentals of Data and Signals
Chapter 2: Fundamentals of Data and Signals TRUE/FALSE 1. The terms data and signal mean the same thing. F PTS: 1 REF: 30 2. By convention, the minimum and maximum values of analog data and signals are
More informationChapter 2. Physical Layer
Chapter 2 Physical Layer Lecture 1 Outline 2.1 Analog and Digital 2.2 Transmission Media 2.3 Digital Modulation and Multiplexing 2.4 Transmission Impairment 2.5 Data-rate Limits 2.6 Performance Physical
More information