| Information Theory and Digital Communications |
|
 |
ELEC S323
Information Theory and Digital Communications
| Introduction |  |
Welcome to ELEC S323 Information Theory and Digital Communications.
This is a two-semester, ten-credit course for The Open University of Hong Kong (OUHK) students seeking the BSc(Hons) in Communications Technology. It is also an optional higher level course of BSc(Hons) in Electronics. The study units, supplementary readings, and activities and self-tests will help you master the topics over a period of around 36 weeks.
About the course
The central topics of the course are fundamental limits in information theory and reliable transmission of digital signals. Four major themes are covered in this course: 1) The overall architecture of digital telecommunication systems; 2) The fundamental limits of transmitting information reliably under noisy environment; 3) The design and analysis techniques to handle the random signals, of the impairments suffered by signals during transmission; 4) The various techniques to minimize the occurrence of errors resulting from noise and distortion during signal transmission.
Purpose of this Course Guide
You have probably taken other courses at the OUHK, so you should now be well aware of both the study skills required for distance learning and how OUHK courses are organized. It is still recommended, though, that you read this Course Guide thoroughly before looking at the study units or supplementary readings.
The Course Guide tells you briefly what the course is about and how you can work your way through the material. It suggests the amount of time you are likely to need to spend in order to complete the course and will give you a general idea when your tutor-marked assignments are due. For detailed information on assignments, however, please refer to the assignment File and, for information on due dates and cut off times for work to be submitted, please refer to the Presentation Schedule.
In distance learning, as practised by the OUHK, the study units (not the tutor) replace the university lecturer. This is one of the great advantages of distance learning: you can read and work through specially designed study material at your own pace at times and places that suit you best. Think of it as reading the lecture instead of hearing it from a lecturer. In the same way that a lecturer might set you some reading to do, the study unit will tell you when to read your textbook or other material. In the same way that a lecturer might give you an in-class exercise, your study unit will have exercises (called by various names) for you to do at appropriate points. You are also likely to find review questions at the end of each unit. Do them all, as these exercises and questions give you the practice necessary to achieve the objectives of the course and to pass the examination.
| Course aims | |
The course aims to:
To develop learners' knowledge in the fundamental limits of information theory.
To provide students with a basic knowledge of the major techniques and types of system used in digital communications, and an understanding of the principles on which they are based.
To enable students to follow and interpret descriptions of systems which are new to them, and which have not been covered specifically in the course.
To develop students' capability to evaluate different digital system designs and propose a system appropriate for a specified application.
| Course learning outcomes | |
After completing the course the learners should be able to:
explain the specialized terminology in information theory and digital communications.
perform simple reliability study and queuing analysis of a communication system.
describe and explain the basic operations of digital signal transmission in the presence of noise.
propose a coding or modulation scheme for a given channel and intended traffic.
recognize when a proposed system is unrealistic or failed to satisfy a specification.
| About the course | |
Course organization
The following chart gives a general overview of the course structure.
Unit Title Weeks Assessment activity
(end of unit)1 Introduction 1 2 Reliability, Traffic and Information 7 3 Digital Signals 3 Assignment 1 4 Noise 3 5 Coding and Modulation 5 Assignment 2 6 Comparison of digital modulation systems 3 7 Channel Coding 4 Assignment 3 8 Source coding and channel capacity 3 9 Equalization 2 10 Spread spectrum communication 3 Assignment 4 Revision 2 TOTAL 36
Unit 1 Introduction
ContentsIntroduction
Historical Review
Why go digital?
Advantages of digital communications
Block diagram of a typical digital communication system
Structure of the course
Unit 2 Reliability, Traffic and Information
Unit 3 Digital SignalsContents
Introduction
Probability
relative frequencies and probabilities
Venn diagram
conditional probabilities
Reliability
basic concepts
failure rate
exponential distribution
average lifetimes
practical applications of failure rate
reliability of systems and systems element
Binomial, Poisson and normal distribution
Binomial distribution
Poisson distribution
normal distribution
Traffic
blocking probability and grade of service
queues
the M/M/1 queue
multiple server queue
Coding and Information
source coding
quantifying information
channel coding: error detection and correction
cyclic redundancy checks
channel capacity
Unit 4 NoiseContents
Introduction
Systems and signals
linear models
Fourier analysis
bilateral spectra
amplitude distortion and phase distortion
Modeling individual pulse
the spectrum of an isolated, rectangular pulse
input-output relationships
other pulse shapes
Aspects of digital signal transmission
threshold detection
intersymbol interference
coding aspects
Pulse code modulation
sampling
encoding and decoding
signal recovery
Systems aspects of pulse code modulation
time division multiplexing
regenerative repeaters
codecs
Contents
Introduction
sources of noise
strategies of optimizing noise performance
structure of the block
Characterizing random noise
time domain measures of a random variable
frequency domain characterization of a random variable
quantisation noise revisited
Sources of noise and distortion
thermal (Johnson) noise
shot noise
1/f noise
other causes of signal degradation
Noise in circuits and systems
basic concepts
noise temperature
noise factor
noise factor or noise temperature
Noise in digital systems
threshold detection and the Gaussian distribution
receiver filtering
Unit 5 Coding and Modulation
Contents
Introduction
Line codes
code radix, redundancy and efficiency
interface codes
block codes
Codes for line systems
metallic line systems
optical fibre line systems
trade-offs in multi-level coding
Decoding, errors and timing
error detection
the decoding of data corrupted by errors
timing information and synchronization
increasing the timing content of codes
block alignment
The spectral characteristics of coded data
code spectrum
low-frequency spectral components
high-frequency spectral components
Differential coding and scrambling
non-redundant coding
differential coding
scrambling
Convolutional coding
encoding
decoding and error detection
practical convolutional codes
Modulation
the modulation of digital data
spectra
noise immunity
comparison of binary OOK, PSK and FSK
multi-level modulation schemes
Demodulation methods
envelope demodulation
homodyne demodulation
carrier extraction for homodyne detection
Modems
modulation schemes
bit rates
equalization
duplex operation
trellis code modulation
Matched filter detection
baseband systems
modulated systems
Practical baseband receivers
intersymbol interference
equalization
pulse shape
crosstalk
cost
Unit 6 Comparison of digital modulation systems
Contents
Introduction
Error rates as a function of signal-to-noise (S/N) ratio per bit
Binary modulation scheme
Multi-level modulation
Bandwidth efficiency for digital modulation schemes
PSK
Multi-level ASK or PAM
QAM
FSK
Channel capacity
Comparison of digital modulation schemes
Unit 7 Channel Coding
Contents
Introduction
Linear Block Codes
Code linearity
Minimum distance and error-correcting capabilities
Coding gain
Generator matrix
Systematic codes
Parity check matrix
Syndrome
Syndrome decoding
Cyclic codes
Generator polynomial
Error detection
Well-known block codes
Performance of block codes
Soft-decision Decoding vs Hard-decision Decoding
Convolutional Codes
Encoding of convolution code
Decoding of convolutional code
Properties of convolutional code
Performance of convolutional code
Tellis-coded Modulation
Interleaving
Block interleaver
Convolutional interleaver
Concatenated block code
Unit 8 Source Coding and Channel Capacity
Contents
Introduction
Measure of Information
Information source
Entropy
Source coding
Source coding theorem
Classes of code
Kraft-McMillan inequality
Huffman coding
Lempel-Ziv coding
Discrete memoryless channels
Noiseless channel
Binary symmetric channel
Channel capacity
Joint and conditional entropy
Mutual information
Channel capacity
Channel coding theorem
Differential entropy
Additive white Gaussian noise channel
Information capacity theorem
Implication of information capacity theorem
Rate distortion theory
Unit 9 Equalization
Contents
Introduction
Linear channel and receiver
Noise and distortion in the channel leading to ISI
Linear equalizer
Zero-forcing equalizer
Effect of noise
Transversal filter
Noise enhancement
Minimum mean-square error criterion
Adaptive equalizer
Method of the steepest descent
Decision feedback equalizer
Unit 10 Spread spectrum communication
Contents
Multiplexing and multiple access
Different types of multiple access
Spread spectrum
Pseudo random sequences
Properties of pseudo random sequences
Direct-sequence spread spectrum
Chip rate
Bandwidth requirement for the DS signal
Spreading effect at the receiver
Processing gain
Frequency hopping
Relationship between hop rate and symbol rate
Processing gain
Comparison between DS and FH
| Course materials | |
Study units
You should read the study units carefully as they can guide your learning and tell you how to approach any assignment related to the unit. Otherwise, you may miss important information. You must read the study units and you should also read all suggested supplementary readings. They are not alternatives. Moreover, you should also read articles in newspapers and journals and other books related to the topics. The more widely you read, the better your appreciation and understanding of the course.
Each unit is divided into a number of sections. The first section provides the objectives of that unit and introduces the materials to be covered. The next section constitutes the contents of the study unit. This section will guide your learning and direct you to complete the activities and self-tests.
There are 10 study units in Information Theory and Digital Communications. Each study unit consists of one to eight weeks' work and includes specific objectives, directions for study, commentaries on readings, additional materials, and summaries of key issues and ideas. The units direct learners to work on exercises related to the required readings and provide practice exercises and self tests, where appropriate. In general, these exercises question learners on the material they have just covered or require them to apply it in some way and, thereby, help learners to assess their progress and to reinforce their understanding of the material. Together with tutor-marked assignments, these exercises will assist learners in achieving the stated learning objectives of the individual units and of the course.
References:
The following books can be used as references to provide further information and they may be helpful for your studies in ELEC S323. However, you are NOT required to buy them.
Simon Haykin, Communication systems, 5/e, Wiley, 2009.
Bernard Sklar, Digital Communications Fundamentals and Applications, 2/e, Prentice Hall, 2001.
Equipment required by students and tutors
The minimum configuration of the computer system is:
Pentium 4 CPU
Microsoft Windows XP
VGA display card & Colour monitor
256 MB RAM
2 GB free space hard disk
CD-ROM Drive (8X or better), sound card & speaker
56 kbps modem.
Mouse + printer
| Course assessment | |
The assessment of this course consists of a continuous assessment component and a final examination. Your progress throughout the course (the continuous assessment component) will be assessed through four assignments. All the 4 assignments will counted for assessment. The assignments are designed to test the students on practical work and analysis skills. At the end of the course you will be required to sit a three-hour examination.
Your overall course mark will be calculated from the results of your assignments and from your examination result as follows:
4 assignments (all equally weighted) 50% Final Examination 50% TOTAL 100% The final examination is a written paper of three hours, and you will attempt the questions without the help of any notes or printed materials relating to the course. A simple scientific calculator is allowed. You will be sent a Specimen Examination Paper, which resembles the actual paper in both style and format, so that you can get some idea of what to expect.
The course grade is mainly determined by the overall course score (CS) yet students are normally required to obtain a minimum in both overall examination score (OES) and overall continuous assessment score (OCAS) set by the University in order to obtain a Pass result. To be awarded a particular course grade, student must meet the minimum CS set by the Award Committee.
Assignments
This course is designed to help you move easily from the required readings to the assignments and examination. You are expected to apply information and techniques presented during the course when completing the assignments.
You must submit assignments to your tutor for formal assessment in accordance with the due dates stated on each assignment. The due dates can also be found on the Presentation Schedule. The self-tests or SAQs are, by definition, not part of your formal assessment, but it is very important that you complete them as you work through the units. They not only expose you to the types of problem you are required to complete for the tutor-marked assignments, but they also reflect the demands of the unit objectives and are designed to help you understand and apply the principles covered in the units.
E-submission of assignments
There are four assignments for this course. These are all assignment exercises. You are required to submit your assignments via the e-submission in the OLE. You must submit the assignments on or before the corresponding due date. You can prepare your assignments using word processing software (e.g. Word) and then upload the pdf file to OLE. Or you can complete your assignments on paper and convert them to soft copy by scanning or taking pictures. The recommended format is pdf, jpeg or png (the file size should be less than 10 Mb). You will be able to complete all forms of assessment from the information and material contained in your study guides; however, it is preferable in all degree-level education to demonstrate that you have read and researched more widely than the required minimum. Using other references gives you a slightly different viewpoint and may give you a deeper understanding of the subject.
Your tutor will mark these assignments. Each assignment has a weighting of 12.5%. The assignment component is worth 50% of the total course mark.
How to do your assignments
For each assignment, first read quickly through the description of the problem in the Assignment File. Make brief notes on what you believe are the key points raised. Next, carefully read the description two or three times while referring to your notes. Make sure that you have identified all the key points. Then, read the instructions that accompany the problem. These explain what you are required to do. Make sure you understand what is required and that your assignment provides what is required.
When you have completed the assignment, you should submit your assignment via the e-submission in the OLE. Make sure that each assignment is uploaded to the OLE before the due date. Marks may be deducted for work that is late without prior authorization. If, for any reason, you cannot complete your work on time, contact your tutor before the assignment is due. This is to discuss the possibility of an extension. Extensions will not be granted after the due date unless there are extremely exceptional circumstances.
You should use references other than your textbooks or work when researching the answers for your assignments. Make sure that you reference your work properly. If you do not, you commit plagiarism, and will be penalized severely. Plagiarism is the theft of somebody else's work or ideas. This applies just as much to using the work of other students as it does to the authors of books. If you use somebody else's ideas in your work, give them credit for it. You do this by referencing. In the body of the work, this appears as (Stallings, 2012) for example. At the end of your assignment, list all of your references alphabetically in a section called 'References'. Include the full name, title and date and place of publication. For instance, one way to cite a reference is:
Stallings, W (2012) Computer Organization and Architecture (9th Edition), Prentice Hall.
| Tutors and Tutorials | |
Tutors
There will be both face-to-face tutorials and on-line tutors support.
Tutorials
There will be regular tutorial sessions with your Tutor, supporting the teaching in the main texts. Exact information about the date, time and place of each tutorial will be provided in Stop Presses. Attendance at tutorials is optional.
Surgeries
Regular surgery sessions are scheduled throughout the course, a tutor will be available to answer specific problems that you may have with the course work. These 'surgery' sessions are intended to provide you with specific assistance with specific problems; the tutor will not run a tutorial, and will not have facilities for talking to more than a few students at a time. The surgeries provide the opportunity for you to resolve difficulties without having to wait for your next scheduled tutorial. Details of these surgery sessions will be sent to you in Stop Presses.
Online support
This course will use the Online Learning Environment (OLE) http://ole.ouhk.edu.hk. Please kindly read the OLE letter and OLE user guide to learn the detailed information which contains information relating to the course. The OLE will be a key resource for the students through the provision of live discussion and support features. The concept of a 'Support Zone' is used.
| Conclusion | |
A note about the developers of this course
Dr Paul Kwok
Paul Kwok graduated from the University of Essex with a BSc degree with first class honours in Telecommunication Engineering in 1976 and obtained his PhD degree in Electrical Engineering from the University of Cambridge in 1979.
He joined the then-Monotype International in the Cambridge Science Park in 1979 and worked on the typesetting of Chinese text in a 1000 dpi laser phototypesetter. He has worked for the then-Hong Kong Polytechnic (Electronic Engineering), The Chinese University of Hong Kong (Computer Science), and the then-University of East Asia (Business Administration). Between 1986 and 1997, he was Associate Professor of Computer Science at the University of Calgary. From 1995-1999 he was Associate Professor and Course Director of Computer Engineering in the Department of Electrical and Electronic Engineering, The University of Hong Kong.
Dr Kwok is Professor in the School of Science & Technology, The Open University of Hong Kong.
Dr Wilson Hon-Wai Chu
Wilson Hon-Wai Chu received his BS (summa cum laude) in Electrical Engineering from Boston University in 1988. He also obtained MSE degrees in Electrical Engineering and Mathematical Science from the Johns Hopkins University in 1990 and 1992, respectively. He earned his PhD degree in Electronic Engineering from the Hong Kong University of Science and Technology in 1996. Dr. Chu was a CW Chu Foundation Scholar from 1985 to 1988.
Dr. Chu is an Assistant Professor in the Engineering Science programmes.
Dr. Chu is a Senior Member of IEEE and a Member of IEICE and Tau Beta Phi.
