Hello, if you have any need, please feel free to consult us, this is my wechat: wx91due
Math 4280: Introduction to Information Theory
• Instructor: Prasad Tetali, tetali-at-math.gatech.edu
• Time/Location: TTH 1:30–2:45pm; Skiles 154
• Office hours: Skiles 132; Tue: 3-5pm, Wed: 2-3pm
• Prerequisites: Undergraduate-level Probability
Recommended Textbook : “Elements of Information Theory” (2nd edition), by Thomas Cover and Joy Thomas, Wiley 2006.
Objectives: To understand quantitative formulations of information, efficiency in communication and compression of information, computational aspects of description of information. To learn of applications of entropy in probability and combinatorics.
List of Topics:
• (1 lecture) Introduction
• (2 lectures) Chapter 2: Entropy, Relative Entropy, Mutual Information
• (2 lectures) Chapter 2 (contd.): Data Processing and Fano’s Inequalities
• (2 lectures) Chapter 3: Asymptotic Equipartition Property (independent r.v.s)
• (1 lecture) Review of main concepts, some example exercises
• (1 lecture) Test 1 (Tuesday, Feb. 5) : in class, closed book, closed notes.
• (2 lectures) Chapter 4: Entropy Rates of Markov Chains (dependent r.v.s)
• (2 lectures) Chapter 5: Coding: Shannon, Huffman codes, Optimality
• (3 lectures) Chapter 7: Communication and Capacity: Channels, Channel coding theorem, Source-channel theorem
• (2 lectures) Polar Codes
• (1 lecture) Test 2 (Tuesday, March 12th) : in class, closed book, closed notes.
• (2 lectures) Entropy and Combinatorial Applications
• (2 lectures) Chapter 8: Differential Entropy (continuous r.v.s)
• (2 lectures) Chapter 9: Gaussian channel
• (1 lecture) Review of main concepts, some example exercises
• (1 lecture) Test 3 (Tuesday, April 16th) : in class, closed book, closed notes.
• (2 lectures) Chapter 14: Kolmogorov (algorithmic) Complexity: Turing machine, Information and Entropy
• Final Exam on Thursday 2nd; 2:40pm – 5:30pm in Skiles 154
Grading policy : Three Tests 70% ; Final 30%
Grade A: 90% and above, Grade F: 50% and below.
Homework will be assigned on a regular basis, but not collected. You are strongly advised to (attempt to) solve all the homework problems, for better understanding of the material and to help prepare for the tests.
Academic Dishonesty: All students are expected to comply with the Georgia Tech Honor Code. Any evidence of cheating or other violations of the Georgia Tech Honor Code will be submitted directly to the Dean of Students. The institute honor code is available at: http://www.policylibrary.gatech.edu/student-affairs/academic-honor-code