Information Theory Research Group
Information Theory Research Group
Faculty and Staff
Joseph A. O'Sullivan
Daniel R. Fuhrmann
Lichun (Andrew) Li
Geoff San Antonio
Aikaterini D. Mandilara
Purpose of Meetings
The purpose of the meetings is to explore deeper results or recent topics in information theory.
The goal is to broaden the knowledge of participants, especially students. Research ideas may
also be inspired.
We will choose a set of
ideas within information theory, and loosely related to information theory, as they relate to
projects in ESSRL and elsewhere on Washington University's campus.
Participants will be called on to
present ideas. These should be relatively polished research
topics or papers from the literature. The definition of
information theory will be modified as needed to accomodate
each paper studied.
There will be two separate themes of the research covered this semester, implying that we will
often separate into two groups. One group will be more focused on the statistics of natural
imagery. One will work through a set of more general topics.
Possible topics (not ordered) include:
- Statistics of Natural Imagery
- Two-Way Communication Channels
- R. Blahut, Principles and practice of information theory, Addison-Wesley, 1987,
- A. Hekstra and F. M. J. Willems, "Dependence balance bounds for single-output two-way
channels," IEEE Transactions on Information Theory, vol. 35, pp. 44-53, no. 1, Jan.
- HB Meeuwissen, J. P. M. Schalkwijk, and A. H. A. Bloemen, "An extension of the achievable
rate region of Schalkwijk's 1983 coding strategy for thebinary multiplying channel," 1995 IEEE
International Symposium on Information Theory, p. 445, Whistler, BC, Canada, Sept. 1995.
- C. E. Shannon, "Two-way communication channels," Proc. 4th Berkeley Symposium Math. Stat.
Prob. vol. 1, pp. 611-644, reprinted in Claude E. Shannon: Collected Papers, pp.
- Z. Zhang, T. Berger, and J. P. M. Schalkwijk "New outer bounds to capacity regions of
two-way channels," IEEE Transactions on Information Theory, vol. 32, pp. 383-386, no. 3, May
- Other unsolved problems in information theory: distributed source coding, broadcast
channel, etc. See T. M. Cover and J. A. Thomas, Elements of Information Theory,
Wiley-Interscience, 1991, chapter 14.
- Data embedding (hiding, steganography, digital watermarking, etc.).
- P. Moulin and J. A.
O'Sullivan, ``Information-Theoretic Analysis of
Information Hiding,'' IEEE Transactions on Information Theory,
Vol. 49, No. 3, pp. 563-593, March 2003.
- B. Chen and G. Wornell papers
- Ton Kalker papers
- books: Ingemar J. Cox, Matthew L. Miller, and Jeffrey A. Bloom, Digital
Watermarking. San Francisco: Morgan Kaufmann, 2002.
- Mauro Barni and franco Bartolini, Watermarking Systems Engineering: Enabling
Digital Assets Security and Other Applications. New York: Marcel Dekker, 2004.
- Joachim Eggers and Bernd Girod, Informed Watermarking. Boston: Kluwer Academic
- Space-Time Coding and Wireless Communication
- A. Paulraj, D. Gore, and R. Nabar, Introduction to Space-Time Wireless
Communications Cambridge University Press, 2003.
- B. Vucetic and J. Yuan, Space-Time Coding. Wiley and Sons, 2003.
- G. L. Stuber, Principles of Mobile Communication, Kluwer Academic Publishers,
- Information Theory and Complexity, communication and computation
E. Kushelevitz and N. Nisan, Communication Complexity. Cambridge University Press,
- G. Chaitin books.
- Andrew Yao's work including: "Computational information theory," in Complexity in
Information Theory, pp. 1-15, Springer-Verlag, 1988.
- C. Papamitriou, Information theory and computational complexity: the expanding
interface, IEEE Information theory society newletter, special golden jubilee issue, pp.
- Stochastic complexity, Rissanen, Schartz, Wallace complexity
- Kolmogorov complexity (see Cover and Thomas)
- Quantum information theory; information theory and physics
- M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantm
Information. Cambridge University Press, 2000.
- Information Physics
- Chris Adami, especially his work on quantum information and black holes.
Meetings will be held on Tuesdays 11:30 a.m. - 1 p.m.
- Feb. 8: Norbert Agbeko, statistics of natural imagery. Paper by Srivastava, Lee, Simoncelli,
and Zhu. pdf file.
- Feb. 15: Po-Hsiang Lai, sparse coding, statistics of natural imagery, a little
- Feb. 22: Lindsey Raddatz, more statistics of natural imagery
- Mar. 1: Geoff San Antonio, MIMO radar
- Mar. 8: Spring Break
- Mar. 15: Steve McLaughlin visits WU, gives seminar in ESE
- Mar. 22: Po-Hsiang, two-way communication channels
- Mar. 29: Dr. O or Brandon W.: other open problems in information theory
(distributed source coding, broadcast channel, etc.)
- Apr. 5: Adam Tyburski, data embedding
- Apr. 12: Shalini Priti, more data embedding
- Apr. 19:
- Apr. 26:
- May 3: finals?
- May 10: finals?
- May 17:
- May 24:
- May 31:
- June 7: no meeting
- June 14: Aikaterini, quantum information theory
- June 21: Aikaterini, quantum information theory
Potentially useful links
Independent Component Analysis(ICA), Principal (PCA), etc.
Max-Plus, Max-Sum, etc., (for those interested)
Algebra working group at INRIA
Paper from INRIA group (S. Gaubert, et
Plus Group at Delft
A Search on Max Plus Algebra