Washington Univ. in St. Louis     School of Eng.     Dept. of Elec. Eng.

Electronic Systems and Signals Research Laboratory

Information Theory Research Group

Information Theory Research Group
Spring 2005


Faculty and Staff
Joseph A. O'Sullivan
Daniel R. Fuhrmann
Po-Hsiang Lai
Xiuxia Du
Jasenka Benac
Brian Fisher
Naveen Singla
Adam Tyburski
Clayton Miller
Lichun (Andrew) Li
Lindsey Raddatz
Debashish Pal
Norbert Agbeko
Geoff San Antonio
Aikaterini D. Mandilara
Liangjun Xie
Shalini Priti

Purpose of Meetings

The purpose of the meetings is to explore deeper results or recent topics in information theory. The goal is to broaden the knowledge of participants, especially students. Research ideas may also be inspired.

We will choose a set of ideas within information theory, and loosely related to information theory, as they relate to research projects in ESSRL and elsewhere on Washington University's campus. Participants will be called on to present ideas. These should be relatively polished research topics or papers from the literature. The definition of information theory will be modified as needed to accomodate each paper studied.

There will be two separate themes of the research covered this semester, implying that we will often separate into two groups. One group will be more focused on the statistics of natural imagery. One will work through a set of more general topics.

Possible topics (not ordered) include:

  1. Statistics of Natural Imagery
  2. Two-Way Communication Channels
    1. R. Blahut, Principles and practice of information theory, Addison-Wesley, 1987, pp. 347-354.
    2. A. Hekstra and F. M. J. Willems, "Dependence balance bounds for single-output two-way channels," IEEE Transactions on Information Theory, vol. 35, pp. 44-53, no. 1, Jan. 1989.
    3. HB Meeuwissen, J. P. M. Schalkwijk, and A. H. A. Bloemen, "An extension of the achievable rate region of Schalkwijk's 1983 coding strategy for thebinary multiplying channel," 1995 IEEE International Symposium on Information Theory, p. 445, Whistler, BC, Canada, Sept. 1995.
    4. C. E. Shannon, "Two-way communication channels," Proc. 4th Berkeley Symposium Math. Stat. Prob. vol. 1, pp. 611-644, reprinted in Claude E. Shannon: Collected Papers, pp. 351-384, 1993.
    5. Z. Zhang, T. Berger, and J. P. M. Schalkwijk "New outer bounds to capacity regions of two-way channels," IEEE Transactions on Information Theory, vol. 32, pp. 383-386, no. 3, May 1986.
  3. Other unsolved problems in information theory: distributed source coding, broadcast channel, etc. See T. M. Cover and J. A. Thomas, Elements of Information Theory, Wiley-Interscience, 1991, chapter 14.
  4. Data embedding (hiding, steganography, digital watermarking, etc.).
    1. P. Moulin and J. A. O'Sullivan, ``Information-Theoretic Analysis of Information Hiding,'' IEEE Transactions on Information Theory, Vol. 49, No. 3, pp. 563-593, March 2003. (pdf version)
    2. B. Chen and G. Wornell papers
    3. Ton Kalker papers
    4. books: Ingemar J. Cox, Matthew L. Miller, and Jeffrey A. Bloom, Digital Watermarking. San Francisco: Morgan Kaufmann, 2002.
    5. Mauro Barni and franco Bartolini, Watermarking Systems Engineering: Enabling Digital Assets Security and Other Applications. New York: Marcel Dekker, 2004.
    6. Joachim Eggers and Bernd Girod, Informed Watermarking. Boston: Kluwer Academic Publishers, 2002.
  5. Space-Time Coding and Wireless Communication
    1. A. Paulraj, D. Gore, and R. Nabar, Introduction to Space-Time Wireless Communications Cambridge University Press, 2003.
    2. B. Vucetic and J. Yuan, Space-Time Coding. Wiley and Sons, 2003.
    3. G. L. Stuber, Principles of Mobile Communication, Kluwer Academic Publishers, 2001.
  6. Information Theory and Complexity, communication and computation
    1. E. Kushelevitz and N. Nisan, Communication Complexity. Cambridge University Press, 1996.
    2. G. Chaitin books.
    3. Andrew Yao's work including: "Computational information theory," in Complexity in Information Theory, pp. 1-15, Springer-Verlag, 1988.
    4. C. Papamitriou, Information theory and computational complexity: the expanding interface, IEEE Information theory society newletter, special golden jubilee issue, pp. 12-13, 1998.
    5. Stochastic complexity, Rissanen, Schartz, Wallace complexity
    6. Kolmogorov complexity (see Cover and Thomas)
  7. Quantum information theory; information theory and physics
    1. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantm Information. Cambridge University Press, 2000.
    2. Information Physics
    3. Chris Adami, especially his work on quantum information and black holes.

Weekly Meetings

Meetings will be held on Tuesdays 11:30 a.m. - 1 p.m.

Potentially useful links

Independent Component Analysis(ICA), Principal (PCA), etc.

Max-Plus, Max-Sum, etc., (for those interested)

Max-Plus Algebra working group at INRIA
Paper from INRIA group (S. Gaubert, et al.).
Max Plus Group at Delft
A Search on Max Plus Algebra

J. A. O'Sullivan Home Page