Ternopil Ivan Puluj National Technical University

Каф. комп'ютерних наук

Fundamentals of Information Theory


1. Educational programs for which discipline is mandatory:

# Educational stage Broad field Major Educational program Course(s) Semester(s)
1 bachelor's 12. Інформаційні технології 122. Комп’ютерні науки та інформаційні технології (бакалавр) 2 4

2. The course is offered as elective for all levels of higher education and all educational programs.

3. Information about the author of the course

Full name Dmytrotsa Lesia
Academic degree PhD
Academic title none
Link to the teacher`s page on the official website of the University
Е-mail (in the domain

4. Information about the course

Study hours structure Lectures: 18
Practical classes: 0
Laboratory classes: 36

Amount of hours for individual work: 96
ECTS credits: 5.0
Teaching language english
Form of final examination exam
Link to an electronic course on the e-learning platform of the university

5. Program of discipline

Description of academic discipline, its goals, subject of study and learning outcomes

The course contains the lectures- theoretical information about the basic concepts in information theory and coding in information systems, the evaluation of quantitative characteristics of the processes of transmission, storage and processing of information, algorithms and software implementations for analyzing information sources, effective coding, basic methods of cryptographic information protection .

The place of academic discipline in the structural and logical scheme of study according to the educational program

Prerequisites. List of disciplines, or knowledge and skills, possession of which students needed (training requirements) for successful discipline assimilation

higher mathematics,
probability theory, probability processes and mathematical statistics,

Contents of the academic discipline

Lectures (titles/topics)

Module 1.
1. Introduction. Foundations: probability, uncertainty, information.
2. Measure of Information
3. Conditional entropy
4. Information Channels.
Module 2.
5. Source Coding.
6. Error correction and detection codes
7. Data compression codes and protocols.
8. Cryptography.
9. Hash function. Digital signature

Laboratory classes (topics)

Module 1.
1. A quantitative measure of information. Entropy and its properties
2. Entropy of Composite Messages of Dependent Sources. Conditional and Mutual Entropy
3. Characteristics of the source of messages and characteristics of communication channel
Module 2.
4. Information encoding. Shannon-Fano coding
5. Huffman Coding
6. Hamming Code.
7. Data compression codes
8. Encryption. Caesar's method.

Learning materials and resources

1. R.M. Gray. Entropy and Information Theory /Springer Science+Business Media, LLC 2011, 409 pages
2. Foundations of Coding: Theory and Applications of Error-Correcting Codes with an Introduction to Cryptography and Information Theory, J. Adamek, Wiley-Interscience 1991
3. Monica Borda. Fundamentals in Information Theory and Coding /Springer-Verlag Berlin, Heidelberg , 2011, 483 pages
4. Cover, T., Thomas, J.: Elements of information theory. Wiley.
5. Richardson, T., Urbanke, R.: Modern Coding Theory. Cambridge University Press.
Approved by the department
(protocol №
on «