Information and Entropy

Shannon and Weaver (S&W) presented an engineering solution to an information problem with something that would eventually be known as information theory. Though very mathematically, its philosophical roots are intrinsically linked to a physical property of matter known as entropy, or the amount of thermal energy, or kinetic energy of the electrons in a closed system. In another definition, entropy also measures a degree of disorder.

S&W equate uncertainty, or information that is not known, to entropy, or degree of disorder in the closed system, by assuming that the probabilistic amount of order in nature with respect to information, or that the natural state of organized information in a closed system is constant and equal to 100%. Thus the goal of their method is to totally reduce, minimize, and eliminate uncertainty.

Let’s consider the next examples. It is also important to know that S&W addressed the problem of noise reduction in electronic transmissions. In one case, a message is correctly transmitted and received. There is no loss of information and uncertainty is equal to 0. In another case, the message is garbled and the information is totally lost, so that uncertainty is equal to 1. Probability is bound by 0 and 1.

The usefulness of S&W appears in all its majestic importance when the message is partially garbled. Is in those cases when the importance of the assumptions used to calculate probabilities emerged. Faster computer processing and inexpensive data storage enabled better corrective solutions. S&W provided the theoretical approach and computer technologies allowed its implementation.

But in general, with respect to computer-based information processing, and in particular with respect to information retrieval, there is great temptation to use the theory without realizing that the environment for which it was developed resulted in a closed system. Language utilization for communication is an open system.

In other words, information theory has limitations. It behooves the researcher and student of information processing to understand that S&W bound this framework and environment in a very particular way. The enticing opportunity of information theory should be balanced with the reality of the results.

For these reasons, one should be asking why is it that systems work at all when using this and other unsuitable theories rather than trying to gain efficiency on processes that are theoretical flawed.

The Information Science Space

Living organisms are consumers of information, indicating the importance of asking basic questions about human consumption of information. Basic human activities, such as breathing require human processing of information.

Processing of information is a cognitive activity that is manifested through thoughts and actions. Cognition is, therefore, an important field of study for Information Science thus part of the Information Science Space.

The expansion of computer technologies in the 1950s boosted Cognitive Science. Cognitive theories, models and systems would be implemented and tested quickly and easily. As computer information systems gained territory in daily life, their role in society cannot be denied.

The field of study known as human-computer interaction (HCI), as several others in the interface of computer technology with humans and with information, has established the bona-fide inclusion of computer technology in the Information Science Space.

NOTE: The study of computer technology takes place in a family of fields under different disciplines such as computer science and information systems.

Because human life does not take place in a vacuum but is part of an interactive community, several other sociological aspects emerge. Various components of this rich human-to-human and human-to information interactions are substantive for multiple disciplines from the humanities to the social sciences. This is the reason why the study of Information Science is a multidisciplinary study.

Several fundamental issues:

• Is it true that living organisms are consumers of information?
• If so, one could ask if there are any types of human processes and behavior not immediately related to information.
• One can define information processing very broadly so as to include breathing, or very narrowly so as to include memorization.
• Information science is not only concerned with information use but also with all processes related to information.
• Understanding the use of information and other information processes remains difficult in part due to the lack of agreement about how specifically information needs to be defined and operate upon.