Probabilistic methods, Estimation of the amount of...

Probabilistic methods

Events that can or do not occur are called random. The probability p is a quantitative measure of the probability of occurrence of an event and estimates the degree of its occurrence, uncertainty.


The probability values ​​are in the range from zero to one: . The value p = 0 refers to an impossible event, the value of p = 1 - to the event that must occur, i.e. to nonrandom events.

Estimating the amount of information for equal events

The probability of occurrence of an event belonging to the set of possible events is defined by the expression


where - the number of events.

For example, dropping a 6-faced cube (N = 6), made of a homogeneous material, the fallout of one or another of its faces is an equally possible event A n of six events . The set of six events is the full group , i.e. When you throw a cube, there is always an event A n. In addition, events are inconsistent, since two or more events can not occur at the same time during the test. Therefore, the probability of an event (falling out of any face when casting) is

For incompatible events that constitute a complete space,

It follows from formula (1.1) that the larger the number of equally probable events, the less the probability and the greater the degree of uncertainty, which determines the amount of information, a logarithmic function with a base a:


The expediency of choosing the measure (1.2) is due to the fact that for N = 1 the degree of uncertainty , the probability of occurrence of the event is equal to p n = 1; at the degree of uncertainty , the probability of occurrence of the event is).

Since there is a relationship between the logarithms with the bases a and b where , then there is also a connection between the uncertainty values ​​for different bases, determined by the constant factor M. For example, at bases 10 and 2, the uncertainty values ​​are related by the relation, since . In computer science logarithms are used for base 2 (log2 N). In this case, the degree of uncertainty H n is expressed in bits. A message that reduces the uncertainty of knowledge in half, carries 1 bit of information.

Expression (1.2) is used to estimate the amount of the so-called own, or individual, information obtained by removing the original uncertainty as a result of an event. Using the expressions (1.1), (1.2), we express the amount of information in terms of probabilities:


The relations (1.2) and (1.3) under the condition (1.1) are called the Hartley measure, which gives an estimate of the amount of information for equally probable events.

thematic pictures

Also We Can Offer!

Other services that we offer

If you don’t see the necessary subject, paper type, or topic in our list of available services and examples, don’t worry! We have a number of other academic disciplines to suit the needs of anyone who visits this website looking for help.

How to ...

We made your life easier with putting together a big number of articles and guidelines on how to plan and write different types of assignments (Essay, Research Paper, Dissertation etc)