Introduction

Information theory as a separate subject is about 70 years old. Since information is energy, therefore it is measured, managed, regulated and controlled for the sake of welfare of humanity. The role of information function is to remove uncertainty and the amount of uncertainty removed is a measure of information.

The concept of information proved to be very important and universally useful. These days language used in telephones, business management, and cybernetics falls under the name “Information Processing”. In addition to this, information theory particularly measures of information have applications in physics, statistical inference, data processing and analysis, accountancy, psychology, etc.

Shannon²⁴ was the first who developed a measure of uncertainty. He was interested in communicating information across the channel in which some information is lost in the process of communication and that was called a noisy channel. His objective was to measure the amount of information lost. He defined a measure of uncertainty of a probability distribution as given below:



where k is an arbitrary positive constant. The measure (1.1) was called entropy. Thereafter, Shannon’s entropy was characterized by various researchers like Khinchin¹], Fadeev⁸. Teverberg²⁵, Chandy and Mcleod⁵, Kendal¹⁵, Lee²⁰, Berges², Cziszar⁷, Cheng⁶, etc. on using different sets of postulates.

The quantity (1.1) measures the amount of information of probability distribution P when effectiveness or importance of the events is not taken into account. In addition to this; some probabilistic problems also play important role. Considering  effectiveness of the outcomes, Belis and Guiasu¹ introduced U= (u₁,u₂,……,u_n) as a utility distribution, where u_i>0, is the usefulness of  an event having probability of occurrence p_i and consequently, “self  useful  information’ is defined as given below:



The measure (1.2) is based two postulated as given below:

P1. In case  all the events of an random experiment have the same utility u>0, then the self used information generated by the product of two statistical independent events E₁ and E₂ can be expressed as the sum of the self-useful information provided by E₁ and E₂  individually i.e.



where P₁*P₂ is the probability of E₁∩ E₂



Further, Belis and Gauisu¹ gave the following qualitative and qualitative information measure:



Longo²¹ called (1.5) as ‘useful’ information  and Guiasu and Picard⁹ called weighted entropy.

In this communication , the ‘useful’ relative information measure is defined and characterized axiomatically in section 2.The new measure thus  introduced is generalized in section 3 with  its and its particular cases are studied in section 4 . The applications of new R-norm information measure are described in section 5. In section 6 its ad joint by taking empirical data is studied with its illustration graphically. In the end the conclusion is given along with an exhaustive list of references.

‘Useful’ Relative Information Measure

Let X be a random variable in an experiment and



be its probability distribution having U= ( u₁, u₂…….., u_n) as a utility distribution, where u_i>0 for each i,  is the utility of an event having probability p­_i .

A ‘useful’ directed divergence measure was defined by Bhaker and Hooda³ and characterized as given below:



where



If we consider a uniform probability



in (2.1), it reduces to log n—H(U;P),



It may be noted that ‘useful’ directed divergence measure D(U;P;Q)  satisfies the following conditions:

D(U;P:Q) ≥ 0

D(U;P:Q)= 0

D(U;P:Q)is a convex function of  q₁,q₂,….,q_n as well as .P₁, P₂,…..,P_n

Further, it is observed that (1.1) is not symmetric in P and Q, since D(U;P:Q)≠ D(U;Q:P) .

Later on a symmetric ‘useful’ J- divergence measure was introduced by Hooda and Ram² as given below:

J(U;P:Q) =D(U;P:Q)+ D(U;Q:P)




where



In case u_i = 1 for i  , then (2.3) reduces to



where (2.4) is a divergence measure is due to  Jeffrey and thus it is called as J-divergence.

Bhaker and Hooda³ also characterized a ‘useful’ relative information measure of order a as given below:



Further Hooda and Ram¹⁰ characterized non-additive ‘useful’ relative information of degree β as given below:  



The measure (2.6) reduces to (2.1) when β = 1, while in case u_i=1  then (2.6) reduces to Kullback- Leibler’s¹⁷ ‘useful’ relative information measure.

Further Boekee and Lubbe⁴ defined R-norm information of the distribution P as



 Kumar et al.[18]  also defined the following ‘useful’ R-norm relative measure:



There are many other generalizations of (2.8) also and one of them is



where



A ‘Useful’ R-norm Relative Information Measure


Theorem 3.1 Let P and Q be two probability distributions attached with a utility distribution U, then the following holds:



? 1 according to  R ? β under the condition



Proof: We know from Holder’s inequality as



where    R >β

, Setting



and



 In Holder’s inequality (3.1), we have



On simplification we have



or



Since 



therefore (3.2) can be written as:



Similarly, we can prove that



For  



for all probability distributions and if R ≠ β and P₁ = q₁ , for each i , i.e. P= Q, then we have



Theorem 2.2  D^β(U; P; Q) is a convex function of P and Q.

Proof:  Let



On differentiating K with regard to  p_i with all q₁  

and u_i having fixed value,  we have

 

fixed and consequently,



is constant

Hence we can write



where



is constant

It implies that



or



or R ­­– β > 0, (2.6) is positive

It implies that



are convex functions of P in view of  



Similarly, for



is also a convex function of P .

For , (3.6) is negative, so



are concave functions of P , since 



Hence for  R – β <0  & R – β >0 , D_R^β( U; P; Q)  is also a convex function of P.

On same lines we can prove that D^β(U;P;Q)  is a convex function of Q for R – β >0 and R – β <0 provided



A Generalized ‘Useful’ R-norm Relative information Measure of Degree β

We consider the following function:



where F(X) is a monotonic increasing function of x.

In view of (3.2), R – β >0 and 



, we have



It implies



Or



Multiplying (4.3) by , we get



Similarly, in view of R – β <0, and



, (4.4)   can be written as:



It implies that



or



Multiplying (4.8) by



, we get



or



Hence from (4.5) and (4.6) together give



It may be noted that (4.11) vanishes when P₁ = q₁ for each i .

It implies that



when  P₁ = q₁ for each i

or



or



In particular when β = 1 and R → 1, then D^β( P: Q; U) reduces to




which is (2.1)

Particular Cases

when



then (4.11) reduces to 




and in case P = C;, D(P;C;U)= 0 .

Further, it can be verified that D(P;C;U) is a convex function of P..

Next, it may be noted that (4.13) can be written as



Next we consider



or



and



Further, if F(x) = x^j, (j ≥ 1),then we have





For j= 1, we get



In case R→ 1 the measure (4.13), (4.17) and (4.5) respectively reduce to





and



It may be noted that (4.21) was defined and characterized by Bhaker and Hooda³.


In case F(x)= log x in (4.1), it reduces to




or



In case u_i = 1 for each i in (4.10), it reduces 




which is well known Renyi’s²³ entropy of order R.  
           

Illustration with an Example

In this section we consider production data of different companies due to Nager and Singh²² represented in Table 5.1. We calculate D^β( P; Q; U) in Table 5.2 and represent graphically in fig.5.1.

Table 5.1: Scanned Copy of Data due to Nager and Singh²²  

Next we compute these values of the generalized ‘useful’ r-norm relative measure when R = 2 and β = 0.5 in the following table:

Now, the graphically representation of the new ‘useful’ R-norm relative information of degree β when R=2 and β=0.5 is given in fig. 5.1.
 

Figure 5.1:  Graph of the New ‘Useful’ R-Norm Relative Information. Click here to view figure

The amount of divergence values can be arranged for forecasting the profit maximization in a table as:

Table 5.3: Data of Divergence Values.

Interpretation

On the basis of values of generalized divergent ‘useful’ R-norm relative information of degree β represented in table 5.3, we can suggest the investor to select the company having maximum divergence value for investment.

Ad Joint of the Generalized Information Measure and its application

On  taking Q and P after interchanging  in (2.9) , we get 




Thus (6.1) is called the ad joint of (2.9). Similarly, we compute these values of the ad joint of generalized measure at R= 2, β = 0.5 and represented table (6.1) as given below:

Table 6.1: Values of ad Joint of Generalized Measure.

Considering the above table 6.1, the graph is drawn as given in the following figure 6.1:
 

Figure 6.1: Graph of Non-symmetric Ad joint. Click here to view figure

Interpretation

The ad joint of’ useful’ R-norm relative information of degree β in decreasing order in the table (6.1 to suggest the investor to make investment in the company of maximum divergence.

Conclusion

In this paper we have defined and characterized the generalized ‘useful’ R-norm relative information of degree β and discussed its particular cases also. The application of this information measure has been studied. The ad joint of this measure is defined and its application in share market and decision making problems are described graphically. The ‘Useful’ R-norm relative information measures of degree and its ad joint are defined and studied in this communication can further be generalized parametrically and applied in planning, forecasting, agriculture, etc.

Conflict of Interest

There is no conflict of interest among the authors of this paper.

References

1. Belis, M. and Guiasu S.,”A quantitative –qualitative measure of information in Cybernatic system”,  IEEE Trans. Information Theory (1968), IT-4,593-94.
   CrossRef
2. Berges, R.,”Zur Herbitung der Shannon scheme informationmath”.Vol.96,282-87.
   CrossRef
3. Bhaker, U.S. and D.S. Hooda, “Mean value characterization of ‘useful’ information measures”, Tamkang Journal of mathematics (1993), vol. 24, 383-394.
   CrossRef
4. Boekee, D.E. and Lubbe, C.A., “The R-norm Information Measure. Information and Control (1980), Vol. 45, 136-155.
   CrossRef
5. Chandy, T.W. and Mcleod, J.B., ”On a functional equation”, Proc. Edinburgh Math. Soc., Edinburg (1960), Vol. 43, 7-8.
   CrossRef
6. Cheng, M.C., “Information theory and some of its applications”,(1976), Vol.10, 159-171.
7. Csiszar,I., “Information measure, a critical survey”,Preprint,Math.Inst. Hung.Acad.Sci.”, (1974 ), 1-24.
8. Fadeev D. K., “On the concept of entropies of a finite probabilistic scheme (Russian)”, Uspechi Math. Nauk(1956), Vol.11, 227-31.
9. Guiasu, S.and Pichard, C.F.,  “ Brone inferieure de la longueur de certain codes”,C.R. Acad.Sci.(1971),273, 248-51.
10. Hooda, D.S. and Anant Ram, “Mean value characterization of non -additive ‘useful’ relative information measures”, Current trends in mathematics, Statistics and O.R. (2001), pp. 74-89.
11. Hooda, D. S, & Kumar, P. Generalized measures of useful directed divergence and Information improvement with applications. Defence Science Journal 54(2), (2004), Pp 125-135.
    CrossRef
12. Hooda,D.S., Upadhaya, K. and Sharma, D. K., “Useful R-norm Information Measure and its Properties”, IOSR Journal of Electronic and Communication Engineering, Vol.8, 52-57.
    CrossRef
13. Hooda,D.S., Upadhaya, K. and Sharma, D. K., “On Parametric Generalization of ‘Useful’ R-norm Information Measure”, British Journal of Mathematics & Computer Science Vol 8(10),(2015), Pp. 1-15.
    CrossRef
14. Jeffery’s, H., “An invariant form for the prior probability in estimation problems”, Proceeding of the royal society of London, vol. 186, pp.453-461.
    CrossRef
15. Kendall, D.G., “Functional equation in information theory”,  Z. Wahrs Verw. Geb.”, (1964), Vol. 2, 225-29.
    CrossRef
16. Khinchin,A.I., “ The concept of entropy in the theory in the theory of probability”, Uspechi Math. Nauk(1953), 8(3), 35.
17. Kulback,  S. and Leibler, R.A. “Information Theory and Statistics”, New York, John Wiley.
18. Kumar, Satish, Gurdas Ram and Vishal Gupta, “On ‘useful’ R-norm relative information and J- divergence measures”, International  journal of pure and applied mathematics(2012),vol.77No.3, 349- 358.
19. Leibler, R.A. and S. Kullback, “On information and sufficiency”, The Annals of Mathematical Statistics (1951), vol.22, pp.79-86.
    CrossRef
20. Lee, P.M., “On axioms of information theory”, Ann. Math. Stat. (1964),35, 414-18.
    CrossRef
21. Longo, G., “A quantitative-qualitative measure of information”, Springer Verlag, 1972.
    CrossRef
22. Nager, Ruchi and R.P. Singh, “Application information Theoretic divergence measures-Forecasting profit maximization in share market”, GSTF journal of mathematics, Statistics and Operation research (2013), Vol.2, pp.45-53.
    CrossRef
23. Renyi, A. “On measure of entropy and information, In: Proc 4-th Barkley symposium”, on Mathematics, Statistics and Probability, University of California Press (1966), 547-561.
24. Shannon, C.E.,  “ A mathematical theory of communication”, Bell System Tech. Journ.(1984), Vol. 27, 379-423.
    CrossRef
25. Tverberg, H., “A new derivative of the information function”, Math. Scand (1958), Vol.6, 297-98.
    CrossRef