Journal of Lost Papers: On the divergence between two distributions and the probability of misclassification of several decision rules

The (virtual) Journal of Lost Papers: Article 001

Once upon a time, many years ago, scientific papers couldn’t be downloaded over the interwebs [1] from the comfort of one’s laptop. Instead papers were printed on actual “paper” (made from dead trees), and stored in large buildings called “libraries”. One had to physically go to the “library”, locate the bound codex in the “stacks”, and then find the desired article by turning to the correct “page number”. If you wanted a copy of the paper, you then lugged the huge codices to a primitive machine called a “photocopier”, which could be induced to provide a bad copy in return for “coins” [3].

Recently, I was searching for inequalities between the Jeffreys divergence and Bayes error rates. I’d cooked up my own inequality, but suspected that a tighter bound should be possible, so I started searching for relevant papers [2]. And I turned up a tantalizing citation to an ancient paper that, amazingly, does not appear to exist in the virtual world. So, for the first time in a long time, I sojourned to an actual library, in this case the Engineering library at UC Berkeley. I’m glad I did. The paper is in an early machine learning conference proceedings, along with other fascinating papers by now-famous pioneers. And I found exactly the inequalities I was looking for.

Thus, for posterities sake, I present anew:

On the divergence between two distributions and the probability of misclassification of several decision rules

Gofried T. Toussaint, Proceedings of the Second International Joint Conference on Pattern Recognition (1974) 27–34

PDF: Toussaint1974a.pdf

[1] It’s a series of tubes.

[2] It has been said that “a week in the lab can save an hour in the library”. On the other hand, it is generally much easier to find relevant papers once you’ve already figured out the answer for yourself.

[3] An ancient form of currency, consisting of physical tokens constructed from disks of metal.