	Contingency Tables.
	(Updated Feb-06-2003)
 	
The Adaptive Filter program may produce up to 3 contingency tables,
written in files whose names are defined by the variables trainTable,
train1Table, and testTable in the config file. (See the sample config
file for syntax).


(a) trainTable: This table is obtained by applying the classifier obtained
after the initial training (or simply "*the* training, in a batch
method) to the entire training set.  We actually do specially score
every training set documents to produce the table, even if, perhaps,
some or all of them have been recently scored for some other purpose
(such as threshold-setting).


(b) train1Table: This table is only produced in the methods employing
pseudo-feedback (such as Rocchio or Paul1; any method whose classifier
class is derived from SumFilterMethod). To obtain it, we take the
model as it exists after the true P and N examples in the training set
have been taken into account, but before pseudo-feedback; combine this
model with the threshold for it comuted by the LOO method on all the
true P and N examples in the training set; and finally apply the
resulting classifier to the entire training set. In practice,
producing this table does not involve an actual re-scoring of the
training set, because we utilize the scores that need to be obtained
anyway for the purpose of pseudo feedback. 

On the rare occasions that there simply are no true positive or
negative training examples for a topic (this would never happen with
full-size RCV1-v1 or RCV1-v2 training sets, but is not impossible with
smaller experimental sets), it is of course not possible to use LOO to
obtain any sensible threshold value for the pre-pseudo-feedback
classifier. For these topics, we print the corresponding rows in the
train1Table under assumption that the classifier discovers nothing
(i.e., zeros in the first 3 numerical columns).


(c) testTable: this table is produced during the actual filtering of
the test set, and represents the results of applying the actually used
classifier (batch [static], or adaptive [adjustable]) to the
entire test set.

In each file, each line has the following 7-column format, suggested
by Alex Genkin in January 2003:

Topic_id  sp_tp sp_tu sp_tn   sn_tp  sn_tu sn_tn

e.g.

R121        3    893      8      0  19155      4

Here sp_tp is the number of examples "discovered" by the system (a.k.a
"system positives") which, according to the Oracle, are truly positive
("true positives"); sp_tn is the number of discovered examples that
are, in fact, negative ("false positives"); sp_tu is the number of
discovered examples for which the Oracle had no judgment ("unjudged";
there are no such examples in fully judged corpora).  sn_tp, sn_tu,
sn_tn are the corresponding P/U/N counts for the examples *not*
discovered by the classifier. The 6 numbers should total to the size
of the training set or test set, as appropriate.

