
	Tanimoto coefficients application.
	=================================

	
	What it does:


This application (Tani2.cpp) computes Tanimoto coefficients
 C[i,j] =  sum_t{ min(f(t,d_i), f(t,d_j)) } / sum_t{ max(f(t,d_i), f(t,d_j)) }
where i,j range over the entire P+N set of a specified topic. 

This is done separately for each topic from the query file in which
|P| >= minP,
|N| >= minN, and
|P+N| <= maxPN.

Two separate out files is produced for each topic; one file with doc
ids and labels, and the other with the coefifcients. In the first
file (Qid.lst), each line has the format

doc-id label

where label is 1 or 0, for documents from P or N, respectively. In the
second file (Qid.tan), each line has the format

d_i: d_1 C[i,1]   d_2 C[i,2] .... d_M C[i,M]

where d_i, d_j are document labels, and C[i,j] are the coeficients.
M = |N+P|.


	How to compile:


This application is implmented as a standard Lemur-using application,
similar to AdaptFilter, and is packaged in filter.zip along with the
adaptive filtering code. The source code is in
filter/src/tanimoto/Tani2.cpp.  

Compile the program with "make Tani2" in filter/src.

Run with e.g.

    $FILTER/src/Tani2 $FILTER/data/tani2_param > tani.log

where $FILTER is the location of your "filter" source directory.  The
parameter file, filter/data/tani2_param, is similar to the one used by
the Adaptive filtering application, but is much shorter. It specifies
the index file from which the TF info is obtained, the query file that
lists topic IDs, and the QREL file that lists relevant and irrelevant
documents (P and N) for each topic. All files are expected to be in
the same format that we used in adaptive filtering.

The parameters tani.minN, tani.minP, tani.maxPN control the selection
of topics to be analyzed. Other topics will be ignored.

The program will report to the standard output some progress
information, namely the query ids, sizes of |P| and |N|, and the
average values of C[p,p], C[n,n], and C[p,n] for each query that it
processes. (The diagonal elements, C[i,i] are excluded from this
averaging). By the way, for each particular query all its 3 averages
seem to be very similar; that is, C[P,N] coefficients are not,
on average, noticeably smaller than the off-diagonal C[i,j]
cpefficients within the C[P,P] or C[N,N] blocks.


     Runtime performance:

It takes about 5 minutes to run the program on an Apple Macintosh
computer (a new "iMac" model) on the RCV1 corpus, processing the 100
"natural" queries (R101-R200). Actually, the program only processes
only 52 topics, ignoring the other 48, becaus ethey have too few
positive documents. The total size of the output files is around 53
Mb.






