David D. Lewis and Vladimir Menkov

Last update: September 24, 2004

Acknowledgments

The work was partially supported under funds provided by the KD-D group for a project at DIMACS on Monitoring Message Streams, funded through National Science Foundation grant EIA-0087022 to Rutgers University. The NSF also partially supported the work through ITR grant DMS-0113236.

Source code used:

• The Lemur Toolkit for Language Modeling and Information Retrieval: http://www-2.cs.cmu.edu/~lemur/

• Numerical Recipes: http://www.library.cornell.edu/nr/bookcpdf.html

References

Project Reports at the main MMS Working Group page ( http://www.stat.rutgers.edu/~madigan/mms/ ):

• Andrei Anghelescu, David D. Lewis, Vladimir Menkov, and Paul Kantor. "Training a Rocchio classifier for adaptive filtering". (TREC 2002).
• David D. Lewis and Vladimir Menkov. "A Closer Look at Nearest Neighbor Text Classification". May 2003.
• David D. Lewis, S. Muthukrishnan, R. Ostrovsky, M. Strauss. "Towards Fast, Effective Nearest Neighbor Text Classification". November 2002. (A number of randomized projection methods referred to in that paper are implemented by this software, but as they are not particularly effective, they are not documented here).

Internal reports on the results of kNN experiments (http://aplab.rutgers.edu/vmenkov/ ):

• kNN runs on RCV1-v2 (January 2003)
• kNN runs on WIPO-alpha (Summer 2003)
• kNN with Inverted List Heuristics (ILH) (Summer-Fall 2003)
• On the optimal neighborhood size k for individual topics (September 2003)

Information on the RCV1-v2 corpus (LYRL):

• David D. Lewis, Yiming Yang, Tony Rose, and Fan Li. RCV1: "A New Benchmark Collection for Text Categorization Research". Journal of Machine Learning Research, 2004. It is expected to appear in early 2004 on the JMLR web site, http://www.jmlr.org/. A draft version of the article is available as a local copy in draft7-pdflatex.pdf.
• Data at David Lewis' site: http://www.daviddlewis.com/resources/testcollections/.

• T. Rose, M. Stevenson, M. Whitehead: "The Reuters Corpus Volume 1 - from Yesterday's News to Tomorrow's Langauage Resources". http://about.reuters.com/researchandstandards/corpus/LREC_camera_ready.pdf

Information on the WIPO-alpha corpus:

• WIPO site: http://www.wipo.int/ibis/datasets/index.html
• Our internal report on corpus preparation: http://aplab.rutgers.edu/vmenkov/wipo/index.html

TREC:

• Text REtrieval Conference (TREC)

Legal Notice

The Adaptive Filtering Software, and this webpage, are covered by the following notice:

Copyright 2005, Rutgers University, New Brunswick, NJ.

All Rights Reserved

Permission to use, copy, modify, and distribute this software and its documentation for any purpose other than its incorporation into a commercial product is hereby granted without fee, provided that the above copyright notice appears in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the names of Rutgers University, DIMACS, and the authors not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission.

RUTGERS UNIVERSITY, DIMACS, AND THE AUTHORS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT SHALL RUTGERS UNIVERSITY, DIMACS, OR THE AUTHORS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

Feedback

Questions, comments, bug reports welcome by email: vmenkov@aplab.rutgers.edu