Contributors: - GK: Gabriella Kazai - BP: Benjamin Piwowarski - AV: Arjen de Vries - ZS: Zoltan Szlavik ========================================================================= Version CVS ======= 2007/11/27 [BP] Automated tests for metrics 2007/11/26 [BP] Fixed bug #1834543 (bad collection handling for runs) 2007/10/31 [BP] Parsing of the INEX 2007 format (handling passages) The default topic submission run result is a Component (was Doxel), from which derive Doxel and Passage Version 1.0.6 ======= 2007/08/11 [BP] Add a warning if the collection of the runs & the collection of the assessments are not the same 2007/08/09 [BP] Added an -alias option for collections in the evaluation task 2007/05/14 [BP] Corrected the captions for precision-recall graphs 2007/03/13 [BP] Fixed a bug that prevented some assessed files from being retrieved during evaluation 2007/03/01 [BP] Added an option (set collection) so that the wikipedia collection can be used as-is (ie, without moving all the files within one directory). See the README file for more information. 2006/11/22 [BP] Fixed a lot of bugs and added new options for evaluation 2006/11/17 [BP] EvalJ now *iterates* on collection files instead of loading the whole file list 2006/11/15 [BP] Changed the README file into a README.html file (the former was getting a little bit overcrowded) 2006/11/15 [BP] Corrected bugs with EPRUM + BEPD metric for the BEC task 2006/11/15 [ZS] Test set for the BEC task 2006/11/10 [BP] Added the BEP behaviour in EPRUM 2006/11/08 [BP] Implemented the BEPD metric Version 1.0.5 ======= 2006/02/10 [GK] Fixed minor issue with rounding Version 1.0.4 ======= 2006/01/06 [GK] Fetchbrowse evaluation, new interpolation for ep/gr Version 1.0.3 ========= 2005/12 [BP] Fixed small bugs (missing jars, etc.) 2006/05/01 [BP] Added a -details option to the evaluate command (outputs topic level results). Version 1.0.2 ========= 2005/12 [BP] Fixed small bugs (missing jars, etc.) Version 1.0 ======= 2005/11/09 [BP] Fixed bugs & prepared the first release 2005/11/09 [GK] fetch&browse evaluation 2005/10/31 [BP] Updated E2PRUM and debugged it (not yet for F+B) 2005/10/25 [BP] * Changed the assessment format in order to store passages * Modified the SubmissionRun and TopicSubmissionRun so that we can iterate over articles (Fetch&Browse task handling) and that doxel are ordered by article first for F&B. 2005/10/24 [GK] Lots of modifications for INEX 2005 (inlc. parsing, new properties, rewrite of XCGEval, bug fixes, etc.) 2005/10/23 [BP] Big changes in evalj: * The metrics definition are no in a file (and not anymore command line options) * The E2PRUM metrics now works (not tested though) * The "ideal target generator" is now an option of PRUM/E2PRUM/GR 2005/10/21 [BP] Try to add instructions to close any opened input stream (bug: too many open files) 2005/10/19 [BP] Fixed a bug due to the format change in assessments (real values instead of an integer value for exh/spe) 2005/10/18 [BP] Added support for INEX 2005 in evalj 2005/08/15 [GK] MAP, Q-measure and R-measure and bug fixes 2005 Aug [GK] overlap:on/off and recallXCG metric 2005/07/23 [BP] fixed bug #1240724 * Corrected the way to compute the number of assessed elements in PR * corrected the number of irrelevant elements at last rank * for inex_eval compatibility, rank is more important than rsv while ordering lists [BP/GK] Corrected a bug in PRng and updated the formula to the latest version of the metric [BP/GK] Corrected a bug with the "assessed document count" in PrecisionRecall [BP] Added some validity checks on collection management and a lot of warning messages helping users understanding what goes wrong [BP] Added support for XCG directly in EvalJ [BP] Added an "about" command that prints the current version of evalj (for bug reports) [BP] Fixed a bug with PRUM (hierarchical behaviour) [BP] Added new options to simulated runs generation (inverse and rank) [BP] Fix a bug (ranks not read) [BP] New options for PRUM (idealQuantisation & relevanceQuantisation) Version pre-3 ============= [BP] Bugfixes Version pre-2 ============= [BP] A lot of bugfixes [BP] Added options for the three metrics [BP] Simulated runs Version pre-1 ============= [BP] First public release