Strange score calculation behavior after boosting documents
0 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Alexey_Gorokhov
Posted On:   Thursday, August 24, 2006 01:15 PM

Hello, I ran into some very strange behavior by Lucene 1.9. Boost factor under 1.3 does not effect the result score! I wrote a simple test to isolate the issue: Writing test index Creating 4 documents with same KEY and boosts of default, 1.1, 1.2, and 1.3 public static void writeTestIndex() throws IOException { &nbsp&nbsp // opening index writer &nbsp&nbsp IndexWriter writer = null; &nbsp&nbsp writer = new IndexWriter("C:\a_temp", new StandardAnalyzer(), true); &nbsp&nbsp Document currentDocument = null; &nbsp&nbsp // creating and adding document with D   More>>

Hello,

I ran into some very strange behavior by Lucene 1.9. Boost factor under 1.3 does not effect the result score! I wrote a simple test to isolate the issue:



Writing test index

Creating 4 documents with same KEY and boosts of default, 1.1, 1.2, and 1.3



public static void writeTestIndex() throws IOException {


&nbsp&nbsp // opening index writer

&nbsp&nbsp IndexWriter writer = null;

&nbsp&nbsp writer = new IndexWriter("C:\a_temp", new StandardAnalyzer(), true);



&nbsp&nbsp Document currentDocument = null;



&nbsp&nbsp // creating and adding document with DEFAULT boost

&nbsp&nbsp currentDocument = new Document();

&nbsp&nbsp currentDocument.add(new Field("KEY", "AA", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.add(new Field("BOOST_FACTOR", "1", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp writer.addDocument(currentDocument);



&nbsp&nbsp // creating and adding document with 1.1 boost

&nbsp&nbsp currentDocument = new Document();

&nbsp&nbsp currentDocument.add(new Field("KEY", "AA", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.add( new Field("BOOST_FACTOR", "1.1", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.setBoost((float)1.1);

&nbsp&nbsp writer.addDocument(currentDocument);



&nbsp&nbsp // creating and adding document with 1.2 boost

&nbsp&nbsp currentDocument = new Document();

&nbsp&nbsp currentDocument.add(new Field("KEY", "AA", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.add( new Field("BOOST_FACTOR", "1.2", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.setBoost((float)1.2);

&nbsp&nbsp writer.addDocument(currentDocument);



&nbsp&nbsp // creating and adding document with 1.3 boost

&nbsp&nbsp currentDocument = new Document();

&nbsp&nbsp currentDocument.add(new Field("KEY", "AA", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.add(new Field("BOOST_FACTOR", "1.3", Field.Store.YES, Field.Index.UN_TOKENIZED));

&nbsp&nbsp currentDocument.setBoost((float)1.3);

&nbsp&nbsp writer.addDocument(currentDocument);



&nbsp&nbsp // optimizing and closing IndexWriter

&nbsp&nbsp writer.optimize();

&nbsp&nbsp writer.close();

}




Test Search

Searching for the KEY value, which is the same in all 4 documents



public static void testIndex() throws IOException {



&nbsp&nbsp // opening IndexSearcher

&nbsp&nbsp IndexSearcher searcher = null;

&nbsp&nbsp searcher = new IndexSearcher("C:\a_temp");



&nbsp&nbsp // searching for KEY

&nbsp&nbsp Hits hits = searcher.search(new TermQuery(new Term("KEY", "AA")));



&nbsp&nbsp // listing documents and their BOOST_FACTOR field

&nbsp&nbsp Document doc = null;

&nbsp&nbsp if (null != hits) {

&nbsp&nbsp logger.debug("Listing results: ");

&nbsp&nbsp for (int i = 0; i < hits.length(); i++) {

&nbsp&nbsp doc = hits.doc(i);

&nbsp&nbsp logger.debug("BOOST_FACTOR field: " + doc.get("BOOST_FACTOR") + " Score: " + hits.score(i));

&nbsp&nbsp }

&nbsp&nbsp }



&nbsp&nbsp // closing IndexSearcher

&nbsp&nbsp searcher.close();

}



Output



BOOST_FACTOR field: 1.3 Score: 0.9710705

BOOST_FACTOR field: 1 Score: 0.7768564

BOOST_FACTOR field: 1.1 Score: 0.7768564

BOOST_FACTOR field: 1.2 Score: 0.7768564



Boost of 1.1 and 1.2 did not effect score for the last 2 documents! Document with boost of 1.3 jumped to the top, but the rest were returned in the order they were added to the index.



What am I missing here? I thought document score would reflect all levels of boost, not just 1.3 and above? Please help.

   <<Less
About | Sitemap | Contact