Parallel Indexing with Threads
1 posts in topic
Flat View  Flat View
Older Post First |   Reply to Topic
TOPIC ACTIONS:
 

Posted By:   kai_tan
Posted On:   Wednesday, October 17, 2001 12:42 AM

I'm using threads to index HTML documents in parallel, but the oddest thing is that it actually takes more time using multi-threading than using a single-threaded model.


I've tried two strategies using multi-threading. One is just invoking multiple threads writing to a single writer. Another is creating multiple writers, writing to multiple indices, then using addIndexes to merge them. No luck with either.



Can anyone help?

Report | Quote This | Send private message | Reply | Print

Re: Parallel Indexing with Threads

Posted By:   Otis_Gospodnetic  
Posted On:   Thursday, October 18, 2001 10:13 PM

According to the reports of a few Lucene users and developers Lucene's indexing process is I/O bound, which means that no matter how many threads you have performing indexing in parallel you won't be able to improve indexing performance.


Multiple threads won't help your hard disc write faster.
Report | Quote This | Send private message | Reply | Print
About | Sitemap | Contact