I made a program that parses an email file and saves the messages into a database. The emails are in maildir format so each message is saved as a file. The program runs well if there are only a few files but quite slow when there are already more than a hundred emails. This would be a problem as the number of users increase and more and more emails come in. To make the program faster, I am thinking of using threads to parse the files but I'm not sure which implementation would be better. I have to methods in mind:

1) direct all emails to only one directory and use threads to parse the files, processing the ones which are not already being parsed by another thread;

2) distribute the emails to many directories and assign a thread to each directory.

Please give your comments on which one is a better implementation with regards to speed, stability and hardware/processor load. Or if you have another idea that might be better, please let me know.

Thanks.