Wednesday, December 27, 2006 09:21 AM
I need to write a program that will process records in a batch basically it look for any records being added in a Database table e.g. Requests. If there are any records picks them up and calls a Web Service gets a response and inserts in some other table e.g Response, updates the column in Requests table so next time the same record is not picked up again. Now the issue is I may have over 8000 records that needs to be processed in a reasonable time and am trying to figure out a better solution so that all my records gets processed in a reasonable time, the Web Service may take 10-15 seconds to send the response back. I will have this program to run as a windows service all the time. Just trying to find out if it's ok to poll on the database to check for new records and in case of errors how do I make sure my program that runs as a service keep running. Or what else can be a better solution.
In my scenario my batch process run every night. From there I need to process all the records and send them to a Web Service that responds back and insert the response in a separate table. The same process can be run in a real time on demand i.e. from a UI a user can select the submit button and behind the secenes the same route is taken inserts a record in a request table and gets the response the whole cycle takes arround 10-15 seconds. My concern is in a batch mode when I have so many records that all needs to be processed within a reasonable time. So wondering what is my best way to do this. As the program will run 24x7 all the time as a Windows Service so wondering if I can make use of threads and have 10 threads at one point running now I don't know how to make use of Database Connection within threads and how often to release a connection and gets a new connection.
Any help in this is really appreciated. If using a program to poll on a table are there be any issues with the database connections and using threading if I have 8000+ records what are chances of failure etc.
Just looking for some ideas that anyone can share.
Re: Threads or JMS (Issue)
Wednesday, December 27, 2006 12:28 PM