Thursday, March 15, 2007 02:37 PM
Can you shift the grouping work to the database?
Write some code to examine the xml files, then call a stored proc and pass it the results of the examination. The stored proc could handle the data retrieval, filtering, grouping, etc. and return one nice result set.
It's not a database-neutral way to go, but may make it possible to do all the work on sets (more efficient for the db) rather than iteratively outside the db.
You may also want to see if views can be written to combine all the various result sets into a single one from the database.