SharedFileInputStream in JavaMail 1.3
0 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   robert_white
Posted On:   Monday, January 27, 2003 03:24 PM

I see a new class in the JavaMail 1.3 jar file called com.sun.mail.util.SharedFileInputStream , but there's no documentation. I needed this to handle really large input files, so I wrote my own version and shoehorned it into the Sun POP3 reference implementation (using the JavaMail 1.2 source code available with the source code for J2EE 1.3.1 SDK). But, of course, I'd prefer to avoid using my own version of the POP3 implementation, if only Sun's version was sufficient. The problem is that the default behavior of Sun's POP3 implementation always reads an entire message into memory (and its default text/* handler does a really bad job with very large text part attachments -- inefficient to the point that it hangs up even a system with 1   More>>

I see a new class in the JavaMail 1.3 jar file called com.sun.mail.util.SharedFileInputStream , but there's no documentation. I needed this to handle really large input files, so I wrote my own version and shoehorned it into the Sun POP3 reference implementation (using the JavaMail 1.2 source code available with the source code for J2EE 1.3.1 SDK).


But, of course, I'd prefer to avoid using my own version of the POP3 implementation, if only Sun's version was sufficient. The problem is that the default behavior of Sun's POP3 implementation always reads an entire message into memory (and its default text/* handler does a really bad job with very large text part attachments -- inefficient to the point that it hangs up even a system with 1 Gigabyte of memory!). What I needed was a version that looks at the size of the message and writes all large messages to a temporary file rather than an in-memory buffer. There are problems with this, but they turned out to be solveable.


As for what "very large" means, I read the threshold from a Properties object passed to the constructor. Other properties specify the location and name of the temporary files.


In order to make this work, I needed to subclass com.sun.mail.pop3.Protocol and com.sun.mail.pop3.POP3Store (and in order to do this effectively, I needed to slightly modify these two classes making certain private things protected ).


The only reason to modify and subclass POP3Store is so that it will use my subclass of Protocol. My Protocol subclass only overrides the constructor (of course) and a method called multilineCommand() . My version of this method will return a SharedFileInputStream that completely mimics the default implementation's SharedByteArrayInputStream (i.e. it implements the SharedInputStream interface).


The tricky part to this implementation is that the default implementation opens a lot of input streams, probably more than it really needs to. There is little cost to this when these streams are ByteArrayStreams, however, when using FileInputStreams, one has to be careful to register all of these so that they can be closed. ByteArrayInputStreams do not have to be closed because when they are all discarded, the memory is freed and that's that. However, FileInputStreams allocate system resources, such as File Handles, that need to be closed and released. Otherwise, the temporary files cannot be deleted.


I solved this problem by having each SharedFileInputStream (and the original SharedFileOutputStream used to write the temporary file) register itself with a Singleton StreamRegistrar class. Then, at a point in my program when I know I have completed processing a given message, I call StreamRegistrar.deleteTempFiles(); to delete all temporary files (there can be more than one).


This approach seems to work, but as I said above, I'd much prefer to use code that had been tested by many. So I pose the question, does anybody have information regarding the SharedFileInputStream in JavaMail 1.3?

   <<Less
About | Sitemap | Contact