serialization code is hanging
2 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   derwood_king
Posted On:   Thursday, July 20, 2006 12:11 PM

Hello, I am serializing some data and I don't know why the program is hanging. It writes out the file, but doesn't want to close the file and finish. I had to allocate a lot of memory at runtime or it wouldn't begin to work. When I run the same program to serialize a smaller dataset it doesn't hang. The code is as follows: //allocate an array report, to hold the exact number of reports received. wrf_report_3d_regular[] report = new wrf_report_3d_regular[i]; for(ii=0;ii <=i-1;ii++){ report[ii] = new wrf_report_3d_regular(t[ii],p[ii],pb[ii],qvapor[ii], qcloud[ii],qrain[ii],qice[ii],qsnow[ii],qgraup[ii]); //System.out.println(report[ii].print()); }    More>>

Hello,
I am serializing some data and I don't know why the program is hanging. It writes out the file, but doesn't want to close the file and finish. I had to allocate a lot of memory at runtime or it wouldn't begin to work. When I run the same program to serialize a smaller dataset it doesn't hang. The code is as follows:




			
//allocate an array report, to hold the exact number of reports received.
wrf_report_3d_regular[] report = new wrf_report_3d_regular[i];
for(ii=0;ii <=i-1;ii++){
report[ii] = new wrf_report_3d_regular(t[ii],p[ii],pb[ii],qvapor[ii],
qcloud[ii],qrain[ii],qice[ii],qsnow[ii],qgraup[ii]);
//System.out.println(report[ii].print());
}


//serialize
try{
try{
ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("serialized_WRF_3d_regular.dat"));
out.writeObject(report);
//out.flush();
out.close();
}catch(FileNotFoundException ex){ex.printStackTrace();}
}catch(IOException ioe){ioe.printStackTrace();}





To run the program I use:

			
java -Xmx1024M -classpath . JAVASPACECODE.readDemoWRF_3d_regular


512M was not enough.

Any ideas are welcome.

TIA,
Derwood    <<Less

Re: serialization code is hanging

Posted By:   Almagest_FUTT  
Posted On:   Friday, July 21, 2006 11:22 AM

  • You quite definitely should flush your stream

  • When using variables in loops, declare them outside the loop whenever possible. This will greatly improve garbage colletion. Like this:


    BAD:
    for(;;){
    OutputStream out = new OutputStream(); // ...
    }

    BETTER:
    OutputStream out;
    for(;;){
    out = new OutputStream(); // ...
    }


  • That's rather a lot of handles you're opening to the very same file, ain't it ? Try the follwoing approach:
    • Open but one file handle (outputstream)

    • Write a short method to get the serialized form (bytes) of an Object (write to a ByteArrayOutputStream, return its content)

    • Get the serialized bytes for each Object, and write them to the one FileOutputStream (which may adequately be buffered)

Re: serialization code is hanging

Posted By:   Jani_Kaarela  
Posted On:   Friday, July 21, 2006 08:51 AM


Not really paying attention here, since I'm at work, but maybe too much I/O chokes your system? Try buffering the output stream:


ObjectOutputStream out = new ObjectOutputStream(new BufferedOutputStream(new FileOutputStream("serialized_WRF_3d_regular.dat")));

This is definitely a wild guess, but it could work, and will improve performance anyway! Also, experiment with different buffer sizes to get ideal performance...

About | Sitemap | Contact