serialization code is hanging
2 posts in topic
Flat View  Flat View
Older Post First |   Reply to Topic
TOPIC ACTIONS:
 

Posted By:   derwood_king
Posted On:   Thursday, July 20, 2006 12:11 PM

Hello, I am serializing some data and I don't know why the program is hanging. It writes out the file, but doesn't want to close the file and finish. I had to allocate a lot of memory at runtime or it wouldn't begin to work. When I run the same program to serialize a smaller dataset it doesn't hang. The code is as follows: //allocate an array report, to hold the exact number of reports received. wrf_report_3d_regular[] report = new wrf_report_3d_regular[i]; for(ii=0;ii <=i-1;ii++){ report[ii] = new wrf_report_3d_regular(t[ii],p[ii],pb[ii],qvapor[ii], qcloud[ii],qrain[ii],qice[ii],qsnow[ii],qgraup[ii]); //System.out.println(report[ii].print()); }    More>>

Hello,
I am serializing some data and I don't know why the program is hanging. It writes out the file, but doesn't want to close the file and finish. I had to allocate a lot of memory at runtime or it wouldn't begin to work. When I run the same program to serialize a smaller dataset it doesn't hang. The code is as follows: 




			

			//allocate an array report, to hold the exact number of reports received.
			

			wrf_report_3d_regular[] report = new wrf_report_3d_regular[i];
			

			for(ii=0;ii
			<=i-1;ii++){
			

			report[ii] = new wrf_report_3d_regular(t[ii],p[ii],pb[ii],qvapor[ii],
			

			qcloud[ii],qrain[ii],qice[ii],qsnow[ii],qgraup[ii]);
			

			//System.out.println(report[ii].print());
			

			}
			

			

			

			//serialize
			

			try{
			

			try{
			

			ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("serialized_WRF_3d_regular.dat"));
			

			out.writeObject(report);
			

			//out.flush();
			

			out.close();
			

			}catch(FileNotFoundException ex){ex.printStackTrace();}
			

			}catch(IOException ioe){ioe.printStackTrace();}





To run the program I use: 

			

			java -Xmx1024M -classpath . JAVASPACECODE.readDemoWRF_3d_regular


512M was not enough.

Any ideas are welcome.

TIA,
Derwood    <<Less
Report | Quote This | Send private message | Reply | Print

Re: serialization code is hanging

Posted By:   Almagest_FUTT  
Posted On:   Friday, July 21, 2006 11:22 AM

  • You quite definitely should flush your stream

  • When using variables in loops, declare them outside the loop whenever possible. This will greatly improve garbage colletion. Like this:


    BAD:
    for(;;){
        OutputStream out = new OutputStream(); // ...
    }

    BETTER:
    OutputStream out;
    for(;;){
        out = new OutputStream(); // ...
    }


  • That's rather a lot of handles you're opening to the very same file, ain't it ? Try the follwoing approach:
    • Open but one file handle (outputstream)

    • Write a short method to get the serialized form (bytes) of an Object (write to a ByteArrayOutputStream, return its content)

    • Get the serialized bytes for each Object, and write them to the one FileOutputStream (which may adequately be buffered)

Report | Quote This | Send private message | Reply | Print

Re: serialization code is hanging

Posted By:   Jani_Kaarela  
Posted On:   Friday, July 21, 2006 08:51 AM


Not really paying attention here, since I'm at work, but maybe too much I/O chokes your system? Try buffering the output stream:


ObjectOutputStream out = new ObjectOutputStream(new BufferedOutputStream(new FileOutputStream("serialized_WRF_3d_regular.dat")));

This is definitely a wild guess, but it could work, and will improve performance anyway! Also, experiment with different buffer sizes to get ideal performance...

Report | Quote This | Send private message | Reply | Print
About | Sitemap | Contact