[Users] BenchIO_HDF5 questions

[Erik Schnetter]

On Apr 18, 2008, at 13:54:30, Avi Purkayastha wrote:

> Thomas,
>
>>> 2) Is there an order of preference based on importance, for these
>>> three
>>> tests -- onefile, eachproc, and 8proc from the Cactus user
>>> community,
>>> or all of equal importance?
>>
>> Nowadays we almost always output in parallel on each processor  
>> because
>> it gives the best performance.
>
> I understand from the message that yours and the communities choice
> for 'eachproc' is due to lack of fast disks in combination with
> parallel f/s not delivering the performance that you are getting from
> 'eachproc'. But in the future if the other choices did deliver
> comparable performance in terms of throughput would that be
> preferable from the algorithmic and implementation standpoint?

Our preferred I/O method would currently be: Each node (if we combine  
OpenMP and MPI) or each processor (if we use only MPI) writes its data  
into a file.  Having a single file would indeed be simpler than having  
N files and would in principle be preferred.  We are using the HDF5  
library for I/O, so this also depends on the performance of the  
parallel HDF5 implementation.  We are currently only using the serial  
version of HDF5.

>> I don't have any results for these benchmarks. You could take a
>> look at
>> the disk I/O scaling tests we have performed on our AEI clusters when
>> they were delivered. They don't use Cactus but mimic Cactus behaviour
>> during a parallel checkpoint.
>
> I looked at www.cactuscode.org but did not find anything there. Maybe
> you can point me to where that data might be?


Thomas is probably referring to <http://numrel.aei.mpg.de/Research/Peyote/Benchmarks/ 
 >.  The AEI uses Cactus for much of its computational work; we have a  
collaboration between LSU and AEI in numerical relativity.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/users/attachments/20080418/fde94a93/attachment.bin