From dprideout at gmail.com  Fri Feb  8 09:15:22 2008
From: dprideout at gmail.com (David Rideout)
Date: Fri, 8 Feb 2008 10:15:22 -0500
Subject: [Developers] CCTK_POSTSTEP
Message-ID: <1ce81abb0802080715p18c49ca8m94a9077f81d71192@mail.gmail.com>

I notice that CCTK_POSTSTEP is scheduled at iteration 0, before the
first CCTK_PRESTEP of iteration 1.  Why is this?  There is no mention
of this in the documentation, that I could find.  Is this a bug?

Thanks,
David

From schnetter at cct.lsu.edu  Fri Feb  8 10:00:10 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 8 Feb 2008 10:00:10 -0600
Subject: [Developers] CCTK_POSTSTEP
In-Reply-To: <1ce81abb0802080715p18c49ca8m94a9077f81d71192@mail.gmail.com>
References: <1ce81abb0802080715p18c49ca8m94a9077f81d71192@mail.gmail.com>
Message-ID: <0A73538F-918F-49C5-9E37-A642107FB392@cct.lsu.edu>

On Feb 8, 2008, at 09:15:22, David Rideout wrote:

> I notice that CCTK_POSTSTEP is scheduled at iteration 0, before the
> first CCTK_PRESTEP of iteration 1.  Why is this?  There is no mention
> of this in the documentation, that I could find.  Is this a bug?


Iteration 0 is actually the initial data setup.  It is run after  
setting up the initial data, right before the analysis routines are  
run.  This is intended.  The reason is that certain quantities (e.g.  
the ADM metric) may need to be calculated from the evolved quantities  
(e.g. the BSSN variables), and this also needs to be done after the  
initial data have been set up.

Reading through the documentation, I agree that it is unclear in this  
respect.  The documentation doesn't clearly show the order in which  
the time bins are executed.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/ 
~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.




-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080208/2e3951a4/attachment.bin 

From jtao at cct.lsu.edu  Fri Feb  8 10:00:42 2008
From: jtao at cct.lsu.edu (Jian Tao)
Date: Fri, 08 Feb 2008 10:00:42 -0600
Subject: [Developers] CCTK_POSTSTEP
In-Reply-To: <1ce81abb0802080715p18c49ca8m94a9077f81d71192@mail.gmail.com>
References: <1ce81abb0802080715p18c49ca8m94a9077f81d71192@mail.gmail.com>
Message-ID: <47AC7CAA.5080308@cct.lsu.edu>

Hi David,

In my case, I schedule a routine to update a GF at CCTK_PRESTEP during
the evolution. I don't think I want that routine to be called at 
somewhere "pre" iteration 0 since the RHS is not initialized at
that point.

However, it is ok to have CCTK_POSTSTEP before the evolution starts.
If it is disable at iteration 0, I will have to schedule my
routines again in CCTK_POSTINITIAL.

Regards,
Jian


David Rideout wrote:
> I notice that CCTK_POSTSTEP is scheduled at iteration 0, before the
> first CCTK_PRESTEP of iteration 1.  Why is this?  There is no mention
> of this in the documentation, that I could find.  Is this a bug?
> 
> Thanks,
> David
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
> 


From d.rideout at imperial.ac.uk  Fri Feb  8 14:39:53 2008
From: d.rideout at imperial.ac.uk (Rideout, David P)
Date: Fri, 8 Feb 2008 20:39:53 -0000
Subject: [Developers] CCTK_POSTSTEP
References: <1ce81abb0802080715p18c49ca8m94a9077f81d71192@mail.gmail.com>
	<47AC7CAA.5080308@cct.lsu.edu>
Message-ID: <32DB9F7418A0734089B73B858EA20FEB88AF@icex6.ic.ac.uk>

Ok, it sounds like a documentation bug then.  Thanks for the explanations.

Cheers,
David


-----Original Message-----
From: developers-bounces at cactuscode.org on behalf of Jian Tao
Sent: Fri 08/02/2008 5:00 PM
To: developers at cactuscode.org
Subject: Re: [Developers] CCTK_POSTSTEP
 
Hi David,

In my case, I schedule a routine to update a GF at CCTK_PRESTEP during
the evolution. I don't think I want that routine to be called at 
somewhere "pre" iteration 0 since the RHS is not initialized at
that point.

However, it is ok to have CCTK_POSTSTEP before the evolution starts.
If it is disable at iteration 0, I will have to schedule my
routines again in CCTK_POSTINITIAL.

Regards,
Jian


David Rideout wrote:
> I notice that CCTK_POSTSTEP is scheduled at iteration 0, before the
> first CCTK_PRESTEP of iteration 1.  Why is this?  There is no mention
> of this in the documentation, that I could find.  Is this a bug?
> 
> Thanks,
> David
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
> 

_______________________________________________
Developers mailing list
Developers at cactuscode.org
http://www.cactuscode.org/mailman/listinfo/developers



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.cactuscode.org/pipermail/developers/attachments/20080208/4217355b/attachment-0001.html 

From schnetter at cct.lsu.edu  Sun Feb 10 15:08:21 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Sun, 10 Feb 2008 15:08:21 -0600
Subject: [Developers] Library-specific linker flags
Message-ID: <2936EFB2-776A-4949-BA60-EA16DBA722C8@cct.lsu.edu>

On Queen Bee and probably on related systems, it is necessary to link  
certain libraries statically, but others dynamically.  The  
configuration mechanism in Cactus can currently not handle this.

In Cactus, external libraries have three configuration variables  
associated with them, one defining the include path, one defining the  
library path, and one listing the library names.  It is not possible  
to add linker flags before or after the library names, since library  
names unconditionally have a "-l" prefix attached.  For example, the  
configuration settings

BLAS_DIR  = /usr/local/compilers/Intel/mkl-10.0/lib/em64t
BLAS_LIBS = mkl_em64t guide   pthread

are translated to

GENERAL_LIBRARIES = -L/usr/local/compilers/Intel/mkl-10.0/lib/em64t - 
lmkl_em64t -lguide -lpthread



I would like to add the flags -Bstatic and -Bdynamic before and after  
the library name, respectively.  I suggest to update the prefix- 
attaching mechanism so that it add prefixes to library names, but  
leaves flags alone.  This allows me to write

BLAS_DIR  = /usr/local/compilers/Intel/mkl-10.0/lib/em64t
BLAS_LIBS = -Bstatic mkl_em64t guide -Bdynamic   pthread

which would be translated to

GENERAL_LIBRARIES = -L/usr/local/compilers/Intel/mkl-10.0/lib/em64t - 
Bstatic -lmkl_em64t -lguide -Bdynamic -lpthread

which is the correct list of options for Queen Bee.



The patch below to lib/make/configure.in achieves the above:

+# How to link in the non-Cactus libraries:
+# Add -L and -l prefixes, but only for file names;
+# allow arbitrary options in between
  AC_SUBST(GENERAL_LIBRARIES)
-: ${GENERAL_LIBRARIES='$(LIBDIRS:%=-L%) $(LIBS:%=-l%)'}
+: ${GENERAL_LIBRARIES='$(patsubst -L-%,-%,$(LIBDIRS:%=-L%)) $ 
(patsubst -l-%,-%,$(LIBS:%=-l%))'}

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/ 
~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.




-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080210/34a174f4/attachment.bin 

From tradke at aei.mpg.de  Tue Feb 12 04:10:45 2008
From: tradke at aei.mpg.de (Thomas Radke)
Date: Tue, 12 Feb 2008 11:10:45 +0100
Subject: [Developers] Library-specific linker flags
In-Reply-To: <2936EFB2-776A-4949-BA60-EA16DBA722C8@cct.lsu.edu>
References: <2936EFB2-776A-4949-BA60-EA16DBA722C8@cct.lsu.edu>
Message-ID: <47B170A5.5080207@aei.mpg.de>

Erik Schnetter wrote:
> On Queen Bee and probably on related systems, it is necessary to link  
> certain libraries statically, but others dynamically.  The  
> configuration mechanism in Cactus can currently not handle this.
> 
> In Cactus, external libraries have three configuration variables  
> associated with them, one defining the include path, one defining the  
> library path, and one listing the library names.  It is not possible  to 
> add linker flags before or after the library names, since library  names 
> unconditionally have a "-l" prefix attached.  For example, the  
> configuration settings
> 
> BLAS_DIR  = /usr/local/compilers/Intel/mkl-10.0/lib/em64t
> BLAS_LIBS = mkl_em64t guide   pthread
> 
> are translated to
> 
> GENERAL_LIBRARIES = -L/usr/local/compilers/Intel/mkl-10.0/lib/em64t - 
> lmkl_em64t -lguide -lpthread
> 
> 
> 
> I would like to add the flags -Bstatic and -Bdynamic before and after  
> the library name, respectively.  I suggest to update the prefix- 
> attaching mechanism so that it add prefixes to library names, but  
> leaves flags alone.  This allows me to write
> 
> BLAS_DIR  = /usr/local/compilers/Intel/mkl-10.0/lib/em64t
> BLAS_LIBS = -Bstatic mkl_em64t guide -Bdynamic   pthread
> 
> which would be translated to
> 
> GENERAL_LIBRARIES = -L/usr/local/compilers/Intel/mkl-10.0/lib/em64t - 
> Bstatic -lmkl_em64t -lguide -Bdynamic -lpthread
> 
> which is the correct list of options for Queen Bee.
> 
> 
> 
> The patch below to lib/make/configure.in achieves the above:
> 
> +# How to link in the non-Cactus libraries:
> +# Add -L and -l prefixes, but only for file names;
> +# allow arbitrary options in between
>  AC_SUBST(GENERAL_LIBRARIES)
> -: ${GENERAL_LIBRARIES='$(LIBDIRS:%=-L%) $(LIBS:%=-l%)'}
> +: ${GENERAL_LIBRARIES='$(patsubst -L-%,-%,$(LIBDIRS:%=-L%)) $ (patsubst 
> -l-%,-%,$(LIBS:%=-l%))'}

Looks good to me. The feature should also be document somewhere in the 
users' guide chapter A2.

-- 
Cheers, Thomas.

From schnetter at cct.lsu.edu  Wed Feb 13 00:57:13 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Wed, 13 Feb 2008 00:57:13 -0600
Subject: [Developers] Library-specific linker flags
In-Reply-To: <47B170A5.5080207@aei.mpg.de>
References: <2936EFB2-776A-4949-BA60-EA16DBA722C8@cct.lsu.edu>
	<47B170A5.5080207@aei.mpg.de>
Message-ID: <A795A736-62FA-4C81-A838-2228C40BA209@cct.lsu.edu>

On Feb 12, 2008, at 04:10:45, Thomas Radke wrote:

> Erik Schnetter wrote:
>> On Queen Bee and probably on related systems, it is necessary to link
>> certain libraries statically, but others dynamically.  The
>> configuration mechanism in Cactus can currently not handle this.
>>
>> In Cactus, external libraries have three configuration variables
>> associated with them, one defining the include path, one defining the
>> library path, and one listing the library names.  It is not  
>> possible  to
>> add linker flags before or after the library names, since library   
>> names
>> unconditionally have a "-l" prefix attached.  For example, the
>> configuration settings
>>
>> BLAS_DIR  = /usr/local/compilers/Intel/mkl-10.0/lib/em64t
>> BLAS_LIBS = mkl_em64t guide   pthread
>>
>> are translated to
>>
>> GENERAL_LIBRARIES = -L/usr/local/compilers/Intel/mkl-10.0/lib/em64t -
>> lmkl_em64t -lguide -lpthread
>>
>>
>>
>> I would like to add the flags -Bstatic and -Bdynamic before and after
>> the library name, respectively.  I suggest to update the prefix-
>> attaching mechanism so that it add prefixes to library names, but
>> leaves flags alone.  This allows me to write
>>
>> BLAS_DIR  = /usr/local/compilers/Intel/mkl-10.0/lib/em64t
>> BLAS_LIBS = -Bstatic mkl_em64t guide -Bdynamic   pthread
>>
>> which would be translated to
>>
>> GENERAL_LIBRARIES = -L/usr/local/compilers/Intel/mkl-10.0/lib/em64t -
>> Bstatic -lmkl_em64t -lguide -Bdynamic -lpthread
>>
>> which is the correct list of options for Queen Bee.
>>
>>
>>
>> The patch below to lib/make/configure.in achieves the above:
>>
>> +# How to link in the non-Cactus libraries:
>> +# Add -L and -l prefixes, but only for file names;
>> +# allow arbitrary options in between
>> AC_SUBST(GENERAL_LIBRARIES)
>> -: ${GENERAL_LIBRARIES='$(LIBDIRS:%=-L%) $(LIBS:%=-l%)'}
>> +: ${GENERAL_LIBRARIES='$(patsubst -L-%,-%,$(LIBDIRS:%=-L%)) $  
>> (patsubst
>> -l-%,-%,$(LIBS:%=-l%))'}
>
> Looks good to me. The feature should also be document somewhere in the
> users' guide chapter A2.


I committed the change and updated the documentation.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080213/1f9411c2/attachment.bin 

From schnetter at cct.lsu.edu  Tue Feb 19 08:59:21 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Tue, 19 Feb 2008 08:59:21 -0600
Subject: [Developers] Making WaveToy use InitBase
Message-ID: <A51B1177-C413-4847-9A90-B4C6499A5F6F@cct.lsu.edu>

The IDScalarWave thorns currently initialise exactly two time levels.   
This is inconvenient for mesh refinement: If three time levels are  
needed, then Carpet has to perform an additional initialisation step  
and cannot rely on the analytic solution for initial data.   
Furthermore, if this is not done, the third time level remains  
uninitialised, leading to problems that are very difficult to detect,  
since people typically don't output past time levels.

I suggest to make IDScalarWave use the InitBase infrastructure to  
detect this problem.  In particular, if InitBase specifies that only a  
single time level is to be initialised, IDScalarWave should do so, and  
if all time levels should be initialised, IDScalarWave should either  
also do so, or abort with an error message during parameter checking.

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080219/60cc2b9a/attachment.bin 

From dprideout at gmail.com  Thu Feb 28 14:35:17 2008
From: dprideout at gmail.com (David Rideout)
Date: Thu, 28 Feb 2008 15:35:17 -0500
Subject: [Developers] memory error in src/main/Parameter.c's GetBaseName()
Message-ID: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>

Hi,

I have a code which runs fine in serial, but seg faults while freeing
grid arrays when run in parallel.  When I run it with memory checking
in ddt, it shows an error at the malloc() call in GetBaseName()
(src/main/Parameters.c line 2374):

Processes 0-1:
Memory error detected: "a previous write overwrote the reserved memory
after the end of a memory allocation"

This call to GetBaseName() arises from a call to CCTK_ParameterData()
during BASEGRID, to get the size of an array parameter.

Any ideas??

Thanks,
David

From schnetter at cct.lsu.edu  Thu Feb 28 14:38:40 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Thu, 28 Feb 2008 14:38:40 -0600
Subject: [Developers] memory error in src/main/Parameter.c's
	GetBaseName()
In-Reply-To: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
References: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
Message-ID: <C7E0F07E-8A92-4325-8108-707489F2B1B5@cct.lsu.edu>

On Feb 28, 2008, at 14:35:17, David Rideout wrote:

> Hi,
>
> I have a code which runs fine in serial, but seg faults while freeing
> grid arrays when run in parallel.  When I run it with memory checking
> in ddt, it shows an error at the malloc() call in GetBaseName()
> (src/main/Parameters.c line 2374):
>
> Processes 0-1:
> Memory error detected: "a previous write overwrote the reserved memory
> after the end of a memory allocation"
>
> This call to GetBaseName() arises from a call to CCTK_ParameterData()
> during BASEGRID, to get the size of an array parameter.
>
> Any ideas??


Are you using mvapich?

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080228/6f6c9569/attachment.bin 

From d.rideout at imperial.ac.uk  Thu Feb 28 15:20:11 2008
From: d.rideout at imperial.ac.uk (Rideout, David P)
Date: Thu, 28 Feb 2008 21:20:11 -0000
Subject: [Developers] memory error in src/main/Parameter.c'sGetBaseName()
References: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
	<C7E0F07E-8A92-4325-8108-707489F2B1B5@cct.lsu.edu>
Message-ID: <32DB9F7418A0734089B73B858EA20FEB88B1@icex6.ic.ac.uk>

No, hp-mpi:

drideout at bl125 ~/Cactus$ make causetsd_amd-configinfo
________________________________________________________________________
Displaying configuration information
# CONFIGURATION  : causetsd_amd
# CONFIG-DATE    : Thu Feb 28 15:22:55 2008 (GMT)
# CONFIG-HOST    : bl125
# CONFIG-STATUS  : 0
# CONFIG-OPTIONS :
CC=pathcc
CXX=pathCC
CXX_OPTIMISE_FLAGS=-O3 -OPT:Ofast -fno-math-errno -TENV:simd_zmask=OFF -TENV:simd_imask=OFF -TENV:simd_omask=OFF
C_OPTIMISE_FLAGS=-O3 -OPT:Ofast -fno-math-errno -TENV:simd_zmask=OFF -TENV:simd_imask=OFF -TENV:simd_omask=OFF
C_WARN_FLAGS=-Wall
DEBUG=yes
MPI=CUSTOM
MPI_INC_DIRS=/opt/hpmpi/include
MPI_LIBS=mpi
MPI_LIB_DIRS=/opt/hpmpi/lib/linux_amd64
OPTIMISE=no



-----Original Message-----
From: developers-bounces at cactuscode.org on behalf of Erik Schnetter
Sent: Thu 28/02/2008 9:38 PM
To: developers at cactuscode.org
Subject: Re: [Developers] memory error in src/main/Parameter.c'sGetBaseName()
 
On Feb 28, 2008, at 14:35:17, David Rideout wrote:

> Hi,
>
> I have a code which runs fine in serial, but seg faults while freeing
> grid arrays when run in parallel.  When I run it with memory checking
> in ddt, it shows an error at the malloc() call in GetBaseName()
> (src/main/Parameters.c line 2374):
>
> Processes 0-1:
> Memory error detected: "a previous write overwrote the reserved memory
> after the end of a memory allocation"
>
> This call to GetBaseName() arises from a call to CCTK_ParameterData()
> during BASEGRID, to get the size of an array parameter.
>
> Any ideas??


Are you using mvapich?

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.





-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/ms-tnef
Size: 3773 bytes
Desc: not available
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080228/de13a6c8/attachment-0001.bin 

From schnetter at cct.lsu.edu  Thu Feb 28 15:28:03 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Thu, 28 Feb 2008 15:28:03 -0600
Subject: [Developers] memory error in src/main/Parameter.c'sGetBaseName()
In-Reply-To: <32DB9F7418A0734089B73B858EA20FEB88B1@icex6.ic.ac.uk>
References: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
	<C7E0F07E-8A92-4325-8108-707489F2B1B5@cct.lsu.edu>
	<32DB9F7418A0734089B73B858EA20FEB88B1@icex6.ic.ac.uk>
Message-ID: <1DEB5034-CD6B-4B59-9072-E349FC6FFAD1@cct.lsu.edu>

It is likely that some part of your code accesses an invalid pointer  
or accesses an array with an invalid index.  This part can be  
anywhere.  I was asking about mvapich because I am just trying to  
track down a problem that may be caused by this MPI library.  I  
haven't come across the error that you report, so I'm afraid I can't  
help further.

-erik

On Feb 28, 2008, at 15:20:11, Rideout, David P wrote:

> No, hp-mpi:
>
> drideout at bl125 ~/Cactus$ make causetsd_amd-configinfo
> ________________________________________________________________________
> Displaying configuration information
> # CONFIGURATION  : causetsd_amd
> # CONFIG-DATE    : Thu Feb 28 15:22:55 2008 (GMT)
> # CONFIG-HOST    : bl125
> # CONFIG-STATUS  : 0
> # CONFIG-OPTIONS :
> CC=pathcc
> CXX=pathCC
> CXX_OPTIMISE_FLAGS=-O3 -OPT:Ofast -fno-math-errno - 
> TENV:simd_zmask=OFF -TENV:simd_imask=OFF -TENV:simd_omask=OFF
> C_OPTIMISE_FLAGS=-O3 -OPT:Ofast -fno-math-errno -TENV:simd_zmask=OFF  
> -TENV:simd_imask=OFF -TENV:simd_omask=OFF
> C_WARN_FLAGS=-Wall
> DEBUG=yes
> MPI=CUSTOM
> MPI_INC_DIRS=/opt/hpmpi/include
> MPI_LIBS=mpi
> MPI_LIB_DIRS=/opt/hpmpi/lib/linux_amd64
> OPTIMISE=no
>
>
>
> -----Original Message-----
> From: developers-bounces at cactuscode.org on behalf of Erik Schnetter
> Sent: Thu 28/02/2008 9:38 PM
> To: developers at cactuscode.org
> Subject: Re: [Developers] memory error in src/main/ 
> Parameter.c'sGetBaseName()
>
> On Feb 28, 2008, at 14:35:17, David Rideout wrote:
>
>> Hi,
>>
>> I have a code which runs fine in serial, but seg faults while freeing
>> grid arrays when run in parallel.  When I run it with memory checking
>> in ddt, it shows an error at the malloc() call in GetBaseName()
>> (src/main/Parameters.c line 2374):
>>
>> Processes 0-1:
>> Memory error detected: "a previous write overwrote the reserved  
>> memory
>> after the end of a memory allocation"
>>
>> This call to GetBaseName() arises from a call to CCTK_ParameterData()
>> during BASEGRID, to get the size of an array parameter.
>>
>> Any ideas??
>
>
> Are you using mvapich?
>
> -erik
>
> -- 
> Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/
>
> My email is as private as my paper mail.  I therefore support  
> encrypting
> and signing email messages.  Get my PGP key from www.keyserver.net.
>
>
>
>
>
> <winmail.dat>_______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers


-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080228/58206efd/attachment.bin 

From tradke at aei.mpg.de  Fri Feb 29 05:09:31 2008
From: tradke at aei.mpg.de (Thomas Radke)
Date: Fri, 29 Feb 2008 12:09:31 +0100
Subject: [Developers] memory error in src/main/Parameter.c's
	GetBaseName()
In-Reply-To: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
References: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
Message-ID: <47C7E7EB.3030201@aei.mpg.de>

David Rideout wrote:
> Hi,
> 
> I have a code which runs fine in serial, but seg faults while freeing
> grid arrays when run in parallel.  When I run it with memory checking
> in ddt, it shows an error at the malloc() call in GetBaseName()
> (src/main/Parameters.c line 2374):
> 
> Processes 0-1:
> Memory error detected: "a previous write overwrote the reserved memory
> after the end of a memory allocation"
> 
> This call to GetBaseName() arises from a call to CCTK_ParameterData()
> during BASEGRID, to get the size of an array parameter.
> 
> Any ideas??

I can't see anything wrong in the calling stack CCTK_ParameterData() -> 
ParameterFind() -> GetBaseName().

Like Erik, I would suspect a memory corruption earlier in the run. Do 
your grid arrays maybe have a size of 0 on some processors, and there is 
code not checking for that ?

-- 
Cheers, Thomas.

From dprideout at gmail.com  Fri Feb 29 10:19:28 2008
From: dprideout at gmail.com (David Rideout)
Date: Fri, 29 Feb 2008 11:19:28 -0500
Subject: [Developers] memory error in src/main/Parameter.c's
	GetBaseName()
In-Reply-To: <47C7E7EB.3030201@aei.mpg.de>
References: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
	<47C7E7EB.3030201@aei.mpg.de>
Message-ID: <1ce81abb0802290819k7e584680o5ac01a2ac4bba4fe@mail.gmail.com>

Thanks!  I think I have found the problem.  It is not that the grid
arrays are of size 0, but that I want distrib=constant grid
_functions_.  Cactus seems to just silently ignore my distrib=constant
request.  Shall we add a check for this to the interface.ccl parser?

Cheers,
David

On Fri, Feb 29, 2008 at 6:09 AM, Thomas Radke <tradke at aei.mpg.de> wrote:
>
> David Rideout wrote:
>  > Hi,
>  >
>  > I have a code which runs fine in serial, but seg faults while freeing
>  > grid arrays when run in parallel.  When I run it with memory checking
>  > in ddt, it shows an error at the malloc() call in GetBaseName()
>  > (src/main/Parameters.c line 2374):
>  >
>  > Processes 0-1:
>  > Memory error detected: "a previous write overwrote the reserved memory
>  > after the end of a memory allocation"
>  >
>  > This call to GetBaseName() arises from a call to CCTK_ParameterData()
>  > during BASEGRID, to get the size of an array parameter.
>  >
>  > Any ideas??
>
>  I can't see anything wrong in the calling stack CCTK_ParameterData() ->
>  ParameterFind() -> GetBaseName().
>
>  Like Erik, I would suspect a memory corruption earlier in the run. Do
>  your grid arrays maybe have a size of 0 on some processors, and there is
>  code not checking for that ?
>
>  --
>  Cheers, Thomas.
>
>
> _______________________________________________
>  Developers mailing list
>  Developers at cactuscode.org
>  http://www.cactuscode.org/mailman/listinfo/developers
>

From rideout at aei.mpg.de  Fri Feb 29 12:31:46 2008
From: rideout at aei.mpg.de (David Rideout)
Date: Fri, 29 Feb 2008 19:31:46 +0100 (CET)
Subject: [Developers] [Flesh-cvs] DEVELOPMENT CVS "Cactus/lib/sbin
	interface_parser.pl"
In-Reply-To: <20080229165119.537F781A6@asylum.cct.lsu.edu>
References: <20080229165119.537F781A6@asylum.cct.lsu.edu>
Message-ID: <Pine.LNX.4.64.0802291926290.28097@sshserv2.aei.mpg.de>

Thanks.  This just broke my code! (because I haven't taken out the 
distrib=constant yet)  :-)

On Fri, 29 Feb 2008, Erik Schnetter wrote:

> Update of /cactusdevcvs/Cactus/lib/sbin
> In directory asylum.cct.lsu.edu:/tmp/cvs-serv7770
>
> Modified Files:
> 	interface_parser.pl
> Log Message:
> Check the dimension and distribution type of grid scalars and the
> distribution type of grid functions
>
>
>
> _______________________________________________
> Flesh-cvs mailing list
> Flesh-cvs at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/flesh-cvs
>

From schnetter at cct.lsu.edu  Fri Feb 29 13:59:57 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 29 Feb 2008 13:59:57 -0600
Subject: [Developers] Support mesh refinement and multi-patch systems in
	CartGrid3D
Message-ID: <8A6268C8-451E-4775-B3A7-74E2CBE676A7@cct.lsu.edu>

Thorn CactusBase/CartGrid3D initialises the coordinate arrays and  
registers the coordinate ranges with the flesh.  It currently assumes  
that it is called on all refinement levels and on all patches of a  
multi-patch evolution.  This is not so with certain AMR schemes (e.g.  
with Paramesh) or if a multi-patch simulation distributes different  
patches to different processors.

Since CartGrid3D does not actually know anything about mesh refinement  
or multi-patch setups, I found no way to make things work cleanly in  
all cases.  I therefore suggest to introduce two new parameters as  
work-around.  These parameters select in which mode the coordinate  
range registration is called: level mode, singlemap mode, or local mode:

BOOLEAN set_ranges_on_all_levels "Set the coordinate ranges on all  
levels"
BOOLEAN set_ranges_on_all_maps "Set the coordinate ranges on all maps"

The default behaviour is the current behaviour, namely initialisation  
in local mode.  Paramesh simulations would use  
"set_ranges_on_all_levels", while multi-patch simulations would use  
"set_ranges_on_all_maps".

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.


cvs diff: Diffing .
Index: param.ccl
===================================================================
RCS file: /cactusdevcvs/CactusBase/CartGrid3D/param.ccl,v
retrieving revision 1.21
diff -u -r1.21 param.ccl
--- param.ccl	11 May 2006 13:51:09 -0000	1.21
+++ param.ccl	29 Feb 2008 19:53:26 -0000
@@ -181,3 +181,13 @@
  {
   : :: "Logical"
  } "no"
+
+private:
+
+BOOLEAN set_ranges_on_all_levels "Set the coordinate ranges on all  
levels"
+{
+} "no"
+
+BOOLEAN set_ranges_on_all_maps "Set the coordinate ranges on all maps"
+{
+} "no"
Index: schedule.ccl
===================================================================
RCS file: /cactusdevcvs/CactusBase/CartGrid3D/schedule.ccl,v
retrieving revision 1.19
diff -u -r1.19 schedule.ccl
--- schedule.ccl	29 Feb 2008 19:30:38 -0000	1.19
+++ schedule.ccl	29 Feb 2008 19:53:26 -0000
@@ -25,10 +25,29 @@
    LANG:C
  } "Check coordinates for CartGrid3D"

-schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+if (set_ranges_on_all_levels)
  {
-  LANG:C
-} "Set up ranges for spatial 3D Cartesian coordinates"
+  schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+  {
+    LANG:C
+    OPTIONS: level
+  } "Set up ranges for spatial 3D Cartesian coordinates"
+}
+else if (set_ranges_on_all_maps)
+{
+  schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+  {
+    LANG:C
+    OPTIONS: singlemap
+  } "Set up ranges for spatial 3D Cartesian coordinates"
+}
+else
+{
+  schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+  {
+    LANG:C
+  } "Set up ranges for spatial 3D Cartesian coordinates"
+}

  schedule CartGrid3D_SetCoordinates as SpatialCoordinates at  
CCTK_BASEGRID
  {
cvs diff: Diffing doc
cvs diff: Diffing doc/fig
cvs diff: Diffing par
cvs diff: Diffing src
Index: src/CartGrid3D.c
===================================================================
RCS file: /cactusdevcvs/CactusBase/CartGrid3D/src/CartGrid3D.c,v
retrieving revision 1.58
diff -u -r1.58 CartGrid3D.c
--- src/CartGrid3D.c	20 Feb 2008 00:29:16 -0000	1.58
+++ src/CartGrid3D.c	29 Feb 2008 19:53:26 -0000
@@ -71,24 +71,41 @@
  @@*/
  void CartGrid3D_SetRanges(CCTK_ARGUMENTS)
  {
-  int i, j, k, idx;
-  int is_coarsest_refinement_level, coord_handle, ierr;
+  int i;
+  int coord_handle, ierr;
    CCTK_REAL this_delta[3], origin[3], min1[3], max1[3];
    CCTK_REAL *coarse_delta[3];
    double lower[3], upper[3];
    int domainsym[6], cntstag[3], loweri[3], upperi[3], do_periodic[3];
    char coord_name[16];
-  DECLARE_CCTK_ARGUMENTS
-  DECLARE_CCTK_PARAMETERS
+  DECLARE_CCTK_ARGUMENTS;
+  DECLARE_CCTK_PARAMETERS;


-  /* ranges need to be set up only once, on the coarsest refinement  
level */
-  is_coarsest_refinement_level = cctk_levfac[0] == 1 &&
-                                 cctk_levfac[1] == 1 &&
-                                 cctk_levfac[2] == 1;
-  if (! is_coarsest_refinement_level)
+  if (set_ranges_on_all_levels)
    {
-    return;
+    /* Ranges must be set up only once, and this must happen on the
+       coarse grid.  However, the coarse grid itself may not actually
+       exist; in this case, use the coarsest existing grid.  We assume
+       that this is the first grid for which this routine is
+       called.  */
+    static int is_coarsest_refinement_level = 1;
+    if (! is_coarsest_refinement_level)
+    {
+      return;
+    }
+    is_coarsest_refinement_level = 0;
+  }
+  else
+  {
+    /* Ranges need to be set up only once (or once per map), on the
+       coarsest refinement level */
+    int const is_coarsest_refinement_level =
+      cctk_levfac[0] == 1 && cctk_levfac[1] == 1 && cctk_levfac[2] ==  
1;
+    if (! is_coarsest_refinement_level)
+    {
+      return;
+    }
    }

    coarse_delta[0] = coarse_dx;

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080229/0a802c63/attachment.bin 

From schnetter at cct.lsu.edu  Fri Feb 29 16:57:24 2008
From: schnetter at cct.lsu.edu (Erik Schnetter)
Date: Fri, 29 Feb 2008 16:57:24 -0600
Subject: [Developers] memory error in src/main/Parameter.c's
	GetBaseName()
In-Reply-To: <1ce81abb0802290819k7e584680o5ac01a2ac4bba4fe@mail.gmail.com>
References: <1ce81abb0802281235u52fdee94hc71b6e2bad01be73@mail.gmail.com>
	<47C7E7EB.3030201@aei.mpg.de>
	<1ce81abb0802290819k7e584680o5ac01a2ac4bba4fe@mail.gmail.com>
Message-ID: <3A4C0661-AB74-4A4B-960F-346D848280D0@cct.lsu.edu>

I added such a check.

-erik

On Feb 29, 2008, at 10:19:28, David Rideout wrote:

> Thanks!  I think I have found the problem.  It is not that the grid
> arrays are of size 0, but that I want distrib=constant grid
> _functions_.  Cactus seems to just silently ignore my distrib=constant
> request.  Shall we add a check for this to the interface.ccl parser?
>
> Cheers,
> David
>
> On Fri, Feb 29, 2008 at 6:09 AM, Thomas Radke <tradke at aei.mpg.de>  
> wrote:
>>
>> David Rideout wrote:
>>> Hi,
>>>
>>> I have a code which runs fine in serial, but seg faults while  
>>> freeing
>>> grid arrays when run in parallel.  When I run it with memory  
>>> checking
>>> in ddt, it shows an error at the malloc() call in GetBaseName()
>>> (src/main/Parameters.c line 2374):
>>>
>>> Processes 0-1:
>>> Memory error detected: "a previous write overwrote the reserved  
>>> memory
>>> after the end of a memory allocation"
>>>
>>> This call to GetBaseName() arises from a call to  
>>> CCTK_ParameterData()
>>> during BASEGRID, to get the size of an array parameter.
>>>
>>> Any ideas??
>>
>> I can't see anything wrong in the calling stack  
>> CCTK_ParameterData() ->
>> ParameterFind() -> GetBaseName().
>>
>> Like Erik, I would suspect a memory corruption earlier in the run. Do
>> your grid arrays maybe have a size of 0 on some processors, and  
>> there is
>> code not checking for that ?
>>
>> --
>> Cheers, Thomas.
>>
>>
>> _______________________________________________
>> Developers mailing list
>> Developers at cactuscode.org
>> http://www.cactuscode.org/mailman/listinfo/developers
>>
> _______________________________________________
> Developers mailing list
> Developers at cactuscode.org
> http://www.cactuscode.org/mailman/listinfo/developers
>


-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080229/12d5f92a/attachment.bin