From goodale at cct.lsu.edu Thu Feb 1 03:16:31 2007 From: goodale at cct.lsu.edu (Tom Goodale) Date: Thu, 1 Feb 2007 09:16:31 +0000 (GMT) Subject: [Developers] flush-ing stdout in cctk_vinfo In-Reply-To: References: <200701311821.48043.szilagyi@aei.mpg.de> Message-ID: On Wed, 31 Jan 2007, Jonathan Thornburg wrote: > Hi, > > I wrote >> In my opinion, the fact that Bela used --buffering line , yet (as I >> understand it) found additional output appearing when he did explicit >> flushes, constitutes a bug of some sort. I'm not quite sure whether >> it's caused by stdout and stderr not being synchronized, or a bug in >> stdio that isn't handling line buffering properly, but from the user's >> perspective, asking for --buffering line and not getting it constitues >> a bug. >> >> Given this, IMHO Bela's suggestion is a reasonable workaround, which >> would certainly avoid the bug in many, albeit not all, cases. That is, >> we would change the code so that if --buffering line is specified, >> CCTK_VInfo() does an explicit fflush(stdout) just before returning. > > Sorry, I now realise bela specified -bno . But I think this > makes the case even stronger -- he asked for *no* output buffering, > but yet there clearly was some buffering (since explicit flushing > gave additional output). There's definitely a bug somewhere, so putting the fflush in would work around it, at least for output using CCTK functions. However I think the problem is with the redirection to the CCTK_Proc... files happening after the buffering has been set - freopen is equivalent to fclose followed by fopen. I think we just need to move some code about to have code which will work even when non CCTK functions output. Cheers, Tom From schnetter at cct.lsu.edu Sun Feb 4 15:19:43 2007 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Sun, 4 Feb 2007 15:19:43 -0600 Subject: [Developers] flush-ing stdout in cctk_vinfo In-Reply-To: References: <200701311821.48043.szilagyi@aei.mpg.de> Message-ID: On Feb 1, 2007, at 03:16:31, Tom Goodale wrote: > On Wed, 31 Jan 2007, Jonathan Thornburg wrote: > >> Hi, >> >> I wrote >>> In my opinion, the fact that Bela used --buffering line , yet (as I >>> understand it) found additional output appearing when he did >>> explicit >>> flushes, constitutes a bug of some sort. I'm not quite sure whether >>> it's caused by stdout and stderr not being synchronized, or a bug in >>> stdio that isn't handling line buffering properly, but from the >>> user's >>> perspective, asking for --buffering line and not getting it >>> constitues >>> a bug. >>> >>> Given this, IMHO Bela's suggestion is a reasonable workaround, which >>> would certainly avoid the bug in many, albeit not all, cases. >>> That is, >>> we would change the code so that if --buffering line is specified, >>> CCTK_VInfo() does an explicit fflush(stdout) just before >>> returning. >> >> Sorry, I now realise bela specified -bno . But I think this >> makes the case even stronger -- he asked for *no* output buffering, >> but yet there clearly was some buffering (since explicit flushing >> gave additional output). > > There's definitely a bug somewhere, so putting the fflush in would > work > around it, at least for output using CCTK functions. However I > think the > problem is with the redirection to the CCTK_Proc... files happening > after > the buffering has been set - freopen is equivalent to fclose > followed by > fopen. I think we just need to move some code about to have code > which > will work even when non CCTK functions output. See the attached patch. Delay setting the output buffering mode until after I/O has been redirected. Okay? -erik -- Erik Schnetter My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from www.keyserver.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: redirect.diff Type: application/octet-stream Size: 1475 bytes Desc: not available Url : http://www.cactuscode.org/pipermail/developers/attachments/20070204/1417be40/attachment-0001.obj -------------- next part -------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://www.cactuscode.org/pipermail/developers/attachments/20070204/1417be40/attachment-0001.bin From goodale at cct.lsu.edu Sun Feb 4 16:49:47 2007 From: goodale at cct.lsu.edu (Tom Goodale) Date: Sun, 4 Feb 2007 22:49:47 +0000 (GMT) Subject: [Developers] flush-ing stdout in cctk_vinfo In-Reply-To: References: <200701311821.48043.szilagyi@aei.mpg.de> Message-ID: Looks good to me. Bela can you verify that this solves your problem ? Cheers, Tom On Sun, 4 Feb 2007, Erik Schnetter wrote: > On Feb 1, 2007, at 03:16:31, Tom Goodale wrote: > >> On Wed, 31 Jan 2007, Jonathan Thornburg wrote: >> >>> Hi, >>> >>> I wrote >>>> In my opinion, the fact that Bela used --buffering line , yet (as I >>>> understand it) found additional output appearing when he did explicit >>>> flushes, constitutes a bug of some sort. I'm not quite sure whether >>>> it's caused by stdout and stderr not being synchronized, or a bug in >>>> stdio that isn't handling line buffering properly, but from the user's >>>> perspective, asking for --buffering line and not getting it constitues >>>> a bug. >>>> >>>> Given this, IMHO Bela's suggestion is a reasonable workaround, which >>>> would certainly avoid the bug in many, albeit not all, cases. That is, >>>> we would change the code so that if --buffering line is specified, >>>> CCTK_VInfo() does an explicit fflush(stdout) just before returning. >>> >>> Sorry, I now realise bela specified -bno . But I think this >>> makes the case even stronger -- he asked for *no* output buffering, >>> but yet there clearly was some buffering (since explicit flushing >>> gave additional output). >> >> There's definitely a bug somewhere, so putting the fflush in would work >> around it, at least for output using CCTK functions. However I think the >> problem is with the redirection to the CCTK_Proc... files happening after >> the buffering has been set - freopen is equivalent to fclose followed by >> fopen. I think we just need to move some code about to have code which >> will work even when non CCTK functions output. > > See the attached patch. > > Delay setting the output buffering mode until after I/O has been redirected. > > Okay? > > -erik > > From tradke at aei.mpg.de Mon Feb 5 10:05:09 2007 From: tradke at aei.mpg.de (Thomas Radke) Date: Mon, 05 Feb 2007 17:05:09 +0100 Subject: [Developers] Patch: pass arrays and hashes by reference in interface parser Message-ID: <45C755B5.4010901@aei.mpg.de> This patch optimises the interface_parser.pl script to pass perl lists and hashes by reference rather than by value. It turned out that most of the runtime of the parameter parser was spent in flattening list/hash arguments during function calls. Using references conveniently solves this performance bottleneck; for the 'PublicThorns' configuration used in the nightly integration tests (with a list of some 120 thorns), the overall CST runtime went from 100s down to 8s [although the same "mistake" of passing lists/hashes is done in the schedule and parameter parser scripts, it's probably not worth the effort fixing it there as well]. -- Cheers, Thomas. --- StripMime Report -- processed MIME parts --- multipart/mixed text/plain (text body -- kept) text/x-patch --- From goodale at cct.lsu.edu Mon Feb 5 10:16:54 2007 From: goodale at cct.lsu.edu (Tom Goodale) Date: Mon, 5 Feb 2007 16:16:54 +0000 (GMT) Subject: [Developers] [Patches] Patch: pass arrays and hashes by reference in interface parser In-Reply-To: <45C755B5.4010901@aei.mpg.de> References: <45C755B5.4010901@aei.mpg.de> Message-ID: Looks good to me. The flattening is left over from the perl 4 days 8-( On Mon, 5 Feb 2007, Thomas Radke wrote: > This patch optimises the interface_parser.pl script to pass perl lists and > hashes by reference rather than by value. > > It turned out that most of the runtime of the parameter parser was spent in > flattening list/hash arguments during function calls. Using references > conveniently solves this performance bottleneck; for the 'PublicThorns' > configuration used in the nightly integration tests (with a list of some 120 > thorns), the overall CST runtime went from 100s down to 8s [although the same > "mistake" of passing lists/hashes is done in the schedule and parameter > parser scripts, it's probably not worth the effort fixing it there as well]. > > From szilagyi at aei.mpg.de Mon Feb 5 07:20:24 2007 From: szilagyi at aei.mpg.de (Bela Szilagyi) Date: Mon, 5 Feb 2007 14:20:24 +0100 Subject: [Developers] flush-ing stdout in cctk_vinfo In-Reply-To: References: <200701311821.48043.szilagyi@aei.mpg.de> Message-ID: <200702051420.24865.szilagyi@aei.mpg.de> The patch seems to give a consistent output behavior across various proccessors, at least for the tests I'm looking at. Thanks for taking care of this. Bela. On Sunday 04 February 2007 23:49, Tom Goodale wrote: > Looks good to me. Bela can you verify that this solves your problem ? > > Cheers, > > Tom > > On Sun, 4 Feb 2007, Erik Schnetter wrote: > > On Feb 1, 2007, at 03:16:31, Tom Goodale wrote: > >> On Wed, 31 Jan 2007, Jonathan Thornburg wrote: > >>> Hi, > >>> > >>> I wrote > >>> > >>>> In my opinion, the fact that Bela used --buffering line , yet (as I > >>>> understand it) found additional output appearing when he did explicit > >>>> flushes, constitutes a bug of some sort. I'm not quite sure whether > >>>> it's caused by stdout and stderr not being synchronized, or a bug in > >>>> stdio that isn't handling line buffering properly, but from the user's > >>>> perspective, asking for --buffering line and not getting it > >>>> constitues a bug. > >>>> > >>>> Given this, IMHO Bela's suggestion is a reasonable workaround, which > >>>> would certainly avoid the bug in many, albeit not all, cases. That > >>>> is, we would change the code so that if --buffering line is > >>>> specified, CCTK_VInfo() does an explicit fflush(stdout) just before > >>>> returning. > >>> > >>> Sorry, I now realise bela specified -bno . But I think this > >>> makes the case even stronger -- he asked for *no* output buffering, > >>> but yet there clearly was some buffering (since explicit flushing > >>> gave additional output). > >> > >> There's definitely a bug somewhere, so putting the fflush in would work > >> around it, at least for output using CCTK functions. However I think the > >> problem is with the redirection to the CCTK_Proc... files happening > >> after the buffering has been set - freopen is equivalent to fclose > >> followed by fopen. I think we just need to move some code about to have > >> code which will work even when non CCTK functions output. > > > > See the attached patch. > > > > Delay setting the output buffering mode until after I/O has been > > redirected. > > > > Okay? > > > > -erik > > _______________________________________________ > Developers mailing list > Developers at cactuscode.org > http://www.cactuscode.org/mailman/listinfo/developers -- Bela Szilagyi ---------------------------------------------------- Max-Planck-Institut f?r Gravitationsphysik Albert-Einstein-Institut Tel: +49 331 567 7189 Fax: +49 331 567 7252 ---------------------------------------------------- From schnetter at cct.lsu.edu Tue Feb 6 08:30:56 2007 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Tue, 6 Feb 2007 08:30:56 -0600 Subject: [Developers] Patch: pass arrays and hashes by reference in interface parser In-Reply-To: <45C755B5.4010901@aei.mpg.de> References: <45C755B5.4010901@aei.mpg.de> Message-ID: <385DEFBC-BF08-43BD-8424-E7CAF203BFF9@cct.lsu.edu> On Feb 5, 2007, at 10:05:09, Thomas Radke wrote: > This patch optimises the interface_parser.pl script to pass perl lists > and hashes by reference rather than by value. > > It turned out that most of the runtime of the parameter parser was > spent > in flattening list/hash arguments during function calls. Using > references conveniently solves this performance bottleneck; for the > 'PublicThorns' configuration used in the nightly integration tests > (with > a list of some 120 thorns), the overall CST runtime went from 100s > down > to 8s [although the same "mistake" of passing lists/hashes is done in > the schedule and parameter parser scripts, it's probably not worth the > effort fixing it there as well]. Thank you. Thank you. Thank you. I had looked at that as well, and I thought it was somehow the O(n^2) algorithm to check for interface consistency. Changing that to an O (n) algorithm would have been complicated, so I gave up. -erik -- Erik Schnetter My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from www.keyserver.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://www.cactuscode.org/pipermail/developers/attachments/20070206/6bc56d42/attachment.bin From schnetter at cct.lsu.edu Tue Feb 6 17:33:01 2007 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Tue, 6 Feb 2007 17:33:01 -0600 Subject: [Developers] Perl modules and global variables Message-ID: <995F3FBA-FBD7-4C61-8EDF-F2E1BB26AD12@cct.lsu.edu> I've tried to introduce a "use strict" to the file interface_parser.pl. One problem is that this file accesses the global variable $cctk_home, which is defined in CST before it requires interface_parser.pl. How do I propagate the variable $cctk_home from CST to interface_parser.pl? -erik -- Erik Schnetter My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from www.keyserver.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://www.cactuscode.org/pipermail/developers/attachments/20070206/24641520/attachment.bin From carsten.schneemann at aei.mpg.de Tue Feb 6 18:19:13 2007 From: carsten.schneemann at aei.mpg.de (Carsten Schneemann) Date: Wed, 07 Feb 2007 01:19:13 +0100 Subject: [Developers] Perl modules and global variables In-Reply-To: <995F3FBA-FBD7-4C61-8EDF-F2E1BB26AD12@cct.lsu.edu> References: <995F3FBA-FBD7-4C61-8EDF-F2E1BB26AD12@cct.lsu.edu> Message-ID: <45C91B01.1000608@aei.mpg.de> Hi! Erik Schnetter wrote: > I've tried to introduce a "use strict" to the file > interface_parser.pl. One problem is that this file accesses the > global variable $cctk_home, which is defined in CST before it requires > interface_parser.pl. > > How do I propagate the variable $cctk_home from CST to > interface_parser.pl? > My Perl knowledge is limited, but AFAIK unless you the required file contains a "package" (which is roughly similar to a Fortran module), which interface_parser.pl doesn't, the global variables from CST *are* in fact visible in / propagated to interface_parser.pl. The problem here is rather that with "use strict" you must either explicitly declare $cctk_home and friends as global in CST (via "our $cctk_home") *or* you have to use the fully qualified name $main::cctk_home when referencing them (instead of the "lazy" form $cctk_home). So one solution would be to declare all global variables in CST as global or replace all references to global variables in interface_parser.pl by their fully qualified names. Since both options are somewhat messy you could try to define a lexical variable with the same name as an alias in interface_parser.pl, e.g. my $cctk_home = $main::cctk_home and similarly for each global variable it uses. There might also be more sophisticated solutions... Bye, Carsten. From schnetter at cct.lsu.edu Tue Feb 6 21:34:01 2007 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Tue, 6 Feb 2007 21:34:01 -0600 Subject: [Developers] Perl modules and global variables In-Reply-To: <45C91B01.1000608@aei.mpg.de> References: <995F3FBA-FBD7-4C61-8EDF-F2E1BB26AD12@cct.lsu.edu> <45C91B01.1000608@aei.mpg.de> Message-ID: <413B9089-04CB-4A90-AE4E-F6CA18D6110B@cct.lsu.edu> On Feb 6, 2007, at 18:19:13, Carsten Schneemann wrote: > Hi! > > Erik Schnetter wrote: >> I've tried to introduce a "use strict" to the file >> interface_parser.pl. One problem is that this file accesses the >> global variable $cctk_home, which is defined in CST before it >> requires >> interface_parser.pl. >> >> How do I propagate the variable $cctk_home from CST to >> interface_parser.pl? >> > My Perl knowledge is limited, but AFAIK unless you the required file > contains a "package" (which is roughly similar to a Fortran module), > which interface_parser.pl doesn't, the global variables from CST *are* > in fact visible in / propagated to interface_parser.pl. The problem > here > is rather that with "use strict" you must either explicitly declare > $cctk_home and friends as global in CST (via "our $cctk_home") *or* > you > have to use the fully qualified name $main::cctk_home when referencing > them (instead of the "lazy" form $cctk_home). > > So one solution would be to declare all global variables in CST as > global or replace all references to global variables in > interface_parser.pl by their fully qualified names. Since both options > are somewhat messy you could try to define a lexical variable with the > same name as an alias in interface_parser.pl, e.g. > my $cctk_home = $main::cctk_home > and similarly for each global variable it uses. > > There might also be more sophisticated solutions... Declaring all global variables in CST would not have been that bad. But declaring them with "our" doesn't make them visible in required files. I then opted to use $main::cctk_home, which works fine. Thanks Larry, -erik -- Erik Schnetter My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from www.keyserver.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://www.cactuscode.org/pipermail/developers/attachments/20070206/bcb272d8/attachment.bin From schnetter at cct.lsu.edu Wed Feb 14 11:45:20 2007 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Wed, 14 Feb 2007 11:45:20 -0600 Subject: [Developers] Fwd: [CactusMaint] CactusWave/2083: Patch: Make IDScalarWaveC numerically stable References: <200702141733.l1EHXf6b027277@cactus.cct.lsu.edu> Message-ID: Begin forwarded message: > From: schnetter at cct.lsu.edu > Date: February 14, 2007 11:33:41 AM CST > To: goodale at cct.lsu.edu, gnats-admin at cactuscode.org, > cactusmaint at cactuscode.org > Subject: [CactusMaint] CactusWave/2083: Patch: Make IDScalarWaveC > numerically stable > Reply-To: bugs at cactuscode.org, CactusMaint > > >> Number: 2083 >> Notify-List: >> Category: CactusWave >> Synopsis: Patch: Make IDScalarWaveC numerically stable >> Confidential: no >> Severity: critical >> Priority: high >> Responsible: goodale >> State: open >> Class: sw-bug >> Submitter-Id: user >> Arrival-Date: Wed Feb 14 11:33:41 -0600 2007 >> Originator: Erik Schnetter >> Release: Cactus 4.0.b16 >> Organization: >> Environment: >> Description: > IDScalarWaveC (and the initial data thorns in the other languages) > use a floating point comparison R == 0 in the Gaussian initial data > to implement a special case for the origin, where a certain formula > is not defined. This is wrong since floating point comparisons > cannot be exact. > > The enclosed patch remedies this. Since the whole file looked > unclean and was a bit strangely formatted (in my taste), I cleaned > it up generally. >> How-To-Repeat: >> Fix: > Unknown >> Unformatted: -erik -- Erik Schnetter My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from www.keyserver.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://www.cactuscode.org/pipermail/developers/attachments/20070214/cbd7347e/attachment-0001.bin From schnetter at cct.lsu.edu Mon Feb 19 11:08:48 2007 From: schnetter at cct.lsu.edu (Erik Schnetter) Date: Mon, 19 Feb 2007 11:08:48 -0600 Subject: [Developers] Fwd: [CactusMaint] Cactus/2084: Patch: Circumvent argc length restrictions on AIX References: <200702191708.l1JH8Dwv012324@cactus.cct.lsu.edu> Message-ID: <9D4CA339-0C0F-4F4B-A895-87E62CD545C1@cct.lsu.edu> Begin forwarded message: > From: schnetter at cct.lsu.edu > Date: February 19, 2007 11:08:13 AM CST > To: goodale at cct.lsu.edu, gnats-admin at cactuscode.org, > cactusmaint at cactuscode.org > Cc: elena at cct.lsu.edu > Subject: [CactusMaint] Cactus/2084: Patch: Circumvent argc length > restrictions on AIX > Reply-To: bugs at cactuscode.org, CactusMaint > > >> Number: 2084 >> Notify-List: elena at cct.lsu.edu >> Category: Cactus >> Synopsis: Patch: Circumvent argc length restrictions on AIX >> Confidential: no >> Severity: critical >> Priority: high >> Responsible: goodale >> State: open >> Class: sw-bug >> Submitter-Id: user >> Arrival-Date: Mon Feb 19 11:08:13 -0600 2007 >> Originator: Erik Schnetter >> Release: Cactus 4.0.b16 >> Organization: >> Environment: >> Description: > AIX has restrictions on the length of the arguments that can be > passed to a subprocess. This makes it impossible to build a > configuration with a large thorn list, which we want to do for the > nightly build tests. The main problem is the call to ar when > creating libCactusBindings.a. > > The enclosed patch circumvents this restriction. Instead of > passing the list of object files directly to ar, this list is > written to a file "objectlist", and the library is created through > xargs. > > Writing the list of object files to a file is not straightforward. > Make has no facilities to write a variable to a file. It is, > obviously, not possible to write this list via a call to echo. > Instead, I create a make rule for each object file which appends > this object file's name to the objectlist file. > > In order to prevent this rule from causing the library to be > rebuilt every time, the objectlist file is created in a recursive > call to make. > > Thorn Formaline has the same problem when it builds its libraries. > For that it needs to call make.configuration recursively. > > > > The clean target used to remove "lib/*"; this list is also to > long. Instead, it now removes the whole directory "lib" and > creates the directory again afterwards. -erik -- Erik Schnetter My email is as private as my paper mail. I therefore support encrypting and signing email messages. Get my PGP key from www.keyserver.net. -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://www.cactuscode.org/pipermail/developers/attachments/20070219/9f31e52c/attachment.bin