[Developers] Support mesh refinement and multi-patch systems in CartGrid3D

[Erik Schnetter]

Thorn CactusBase/CartGrid3D initialises the coordinate arrays and  
registers the coordinate ranges with the flesh.  It currently assumes  
that it is called on all refinement levels and on all patches of a  
multi-patch evolution.  This is not so with certain AMR schemes (e.g.  
with Paramesh) or if a multi-patch simulation distributes different  
patches to different processors.

Since CartGrid3D does not actually know anything about mesh refinement  
or multi-patch setups, I found no way to make things work cleanly in  
all cases.  I therefore suggest to introduce two new parameters as  
work-around.  These parameters select in which mode the coordinate  
range registration is called: level mode, singlemap mode, or local mode:

BOOLEAN set_ranges_on_all_levels "Set the coordinate ranges on all  
levels"
BOOLEAN set_ranges_on_all_maps "Set the coordinate ranges on all maps"

The default behaviour is the current behaviour, namely initialisation  
in local mode.  Paramesh simulations would use  
"set_ranges_on_all_levels", while multi-patch simulations would use  
"set_ranges_on_all_maps".

-erik

-- 
Erik Schnetter <schnetter at cct.lsu.edu>   http://www.cct.lsu.edu/~eschnett/

My email is as private as my paper mail.  I therefore support encrypting
and signing email messages.  Get my PGP key from www.keyserver.net.


cvs diff: Diffing .
Index: param.ccl
===================================================================
RCS file: /cactusdevcvs/CactusBase/CartGrid3D/param.ccl,v
retrieving revision 1.21
diff -u -r1.21 param.ccl
--- param.ccl	11 May 2006 13:51:09 -0000	1.21
+++ param.ccl	29 Feb 2008 19:53:26 -0000
@@ -181,3 +181,13 @@
  {
   : :: "Logical"
  } "no"
+
+private:
+
+BOOLEAN set_ranges_on_all_levels "Set the coordinate ranges on all  
levels"
+{
+} "no"
+
+BOOLEAN set_ranges_on_all_maps "Set the coordinate ranges on all maps"
+{
+} "no"
Index: schedule.ccl
===================================================================
RCS file: /cactusdevcvs/CactusBase/CartGrid3D/schedule.ccl,v
retrieving revision 1.19
diff -u -r1.19 schedule.ccl
--- schedule.ccl	29 Feb 2008 19:30:38 -0000	1.19
+++ schedule.ccl	29 Feb 2008 19:53:26 -0000
@@ -25,10 +25,29 @@
    LANG:C
  } "Check coordinates for CartGrid3D"

-schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+if (set_ranges_on_all_levels)
  {
-  LANG:C
-} "Set up ranges for spatial 3D Cartesian coordinates"
+  schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+  {
+    LANG:C
+    OPTIONS: level
+  } "Set up ranges for spatial 3D Cartesian coordinates"
+}
+else if (set_ranges_on_all_maps)
+{
+  schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+  {
+    LANG:C
+    OPTIONS: singlemap
+  } "Set up ranges for spatial 3D Cartesian coordinates"
+}
+else
+{
+  schedule CartGrid3D_SetRanges at CCTK_BASEGRID before  
SpatialCoordinates
+  {
+    LANG:C
+  } "Set up ranges for spatial 3D Cartesian coordinates"
+}

  schedule CartGrid3D_SetCoordinates as SpatialCoordinates at  
CCTK_BASEGRID
  {
cvs diff: Diffing doc
cvs diff: Diffing doc/fig
cvs diff: Diffing par
cvs diff: Diffing src
Index: src/CartGrid3D.c
===================================================================
RCS file: /cactusdevcvs/CactusBase/CartGrid3D/src/CartGrid3D.c,v
retrieving revision 1.58
diff -u -r1.58 CartGrid3D.c
--- src/CartGrid3D.c	20 Feb 2008 00:29:16 -0000	1.58
+++ src/CartGrid3D.c	29 Feb 2008 19:53:26 -0000
@@ -71,24 +71,41 @@
  @@*/
  void CartGrid3D_SetRanges(CCTK_ARGUMENTS)
  {
-  int i, j, k, idx;
-  int is_coarsest_refinement_level, coord_handle, ierr;
+  int i;
+  int coord_handle, ierr;
    CCTK_REAL this_delta[3], origin[3], min1[3], max1[3];
    CCTK_REAL *coarse_delta[3];
    double lower[3], upper[3];
    int domainsym[6], cntstag[3], loweri[3], upperi[3], do_periodic[3];
    char coord_name[16];
-  DECLARE_CCTK_ARGUMENTS
-  DECLARE_CCTK_PARAMETERS
+  DECLARE_CCTK_ARGUMENTS;
+  DECLARE_CCTK_PARAMETERS;


-  /* ranges need to be set up only once, on the coarsest refinement  
level */
-  is_coarsest_refinement_level = cctk_levfac[0] == 1 &&
-                                 cctk_levfac[1] == 1 &&
-                                 cctk_levfac[2] == 1;
-  if (! is_coarsest_refinement_level)
+  if (set_ranges_on_all_levels)
    {
-    return;
+    /* Ranges must be set up only once, and this must happen on the
+       coarse grid.  However, the coarse grid itself may not actually
+       exist; in this case, use the coarsest existing grid.  We assume
+       that this is the first grid for which this routine is
+       called.  */
+    static int is_coarsest_refinement_level = 1;
+    if (! is_coarsest_refinement_level)
+    {
+      return;
+    }
+    is_coarsest_refinement_level = 0;
+  }
+  else
+  {
+    /* Ranges need to be set up only once (or once per map), on the
+       coarsest refinement level */
+    int const is_coarsest_refinement_level =
+      cctk_levfac[0] == 1 && cctk_levfac[1] == 1 && cctk_levfac[2] ==  
1;
+    if (! is_coarsest_refinement_level)
+    {
+      return;
+    }
    }

    coarse_delta[0] = coarse_dx;

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 194 bytes
Desc: This is a digitally signed message part
Url : http://www.cactuscode.org/pipermail/developers/attachments/20080229/0a802c63/attachment.bin