[santacruzgalaxy-list] Grackle proposal
Nick Gnedin
ngnedin at gmail.com
Wed Mar 19 14:44:25 PDT 2014
Cooling is done element by element, so there should never be any need to
send in anything but a single element - for all realistic situations the
overhead of a function call is negligible compared to the time it takes
to query a table and evolve internal energy.
If you need to send in the whole array, you are not pre-computing all
the factors properly.
Let's arrange for a place somewhere where we can all edit API
declarations and leave comments. I can make a public repository on
BitBucket, but there may be a better option.
On 03/19/2014 04:35 PM, Matthew Turk wrote:
> On Wed, Mar 19, 2014 at 5:26 PM, Romain Teyssier
> <romain.teyssier at gmail.com> wrote:
>> Hi Matt,
>>
>>
>> For example:
>>
>> int solve_chemistry(chemistry_data &my_chemistry, ???
>> code_units &my_units, OK set to 1
>> gr_float a_value, gr_float dt_value, why a ?
>> gr_int grid_rank, gr_int *grid_dimension, ????
>> gr_int *grid_start, gr_int *grid_end, ????
>> gr_float *density, gr_float *internal_energy, OK
>> gr_float *x_velocity, gr_float *y_velocity, gr_float *z_velocity, ???
>> gr_float *HI_density, gr_float *HII_density, gr_float *HM_density, too much
>> gr_float *HeI_density, gr_float *HeII_density, gr_float *HeIII_density, too much
>> gr_float *H2I_density, gr_float *H2II_density, too much
>> gr_float *DI_density, gr_float *DII_density, gr_float *HDI_density, too much
>> gr_float *e_density, gr_float *metal_density); OK
>>
>> What are the *grid related variables ?
>
> Good question! These, like the code units variables, are designed to
> minimize the overhead of any simulation that wants to put its code
> into it. For instance, if one had a patch/block-based code (like
> Enzo, FLASH, Nyx, etc) or an Octree code where the 2x2x2 zones were
> included in (2+2*NGZ, 2+2*NGZ, 2+2*NGZ) blocks and one didn't want to
> spend time solving the ghost zones, they can be masked out. These are
> the variables:
>
> grid_start => array of size (grid_rank) indicating the indices to
> start at in each rank of dimensionality
> grid_end => array of size (grid_rank) indicating the indices to *end*
> at in each rank of dimensionality
> grid_dimension => size of the block of data in each dimension
> grid_rank => dimensionality; if you're supplying a pencil beam, this would be 1.
>
> While these typically are better for block or patch based solvers, I
> think that the primary goal -- *minimizing* the overhead to using a
> new code -- is met with them.
>
> As I mentioned in our previous email, the velocities are supplied for
> the Sobolev approximation. You can see the full discussion on the
> public Grackle mailing list here:
>
> https://groups.google.com/forum/#!searchin/grackle-cooling-users/velocity/grackle-cooling-users/Dr77TM2te9g/225RNzoZADEJ
>
>>
>> If chemistry_data is an external variable, this is a pain because this variable need to be declared in the main code.
>> Same thing for code_units.
>
> The alternative is that it be a global defined in a different
> namespace, loading in by the dlloader at runtime. Not sure that's
> substantially different.
>
> Code units, if you want to convert to CGS, will just be 1.0 for all of
> them, which can be constant.
>
>> This is all too ENZO specifics.
>
> I'm not sure I believe that argument. Perhaps the names of the
> variables share too much with Enzo naming schemes, but I think it's
> rather intuitive to think of the 3D dataset as a base starting point
> and then reducing overhead by supplying 1-dimensional arrays.
>
> -Matt
>
>>
>> Romain
>>
>>
>>
>> On 19 Mar 2014, at 22:08, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> On Wed, Mar 19, 2014 at 4:58 PM, Romain Teyssier
>>> <romain.teyssier at gmail.com> wrote:
>>>>
>>>> I totally agree with the 3 routines that need to be used.
>>>> Nothing else should be required on the code's side, except passing the required variables.
>>>>
>>>>
>>>>> grackle_init_run()
>>>>> called once per simulation
>>>>>
>>>>
>>>> The parameters that should be passed here are
>>>> - some cosmological model parameters
>>>> - UV background model (spectrum, reionization redshift...)
>>>> - starting redshift to compute the initial temperature and ionisation state
>>>>
>>>>> grackle_init_step(aexp,...)
>>>>> called once per each time-step, uses the value of the scale
>>>>> factor and other parameters as needed
>>>>
>>>> In principle at this stage, only aexp is required for cosmo runs.
>>>> For non cosmo runs, the cooling tables do not need to be modified.
>>>>
>>>>>
>>>>> grackle_solve_chemistry(dt,den,tem,...)
>>>>> called for each resolution element one or more times per one
>>>>> global time-step, uses gas properties to update internal energy.
>>>>> Input gas properties may be required to have specific units and
>>>>> be of specific data type. It should internally sense if it
>>>>> runs within an OpenMP construct and support OpenMP
>>>>> parallelization.
>>>>>
>>>>
>>>> I would rather give the possibility of passing an array of cell.
>>>> Users could also set the array size to 1 to deal with cells one by one.
>>>> The required information could be
>>>> nH, T or Tovermu or specific energy, metallicity in some units, dt_hydro, ncell
>>>> On output, Delta T or Delta Tovermu or Delta specific energy
>>>> I suggest we use fixed units (cgs or mks) for all input and output variables.
>>>>
>>>> These 3 routines are the only one that the user should care about.
>>>>
>>>
>>> This is awfully similar to the existing API:
>>>
>>> https://bitbucket.org/brittonsmith/grackle/src/642cd133535a2dd3c57626b768c7c8c107096ac7/src/clib/grackle.h?at=default
>>>
>>> The only necessary functions, according to
>>> http://grackle.readthedocs.org/en/latest/Integration.html , are:
>>>
>>> initialize_chemistry_data
>>> solve_chemistry or solve_chemistry for tabular data
>>>
>>> Looking at the example (non-equilibrium) executable:
>>>
>>> https://bitbucket.org/brittonsmith/grackle/src/642cd133535a2dd3c57626b768c7c8c107096ac7/src/example/example.C?at=default
>>>
>>> those are the only *functional* routines. Everything else is to
>>> provide additional information, *not* to actually do any computation.
>>> I think that we have actually met these standards. All of the units
>>> are provided so as the *avoid* any boilerplate -- but if you want to
>>> supply in CGS, you can set the input unit conversions to 1.0.
>>>
>>> To compute the equilibrium tables:
>>>
>>> https://bitbucket.org/brittonsmith/grackle/src/642cd133535a2dd3c57626b768c7c8c107096ac7/src/example/table_example.C?at=default
>>>
>>> it's even simpler. The only *active* routine is called on line 120,
>>> "solve_chemistry". I am genuinely being earnest when I ask, from the
>>> .h file linked above, which arguments to the functions would you like
>>> to see removed? I've included the full function signatures below. If
>>> you want to get *out* various things, you can utilize the other
>>> functions -- like the cooling time function and so on.
>>>
>>> -Matt
>>>
>>> int initialize_chemistry_data(chemistry_data &my_chemistry,
>>> code_units &my_units, gr_float a_value);
>>>
>>> int solve_chemistry(chemistry_data &my_chemistry,
>>> code_units &my_units,
>>> gr_float a_value, gr_float dt_value,
>>> gr_int grid_rank, gr_int *grid_dimension,
>>> gr_int *grid_start, gr_int *grid_end,
>>> gr_float *density, gr_float *internal_energy,
>>> gr_float *x_velocity, gr_float *y_velocity, gr_float *z_velocity,
>>> gr_float *HI_density, gr_float *HII_density, gr_float *HM_density,
>>> gr_float *HeI_density, gr_float *HeII_density, gr_float *HeIII_density,
>>> gr_float *H2I_density, gr_float *H2II_density,
>>> gr_float *DI_density, gr_float *DII_density, gr_float *HDI_density,
>>> gr_float *e_density, gr_float *metal_density);
>>>
>>> int solve_chemistry(chemistry_data &my_chemistry,
>>> code_units &my_units,
>>> gr_float a_value, gr_float dt_value,
>>> gr_int grid_rank, gr_int *grid_dimension,
>>> gr_int *grid_start, gr_int *grid_end,
>>> gr_float *density, gr_float *internal_energy,
>>> gr_float *x_velocity, gr_float *y_velocity,
>>> gr_float *z_velocity,
>>> gr_float *metal_density);
>>>
>>>> Cheers,
>>>> Romain
>>>>
>>>>
>>>>
>>>>> The API can also define its own data types with natutal conversion from
>>>>> standard types, for example the following code should be valid:
>>>>>
>>>>> float f;
>>>>> double d;
>>>>> gr_float gr_f;
>>>>>
>>>>> gr_f = f;
>>>>> gr_f = d;
>>>>>
>>>>> An analogous API should be provided for F77.
>>>>>
>>>>> Then in a code the API will be implemented as follows:
>>>>>
>>>>> Begin_code
>>>>>
>>>>> grackle_init_run()
>>>>>
>>>>> Loop_over_timesteps(aexp)
>>>>>
>>>>> grackle_init_step(aexp,...)
>>>>>
>>>>> #pragma omp parallel loop
>>>>> Loop_over_resolution_elements(elem)
>>>>> {
>>>>> gr_float dt = code_time_step*time_unit;
>>>>> gr_float den = code_density(elem)*den_unit;
>>>>> gr_float tem = code_temperature(elem)*tem_unit;
>>>>> grackle_solve_chemistry(dt,den,tem,...)
>>>>> }
>>>>>
>>>>> End_loop
>>>>>
>>>>> End_code
>>>>>
>>>>> Let's converge on the API, and then the Grackle team will be able to
>>>>> write a couple of wrappers that will suit everyone.
>>>>>
>>>>> Given the wrappers, if Grackle is installed as an external library and
>>>>> provides a proper Linus-style installer (that will handle all HDF5 and
>>>>> other dependencies), then it should be trivial to integrate it in any code.
>>>>>
>>>>> Nick
>>>>>
>>>>> To unsubscribe from this group and stop receiving emails from it, send an email to santacruzgalaxy-list+unsubscribe at ucsc.edu.
>>>>> _______________________________________________
>>>>> santacruzgalaxy-list mailing list
>>>>> santacruzgalaxy-list at ucsc.edu
>>>>> https://lists.ucsc.edu/mailman/listinfo/santacruzgalaxy-list
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "Grackle Cooling Library Users" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to grackle-cooling-users+unsubscribe at googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>
To unsubscribe from this group and stop receiving emails from it, send an email to santacruzgalaxy-list+unsubscribe at ucsc.edu.
More information about the santacruzgalaxy-list
mailing list