[santacruzgalaxy-list] Grackle proposal

[Matthew Turk]

Hi Nick,

Thanks for your detailed proposal.  I agree, there's a lot to think
about here, and from my own perspective I want to see grackle be as
easy to use as possible.

On Wed, Mar 19, 2014 at 2:03 PM, Nick Gnedin <ngnedin at gmail.com> wrote:
>
>
> For those who see this for the first time: there was an e-mail exchange
> about Grackle implementation that has almost reached an explosion point.
> Hence, this proposal to focus the discussion and to formulate the whole
> approach better.
>
>
>
> The root of our problems with Grackle is that we approached the whole
> question from the tail (finished library), not the head (the API). To
> fix it, I propose an API that, I believe, everyone can implement easily.

Having read this over, I think that this can all be approached as a
relatively simple wrapper around the existing Grackle API.  We should
perhaps view this as the "simple" version of the potentially more
complex API that enables greater control.

>
> API includes 1 header file, say "grackle.h", that exposes just 3 functions:
>
> grackle_init_run()
>      called once per simulation

Would the necessary header information and simulation-specific
parameters be stored in a global variable?

>
> grackle_init_step(aexp,...)
>      called once per each time-step, uses the value of the scale
>      factor and other parameters as needed
>
> grackle_solve_chemistry(dt,den,tem,...)
>      called for each resolution element one or more times per one
>      global time-step, uses gas properties to update internal energy.
>      Input gas properties may be required to have specific units and
>      be of specific data type. It should internally sense if it
>      runs within an OpenMP construct and support OpenMP
>      parallelization

Calling this for each resolution element would lead to lots of cache
misses for those codes where the resolution elements that are solved
simultaneously are large in number.  My concern is that the typical
step involves interpolation and application for equilibrium, and then
for non-equilibrium it involves interpolation, update, and then
copy-out.  I am not entirely satisfied with the idea of having heavy
function overhead in this case, either.  Allocation of temporary
values for those situations where there *are* a large number of cells
can also then lead to careful thread safety problems.

Because getting the OpenMP pragmas in place properly can be difficult,
I think that what *could* be done is have a simple wrapper that does
what you're prescribing, but still enable the full "evolve over dt"
mechanism which is presently the primary interface.

>
> The API can also define its own data types with natutal conversion from
> standard types, for example the following code should be valid:
>
> float f;
> double d;
> gr_float gr_f;
>
> gr_f = f;
> gr_f = d;

This is one of the reasons the types are defined at compile time, to
ensure that the fortran data types are of the correct size.  The lack
of interface checking in most fortran compilers results in problems
like this that can be difficult to diagnose.  I'm not sure this is an
improvement over the existing system?

>
> An analogous API should be provided for F77.
>
> Then in a code the API will be implemented as follows:
>
> Begin_code
>
>      grackle_init_run()
>
>      Loop_over_timesteps(aexp)
>
>          grackle_init_step(aexp,...)
>
> #pragma omp parallel loop
>          Loop_over_resolution_elements(elem)
>          {
>               gr_float dt = code_time_step*time_unit;
>               gr_float den = code_density(elem)*den_unit;
>               gr_float tem = code_temperature(elem)*tem_unit;
>               grackle_solve_chemistry(dt,den,tem,...)
>          }
>
>      End_loop
>
> End_code
>
> Let's converge on the API, and then the Grackle team will be able to
> write a couple of wrappers that will suit everyone.

I'm happy to provide support, but unfortunately my own time is quite
limited.  I think it's important to note that the Grackle code is open
source, has a mechanism for evaluating changes, and that there are a
number of invested parties.  Britton and I are both happy to provide
ideas and suggestions, but I would really like to ask that some
outside contributions be made.  The lion's share has been done by
Britton, who has responded incredibly quickly to many different
requests and suggestions, and I think having light wrappers on top is
a worthy addition that would be well-suited to outside contribution.

>
> Given the wrappers, if Grackle is installed as an external library and
> provides a proper Linus-style installer (that will handle all HDF5 and
> other dependencies), then it should be trivial to integrate it in any code.

I agree!  This sounds great.

Thanks again for your detailed proposal, Nick.

-Matt

>
> Nick
>
> To unsubscribe from this group and stop receiving emails from it, send an email to santacruzgalaxy-list+unsubscribe at ucsc.edu.
> _______________________________________________
> santacruzgalaxy-list mailing list
> santacruzgalaxy-list at ucsc.edu
> https://lists.ucsc.edu/mailman/listinfo/santacruzgalaxy-list

To unsubscribe from this group and stop receiving emails from it, send an email to santacruzgalaxy-list+unsubscribe at ucsc.edu.