Better support for tuple and complex variables in fits, data and inits

[jgabry]

Submission Checklist

• Run unit tests
• Declare copyright holder and agree to license (see below)

Summary

AI was used in writing the code for this PR and the summary below. I have reviewed all code.

---

Supersedes #1174 and reuses its test model. Closes #925.

CmdStan names a tuple element's columns with a colon (b_tuple:2.1.1)
and a complex number's parts with .real and .imag. Our name repair
and variable_dims() assumed every column was name.i.j, so a fit with
such variables warned about NA sizes, its metadata was wrong and a
variable filter could not select a tuple. Two more layers had the same
gap: the model methods rebuilt constrained values through
utils::relist() from the compiled model's own dotted names, which
mislabel arrays of tuples, and write_stan_json() could not write a
tuple at all, so tuple data and inits were unusable.

What changes:

• Column names are repaired piece by piece and keep the colons:
  b_tuple:2[1,1], arr_pair[1]:2, z[real], zv[1,imag]. This is what
  posterior already does with them. A variable is the name before the
  first [ or : everywhere names are filtered or sized, so
  fit$draws("b_tuple") selects all of a tuple's columns.
  stan_variable_sizes gives a complex variable a trailing 2 and a
  tuple the size of its outer array.
• $constrain_variables() rebuilds its result from the model's
  constrained_param_names(): a complex value comes back as an R
  complex number or array, a tuple as an unnamed list of its elements.
  $unconstrain_variables() and $unconstrain_draws() accept the same
  shapes. $variable_skeleton() is removed along with the C++ that fed
  it; utils::relist() is no longer needed. The one shape change for
  existing models is that a 1-d variable now comes back as a 1-d array
  instead of a bare vector.
• write_stan_json() gains a variables argument taking the
  declarations from $variables() or variables_stan_file(). With them
  an unnamed list is written as a tuple; without them it is still
  converted to an array as before. Complex values are written as
  [re, im] pairs. The fitting methods pass the model's declarations
  for data and inits, which also moves the per-variable data
  conversions from process_data() into the writer, and init = fit
  carries tuple parameters over.

One caveat found while writing the test model: CmdStan 2.40's JSON
reader misreads a complex value inside an array of tuples (a bare
tuple(real, complex) and complex_vector read fine). The test model's
array[2] tuple(real, real) data avoids it and the model notes why.
See stan-dev/stan#3437.

Tests: unit tests for the name repair, sizes, filters and the
flatten/unflatten pair; write_stan_json() with tuples and complex
values; and test-tuple-complex.R, one fit of the model exercising the
metadata, filters, data round trip, inits from a fit and from a list,
and the model methods.

Copyright and Licensing

Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
Jonah Gabry

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

• Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
• Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 98.02956% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (v1.0@f4bb79c). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
R/csv.R	96.00%	4 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             v1.0    #1287   +/-   ##
=======================================
  Coverage        ?   93.13%           
=======================================
  Files           ?       19           
  Lines           ?     7372           
  Branches        ?        0           
=======================================
  Hits            ?     6866           
  Misses          ?      506           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jgabry]

@SteveBronder this closes #925, which is an issue you opened a while ago. Do you have time to briefly try out this branch just to see if it behaves as you'd expect for complex and tuple types?