Hi everyone,
I am new to amortized Bayesian inference (ABI) and, in fact, I have so far not tried to fit any model, e.g. using your software, yet. So far, I have “only” read a few papers on ABI/SBI to get an initial understanding of the topic, including the paper on Amortized Bayesian Multilevel Models.
I am interested in ABI and BayesFlow solely because of the amortization aspect, because I need to fit my models repeatedly to many data sets. Other reasons for considering ABI (e.g. intractable likelihood in complicated/mechanistic models) do not matter to me at all (at least for now). Hence, the models I am interested in generally come from the class of generalized linear (mixed) models. Concrete examples could include a negative binomial regression model for count data with only few (say 1-3) covariates or a linear mixed model for longitudinal observations such as a hierarchical intercept-slope model. Because most examples I have seen so far (one exception) focus on more complicated models I was hoping to get some answers to a couple of questions regarding the use of ABI for such simpler models and maybe some experiences here.
Summary network
I was wondering whether for (hierarchical) GLMs would it make more sense to handcraft summary statistics instead of specifying and training a summary network? E.g. for the simplest case of a linear model with Gaussian error term, the sufficient statistics are available analytically, right? Perhaps to broaden the question: When would you recommend to handcraft summary statistics and when would you recommend to use a summary network instead?
Inference network
Taking into account the relative simplicity of my models of interest, are there specific inference network architectures that you would recommend? I.e. are there certain networks for which I can expect the training to be notably faster without sacrificing inferential accuracy compared to other alternatives?
Inferential accuracy
Do you have experience with the inferential accuracy of ABI in these settings compared to gold-standards like Stan/NUTS? Importantly, I do not only/mainly care about means/medians but very much about tail probabilities (e.g. 98% quantile).
Simulation gap
I have read about the potential problem of having a simulation gap and ABI not being able to estimate the posterior reliably in such scenarios. I would like to know “how large” this gap would need to be to cause problems in practical applications. I am asking this because I may want to deliberately specify a prior for a parameter that is not well aligned with the observed data. The prior would still have some probability mass in the relevant regions and be able to sample parameters that are aligned with the observed data, so there would only be misalignment but not an actual complete gap.
Thank you very much and best wishes
David