When the posterior has different modes (like a very high peak with some minor but long tales), what would be a better alternative for calculating the aggregate value of samples other than ‘mean’ or ‘median’? I remember I had seen some suggestions in the documentation but I’m having a hard time finding it. Is there a method to calculate the point estimate of the posterior for the most relevant mode of the samples without considering the less relevant area (like the tails)?
Thank you so much,
Ali
PS1: Congratulations on the new version! I’m really excited to testing it.
PS2: I just realized that I’ve never shared our paper on integrating BayesFlow with System Dynamics models and their estimation tasks. I should thank you all for all the support you’ve provided on this forum too. Here’s the paper: https://onlinelibrary.wiley.com/doi/full/10.1002/sdr.1798
for multimodal distributions, no single point estimate really covers the distribution well. In these cases, I would suggest just plotting the distribution in the form of a density, for example. Can you share some of these multimodal distributions. Perhaps I have some more concrete thoughts how to summarize it. In any case, using tail quantiles to summarize the range of the distribution (e.g., 5% and 95% quantile) remain rensible even in multimodal settings.
Thanks for the suggestion, Paul! I don’t have clean example in code, but here’s a relatively similar situation. In dimensions that I’ve highlighted, taking the mean of samples would give a point estimate that is further from the ground truth. What I meant in my original post is similar situations that can happen anywhere in the space. They
Another example is this, but it is not a well-calibrated model and there are only a few samples collected. However, I had the highlighted situations conceptually in mind. Thanks again a lot for your suggestions.
When a variable is bounded and the true value is close to the boundary, it is expected that the mean is biased. Which doesn’t make it a bad estimate necessarly. Same with the median. You can of course attempt to use the mode, but as you say this will struggle with multimodality. And also, for highly skewed posteriors, the mode will likely be somewhere close to the tails of the distribution, so it is not really a “measure of central tendency” in this case. What I am trying to say here is that there is no single estimate that is able to capture all these cases. Rather, illustrating the distributions directly, perhaps supported with a few quantiles (+ mean perhaps) will often provide a good overview of the estimated posterior.
