Hello!
I am new to Normalizing Flows and am trying to use a conditional Masked Autoregressive Flow, where the data that will be “normalized” is a 1 dimensional variable. More specifically it is a very highly zero inflated rate (~90% of values are 0.0). Do you have any suggestions how to address the problem of high zero inflation? Subsampling the zero counts is an option, but then a lot of data is lost and the zeros have a meaning. Thanks!
Jakub
Welcome to the bayesflow forums! Can you share a histogram of your data perhaps? So we can get an idea of how your data looks like. Also what would you use as conditions (you put conditions in the title)?
Additionally, we don’t support MAFs in bayesflow. I would recommend going with a spline flow as a more performant option. Check out our regression tutorial, which may be of help.
Thank you!
The distribution histogram looks like a heavily zero inflated beta distribution (I can share the histogram tomorrow). I conditon on manually handcrafted features extracted from genomic alignment data (each observation is a numeric vector of length 8). I have also tried to train a simple embedding MLP jointly with the normalizing flow and condition on the MLP embedding and not the raw data.
Thank you, I will try that out! I initially chose MAF’s, because it was the most intuitive normalizing flow for my level of understanding.
So if you are planning to model a 1 dimensional variable that looks like a zero-inflated beta distribution, why would you want to use any normalizing flow in the first place? It may be easier and cleaner to just use distributional regression with a zero-inflated beta distribution for your purposes. I can help you set up a model if you like (not in bayesflow or with any other abi/sbi library though)?
Thanks,
this is the histogram of the data:
I want to use conditional normalizing flows as a potentially interesting idea that might or might not work. In general I want the flow to learn the distribution of the error rate (of the genomic sequencing+alignment process) conditional for features of the sample and of the reference genome context. I train the normalizing flow on data where I assume that the only variance I observe is technical error. Then during inference, I plan to condition the reverse flow on a new sample and get the posterior error rate distribution conditional on this sample’s features. Finally, when I have obtained that posterior, I have a rough proxy for how much of the observed variability in that sample is due to technical noise (the rest is then the biological signal of interest).
PS: I have also used a zero-inflated binomial mixed model for this task and got some decent results. But there I got one point estimate of the error rate for a new sample. Therefore I wanted to try out normalizing flows to obtain not just a point estimate but a whole (conditional) posterior for a sample.
The problem is indeed the exact zeros which will mess badly with your normalizing flows since they assume continuous data as you know. My suggestions would be to use Bayesian zero-inflated binomial models, say, in the R package brms:
fit <- brm(bf(y ~ , zi ~ ), data = , family = zero_inflated_binomial())
This would give your full distributions of whatever predictions you seek.
If you want model overdispersion you can ask AI to help you write a zero_inflated_beta_binomial custom family for brms.
Apologies if that does not go in the direction you want to go. If you insist on using normalizing flows for this case, perhaps @KLDivergence has some ideas.
Thank you very much!
I will try that out.
Alternatively I can get rid of exact zeros. This would then change slightly the biological interpretation and potentially harm the sensitivity of the method to detect the biological signal. But these downsides are hypothetical as of now and I will validate them, maybe they are negligible in practice.
Thanks again for the input.
