Error when using loaded approximator

General
1

When I try to use .sample when using a loaded approximator I get the following error:

ValueError: Cannot call forward with strict=False before calling forward with strict=True.

The issue does not come up when I try to use .sample straight after training the model (without first saving and loading it back in). I am using BayesFlow version 2.0.3.

2

I will investigate and get back to you. Can you show me the adapter you are using?

3

Hi Stefan, thanks for checking it out! Iā€™m using the following adapter:

adapter = (
bf.Adapter()
.to_array()
.convert_dtype(ā€œfloat64ā€, ā€œfloat32ā€)
.concatenate([ā€œE_barā€, ā€œE_cableā€, ā€œdamagesā€], into=ā€œinference_variablesā€)
.concatenate([ā€œpred_vectorā€], into=ā€œsummary_variablesā€)
)

and the following approximator:

approximator = bf.BasicWorkflow(
simulator=simulator,
adapter=adapter,
summary_network=summary_network,
inference_network=inference_network,
checkpoint_filepath=model_folder_path
)

I fit the model using:

history = approximator.fit_offline(standardized_training_data, epochs=epochs, num_batches=num_batches, batch_size=batch_size, validation_data=standardized_validation_data, callbacks=[StepHistory()])

I load pretrained models using:

pretrained_approximator = keras.saving.load_model(os.path.join(model_folder_path, ā€˜model.kerasā€™))

4

Hi Tom,

Thanks for the context info! This reminds me of this issue that was also using offline training - could it be a similar problem that we have two disconnected adapters in the OfflineDataset and approximator here? I.e., I donā€™t know about the rest of your workflow but do you also load a saved dataset here?

If the source of the error is still unclear, it would help us a lot of you could open an issue with a minimal reproducible code example so that we can track down the issue. In the meantime, a quick bandaid fix so that you can continue with your project would be to manually call the adapter once (e.g., _ = adapter(data)) before sampling.

Best,
Lasse

1 Like
5

Hi Lasse, thanks for the swift reply!

I think indeed the issue is similar, I create an offline dataset separately and load in this dataset using pickle. For now Iā€™ve opted for the bandaid fix, which works fine for me.

Cheers, Tom

1 Like
6

Hi Tom, just to confirm. Are you using approximator.sample() or workflow.sample() after loading the model?

7

Hi Stefan,

I use the following:

approximator = bf.BasicWorkflow(
simulator=simulator,
adapter=adapter,
summary_network=summary_network,
inference_network=inference_network,
checkpoint_filepath=model_folder_path
)

approximator = keras.saving.load_model(os.path.join(model_folder_path, ā€˜model.kerasā€™))

post_draws = approximator.sample(conditions=standardized_standard_test_data, num_samples=1000)

8

I am not able to reproduce the problem in any of my custom training pipelines. Does the problem still persist on dev?

9

Hi Stefan,

The problem is resolved on dev, thanks for checking it out!

10

Cool, thanks for confirming. dev is becoming the new release later today.
S.