Revisiting the Calibration Conundrum: Our SASA 2025 Experience
We were greatly honoured to be informed in September this year that we had received first prize in the SASA (South African Statistical Association) 2025 Honours Project Competition for our project, "A Critical Analysis of History Matching and Emulation with hmer". Our project dived into the History Matching and Emulation (HME) method for model calibration, with a focus on its application to a complex epidemiological model. HME iteratively rules out regions of input parameter space that cannot plausibly produce model output that matches observed data. It uses emulation and a negative criterion - 'implausibility' - to improve the computational feasibility of complex model calibration. We used the hmer R package to apply HME to an epidemiological model, identifying challenges to its performance in a complex calibration setting.
Key issues encountered in our application included overconfidence in emulator estimations due to false homogeneity of output variables, and misleading Bayesian Linear Analysis (BLA) over irregular regions of the non-implausible space. We explored various strategies to address challenges faced in the application of HME, such as omitting the BLA and adjusting the initial design points and implausibility cut-offs. We concluded that HME can successfully calibrate a complex model, but it requires the modeller to have a deep understanding of both the model and the HME method. Our analysis explored practical constraints of HME and offered recommendations for its application in real-life complex modelling contexts.
Along with our award came an invitation to the SASA 2025 conference, which we had the pleasure of attending between the 24th and 28th of November. It began with two days of workshops, during which we got hands-on introductions to topics such as using Stan in R for Bayesian modelling, creating websites and interactive slide shows using Quarto, working with functional data, and using LLMs with the R package ellmer. Attending the presentations of other students and academics over the final few days was interesting and inspiring. The options for research in this field are so incredibly broad, and this was showcased by the diversity of incredible projects and collaborations being developed by the community.
On the 26th, we had an opportunity to participate in the poster presentation event. We loved getting to revisit our project, particularly through such engaging conversation with the event attendees. We were lucky enough to take home second place for the young statisticians poster presentation competition!
More than anything, we loved the opportunity to meet and make connections with statisticians of all ages and all walks of life, from a variety of institutions throughout South Africa and beyond. We are already looking forward to SASA 2026!
Thank you for all of the guidance from our supervisors Professor Sheetal Silal, Mr. Jared Norman and Associate Professor Allan Clark, and thank you to Mr Stefan Britz, Dr Sebnem Er, Associate Professor Allan Clark and Ms Delene van Wyk for organising the UCT delegation to the conference and for taking such good care of us. A particular thank you goes out to MASHA for funding our attendance at the conference. It really was an incredible experience!
