In one of our recent papers, we have analyzed the applicability of the Akaike (AIC) and Bayesian Information Criterion (BIC) to multi-level mechanistic explanations. While the AIC and BIC are useful tools to weigh other models’ simplicity against their goodness-of-fit to find the best one under consideration, both criteria face significant difficulties when applying them to complex mechanistic models.
Certainly, finding a satisfactory degree of simplicity is relevant in mechanistic models. It may even be particularly important to find rules that determine the desirable simplicity or necessary complexity of mechanistic explanations, as they can be extended easily by adding more and more layers. For instance, in our paper, we introduced a mechanistic model that explains human behavior at different levels. The levels comprise a good-based model at the highest level and extend all the way to the molecular level. Based on what conditions can a scientist decide at which point a model becomes too complex as its explanatory power is reduced by adding more levels?
In our recent paper, we concluded that the AIC and BIC are applicable only under very specific conditions as they require fully quantifiable parameters and data points that allow an evaluation of the models’ goodness-of-fit and simplicity values. In mechanistic models, this condition is satisfied scarcely ever. At best, it may apply to individual levels. Does this circumstance render the AIC and BIC completely useless? Can we conceive alternative existing or hypothetical criteria that may be more appropriate for mechanistic model selection? We aim to answer these questions in our next paper and share some of our initial thoughts in this blog.
In the context of criteria such as the AIC and BIC, the number of adjustable parameters determines a model’s simplicity. For multi-level mechanistic models, this measure may be neither ascertainable nor well-founded. On the one hand, quantifiable variables are not used on many levels. On the other hand, it is doubtful that the number of parameters should be the single determinant of simplicity. Mechanistic models’ complexity is, to a large extent, determined by the number of levels, which may or may not have quantitative parameters. Consequently, a proper criterion would need to incorporate a measure for the number of levels and the complexity of individual levels, as weighing all levels equally would certainly not be adequate.
Furthermore, it may be a partial solution to apply the AIC or BIC to those levels which are fully quantitative. Is using the AIC or BIC in some cases better than not using it at all? This may or may not be the case. Occasionally, the inclusion of variable X in level A may depend on, or influence, the inclusion of variable Y in level B. How can a modeler deal with these complex interdependencies?
We will discuss these and various other issues in our next paper.
