Bayesian networks (BNs) are graphical \emph{first-order} probabilistic models that allow for a compact representation of large probability distributions, and for efficient inference, both exact and approximate. We introduce a \emph{higher-order} programming language which we prove complete w.r.t. Bayesian networks: each (possibly higher-order) program of ground type \emph{compiles} into a BN. This language allows for the specification of recursive probability models and hierarchical structures. Moreover, we provide a \emph{compositional} and \emph{cost-aware} semantics, which is based on factors, a standard mathematical notion used in Bayesian inference. Our results rely on advanced techniques rooted into linear logic, intersection types, rewriting theory, and Girard’s geometry of interaction, which are here combined in a novel way.