Scalable and reliable probabilistic inference remains a significant barrier for applying and using probabilistic programming languages (PPLs) in practice. The core of the inference challenge is that there is no universal approach to inference: different kinds of inference strategies are specialized for different kinds of probabilistic programs. For example, Stan’s inference strategy is highly effective on continuous and differentiable programs such as hierarchical Bayesian models, but struggles on programs with high-dimensional discrete structure. On the other extreme, languages like Dice and ProbLog scale well on purely-discrete problems, but the price for this scalability is that they must forego any support whatsoever of continuous probability.

The landscape of today’s probabilistic programming languages is fragmented: each language exists in isolation and performs well on certain kinds of programs, and programmers must choose up-front which language they want to use for their problem. This is clearly problematic from a usability perspective, but there is a deeper issue: \emph{program heterogeneity}. It is often the case that realistic programs have sub-programs that are best handled by \emph{different} languages.

This situation stands in contrast to traditional programming languages, which typically have support for \emph{language interopertion}: for example, Python programmers regularly invoke C code when they need high-performance computation. The key to interoperation is a mechanism for safely transferring data and execution between languages. The question of safe language interoperability has been extensively explored in the context of non-probabilistic languages, so we ask: \emph{What does safe interoperation look like for PPLs}?