Program synthesis is the task of automatically generating a program that matches some specification, e.g., a set of input-output examples. A key challenge for efficient search in the vast space of programs is telling when a partial program is on the right track, because it cannot be directly tested against the input-output examples. In this work, we propose to address this challenge using probabilistic program synthesis. We begin by synthesizing simple, “coarse” probabilistic programs that use randomness liberally as a temporary stand-in for as-yet-unknown input-dependent logic. We then iteratively refine these programs by replacing high-entropy expressions with low-entropy ones. A key benefit of this approach is that even coarse programs have well-defined (probabilistic) semantics, enabling us to evaluate likelihoods of input-output examples and identify promising candidates for further refinement. This refinement process can either be used to synthesize \textit{deterministic} programs, by iteratively refining away \textit{all} of the stochastic primitives; or alternatively, synthesis can be stopped early to yield a learned probabilistic model of the conditional distribution on outputs given inputs. We present preliminary evidence that likelihoods of coarse probabilistic programs are useful for guiding synthesis in both settings.