After you fight the vampire spawn at 3rd level and flee, and the vampires then kill all the guards, wipe out half of Barovia and you continue on your adventures, the Church of Barovia is where all the villagers move to in order to stay safe.

Or at least, that's what's happening in our version of Ravenloft!