Think Bayesian vs frequentist. Obviously from the perspective of "optimize against a loss function" they're both special cases, but one is mathematically and philosophically parsimonious, the other... Less so.