In this work we present a new approach for the characterization of the Pareto front for multi-objective optimal control problems with state constraints. Our approach is based on the HJB (Hamilton-Jacobi-Bellman) theory. A general bi-objective constrained optimal control problem is considered. First, we introduce an auxiliary optimal control problem for an augmented dynamical system. It turns out that the Pareto front for the multi-objective optimal control problem is characterized as a subset of the zero-level set of the value function associated with the auxiliary control problem. This characterization leads to a numerical procedure for computing the Pareto front and the corresponding optimal trajectories from the augmented value function. A Pareto front characterization is also derived in a special case where one of the criteria is a minimum time to reach a target set. Some numerical examples are presented in this work to show the relevance of our approach.