The oriented chromatic number of an oriented graph $\vec G$ is the minimum order of an oriented graph $\vev H$ such that $\vec G$ admits a homomorphism to $\vev H$. The oriented chromatic number of an undirected graph $G$ is then the greatest oriented chromatic number of its orientations. In this paper, we introduce the new notion of the upper oriented chromatic number of an undirected graph $G$, defined as the minimum order of an oriented graph $\vev U$ such that every orientation $\vec G$ of $G$ admits a homomorphism to $\vec U$. We give some properties of this parameter, derive some general upper bounds on the ordinary and upper oriented chromatic numbers of Cartesian, strong, direct and lexicographic products of graphs, and consider the particular case of products of paths.