A dynamic bipartite matching model is given by a bipartite matching graph which determines the possible matchings between the various types of supply and demand items. Both supply and demand items arrive to the system according to a stochastic process. Matched pairs leave the system and the others wait in the queues, which induces a holding cost. We model this problem as a Markov Decision Process and study the discounted cost and the average cost problem. We assume that the cost function is linear on the queue sizes. We show that for the -shaped matching graph, an optimal matching control prioritizes the matchings in the pendant edges and is of threshold type for the diagonal edge. In addition, for the average cost problem, we compute the optimal threshold value. We then show how the obtained results can be used to characterize the structure of an optimal matching control for a quasi-complete graph with an arbitrary number of nodes. For arbitrary bipartite graphs, we show that, when the cost of the pendant edges is larger than in the neighbors, an optimal matching policy prioritizes the items in the pendant edges. We also study the -shaped matching graph and, when the cost of the pendant edges is larger than the cost of the middle edge, we conjecture that an optimal matching policy is also of threshold type with priority to the pendant edges; however, when the cost of the middle edge is larger, we present simulations that show that it is not optimal to prioritize items in the pendant edges.