This paper characterizes the capacity region of Gaussian MIMO broadcast channels (BCs) with per-antenna power constraint (PAPC). While the capacity region of MIMO BCs with a sum power constraint (SPC) was extensively studied, that under PAPC has received less attention. A reason is that efficient solutions for this problem are hard to find. The goal of this paper is to devise an efficient algorithm for determining the capacity region of Gaussian MIMO BCs subject to PAPC, which is scalable to the problem size. To this end, we first transform the weighted sum capacity maximization problem, which is inherently nonconvex with the input covariance matrices, into a convex formulation in the dual multiple access channel by minimax duality. Then we derive a computationally efficient algorithm combining the concept of alternating optimization and successive convex approximation. The proposed algorithm achieves much lower complexity compared to an existing interiorpoint method. Moreover, numerical results demonstrate that the proposed algorithm converges very fast under various scenarios.