The problem of finding decentralized transmission policies in a wireless communication network with energy harvesting constraints is formulated and solved using the decentralized Markov decision process framework. The proposed policy defines the transmission strategies of all devices so as to correctly balance the collision probabilities with the energy constraints. After an initial coordination phase, in which the network parameters are initialized for all devices, every node proceeds in a fully decentralized fashion. We numerically show that, unlike in the case without energy constraints where a fully orthogonal scheme can be shown to be optimal, in the presence of energy harvesting this is no longer the best choice, and the optimal strategy lies between an orthogonal and a completely symmetric system.