The modelling of modern power markets requires the representation of the following main features: (i) a stochastic dynamic decision process, with uncertainties related to renewable production and fuel costs, among others; and (ii) a game-theoretic framework that represents the strategic behaviour of multiple agents, for example in daily price bids. These features can be in theory represented as a stochastic dynamic programming recursion, where we have a Nash equilibrium for multiple agents. However, the resulting problem is very challenging to solve. This work presents an iterative process to solve the above problem for realistic power systems. The proposed algorithm is consist of a fixed point algorithm, in which, each step is solved via stochastic dual dynamic programming method. The application of the proposed algorithm are illustrated in case studies with the real power systems.