I've been working on modeling some phenomena involving real-time control in an environment with inherent rewards (specifically, playing a 'pong'-like game), and it's increasingly looking like reinforcement learning by itself won't cut it computationally (I'm currently using a temporal difference back propagation neural network).
One possible supplemental learning mechanism is to also have the model predict the environment's future state, from which it can learn from in a supervised manner using standard feed forward back propagation.
My current thinking on synthesizing these mechanisms is to have the input layer feed into a hidden layer, which in turn feeds into both a reward predicting layer and a separate state predicting layer. In training this net, I simply change the weights via reinforcement learning first and then change them again to account for the state prediction error via back prop.
So my question is this: Are there any problems you can foresee arising from this architecture? Additionally, has this combination of learning mechanisms been done before, and if so was it done in a similar way?