Reinforcement learning is learning what to do -- how to map situations to actions -- so as to maximize a numerical reward signal. The learner is not told which actions to take, as in most forms of machine learning, but instead must discover which actions yield the most reward by trying them. In the most interesting and challenging cases, actions may affect not only the immediate reward, but also the next situation and, through that, all subsequent rewards. 

In many tasks to which we would like to apply reinforcement learning, most states encountered will never have been experienced exactly before. This will almost always be the case when the state or action spaces include continuous variables or large number of sensors, such as a visual image. The only way to learn anything at all on these tasks is to generalize from previously experienced states to ones that have never been seen. 

The two main kinds of generalization (function approximation) architectures are local, such as fuzzy rules, and global, such as neural networks. Local architectures are very efficient for state vectors in low dimensions but they suffer from the "curse of dimensionality". Global architectures can be applied to large-dimensional problems, but they have to be trained using nonlinear programming methods, which suffer from the exponential growth of local optima as the input dimension increases. Also, the results of learning cannot be easily interpreted by the modeler. In the work we have done so far, we have used local function approximation architectures based on feature extraction using fuzzy logic. However, our latest results on multi-agent coordination do not depend on the details of the individual learning algorithms and can work just as well with global function approximation. 

Recently, were able to prove convergence of an actor-critic form of Fuzzy Reinforcement Learning. As a result, we now have an unprecedented tool for developing autonomous intelligent agents in a variety of real world domains. 

Recent Publications

• Hamid R. Berenji, David A. Vengerov. (2001) "On Convergence of Fuzzy Reinforcement Learning," Submitted to the 10th IEEE International Conference on Fuzzy Systems (FUZZ-IEEE '01).

For more information, please contact Dr. Hamid Berenji at IIS Corp. Email: berenji@iiscorp.comPhone: (408)730-8345