Individuals in the swarm intelligence systems are generally designed to be able to perform cooperative behaviors. However, those individual are usually with simple structures, i.e., there are few models of individuals with high cognitive functions, e.g., pattern recognition, adaptive learning, self-organizing and so on. In this paper, we propose a neuro-fuzzy reinforcement learning system as a common internal model of the intelligent individuals, i.e., the intelligent agents or multiple autonomous mobile robots. In the proposed model, the local environment information observed by a learner is recognized by a self-structuring neuro-fuzzy network (Fuzzy Net), and a conventional reinforcement learning algorithm named “sarsa” is adopted into the system for modifying the connections between the part of Fuzzy Net and state-action value functions to acquire adaptive behaviors. Swarm of agent is also available to be formed by the proposed method adopting reward/punishment during the learning process. According to the results of simulations of dealing with goal-navigation exploration problems, “swarm learning” i.e., suitable distances between individuals are evaluated with positive rewards during the learning process, showed higher efficiency compared with the opposite case of “individual learning”.