© 2017 MVA Organization All Rights Reserved. This paper presents a self-learning structure for text localization. The proposed system has an ability to improve itself automatically by analyzing unlabelled images. The system consists of three classification modules called component grader, component linker, and group classifier. Firstly, the image is analyzed to obtain the character candidate components. Then, the grader evaluates the possibility of text for every component by considering their properties individually while the linker classifies the degree of connection for every two components and groups all linked components together. Then, the groups of components are classified as text or non-text by the group classifier. Since all three modules work almost independently, we can update one module by using results from the other modules. This paper also presents a strategy for updating all modules by using unlabelled images. The experiment is given to show that the grader and the linker can be initialized by using few labeled training samples and then the system can automatically collect more samples from unlabelled images by using the results from three modules.