There is a growing concern over the adverse health effects of exposure to high concentration of mould spores in the indoor environments. Copious epidemolo-gical studies have shown a direct relationship between the exposure to indoor mould and several adverse health effects. The phenomenon of Sick building syndrome (SBS) and Building Related Illness (BRI) have also been attrib-uted to moulds exposure in the indoor environment. In spite of this growing concern, little have been reported on the development of an objective mould assessment particularly criteria for visual inspection of mould growth on building materials. The main premise of this study is that visual inspection related with mould damaged material can lead to objective ranking of the severity of damaged material, and reduce the subjective nature of mould dam-aged estimation by the use digital image processing (DIP) techniques. A four stage technique procedure, invol-ving image preprocessing, Image segmentation and mould analysis and classification stage for the detection of mould growth is examined in this paper. Results obtained when this proposed algorithm was applied to acquired digital images collected from different infested building materials indicates the appropriateness of this method in enhan-cing the visual assessment and grading associated with mould growth on building materials.