Application of Artificial Intelligence (AI) in Food Recognition: Creating the first Central Asia Food Scenes Dataset for delivering personalized nutrition and health promotion in Kazakhstan

Faculty Development Competitive Research Grant Program (AI and Data Science) 2024-2026

Photographing and sharing food images on social media has the potential to revolutionize personalized dietary interventions. Real-time food recognition can replace laborious process of manually recording and coding food diaries, along with the tedious need for individually inputting each food or beverage item consumed. However, it is noteworthy that there exists a significant gap in published scientific literature concerning the dietary habits and food preferences of individuals in the Central Asia region, despite this region's rich cultural diversity.

In Central Asia, including Kazakhstan, there is a pressing concern regarding premature mortality resulting from non-communicable diseases (NCDs) such as cardiovascular diseases, diabetes, and certain types of cancer. A significant contributor to the prevalence of these NCDs is poor dietary habits. Evidence indicates that Central Asia experiences one of the highest burdens of diet-related deaths globally. These adverse health outcomes not only hinder socioeconomic development but also impede progress toward sustainable development goals (SDGs). Exploring the links between dietary intake, lifestyle, and cardio-metabolic risk in adult Central Asians is crucial for cost-effective public health policies. Yet, the lack of a comprehensive and representative food dataset and current labor-intensive and inaccurate dietary assessment methods necessitate the urgency of developing a contemporary and efficient dietary assessment for robust health surveillance and policy development.

The emergence of artificial intelligence (AI) and its integration into the food industry has brought about promising avenues for enhancing various aspects of farm to fork-related processes (e.g., improving food product characteristics, recipe evaluation, food identification, and dietary analysis). Notably, advancements in computer vision (CV) and the widespread use of smartphones and devices has paved the way for food image recognition. The ability to recognize objects with precision exceeding that of humans has laid the foundation for efficient food recognition models. Integrating AI in dietary assessment can provide insights from big datasets that surpass human capabilities, reducing the burden on healthcare professionals. Additionally, AI-powered smartphone applications can contribute to improved nutrition literacy among the population. The field of automatic food image recognition and classification holds potential benefits for nutritional records, accessibility for visually impaired individuals, and overcoming memory biases. It holds promise for supporting patients and caregivers in dietary management and nutrition enhancement.
Several datasets have been compiled for food classification, with a focus on different cuisines. However, a comprehensive dataset specific to Central Asian foods has been lacking to date. To address this gap, our overarching objective was to establish a comprehensive and easily accessible dataset of regional foods that could benefit both the general public and researchers alike. This will aid further research into examining the association between food intakes, lifestyle factors and their associations with cardio-metabolic risk factors in CA adults. This endeavor marked our research team’s inaugural attempt to construct a Central Asian Food Dataset (CAFD) encompassing 42 distinct food categories, comprising over 16,000 images showcasing unique national dishes from the region. Our implementation of the ResNet152 neural network model yielded an impressive accuracy rate of 88.70% for these 42 categories. This accomplishment underscores our team’s remarkable potential and application of computer vision techniques in the domain of food recognition. This dataset serves as a foundation for enhancing future deep learning-based food recognition models.

Our ongoing work involves refining the classification of some of the less accurately recognized food items through neural network architecture variations and data augmentation techniques. Despite its potential, food recognition remains a complex and multidimensional challenge that distinguishes itself from typical image classification tasks. In contrast to simpler computer vision assignments, food recognition encounters several challenges which include inaccurate food classification resulting from diverse cooking and preparation techniques, ingredient concealment, and variations in photo quality due to varying lighting and image orientations. The presence of inter-class ambiguity within food groups further exacerbates the recognition problems, especially concerning visually similar items like kebabs, salads, or soups. Additionally, the estimation of food volume presents yet another daunting task to researchers working in food recognition domains. Nonetheless, the resolution of these challenges can be overcome by meticulously planned research endeavors aimed at achieving effective classification and optimizing attributes. This is where our research proposal endeavors to bridge these gaps by delving into multi-label classification techniques and employing trained human annotators to categorize food images within their respective food groups. Although this initial process may be tedious and labor-intensive, its implementation promises to significantly enhance the performance of our automatic food recognition model. Furthermore, we intend to expand and explore food localization and scene recognition, enabling accurate nutrient intake estimation from images capturing multiple food items in a single image, resulting in a more accurate portrayal of actual dietary patterns. In our earlier work, we have only worked with classification models for one food item per image. Additionally, the dataset containing food scenes will contain more food classes since typical food scenes usually include local national dishes consumed with other Western or Asian foods. Based on the additional food classes, we will be able to extend the current food categories. This practical extension of our current research aims to further enhance the utility of our dataset and help to consolidate our role as leaders in the field of food recognition within this region.

Another current limitation stemming from the application of AI approaches in computer vision dietary recommendations is the lack of transparency and evidence-based factors utilized in the decision-making process of the algorithm. This limitation presents a crucial bottleneck in engaging healthcare professionals and the general public with these AI-based solutions. It is imperative to comprehend the insights and experience of healthcare professionals, which will aid in elucidating and designing our AI-generated dataset and its subsequent deployment. To address this concern, we are conducting a survey involving healthcare professionals. This initiative aims to further enhance the reliability of our dataset and its subsequent application.

Therefore, to address these gaps, these are the following research objectives:
1.To create the Central Asian Food Scenes (CAFS) dataset by collecting images with multiple food items and carefully annotating these images with bounding boxes.
2.To develop a food object recognition model based on the state-of-the-art object recognition models using transfer learning.
3.To identify shared requirements of healthcare professionals (e.g., dietitians, doctors and nurses) and ensure the CAFD can meet specific patient care needs.
4.To develop a smartphone application utilizing the food recognition model trained with CAFS to capture the lifestyle and other dietary habits of the population living in CA
5.To disseminate research outputs to unveil the CA’s unique culinary tapestry and diet-related risk factors in CA