Paper Presents empirical evaluation of using lightweight signal features from a set of complex activities during the meal (e.g., clattering sound, arm gestures of eating, human voice, TV sound) and fusing built-in sensor data of multiple mobile devices available in a family with a CRFs-based classifier.

Abstract. Monitoring the family mealtime activities enables the analysis of the previous daily routine, hence the positive changes can be made towards better relationships among family members and better physical/mental health. Moreover, the details of family mealtime activities provide important information for study in sociology and culture. This paper presents FamilyLog — a practical system to log family mealtime activities using smartphones and smartwatches. FamilyLog automatically detects and logs details of activities during the mealtime, including occurrence and duration of meal, conversations, participants, TV viewing etc., in an unobtrusive manner. Based on the sensor data collected from real families, we carefully design robust yet lightweight signal features from a set of complex activities during the meal, including clattering sound, arm gestures of eating, human voice, TV sound, etc. Moreover, FamilyLog opportunistically fuses data from built-in sensors of multiple mobile devices available in a family with a CRFs-based classifier. To evaluate the real-world performance of FamilyLog, we perform extensive experiments that consist of 77 days of sensor data from 37 subjects in 8 families with children. FamilyLog can detect those events with high accuracy across different families and home environments.