Ahmed Alateeq
ABSTRACT
Voxento is an interactive voice-based retrieval system for lifelogs which has been redeveloped and optimised to participate in the fifth Lifelog Search Challenge LSC'22, at ACM ICMR'22. Based on the previous experience in the LSC competition and ranked in the top 4 in the last LSC'21 competition among 17 participants, we present a revised version of Voxento to address the critical points to improve the efficiency of retrieval tasks in lifelog datasets. Basically, Voxento provides a spoken interface to the lifelog data, which facilitates an expert and novice user to interact with a personal lifelog using a range of vocal commands and interactions. Briefly, we made some important improvements to support both the retrieval of content and system interaction. This latest version has been enhanced with the addition of a text-based search feature, new filters based on new metadata provided in lifelog data, rich visual information and features and enhanced speech query. Also, the data preparation tasks comprised a new function to reduce the number of non-relevant images and the latest CLIP model version used to derive features from images. The long term development of Voxento includes a lifelog retrieval that supports speech and conversation interaction with less physical actions required by users such as using a mouse. The system presented here uses a desktop computer in order to participate in the LSC'22 competition with the option to use voice interaction or standard text-based retrieval.
- Naushad Alam, Yvette Graham, and Cathal Gurrin. 2021. Memento: A Prototype Lifelog Search Engine for LSC'21. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 53--58. https://doi.org/10.1145/3463948.3469069Google Scholar
Digital Library
- Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2020. Voxento: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In Proceedings of the Third Annual Workshop on the Lifelog Search Challenge (LSC'20) (Dublin, Ireland). Association for Computing Machinery, New York, NY, USA, 77--81. https://doi.org/10.1145/3379172.3391728Google ScholarDigital Library
- Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2021. Voxento 2.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 65--70. https://doi.org/10.1145/3463948.3469071Google ScholarDigital Library
- Aaron Duane and Bjorn THORNór Jónsson. 2021. ViRMA: Virtual Reality Multimedia Analytics at LSC 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 29--34. https://doi.org/10.1145/3463948.3469067Google ScholarDigital Library
- Jim Gemmell, Gordon Bell, and Roger Lueder. 2006. MyLifeBits: A personal database for everything. Commun. ACM 49, 1 (2006), 88--95. https://doi.org/10.1145/1107458.1107460Google ScholarDigital Library
- Cathal Gurrin, Alan F. Smeaton, and Aiden R. Doherty. 2014. LifeLogging: Personal big data. Vol. 8. Now Publishers. 1--125 pages. https://doi.org/10.1561/1500000033Google ScholarDigital Library
- Cathal Gurrin, Liting Zhou, Graham Healy, Björn Por Þór Jónsson, Duc-Tien Dang-Nguyen, Jakub Lokoc, Minh-Triet Tran, Wolfgang Hürst, Luca Rossetto, and Klaus Schöffmann. 2022. Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22. In Proc. International Conference on Multimedia Retrieval (ICMR'22). Association for Computing Machinery, New York, NY, USA.Google ScholarDigital Library
- Omar Shahbaz Khan, Aaron Duane, Björn THORNór Jónsson, Jan Zahálka, Stevan Rudinac, and Marcel Worring. 2021. Exquisitor at the Lifelog Search Challenge 2021: Relationships between Semantic Classifiers. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 3--6. https://doi.org/10.1145/3463948.3469255Google ScholarDigital Library
- Emil Knudsen, Thomas Holstein Qvortrup, Omar Shahbaz Khan, and Björn THORNór Jónsson. 2021. XQC at the Lifelog Search Challenge 2021: Interactive Learning on a Mobile Device. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 89--93. https://doi.org/10.1145/3463948.3469063Google ScholarDigital Library
- Andreas Leibetseder and Klaus Schoeffmann. 2021. LifeXplore at the Lifelog Search Challenge 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 23--28. https://doi.org/10.1145/3463948.3469060Google ScholarDigital Library
- Jakub Lokoc, Frantiek Mejzlik, Patrik Veselý, and Tomá Soucek. 2021. Enhanced SOMHunter for Known-item Search in Lifelog Data. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 71--73. https://doi.org/10.1145/3463948.3469074Google ScholarDigital Library
- Thao Nhu Nguyen, Tu Khiem Le, Van Tu Ninh, Minh Triet Tran, Nguyen Thanh Binh, Graham Healy, Annalina Caputo, and Cathal Gurrin. 2021. LifeSeeker 3.0: An Interactive Lifelog Search Engine for LSC'21. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 41--46. https://doi.org/10.1145/3463948.3469065Google ScholarDigital Library
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.00020 http://arxiv.org/abs/2103.00020Google Scholar
- Jihye Shin, Alexandra Waldau, Aaron Duane, and Björn THORNór Jónsson. 2021. PhotoCube at the Lifelog Search Challenge 2021. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 59--63. https://doi.org/10.1145/3463948.3469073Google ScholarDigital Library
- Florian Spiess, Ralph Gasser, Silvan Heller, Luca Rossetto, Loris Sauter, Milan Van Zanten, and Heiko Schuldt. 2021. Exploring Intuitive Lifelog Retrieval and Interaction Modes in Virtual Reality with vitrivr-VR. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 17--22. https://doi.org/10.1145/3463948.3469061Google ScholarDigital Library
- Ly Duyen Tran, Manh Duy Nguyen, Nguyen Thanh Binh, Hyowon Lee, and Cathal Gurrin. 2021. Myscéal 2.0: A Revised Experimental Interactive Lifelog Retrieval System for LSC'21. In LSC 2021 - Proceedings of the 4th Annual Lifelog Search Challenge (Taipei, Taiwan). Association for Computing Machinery, New York, NY, USA, 11--16. https://doi.org/10.1145/3463948.3469064Google ScholarDigital Library
Index Terms
- Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog
Recommendations
LifeSeeker 4.0: An Interactive Lifelog Search Engine for LSC'22
LSC '22: Proceedings of the 5th Annual on Lifelog Search ChallengeIn this paper, we introduce LifeSeeker 4.0 - an interactive lifelog retrieval system developed for the fifth annual Lifelog Search Challenge (LSC'22). In LifeSeeker 4.0, we focus on enhancing our previous system to allow users who have little to no ...
Voxento 2.0: A Prototype Voice-controlled Interactive Search Engine for Lifelogs
LSC '21: Proceedings of the 4th Annual on Lifelog Search ChallengeIn this paper, we describe an extended version of Voxento which is an interactive voice-based retrieval system for lifelogs that has been developed to participate in the fourth Lifelog Search Challenge LSC'21, at ACM ICMR'21. Voxento provides a spoken ...
Voxento 4.0: A More Flexible Visualisation and Control for Lifelogs
LSC '23: Proceedings of the 6th Annual ACM Lifelog Search ChallengeIn this paper, we introduce Voxento 4.0 – an interactive voice-based retrieval system for lifelogs which has been developed to participate in the sixth Lifelog Search Challenge LSC’23, at ACM ICMR’23. Voxento has participated three times in the LSC ...
Comments