Rokaia Jedir is a first year postgraduate research student at Limerick Institute of Technology, and the newest member of the Interactive Systems Research Group (ISRG), led by Dr. Flaithri Neff. Rokaia’s research is inter-disciplinary by nature. Her main research topic proposes contributions to the current understanding of human working memory (WM) as it pertains to the auditory domain, but also expands to a number of inter-disciplinary research topics, such as music psychology, learning and educational development, and even machine-learning. To date, her preliminary research has built upon existing models of WM by incorporating principles associated with the hearing system. Although a new addition to L.I.T.’s postgraduate research society, Rokaia has been somewhat involved in with the ISRG as a research assistant throughout the final years of her undergraduate studies in Music Technology and Production:

“My undergraduate work to date has presented a review of various proposed models of working memory, with particular focus on a theory developed by Dylan Jones, referred to as the Changing-State Hypothesis. Research to date has focused on revealing overlaps in function between the hearing system (psychoacoustic phenomena) and working memory (based on existing theories developed in cognitive psychology).”

In joining the ISRG, Rokaia intends to spend the duration of her postgraduate studies in collaboration with a team of researchers, including staff and other postgraduate researchers in L.I.T. The Group’s current research area is the development of multimodal frameworks for a variety of interactive applications, with the overarching goal of facilitating the advancement of multimodal, computer interfaces. Multimodal interfaces denote a new system of interacting with technology beyond the visual presentation of information and standard keyboard/mouse input. Such interfaces utilise a combination of human senses, such as seeing, hearing, gesture, and touch. It aims to generate interactive processes between humans and machines that are more reflective of the multidimensional communication inherent between people. It also furthers research in making computers more accessible by allowing different modalities of interaction.

Rokaia’s research topics pertains to a number of crucial matters being investigated by the ISRG. Currently, the Group’s primary research focus is the modelling of human auditory perceptual processes so that computer interfaces can accurately mirror how humans process non-speech auditory information. In this context, Rokaia intends to focus on aspects of working memory, and the disruption of cognitive focus caused by characteristics of the auditory perceptual system:

“Working memory is a brain-system that is always at work, allowing temporary storage and manipulation of information. It provides a means for us to use this information, and contextualise during cognitive tasks, such as reading, memorising information, or even simple math problems. As part of my undergraduate final year project, I worked towards a comprehensive review of research on working memory and auditory perception, with a hope to form a basis for my current postgraduate studies. This year, I am working on the proposal of a new perceptual model detailing the flow of information through the separate mechanisms responsible for working memory and auditory perceptual processing. Later during my postgraduate studies alongside the ISRG, I plan to integrate the proposed model’s functionalities with multimodal interface design concepts, using findings as a basis for effectively delivering sonified data to users of technology.”

People’s innate ability to parse and process the complex auditory scenes we experience everyday in our environment means that there is potentially an alternative, but rich and effective, means to present content- and context-rich data to users of technology. This area of research requires a team that brings together many different disciplines. The ISRG relies on members with expertise in Human-Computer Interaction (HCI); auditory perception; cognitive psychology; acoustics; audio technology; statistics; machine learning; and computer programming.

Dr. Flaithri Neff suggests that Rokaia’s research has many industrial implications that are directly relevant to inter-disciplinary topics covered being investigated by the ISRG, under his supervision:

 “The team frequently collaborates with industry on technically-challenging, novel, and interdisciplinary projects, and is currently working towards the development of intelligent acoustic analysis systems, strongly informed by psychoacoustic principles, for Smart City and Industry 4.0 applications. With members stemming from the Music Technology programme at LIT, the Group also apply their research to diverse creative projects in multimedia and interactive performance in the arts. Rokaia’s research fills an important gap in terms of modelling the auditory perceptual system, and applying these concepts to industrial applications. Capabilities of mobile and wearable devices is constantly increasing in terms of processing multiple channels of complex data, but simultaneously, screen real-estate decreases with every new piece of technology released, and is even eliminated in some form factors. Therefore, the sonification of complex data is garnering interest as an alternative means for relaying complex information to users, or at least as a supplementary channel alongside limited visual presentation on these devices. More robust perceptual design models are required to ensure the effective use of sonification in such contexts, and Rokaia aims to contribute to such models in her proposed research.”

Collaborations between the ISRG and research groups at other academic institutions, as well as key industry partners, continue to be developed. Members of the ISRG also have a keen interest in the important work of standardisation, and are members of a National Standards Authority of Ireland mirror committee reviewing audio, picture, multimedia, and hypermedia information standards. The Group’s publication record demonstrates their interdisciplinary research practice and application, with recent presentations at the Interactive Audio Systems Symposium (University of York, UK), the 15th International Conference on Music Perception and Cognition (University of Graz, Austria), and the Annual British Human-Computer Interaction Conference (Fisherwick Place, Belfast). 

More information on current research and publications by the ISRG is available at www.interactivesystems.ie.