Tools
Tools
Wegner-Clemens, Kira, Malcolm, George 
ORCID: https://orcid.org/0000-0003-4892-5961 and Shomstein, Sarah
(2022)
How much is a cow like a meow? A novel database of human judgements of audiovisual semantic relatedness.
Attention, Perception & Psychophysics, 84 (4).
1317–1327.
ISSN 1943-3921
Preview |
PDF (Wegner-Clemens.APP.2022)
- Accepted Version
Download (1MB) | Preview |
Abstract
Semantic information about objects, events, and scenes influences how humans perceive, interact with, and navigate the world. The semantic information about any object or event can be highly complex and frequently draws on multiple sensory modalities, which makes it difficult to quantify. Past studies have primarily relied on either a simplified binary classification of semantic relatedness based on category or on algorithmic values based on text corpora rather than human perceptual experience and judgement. With the aim to further accelerate research into multisensory semantics, we created a constrained audiovisual stimulus set and derived similarity ratings between items within three categories (animals, instruments, household items). A set of 140 participants provided similarity judgments between sounds and images. Participants either heard a sound (e.g., a meow) and judged which of two pictures of objects (e.g., a picture of a dog and a duck) it was more similar to, or saw a picture (e.g., a picture of a duck) and selected which of two sounds it was more similar to (e.g., a bark or a meow). Judgements were then used to calculate similarity values of any given cross-modal pair. An additional 140 participants provided word judgement to calculate similarity of word-word pairs. The derived and reported similarity judgements reflect a range of semantic similarities across three categories and items, and highlight similarities and differences among similarity judgments between modalities. We make the derived similarity values available in a database format to the research community to be used as a measure of semantic relatedness in cognitive psychology experiments, enabling more robust studies of semantics in audiovisual environments.
| Item Type: | Article |
|---|---|
| Additional Information: | Funding Information: This research was supported by NSF BCS-1921415 and NSF BCS-2022572 to SS. |
| Uncontrolled Keywords: | audiovisual,multisensory,naturalistic stimulus set,semantics,experimental and cognitive psychology,language and linguistics,sensory systems,linguistics and language,3* ,/dk/atira/pure/subjectarea/asjc/3200/3205 |
| Faculty \ School: | Faculty of Social Sciences > School of Psychology |
| UEA Research Groups: | Faculty of Social Sciences > Research Groups > Cognition, Action and Perception Faculty of Social Sciences > Research Groups > Developmental Science |
| Related URLs: | |
| Depositing User: | LivePure Connector |
| Date Deposited: | 29 Mar 2022 13:30 |
| Last Modified: | 04 Dec 2024 01:36 |
| URI: | https://ueaeprints.uea.ac.uk/id/eprint/84327 |
| DOI: | 10.3758/s13414-022-02488-1 |
Downloads
Downloads per month over past year
Actions (login required)
 |
View Item |