Skip to main content

The influence of object-location binding mental load effects on the visual N1 and N2 Event-related Potentials

Abstract

Objective

This study aimed to analyze the effect of object-location binding on the visual working memory workload. For this study, thirty healthy subjects were recruited, and they performed the “What was where” task, which was modified to evaluated object-location binding memory. We analyzed their ERP and behavior response.

Results

Object memory and location memory were preserved during the task, but binding memory decreased significantly when more than four objects were presented. These results indicate that the N1 amplitude is related to the object-only load effect, and the posterior N2 amplitude is a binding-dependent ERP component.

Introduction

VWM is essential for perception and information processing in the real visual world. VWM has a limited capacity determined by visual feature binding, not by the number of individual features, and is generally known to comprise three or four objects [1,2,3]. This binding memory function enables visual information processing even with a limitative VWM capacity. The effect of feature bindings on capacity has been investigated in several psychological studies. Feature bindings represent an object's identity-related features, such as color, shape, and orientation [4].However, the binding of object features and object location on VWM capacity has not been extensively studied. The VWM functional pathways can be divided into ventral stream features and dorsal stream features related to object identity and location [5,6,7]. Because object features and object location information are handled differently in the ventral and dorsal streams, the binding between the object features and the object location could be approached differently.

Most existing studies on object-location binding and VWM capacity have used change detection tasks [8,9,10]. They assessed object-location binding ability by how the ability to sense changes in whole-display or single-probe varied depending on how many objects were presented. However, it has been questioned whether the VWM load and capacity assessment using the change detection task is accurate [11]. To prevent location information from being encoded in an abstract configuration, such as a relative inter-object position, spatial reconstruction allows each object's actual coordinates to be memorized directly is better [9, 12]. Pertzov reported a new task to directly address a memorized object's identity and location that should reveal object-location binding ability more effectively [13, 14]. In Pertzov’s “What was where” task, participants are asked to identify an object in a memory array between a foil object and one of the presented objects and localize its original location. They calculated the memory accuracy for both the identification-object memory and the localization-location memory and also calculated the object-location binding memory accuracy when both identification and localization were accurate.

It is well known that the ventral stream and the dorsal stream converge in the medial temporal lobe [15]. Lesion or functional brain-imaging studies have shown that the hippocampus is the most important region for object-location binding [16,17,18]. However, another study reported that VWM capacity is intact after damage to the medial temporal lobe structures [19].

In the EEG studies, there are several well-known ERP components correlated with visual information processing. The visual N1 is elicited from the primary visual cortex in the early stage of visual information processing and is posteriorly distributed at 150–200 ms [20,21,22,23,24]. It represents the orienting of attention [20], discrimination mechanisms [21], and object-based selection [22]. Also, there is a posterior N2 (also known as the N2c), and it is one of the negative subcomponents with a near 200-ms latency [25]. The posterior N2 is known to reflect the stimulus classification subprocess and is observed in visual attention paradigms but has not been studied as much as other N2 subcomponents [25,26,27,28].

Based on these results, we assumed that object-location binding load and capacity might be correlated with an earlier occipital stage of visual information processing. Thus, this study aimed to analyze the effect of object-location binding on the VWM workload by analyzing behavioral data and ERP response in the occipital region while performing an object-location binding memory task.

Main text

Material and methods

Participants

The protocol for this study was approved by Hanyang University Institutional Review Board (HYI-17-048-4). Thirty people (15 males, 15 females) aged 19–28 years (mean age: 21.73 ± 2.44 years) were recruited voluntarily. All of the participants were university students who had no neurological problems.

Experimental paradigm

For this study, the "What was where" task was modified to observe the mental load effect, similar to our previous study [29]. Participants performed a modified Pertzov’s paradigm to evaluate object-location binding memory. The task was presented using MATLAB (Natick, Massachusetts: The MathWorks Inc.). In this task, a few fractal objects (130 × 130 pixels) were displayed in random locations on a 53.2 × 29.9 cm (1920 × 1080 pixels) touchscreen monitor for 3–5 s (Fig. 1A). The length of time that objects were presented on the screen increased in proportion to the number of objects. We showed two objects after a slight delay: the displayed fractal object and a foil fractal object. Then, participants had to select one object and drag it to the remembered location using the touchscreen. The total task consisted of three sessions, and each session had 150 trials. Three, four, or five objects were presented in each session. The order of the session was chosen at random. This change in the number of presented objects was carried out to observe the mental load's effect during encoding. Response coordinates were recorded to calculate the error distance and memory accuracy. The correct localization thresholds for location memory and binding memory were determined using a visual angle of 4.5° [13, 14, 29]. The corresponding approximate threshold in pixels was 125 pixels, and it was calculated by considering the viewing distance of 42 cm and the size of the viewing monitor (Fig. 1B).

Fig. 1
figure 1

A Examples of a single trial of task. Trial consists of memory array, delay, test array (identification), and spatial reconstruction (localization). B Measurements of behavioral performance; Object memory, location memory and error distance

Measurement and analysis

To assess each participant's performance, the object memory accuracy was measured using the percentage of correctly identified fractals in the test arrays (Fig. 1B). The accuracy of location memory was determined by the percentage of correctly identified locations of selected fractals. The binding memory accuracy was defined as the number of trials in which participants located a target fractal object in its original location. Swap error ratio was defined as ratio of trials in which participants located a target fractal object in other objects’ location. We analyzed the behavioral performance according to the number of objects in the behavioral data. ANOVA and independent t-test for each pair were performed using with the Statistical Package for the Social Sciences (SPSS) Statistics Release 24 software (IBM, Somers, NY, USA).

The EEG was recorded during the task using a gel type 64-channel 10–20 system EEG cap with a BRAINBOX EEG-1166 amplifier system (Braintronics, Almere, FL, NL). EEG data were sampled at 1024 Hz, and impedances for all electrodes was kept below 10 kΩ. We used active G1/G2 ground reference (G1: FPz, G2: AFz) of BRAINBOX system for ground and reference. Common averaging reference and bandpass filtering (1–30 Hz) were performed. Eye movements and blinks were removed using independent component analysis (ICA) [30]. Then we segmented EEG data of the trial in which both the identity and the location of the object were correctly answered into encoding-lock epochs from -200 ms to 800 ms. Baseline removal and amplitude normalization were performed for each epoch. Contaminated epochs were rejected automatically with FASTER [31] which is fully automated statistical thresholding for the EEG artifact rejection algorithm. These data were used to calculate the encoding-lock ERPs.

Results and discussion

Behavioral performance

As shown in Fig. 2A, in object memory, statistically, significant differences were shown only when three objects were presented and when five objects were presented (p = 0.0014). In location memory, no statistically significant change was observed even when the number of objects increased. In the case of binding memory and swap error ratio, there were statistically significant differences when three objects were presented, and four/five objects were presented (Binding memory: p = 0.0391/p = 0.0032, Swap error ratio: p < 0.0005/ p < 0.0005), but there was no difference between four and five object presentation. As the number of presented objects increases, the error distance between the target and actual response coordinates tended to increase. However, the error distance from the nearest neighborhood did not increase even if the number of objects increased as shown in the Fig. 2C

Fig. 2
figure 2

The mean value and memory accuracy for (A) object memory, (B) location memory and error distance, (C) object-location binding memory and (D) swap error ratio (*p < 0.05, ***p < 0.0005)

ERP

We identified two distinct ERP components on the posterior electrode sites (Fig. 3). Two consecutive negative peaks at N1 (150–200 ms) and N2 (200–240 ms) appeared at parieto-occipital electrode sites. We visually checked single-trial ERPs and confirmed that these were not the same peak with a different latency. The amplitudes of N1 tended to increase when the number of presented objects increased from 3 to 4 (p < 0.0001) and from 4 to 5 (p = 0.0478). The amplitude of N2 increased as the number of presented objects increased from 3 to 4 (p = 0.0286) and from 3 to 5 (p = 0.03). In contrast to N1, when the number of presented objects increased from 4 to 5, a significant change was not observed (p = 0.9325).

Fig. 3
figure 3

A Encoding-lock ERPs on posterior electrode site (POz). The ERPs' values were calculated and illustrated separately for each presented object (green line: 3 objects, red line: 4 objects, blue line: 5 objects). B The mean value and standard error of the mean of the ERP components’ normalized amplitude on POz

Discussion

In the object memory accuracy, there was a significant difference between when three objects were presented and when five objects were presented. However, regardless of the number of objects presented, the object memory accuracy was more than 90 percent under all conditions, which means that only the objects’ identity was memorized, but the VWM capacity limit was not reached even though the object presented in this paradigm was complex.

Location memory accuracy was somewhat less accurate for all set sizes. It can be interpreted that there was a partial loss of location memory in all cases, but the error distance shows that the minimum distance did not increase even if the number of presented objects increased to five. This means that the location memory accuracy was evaluated much lower than actual, because of we set very tight localization threshold to detect binding memory and swap error, and in fact location memory was kept almost constant with no difference in all cases.

Binding memory accuracy and swap error ratio are related to object-location binding memory, and both of their values were low because of the tight localization threshold, which was similar to that of localization memory. They showed a significant difference between 3 and 4 but not between 4 and 5. This pattern is similar to the commonly known VWM capacity limit [2, 4]. Our findings for object memory, location memory, and binding memory indicated that only binding memory ERP components elicited from the presented objects provided more information.

Two consecutive ERP components, N1 and N2 was elicited on parieto-occipital site by the paradigm. The amplitude of N1 increased in proportion to the number of presented objects. It can be said that there was a load effect with an increasing number of presented objects in N1. This result agrees with the consecutive results of studies that have reported perceptual load effects on the N1 [32]. As noted for the behavioral results' object memory accuracy, a capacity limit at the N1 load effect was not found. Based on the similarity of these trends and previous research on visual N1, we think that N1 is related to the object-only load effect. In N2, the amplitude increased when the number of presented objects increased from 3 to 4, but not when it increased from 4 to 5. This change can be interpreted in terms of object-location binding memory accuracy variability with an increase in the number of objects presented from 3 to 4, but not when the number was increased from 4 to 5. This pattern was different from the pattern in which the N1 and object memory accuracy changed with an increasing number of presented objects. Therefore, we assumed that the N2 is an object-location binding dependent component, and its amplitude might reflect the task difficulty of the binding process. And also this N2 related process occurs in the parieto-occipital cite, as early as 200 ms after seeing the objects.

The amplitude of N2 was saturated when four objects were presented and there were no significant changes between four and five objects, as same as binding memory accuracy and swap error ratio. However, the amplitude of N1 and object memory accuracy did not show the same change. This seems to have occurred because the object-only difficulty was easy despite the load effect.

In conclusion, the N1 amplitude is related to the object-only load effect, and the posterior N2 amplitude is a binding-dependent ERP component.

Limitations

In this study, there is only ERP analysis about electroencephalogram without source localization analysis like Low Resolution Electromagnetic Tomography. Source localization analysis might help to figure out each step of object-location binding memory process.

Availability of data and materials

The datasets used and analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

VWM:

Visual working memory

EEG:

Electroencephalography

ERP:

Event-related potentials

ICA:

Independent component analysis

FASTER:

Fully automated statistical thresholding for EEG artifact rejection algorithm

References

  1. Luck SJ, Vogel EK. The capacity of visual working memory for features and conjunctions. Nature. 1997;390(6657):279–81.

    Article  CAS  Google Scholar 

  2. Cowan N. The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci. 2001;24(1):87–114.

    Article  CAS  Google Scholar 

  3. Xu Z, et al. The reliability and stability of visual working memory capacity. Behav Res Methods. 2018;50(2):576–88.

    Article  CAS  Google Scholar 

  4. Wheeler ME, Treisman AM. Binding in short-term visual memory. J Exp Psychol Gen. 2002;131(1):48–64.

    Article  Google Scholar 

  5. Tresch MC, Sinnamon HM, Seamon JG. Double dissociation of spatial and object visual memory: evidence from selective interference in intact human subjects. Neuropsychologia. 1993;31(3):211–9.

    Article  CAS  Google Scholar 

  6. Ettlinger G. “Object vision” and “spatial vision”: the neuropsychological evidence for the distinction. Cortex. 1990;26(3):319–41.

    Article  CAS  Google Scholar 

  7. Hollingworth A, Rasmussen IP. Binding objects to locations: the relationship between object files and visual working memory. J Exp Psychol Hum Percept Perform. 2010;36(3):543–64.

    Article  Google Scholar 

  8. van Hoogmoed AH, van den Brink D, Janzen G. Electrophysiological correlates of object location and object identity processing in spatial scenes. PLoS ONE. 2012;7(7): e41180.

    Article  Google Scholar 

  9. Yang P, et al. The effects of changes in object location on object identity detection: a simultaneous EEG-fMRI study. Neuroimage. 2017;157:351–63.

    Article  Google Scholar 

  10. Li J, et al. Visual working memory for dynamic objects: impaired binding between object feature and location. Vis Cogn. 2015;23(3):357–78.

    Article  Google Scholar 

  11. Keshvari S, van den Berg R, Ma WJ. No evidence for an item limit in change detection. PLoS Comput Biol. 2013;9(2): e1002927.

    Article  CAS  Google Scholar 

  12. Horecka KM, et al. Reconstructing relational information. Hippocampus. 2018;28(2):164–77.

    Article  Google Scholar 

  13. Pertzov Y, et al. Binding deficits in memory following medial temporal lobe damage in patients with voltage-gated potassium channel complex antibody-associated limbic encephalitis. Brain. 2013;136(Pt 8):2474–85.

    Article  Google Scholar 

  14. Pertzov Y, et al. Forgetting what was where: the fragility of object-location binding. PLoS ONE. 2012;7(10): e48214.

    Article  CAS  Google Scholar 

  15. Ungerleider LG, Haxby JV. “What” and “where” in the human brain. Curr Opin Neurobiol. 1994;4(2):157–65.

    Article  CAS  Google Scholar 

  16. Ma Q, et al. Extensive cortical functional connectivity of the human hippocampal memory system. Cortex. 2022;147(83):101.

    Google Scholar 

  17. Crane J, Milner B. What went where? Impaired object-location learning in patients with right hippocampal lesions. Hippocampus. 2005;15(2):216–31.

    Article  Google Scholar 

  18. Hannula DE, Ranganath C. Medial temporal lobe activity predicts successful relational memory binding. J Neurosci. 2008;28(1):116–24.

    Article  CAS  Google Scholar 

  19. Jeneson A, et al. Visual working memory capacity and the medial temporal lobe. J Neurosci. 2012;32(10):3584–9.

    Article  CAS  Google Scholar 

  20. Luck SJ, et al. Visual event-related potentials index focused attention within bilateral stimulus arrays. II. Functional dissociation of P1 and N1 components. Electroencephalogr Clin Neurophysiol. 1990;75(6):528–42.

    Article  CAS  Google Scholar 

  21. Vogel EK, Luck SJ. The visual N1 component as an index of a discrimination process. Psychophysiology. 2000;37(2):190–203.

    Article  CAS  Google Scholar 

  22. He X, et al. Cue validity and object-based attention. J Cogn Neurosci. 2004;16(6):1085–97.

    Article  Google Scholar 

  23. Clark VP, Fan S, Hillyard SA. Identification of early visual evoked potential generators by retinotopic and topographic analyses. Hum Brain Mapp. 1994;2(3):170–87.

    Article  Google Scholar 

  24. Vazquez-Marrufo M, et al. Reliability analysis of individual visual P1 and N1 maps indicates the heterogeneous topographies involved in early visual processing among human subjects. Behav Brain Res. 2021;397: 112930.

    Article  Google Scholar 

  25. Folstein JR, Van Petten C. Influence of cognitive control and mismatch on the N2 component of the ERP: a review. Psychophysiology. 2008;45(1):152–70.

    PubMed  Google Scholar 

  26. Luck SJ. An introduction to the event-related potential technique. Cambridge, Mass: MIT Press; 2005.

    Google Scholar 

  27. Renault B, et al. Onset and offset of brain events as indices of mental chronometry. Science. 1982;215(4538):1413–5.

    Article  CAS  Google Scholar 

  28. Kostyrka-Allchorne K, et al. The Short-term effect of video editing pace on children’s inhibition and N2 and P3 ERP components during visual Go/No-Go task. Dev Neuropsychol. 2019;44(4):385–96.

    Article  Google Scholar 

  29. Park J, et al. Sex differences of cognitive load effects on object-location binding memory. Biomed Eng Lett. 2017;7(4):305–9.

    Article  Google Scholar 

  30. Jung TP, et al. Removing electroencephalographic artifacts by blind source separation. Psychophysiology. 2000;37(2):163–78.

    Article  CAS  Google Scholar 

  31. Nolan H, Whelan R, Reilly RB. FASTER: fully automated statistical thresholding for EEG artifact rejection. J Neurosci Methods. 2010;192(1):152–62.

    Article  CAS  Google Scholar 

  32. Luo YJ, Greenwood PM, Parasuraman R. Dynamics of the spatial scale of visual attention revealed by brain event-related potentials. Brain Res Cogn Brain Res. 2001;12(3):371–81.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This research was supported by the Brain Convergence Research Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. NRF-2021M3E5D2A01022391).

Funding

This research was supported by the Brain Convergence Research Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. NRF-2021M3E5D2A01022391).

Author information

Authors and Affiliations

Authors

Contributions

Solwoong Song: Software, Formal analysis, Investigation, Data Curation, Writing—Original Draft. Jinsick Park: Conceptualization, Methodology, Software, Writing—Review & Editing. Young Min Park: Software, Investigation, Validation, Data Curation. In Young Kim: Resources, Supervision, Project administration. Dong Pyo Jang: Validation, Investigation, Supervision, Funding acquisition, Project administration, Writing—Review & Editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Dong Pyo Jang.

Ethics declarations

Ethics approval and consent to participate

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Hanyang University Institutional Review Board (HYI-17-048-4). Written consent was obtained from all individual participants included in the study.

Consent for publication

It was obtained written from all individual participants included in the study. This manuscript doesn’t contain any individual person’s data in any form.

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Song, S., Park, J., Park, Y.M. et al. The influence of object-location binding mental load effects on the visual N1 and N2 Event-related Potentials. BMC Res Notes 15, 217 (2022). https://doi.org/10.1186/s13104-022-06086-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13104-022-06086-0

Keywords

  • EEG
  • ERP
  • Object-location binding memory
  • N2