- Research note
- Open Access
The influence of object-location binding mental load effects on the visual N1 and N2 Event-related Potentials
BMC Research Notes volume 15, Article number: 217 (2022)
This study aimed to analyze the effect of object-location binding on the visual working memory workload. For this study, thirty healthy subjects were recruited, and they performed the “What was where” task, which was modified to evaluated object-location binding memory. We analyzed their ERP and behavior response.
Object memory and location memory were preserved during the task, but binding memory decreased significantly when more than four objects were presented. These results indicate that the N1 amplitude is related to the object-only load effect, and the posterior N2 amplitude is a binding-dependent ERP component.
VWM is essential for perception and information processing in the real visual world. VWM has a limited capacity determined by visual feature binding, not by the number of individual features, and is generally known to comprise three or four objects [1,2,3]. This binding memory function enables visual information processing even with a limitative VWM capacity. The effect of feature bindings on capacity has been investigated in several psychological studies. Feature bindings represent an object's identity-related features, such as color, shape, and orientation .However, the binding of object features and object location on VWM capacity has not been extensively studied. The VWM functional pathways can be divided into ventral stream features and dorsal stream features related to object identity and location [5,6,7]. Because object features and object location information are handled differently in the ventral and dorsal streams, the binding between the object features and the object location could be approached differently.
Most existing studies on object-location binding and VWM capacity have used change detection tasks [8,9,10]. They assessed object-location binding ability by how the ability to sense changes in whole-display or single-probe varied depending on how many objects were presented. However, it has been questioned whether the VWM load and capacity assessment using the change detection task is accurate . To prevent location information from being encoded in an abstract configuration, such as a relative inter-object position, spatial reconstruction allows each object's actual coordinates to be memorized directly is better [9, 12]. Pertzov reported a new task to directly address a memorized object's identity and location that should reveal object-location binding ability more effectively [13, 14]. In Pertzov’s “What was where” task, participants are asked to identify an object in a memory array between a foil object and one of the presented objects and localize its original location. They calculated the memory accuracy for both the identification-object memory and the localization-location memory and also calculated the object-location binding memory accuracy when both identification and localization were accurate.
It is well known that the ventral stream and the dorsal stream converge in the medial temporal lobe . Lesion or functional brain-imaging studies have shown that the hippocampus is the most important region for object-location binding [16,17,18]. However, another study reported that VWM capacity is intact after damage to the medial temporal lobe structures .
In the EEG studies, there are several well-known ERP components correlated with visual information processing. The visual N1 is elicited from the primary visual cortex in the early stage of visual information processing and is posteriorly distributed at 150–200 ms [20,21,22,23,24]. It represents the orienting of attention , discrimination mechanisms , and object-based selection . Also, there is a posterior N2 (also known as the N2c), and it is one of the negative subcomponents with a near 200-ms latency . The posterior N2 is known to reflect the stimulus classification subprocess and is observed in visual attention paradigms but has not been studied as much as other N2 subcomponents [25,26,27,28].
Based on these results, we assumed that object-location binding load and capacity might be correlated with an earlier occipital stage of visual information processing. Thus, this study aimed to analyze the effect of object-location binding on the VWM workload by analyzing behavioral data and ERP response in the occipital region while performing an object-location binding memory task.
Material and methods
The protocol for this study was approved by Hanyang University Institutional Review Board (HYI-17-048-4). Thirty people (15 males, 15 females) aged 19–28 years (mean age: 21.73 ± 2.44 years) were recruited voluntarily. All of the participants were university students who had no neurological problems.
For this study, the "What was where" task was modified to observe the mental load effect, similar to our previous study . Participants performed a modified Pertzov’s paradigm to evaluate object-location binding memory. The task was presented using MATLAB (Natick, Massachusetts: The MathWorks Inc.). In this task, a few fractal objects (130 × 130 pixels) were displayed in random locations on a 53.2 × 29.9 cm (1920 × 1080 pixels) touchscreen monitor for 3–5 s (Fig. 1A). The length of time that objects were presented on the screen increased in proportion to the number of objects. We showed two objects after a slight delay: the displayed fractal object and a foil fractal object. Then, participants had to select one object and drag it to the remembered location using the touchscreen. The total task consisted of three sessions, and each session had 150 trials. Three, four, or five objects were presented in each session. The order of the session was chosen at random. This change in the number of presented objects was carried out to observe the mental load's effect during encoding. Response coordinates were recorded to calculate the error distance and memory accuracy. The correct localization thresholds for location memory and binding memory were determined using a visual angle of 4.5° [13, 14, 29]. The corresponding approximate threshold in pixels was 125 pixels, and it was calculated by considering the viewing distance of 42 cm and the size of the viewing monitor (Fig. 1B).
Measurement and analysis
To assess each participant's performance, the object memory accuracy was measured using the percentage of correctly identified fractals in the test arrays (Fig. 1B). The accuracy of location memory was determined by the percentage of correctly identified locations of selected fractals. The binding memory accuracy was defined as the number of trials in which participants located a target fractal object in its original location. Swap error ratio was defined as ratio of trials in which participants located a target fractal object in other objects’ location. We analyzed the behavioral performance according to the number of objects in the behavioral data. ANOVA and independent t-test for each pair were performed using with the Statistical Package for the Social Sciences (SPSS) Statistics Release 24 software (IBM, Somers, NY, USA).
The EEG was recorded during the task using a gel type 64-channel 10–20 system EEG cap with a BRAINBOX EEG-1166 amplifier system (Braintronics, Almere, FL, NL). EEG data were sampled at 1024 Hz, and impedances for all electrodes was kept below 10 kΩ. We used active G1/G2 ground reference (G1: FPz, G2: AFz) of BRAINBOX system for ground and reference. Common averaging reference and bandpass filtering (1–30 Hz) were performed. Eye movements and blinks were removed using independent component analysis (ICA) . Then we segmented EEG data of the trial in which both the identity and the location of the object were correctly answered into encoding-lock epochs from -200 ms to 800 ms. Baseline removal and amplitude normalization were performed for each epoch. Contaminated epochs were rejected automatically with FASTER  which is fully automated statistical thresholding for the EEG artifact rejection algorithm. These data were used to calculate the encoding-lock ERPs.
Results and discussion
As shown in Fig. 2A, in object memory, statistically, significant differences were shown only when three objects were presented and when five objects were presented (p = 0.0014). In location memory, no statistically significant change was observed even when the number of objects increased. In the case of binding memory and swap error ratio, there were statistically significant differences when three objects were presented, and four/five objects were presented (Binding memory: p = 0.0391/p = 0.0032, Swap error ratio: p < 0.0005/ p < 0.0005), but there was no difference between four and five object presentation. As the number of presented objects increases, the error distance between the target and actual response coordinates tended to increase. However, the error distance from the nearest neighborhood did not increase even if the number of objects increased as shown in the Fig. 2C
We identified two distinct ERP components on the posterior electrode sites (Fig. 3). Two consecutive negative peaks at N1 (150–200 ms) and N2 (200–240 ms) appeared at parieto-occipital electrode sites. We visually checked single-trial ERPs and confirmed that these were not the same peak with a different latency. The amplitudes of N1 tended to increase when the number of presented objects increased from 3 to 4 (p < 0.0001) and from 4 to 5 (p = 0.0478). The amplitude of N2 increased as the number of presented objects increased from 3 to 4 (p = 0.0286) and from 3 to 5 (p = 0.03). In contrast to N1, when the number of presented objects increased from 4 to 5, a significant change was not observed (p = 0.9325).
In the object memory accuracy, there was a significant difference between when three objects were presented and when five objects were presented. However, regardless of the number of objects presented, the object memory accuracy was more than 90 percent under all conditions, which means that only the objects’ identity was memorized, but the VWM capacity limit was not reached even though the object presented in this paradigm was complex.
Location memory accuracy was somewhat less accurate for all set sizes. It can be interpreted that there was a partial loss of location memory in all cases, but the error distance shows that the minimum distance did not increase even if the number of presented objects increased to five. This means that the location memory accuracy was evaluated much lower than actual, because of we set very tight localization threshold to detect binding memory and swap error, and in fact location memory was kept almost constant with no difference in all cases.
Binding memory accuracy and swap error ratio are related to object-location binding memory, and both of their values were low because of the tight localization threshold, which was similar to that of localization memory. They showed a significant difference between 3 and 4 but not between 4 and 5. This pattern is similar to the commonly known VWM capacity limit [2, 4]. Our findings for object memory, location memory, and binding memory indicated that only binding memory ERP components elicited from the presented objects provided more information.
Two consecutive ERP components, N1 and N2 was elicited on parieto-occipital site by the paradigm. The amplitude of N1 increased in proportion to the number of presented objects. It can be said that there was a load effect with an increasing number of presented objects in N1. This result agrees with the consecutive results of studies that have reported perceptual load effects on the N1 . As noted for the behavioral results' object memory accuracy, a capacity limit at the N1 load effect was not found. Based on the similarity of these trends and previous research on visual N1, we think that N1 is related to the object-only load effect. In N2, the amplitude increased when the number of presented objects increased from 3 to 4, but not when it increased from 4 to 5. This change can be interpreted in terms of object-location binding memory accuracy variability with an increase in the number of objects presented from 3 to 4, but not when the number was increased from 4 to 5. This pattern was different from the pattern in which the N1 and object memory accuracy changed with an increasing number of presented objects. Therefore, we assumed that the N2 is an object-location binding dependent component, and its amplitude might reflect the task difficulty of the binding process. And also this N2 related process occurs in the parieto-occipital cite, as early as 200 ms after seeing the objects.
The amplitude of N2 was saturated when four objects were presented and there were no significant changes between four and five objects, as same as binding memory accuracy and swap error ratio. However, the amplitude of N1 and object memory accuracy did not show the same change. This seems to have occurred because the object-only difficulty was easy despite the load effect.
In conclusion, the N1 amplitude is related to the object-only load effect, and the posterior N2 amplitude is a binding-dependent ERP component.
In this study, there is only ERP analysis about electroencephalogram without source localization analysis like Low Resolution Electromagnetic Tomography. Source localization analysis might help to figure out each step of object-location binding memory process.
Availability of data and materials
The datasets used and analysed during the current study are available from the corresponding author on reasonable request.
Visual working memory
Independent component analysis
Fully automated statistical thresholding for EEG artifact rejection algorithm
Luck SJ, Vogel EK. The capacity of visual working memory for features and conjunctions. Nature. 1997;390(6657):279–81.
Cowan N. The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav Brain Sci. 2001;24(1):87–114.
Xu Z, et al. The reliability and stability of visual working memory capacity. Behav Res Methods. 2018;50(2):576–88.
Wheeler ME, Treisman AM. Binding in short-term visual memory. J Exp Psychol Gen. 2002;131(1):48–64.
Tresch MC, Sinnamon HM, Seamon JG. Double dissociation of spatial and object visual memory: evidence from selective interference in intact human subjects. Neuropsychologia. 1993;31(3):211–9.
Ettlinger G. “Object vision” and “spatial vision”: the neuropsychological evidence for the distinction. Cortex. 1990;26(3):319–41.
Hollingworth A, Rasmussen IP. Binding objects to locations: the relationship between object files and visual working memory. J Exp Psychol Hum Percept Perform. 2010;36(3):543–64.
van Hoogmoed AH, van den Brink D, Janzen G. Electrophysiological correlates of object location and object identity processing in spatial scenes. PLoS ONE. 2012;7(7): e41180.
Yang P, et al. The effects of changes in object location on object identity detection: a simultaneous EEG-fMRI study. Neuroimage. 2017;157:351–63.
Li J, et al. Visual working memory for dynamic objects: impaired binding between object feature and location. Vis Cogn. 2015;23(3):357–78.
Keshvari S, van den Berg R, Ma WJ. No evidence for an item limit in change detection. PLoS Comput Biol. 2013;9(2): e1002927.
Horecka KM, et al. Reconstructing relational information. Hippocampus. 2018;28(2):164–77.
Pertzov Y, et al. Binding deficits in memory following medial temporal lobe damage in patients with voltage-gated potassium channel complex antibody-associated limbic encephalitis. Brain. 2013;136(Pt 8):2474–85.
Pertzov Y, et al. Forgetting what was where: the fragility of object-location binding. PLoS ONE. 2012;7(10): e48214.
Ungerleider LG, Haxby JV. “What” and “where” in the human brain. Curr Opin Neurobiol. 1994;4(2):157–65.
Ma Q, et al. Extensive cortical functional connectivity of the human hippocampal memory system. Cortex. 2022;147(83):101.
Crane J, Milner B. What went where? Impaired object-location learning in patients with right hippocampal lesions. Hippocampus. 2005;15(2):216–31.
Hannula DE, Ranganath C. Medial temporal lobe activity predicts successful relational memory binding. J Neurosci. 2008;28(1):116–24.
Jeneson A, et al. Visual working memory capacity and the medial temporal lobe. J Neurosci. 2012;32(10):3584–9.
Luck SJ, et al. Visual event-related potentials index focused attention within bilateral stimulus arrays. II. Functional dissociation of P1 and N1 components. Electroencephalogr Clin Neurophysiol. 1990;75(6):528–42.
Vogel EK, Luck SJ. The visual N1 component as an index of a discrimination process. Psychophysiology. 2000;37(2):190–203.
He X, et al. Cue validity and object-based attention. J Cogn Neurosci. 2004;16(6):1085–97.
Clark VP, Fan S, Hillyard SA. Identification of early visual evoked potential generators by retinotopic and topographic analyses. Hum Brain Mapp. 1994;2(3):170–87.
Vazquez-Marrufo M, et al. Reliability analysis of individual visual P1 and N1 maps indicates the heterogeneous topographies involved in early visual processing among human subjects. Behav Brain Res. 2021;397: 112930.
Folstein JR, Van Petten C. Influence of cognitive control and mismatch on the N2 component of the ERP: a review. Psychophysiology. 2008;45(1):152–70.
Luck SJ. An introduction to the event-related potential technique. Cambridge, Mass: MIT Press; 2005.
Renault B, et al. Onset and offset of brain events as indices of mental chronometry. Science. 1982;215(4538):1413–5.
Kostyrka-Allchorne K, et al. The Short-term effect of video editing pace on children’s inhibition and N2 and P3 ERP components during visual Go/No-Go task. Dev Neuropsychol. 2019;44(4):385–96.
Park J, et al. Sex differences of cognitive load effects on object-location binding memory. Biomed Eng Lett. 2017;7(4):305–9.
Jung TP, et al. Removing electroencephalographic artifacts by blind source separation. Psychophysiology. 2000;37(2):163–78.
Nolan H, Whelan R, Reilly RB. FASTER: fully automated statistical thresholding for EEG artifact rejection. J Neurosci Methods. 2010;192(1):152–62.
Luo YJ, Greenwood PM, Parasuraman R. Dynamics of the spatial scale of visual attention revealed by brain event-related potentials. Brain Res Cogn Brain Res. 2001;12(3):371–81.
This research was supported by the Brain Convergence Research Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. NRF-2021M3E5D2A01022391).
This research was supported by the Brain Convergence Research Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. NRF-2021M3E5D2A01022391).
Ethics approval and consent to participate
This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Hanyang University Institutional Review Board (HYI-17-048-4). Written consent was obtained from all individual participants included in the study.
Consent for publication
It was obtained written from all individual participants included in the study. This manuscript doesn’t contain any individual person’s data in any form.
The authors have no relevant financial or non-financial interests to disclose.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Song, S., Park, J., Park, Y.M. et al. The influence of object-location binding mental load effects on the visual N1 and N2 Event-related Potentials. BMC Res Notes 15, 217 (2022). https://doi.org/10.1186/s13104-022-06086-0
- Object-location binding memory