Decoding Meaning from Mind: A Comprehensive Guide to Semantic Neural Decoding with Simultaneous EEG-fNIRS

Abstract

This article provides a comprehensive exploration of semantic neural decoding using simultaneous electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). It covers the foundational principles of how these complementary non-invasive neuroimaging modalities capture the brain's electrical and hemodynamic responses during semantic processing. The scope extends to methodological frameworks for data acquisition and analysis, practical troubleshooting for optimizing signal quality, and a critical validation of EEG-fNIRS against other neuroimaging techniques. Tailored for researchers, scientists, and drug development professionals, this review synthesizes current advancements and practical insights to propel the development of brain-computer interfaces and novel clinical biomarkers for neurological disorders.

The Neural Basis of Semantic Decoding: Unraveling How EEG and fNIRS Capture Meaning

Defining Semantic Neural Decoding and Its Potential for Next-Generation Brain-Computer Interfaces

Application Notes: Core Principles and Quantitative Landscape

Semantic Neural Decoding is defined as the computational process of reconstructing or classifying the meaning of a subject's perceptual or imagined experiences from brain activity, as opposed to decoding acoustic or motor features of speech [1] [2]. This approach is foundational for developing intuitive Brain-Computer Interfaces (BCIs) that can directly interpret a user's intended communication. The core principle involves mapping non-invasive brain signals onto a semantic feature space, which is then translated into continuous language using a generative language model [1]. This methodology allows the decoded output to capture the gist of the stimulus, even when the exact words are not recovered, enabling applications such as decoding perceived speech, imagined speech, and the meaning inferred from silent videos with a single decoder [1].

The table below summarizes key performance metrics from recent seminal studies in the field, providing a benchmark for current capabilities.

Table 1: Performance Metrics of Non-Invasive Semantic Decoding Approaches

Study / Reference	Neural Modality	Stimulus/Task	Key Performance Metric	Reported Value
Tang et al. (fMRI) [1]	fMRI	Listening to narratives	BERTScore (vs. chance)	~0.812 vs. ~0.790 (null)
			Word Error Rate (WER)	0.924 - 0.941
			Reading Comprehension (from decoded text)	9/16 questions answered correctly
Nature Communications (2025) [3]	EEG/MEG	Reading/Listening to sentences	Top-10 Accuracy (Single-trial, 250-word set)	Up to 37%
			Top-10 Accuracy (Averaged trials)	~80% (after 8-trial average)
Lee et al. (EEG) [2]	EEG	Imagined Speech & Visual Imagery	13-class Classification Accuracy	~40%

The performance of semantic decoders is highly dependent on the experimental protocol and data parameters. The following table outlines the impact of these factors, which must be considered when designing BCI experiments.

Table 2: Impact of Experimental Protocol and Data Scaling on Decoding Performance

Factor	Impact on Decoding Performance	Key Findings
Perception Modality [3]	Reading > Listening	Significantly better decoding for reading vs. listening with identical stimuli (p < 10⁻¹⁶).
Recording Device [3]	MEG > EEG	MEG's higher signal-to-noise ratio yields superior performance compared to EEG.
Training Data Volume [1] [3]	Positive Log-Linear Correlation	Performance steadily increases with more training data, highlighting scalability.
Test-Time Averaging [3]	Positive Log-Linear Correlation	Averaging predictions from multiple trials of the same word can double performance (e.g., from ~37% to ~80% top-10 accuracy).
Subject Cooperation [1]	Required	Successful decoding requires subject cooperation for both training and applying the decoder, a key finding for mental privacy.

Experimental Protocols for Multimodal EEG-fNIRS Semantic Decoding

This section provides a detailed methodology for a core experiment aimed at decoding imagined speech using simultaneous EEG-fNIRS, a multimodal approach that leverages the complementary strengths of both techniques [4] [5].

Protocol: Multimodal Decoding of Imagined Speech

Objective: To acquire a synchronized dataset of electrophysiological (EEG) and hemodynamic (fNIRS) brain activity during imagined speech for training and testing a semantic neural decoder.

Experimental Workflow:

The end-to-end process for acquiring and processing multimodal data for semantic decoding is outlined below.

Detailed Methodology:

• Participants: 20-60 healthy adult participants. Exclude for history of neurological disorder or concussion within 12 months. Secure informed consent and IRB approval [4].
• Stimuli & Paradigm:
  • Stimulus Set: 10-15 words/phrases critical for patient communication (e.g., "yes," "no," "water," "help") [2]. The set can be expanded to include concrete and abstract concepts to probe their differential impact on decoding [2].
  • Task Design: A trial-structured, event-related paradigm is used.
    • Cue (2s): A word/phrase is presented visually (text) or audibly.
    • Overt Repetition (3s): Participant says the word aloud. This serves as a baseline for localizing speech-related networks.
    • Imagined Speech (3s): Participant imagines saying the word without any articulation or movement.
    • Rest (10-15s): A cross-hair is displayed to allow hemodynamic responses to return to baseline [1] [4].
  • Each word should be repeated across 20-30 trials to allow for test-time averaging, which significantly boosts performance [3].
• Simultaneous Data Acquisition:
  • EEG Recording: Use a high-density (e.g., 128-channel) EEG system with a sampling rate ≥ 500 Hz. The system should be synchronized with the fNIRS device and stimulus presentation computer.
  • fNIRS Recording: Use a continuous-wave (CW) fNIRS system. Configure optodes over brain regions critical for semantic processing:
    • Prefrontal Cortex: Involved in high-level semantic integration [1].
    • Temporal Cortex (including Wernicke's area): Central to language comprehension [1] [6].
    • Inferior Frontal Gyrus (including Broca's area): Key for speech production and imagery [2] [6].
    • Inferior Parietal Lobule: Part of the Action Observation Network (AON), active during motor imagery and observation [4].
  • Optodes should be embedded within the EEG cap. Use multiple source-detector distances (e.g., 1.5 cm and 3.0 cm) to separate cortical from extracerebral hemodynamic signals [5]. Record changes in oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR).
• Data Preprocessing:
  • EEG Preprocessing: Apply band-pass filtering (e.g., 0.5-40 Hz), remove line noise, and reject artifacts (e.g., ocular, muscle). Perform Independent Component Analysis (ICA) to remove heart and blink artifacts. Epoche data around stimulus and task onsets.
  • fNIRS Preprocessing: Convert raw light intensity to optical density, then to HbO and HbR concentration changes using the Modified Beer-Lambert Law. Apply a band-pass filter (e.g., 0.01-0.2 Hz) to remove physiological noise (cardiac, respiratory) and slow drifts. Epoch the hemodynamic data.
• Multimodal Data Fusion and Feature Extraction:
  • Feature Extraction: For EEG, extract spectral power features in key frequency bands (e.g., theta, alpha, beta, gamma) from relevant epochs, as high-frequency bands can be particularly informative [2]. For fNIRS, use the mean HbO concentration during the task window as the primary feature.
  • Fusion Method: Employ Structured Sparse Multiset Canonical Correlation Analysis (ssmCCA) to fuse the preprocessed EEG and fNIRS data [4]. This method identifies components that are maximally correlated across the two modalities, effectively pinpointing brain regions where both electrical and hemodynamic activity are consistently detected during the imagined speech task. This fused neural signature serves as a robust input for the decoder.

Protocol: Decoder Architecture and Training

Objective: To reconstruct the semantic content of the imagined speech from the fused EEG-fNIRS features.

Decoder Workflow and Signaling Pathway:

The following diagram illustrates the core signaling pathway from multimodal brain data to reconstructed language.

Detailed Methodology:

• Semantic Feature Extraction: Represent each target word/phrase from the stimulus set as a numerical vector using a pre-trained model (e.g., BERT, GloVe). These vectors encapsulate the semantic meaning of the words [1].
• Training the Encoding Model:
  • The goal is to learn a mapping from the fused neural features (from ssmCCA) to the semantic feature vectors.
  • A model (e.g., ridge regression or a deep neural network with a subject-specific layer [3]) is trained on the trial data. The input is the neural feature vector from a trial, and the target is the semantic vector of the corresponding word.
  • This model learns to predict "how the subject's brain would represent the meaning" of any given word sequence [1].
• Decoding with a Language Model:
  • During inference (on held-out test data), the goal is to find the most likely word sequence given the novel brain recordings.
  • A beam search algorithm is used [1]. The process is as follows:
    • The decoder maintains a small set (the "beam") of the most likely candidate word sequences.
    • At each step, the language model proposes plausible next words for each candidate.
    • The encoding model scores how well the predicted semantic representation of each new candidate sequence matches the actual recorded brain responses.
    • Only the top-k scoring sequences are retained for the next iteration.
  • This iterative process efficiently narrows the vast space of possible word sequences to those that are both semantically consistent with the brain data and grammatically coherent according to the language model.

The Scientist's Toolkit: Research Reagent Solutions

The following table details essential materials, computational tools, and analytical methods required for implementing the protocols described above.

Table 3: Essential Research Tools for Multimodal Semantic Decoding

Category / Item	Function / Application Note
Hardware
High-Density EEG System (128ch+)	Captures millisecond-scale electrical brain dynamics. Critical for analyzing spectral features in high-frequency bands linked to speech imagery [2] [6].
Continuous-Wave (CW) fNIRS System	Measures hemodynamic (HbO/HbR) responses in cortical areas with better spatial resolution than EEG and tolerance for movement [4] [5].
Integrated EEG-fNIRS Cap	Allows simultaneous data acquisition with co-registered optodes and electrodes, essential for data fusion [4] [5].
Software & Computational Models
Structured Sparse Multiset CCA (ssmCCA)	Core algorithm for fusing EEG and fNIRS data to find components with consistent activity across modalities [4].
Pre-trained Language Model (e.g., GPT-2, BERT)	Provides generative capability and linguistic constraints. BERT is also used for evaluating semantic similarity via BERTScore [1].
Semantic Feature Model (e.g., BERT, GloVe)	Converts words into numerical vectors that represent their meaning, forming the target for the encoding model [1].
Deep Learning Pipeline (e.g., Transformer-based)	State-of-the-art for decoding individual words from M/EEG; can be adapted for fused features. Includes subject-specific layers for cross-participant training [3].
Analytical & Experimental
Beam Search Algorithm	An efficient search algorithm that, guided by the language and encoding models, generates the most likely word sequences from brain data [1].
Imagined Speech Paradigm	Experimental protocol where users mentally articulate words without movement. A key intuitive BCI paradigm for direct communication [7] [2] [6].
Action Observation Network (AON) Localizer	A task involving motor execution, observation, and imagery to identify brain regions (e.g., IPL, SPL) that are also engaged in speech imagery, informing optode placement [4].

The human brain processes complex semantic information—such as differentiating between imagined animals and tools—on a millisecond timescale. Electroencephalography (EEG) provides the temporal resolution necessary to capture these rapid neural dynamics, offering a window into the electrical signatures of thought itself. When integrated with functional near-infrared spectroscopy (fNIRS), which tracks slower hemodynamic responses, these modalities create a comprehensive picture of brain activity across multiple physiological domains [8] [9]. This multimodal approach is particularly valuable for semantic decoding research, which aims to identify the specific concepts an individual is processing based solely on neural signals [8].

The fundamental electrical activity measured by EEG originates primarily from cortical pyramidal neurons, where synchronized postsynaptic potentials from thousands of coherently firing neurons generate electrical signals detectable at the scalp [9] [10]. These signals represent large-scale neural oscillatory activity across characteristic frequency bands including theta (4-7 Hz), alpha (8-14 Hz), beta (15-25 Hz), and gamma (>25 Hz) rhythms [9]. Each frequency band carries information about different aspects of neuronal processing, with gamma oscillations, for instance, being implicated in distinguishing between true and false memories [11].

Semantic decoding for brain-computer interfaces (BCIs) represents a paradigm shift from current character-by-character spelling systems toward direct communication of conceptual meaning [8]. This application demands the precise temporal tracking that only EEG can provide, enabling researchers to follow the rapid sequence of neural events as the brain processes categories like animals versus tools during mental imagery tasks [8].

Table 1: Neural Oscillation Bands and Their Functional Correlates in Semantic Processing

Frequency Band	Range (Hz)	Primary Functional Correlates in Semantic Processing
Theta	4-7	Memory encoding, semantic retrieval
Alpha	8-14	Inhibition of task-irrelevant regions
Beta	15-25	Sensorimotor integration, categorical processing
Gamma	>25	Feature binding, memory distinction, concept integration

Technical Principles of EEG for Capturing Neural Dynamics

EEG operates on the principle of differential amplification, recording voltage differences between active exploring electrodes and reference sites placed on the scalp [10]. When the active electrode is more negative than the reference, the EEG potential deflects upward, while the opposite polarity produces a downward deflection [10]. This measurement approach enables EEG to achieve millisecond-level temporal precision, far exceeding the temporal resolution of hemodynamic-based neuroimaging methods [9].

The electrical signals of cerebral origin must pass through multiple biological layers—including the brain itself, cerebrospinal fluid, meninges, skull, and skin—before reaching recording electrodes on the scalp surface [10]. This biological filtering attenuates signal amplitude and spreads EEG activity more widely than its original source [10]. A significant challenge in EEG interpretation is that cerebral activity may be overwhelmed by other biological electrical sources, including scalp muscles, eyes, tongue, and heart, which can generate massive voltage potentials that obscure neural signals [10].

Despite these limitations, technological and analytical advances continue to enhance EEG's utility for semantic decoding research. The application of digital filters can reduce artifacts, though they must be used cautiously as they may also distort EEG activity of interest [10]. The portability, non-invasiveness, and relatively low cost of EEG systems make them particularly suitable for BCI applications where ecological validity and practical implementation are important considerations [8] [9].

Table 2: Technical Advantages and Limitations of EEG for Semantic Decoding Research

Parameter	Advantages	Limitations
Temporal Resolution	Millisecond precision, ideal for tracking neural dynamics of thought	Limited by neurovascular coupling in metabolic response correlation
Spatial Resolution	~2 cm resolution, adequate for cortical localization	Signals attenuated and spread by biological tissues
Practical Implementation	Portable, cost-effective, suitable for real-world BCI applications	Vulnerable to motion artifacts and biological interference
Signal Origin	Direct measurement of neuronal electrical activity	Requires synchronization of large neuronal populations for detection

Integrated EEG-fNIRS Approaches for Semantic Decoding

The combination of EEG and fNIRS leverages their complementary strengths for semantic decoding research. While EEG provides exquisite temporal resolution for tracking the rapid sequence of neural events during thought processes, fNIRS offers better spatial resolution and measures the hemodynamic response coupled to neural activity [9] [4]. This multimodal approach is grounded in the physiological phenomenon of neurovascular coupling, where neural activity triggers localized increases in cerebral blood flow to meet metabolic demands, resulting in measurable hemoglobin concentration changes [9].

Simultaneous EEG-fNIRS recordings have demonstrated particular utility for investigating the neural basis of cognitive-motor processes including motor execution, observation, and imagery [4]. These processes share similarities with semantic imagery tasks, as all involve internal representations without immediate external stimuli. Research has shown that while unimodal analyses reveal differentiated activation between conditions, the activated regions don't fully overlap across EEG and fNIRS modalities [4]. However, when data is fused using advanced analytical techniques like structured sparse multiset Canonical Correlation Analysis (ssmCCA), consistent activation patterns emerge across modalities, particularly in parietal regions associated with the Action Observation Network [4].

For semantic decoding specifically, simultaneous EEG-fNIRS has been employed to differentiate between semantic categories such as animals and tools during silent naming and sensory-based imagery tasks [8]. This approach capitalizes on the fact that fNIRS can address EEG's spatial limitations while EEG compensates for fNIRS's poor temporal resolution, together offering a synergistic method for improving the ecological validity and practicality of semantic BCIs [8].

Experimental Protocols for Semantic Decoding Using EEG-fNIRS

Participant Selection and Preparation

For semantic decoding studies involving silent naming tasks, participants should be native speakers of the target language to minimize variability in neural representation of semantic concepts caused by language differences [8]. In studies focusing solely on sensory imagery tasks without verbal components, native language fluency may be less critical [8]. A standardized screening process should assess handedness, visual acuity (normal or corrected-to-normal), and history of neurological or psychiatric conditions that might confound results.

During preparation, participants should be fitted with a compatible EEG-fNIRS cap system, with electrode and optode placement following the international 10-20 system or similar standardized positioning [4] [12]. fNIRS optodes should be digitized in reference to anatomical landmarks (nasion, inion, preauricular points) using a 3D magnetic space digitizer to account for individual variations in head size and cap positioning [4]. Inter-optode spacing typically ranges from 2.16-3.26 cm, with a mean of approximately 2.88 cm [4].

Stimulus Presentation and Mental Tasks

The experimental protocol should employ carefully selected stimuli representing the semantic categories of interest. For animal-tool differentiation studies, researchers have successfully used 18 animals and 18 tools selected for their suitability across multiple mental tasks and general recognizability [8]. Images should be converted to grayscale, cropped, resized consistently (e.g., 400 × 400 pixels), contrast-stretched, and presented against a neutral background to minimize low-level visual confounds [8].

Participants should perform multiple mental tasks in randomized order across blocks:

• Silent Naming: Participants silently name the displayed object in their mother tongue without vocalization [8]
• Visual Imagery: Participants visualize the object in their minds, focusing on their own mental representation rather than the specific image presented [8]
• Auditory Imagery: Participants imagine sounds associated with the object (e.g., animal sounds or tool noises) [8]
• Tactile Imagery: Participants imagine the feeling of touching the object (e.g., petting an animal or gripping a tool) [8]

Before the experiment, researchers should provide clear descriptions and examples of each mental task, while encouraging participants to use imagery strategies that feel most natural to them [8]. To minimize movement artifacts, participants should be instructed to remain engaged for the full task duration (typically 3-5 seconds) while minimizing eye movements, facial expressions, and head or body motions [8].

Data Acquisition Parameters

For simultaneous EEG-fNIRS acquisition, researchers should employ a continuous-wave fNIRS system measuring at least two wavelengths in the near-infrared region (typically 695 nm and 830 nm) at a sampling rate of ≥10 Hz to capture hemodynamic responses [4]. The EEG system should record from a sufficient number of channels (e.g., 24-128 electrodes) at a minimum sampling rate of 500 Hz to adequately capture neural oscillations across all frequency bands of interest [4] [12].

The bilateral fNIRS probe configuration should include sources and detectors positioned to cover brain regions of interest for semantic processing, typically including sensorimotor and parietal cortices [4]. The specific montage should be designed to target regions associated with the Action Observation Network and semantic processing, including premotor cortex, supplementary motor areas, primary motor cortex, and inferior/superior parietal lobules [4].

Table 3: Data Acquisition Parameters for Simultaneous EEG-fNIRS Semantic Decoding

Parameter	EEG Specifications	fNIRS Specifications
Sampling Rate	≥500 Hz	≥10 Hz
Channel Count	24-128 electrodes	24 channels (8 sources, 10 detectors)
Key Regions	Whole scalp coverage	Sensorimotor and parietal cortices
Signal Types	Electrical potentials (μV)	Oxygenated (HbO) and deoxygenated hemoglobin (HbR) concentration changes
Additional Measures	Electrode impedance monitoring	Optode digitization relative to anatomical landmarks

Data Analysis and Interpretation Framework

Preprocessing Pipelines

EEG preprocessing should include high-pass filtering (≥0.5 Hz) to remove slow drifts, low-pass filtering (≤100 Hz) to eliminate high-frequency noise, and notch filtering (50/60 Hz) to reduce line noise [13]. Researchers should implement artifact removal techniques for ocular, cardiac, and muscle artifacts, with visual inspection to ensure data quality. For fNIRS data, processing should convert raw light intensity measurements to optical density, then to hemoglobin concentration changes using the Modified Beer-Lambert Law [9]. Bandpass filtering (0.01-0.2 Hz) should be applied to remove physiological noise from cardiac cycles, respiration, and blood pressure oscillations [9].

Feature Extraction and Multimodal Fusion

For EEG analysis, feature extraction should focus on time-frequency representations of neural oscillations across theta, alpha, beta, and gamma bands, particularly event-related synchronization/desynchronization (ERS/ERD) patterns [12]. Time-domain features such as event-related potentials (ERPs) should also be extracted, with specific attention to components associated with semantic processing (e.g., N400). For fNIRS, features should include mean, peak, and slope values of HbO and HbR concentration changes during task periods relative to baseline [12].

Multimodal fusion can be implemented using structured sparse multiset Canonical Correlation Analysis (ssmCCA), which identifies relationships between electrical and hemodynamic response patterns to pinpoint brain regions consistently detected by both modalities [4]. This approach has successfully identified shared neural regions associated with the Action Observation Network during motor execution, observation, and imagery tasks [4], with similar principles applying to semantic imagery paradigms.

Statistical Analysis and Machine Learning Classification

For group-level analysis, general linear models (GLM) or mixed-effects models should be employed to identify significant differences in neural responses between semantic categories (animals vs. tools) across mental tasks. Multiple comparison corrections (e.g., FDR, FWE) should be applied to address the high-dimensional nature of EEG-fNIRS data.

For single-trial classification relevant to BCI applications, machine learning approaches such as support vector machines (SVM), linear discriminant analysis (LDA), or deep learning models can be trained to differentiate semantic categories based on combined EEG-fNIRS features. Classification performance should be evaluated using cross-validation techniques, with separate training, validation, and test sets to avoid overfitting.

Table 4: Key Analytical Approaches for EEG-fNIRS Semantic Decoding

Analysis Stage	EEG Methods	fNIRS Methods	Multimodal Integration
Preprocessing	High-pass/Low-pass/Notch filtering, Artifact removal	Modified Beer-Lambert Law conversion, Bandpass filtering	Temporal synchronization, Epoch segmentation
Feature Extraction	Time-frequency analysis (ERS/ERD), Event-related potentials (ERPs)	HbO/HbR concentration changes (mean, peak, slope)	Joint feature spaces, Parallel independent analysis
Pattern Classification	SVM, LDA, Deep learning on spectral features	SVM, LDA on hemodynamic features	ssmCCA, EEG-informed fNIRS analysis, Concatenated feature vectors

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 5: Essential Research Materials for EEG-fNIRS Semantic Decoding Studies

Item	Specification	Research Function
Simultaneous EEG-fNIRS System	24-128 channel EEG, 24-channel fNIRS (e.g., Hitachi ETG-4100)	Simultaneous acquisition of electrical and hemodynamic brain activity
Integrated EEG-fNIRS Cap	Elastic cap with embedded electrodes and optodes	Standardized positioning of recording sensors relative to scalp landmarks
3D Magnetic Space Digitizer	Fastrak (Polhemus) or similar system	Precise mapping of optode/electrode positions relative to anatomical landmarks
Stimulus Presentation Software	MATLAB Psychtoolbox, PsychoPy, E-Prime	Precise timing control for visual stimulus delivery and task synchronization
fNIRS Light Sources	Laser/LED sources at 695 nm and 830 nm wavelengths	Penetration of biological tissues to measure hemoglobin concentration changes
Electrolyte Gel	Conductive EEG gel	Ensuring low impedance electrical connection between electrodes and scalp
Data Analysis Suite	EEGLAB, NIRS-KIT, Homer2, SPM, FieldTrip	Preprocessing, feature extraction, and statistical analysis of multimodal data

EEG provides an indispensable tool for capturing the millisecond-scale neural dynamics underlying semantic thought processes. When integrated with fNIRS in a multimodal framework, researchers can leverage complementary information from electrical and hemodynamic responses to advance semantic decoding capabilities. The experimental protocols and analytical frameworks outlined in this application note provide a foundation for investigating the neural basis of semantic categorization during mental imagery tasks. These approaches hold significant promise for developing more intuitive brain-computer interfaces that communicate conceptual meaning directly, potentially bypassing the character-by-character spelling approaches used in current systems [8]. As methodological refinements continue to enhance the spatiotemporal precision and classification accuracy of these techniques, semantic neural decoding may transition from laboratory demonstration to practical application in both clinical and communication domains.

Functional Near-Infrared Spectroscopy (fNIRS) has emerged as a powerful neuroimaging tool for investigating the hemodynamic correlates of cognitive processes. This non-invasive technique measures cortical activity by detecting changes in hemoglobin concentration, providing a portable and flexible alternative to traditional imaging methods. Within the broader context of semantic decoding research using simultaneous EEG-fNIRS recordings, understanding the principles of fNIRS is paramount for designing robust experiments that can localize cortical activity during complex cognitive tasks. The technology leverages the principle of neurovascular coupling, where neuronal activity triggers a hemodynamic response that can be quantified through near-infrared light absorption [14]. This application note details the core principles, experimental protocols, and analytical frameworks necessary for employing fNIRS to investigate the "hemodynamic footprint" of cognition, with particular emphasis on its application in semantic neural decoding research.

Core Principles and Quantitative Foundations of fNIRS

fNIRS operates on the modified Beer-Lambert law, which relates the attenuation of near-infrared light passing through biological tissue to the concentration of chromophores, primarily oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) [15]. The hemodynamic response function (HRF) serves as the critical link between neural activity and the measured fNIRS signal, typically characterized by an increase in HbO and a decrease in HbR following neuronal firing [14]. The spatial specificity of fNIRS is a key consideration for localizing cognitive functions, with its sensitivity generally limited to the cortical surface up to a depth of approximately 1.5-2 cm beneath the scalp [14] [16].

Table 1: Key fNIRS Signal Components and Their Interpretation in Cognitive Studies

Signal Component	Typical Response to Neural Activation	Physiological Interpretation	Considerations for Semantic Tasks
Oxygenated Hemoglobin (HbO)	Increase	Increased cerebral blood flow and oxygen delivery	Stronger correlation with BOLD fMRI signal; often shows higher sensitivity to cognitive load
Deoxygenated Hemoglobin (HbR)	Decrease	Increased oxygen extraction and consumption	Better specificity to activated region; less susceptible to systemic confounds
Total Hemoglobin (HbT)	Increase	Net increase in regional cerebral blood volume	Considered a proxy for cerebral blood volume; sum of HbO and HbR
Hemodynamic Response Latency	Peak 5-6 seconds after stimulus onset	Neurovascular coupling delay	Must be accounted for in event-related designs for semantic decoding

The successful application of fNIRS in cognitive neuroscience, particularly for semantic decoding, requires careful consideration of its capabilities and limitations. Compared to fMRI, fNIRS offers superior portability and motion tolerance, enabling studies of more naturalistic behaviors and social interactions [14]. However, this advantage comes with the trade-off of limited depth penetration and spatial resolution. Contemporary continuous-wave fNIRS systems typically achieve a spatial resolution of 25-30 mm, though advanced diffuse optical tomography configurations can improve this to approximately 6 mm [14]. The temporal resolution of fNIRS (typically 5-10 Hz) surpasses that of fMRI, allowing for better characterization of the hemodynamic response shape and more effective separation of physiological confounds such as heart rate and respiration [14].

Experimental Design for Semantic Decoding Studies

Paradigm Selection and Optimization

Designing effective fNIRS experiments for semantic decoding requires strategic paradigm selection tailored to the research questions. Two primary approaches dominate: block designs and event-related designs. Block designs, featuring alternating periods of task and rest (e.g., 30-second blocks), maximize statistical power and are ideal for initial localization of semantic processing regions [14]. Event-related designs with irregular timing allow for analysis of single-trial responses and are better suited for investigating the time course of semantic processing or for paradigms where randomized stimulus presentation is essential [14].

For semantic decoding research, a typical paradigm might involve presenting participants with stimuli from different semantic categories (e.g., animals vs. tools) while they perform specific mental tasks such as silent naming, visual imagery, auditory imagery, or tactile imagery [8]. The block design is particularly effective for establishing category-specific activation patterns, while event-related designs can probe more dynamic aspects of semantic retrieval and processing.

Control Conditions and Confounding Factors

Appropriate control conditions are critical for isolating semantic processing from general cognitive demands. Control tasks should engage similar perceptual, attentional, and response processes without activating the semantic networks of interest. For instance, when studying tool-related semantics, a appropriate control might involve viewing tool images but performing a perceptual judgment (e.g., orientation detection) rather than semantic categorization [8].

A significant challenge in fNIRS research is distinguishing cortical hemodynamic responses from confounding systemic signals originating from extracerebral tissues or global physiological fluctuations. Several strategies can address this issue:

• Short-channel regression: Incorporating short source-detector separations (≤15 mm) to regress out superficial signals [17]
• Physiological monitoring: Recording heart rate, blood pressure, and respiration to model and remove physiological noise [18]
• Advanced processing techniques: Utilizing methods like lock-in amplification to enhance depth sensitivity and suppress artifacts [17]

Table 2: Key Experimental Parameters for fNIRS Studies in Semantic Decoding

Parameter	Recommended Specification	Rationale	Example from Literature
Source-Detector Distance	3.0 cm for standard channels; 0.8 cm for short channels	Optimizes gray matter sensitivity while enabling superficial signal regression [16]	3.0 cm used in prefrontal studies of drug craving [15]
Wavelengths	730 nm & 850 nm (common)	Targets absorption peaks of HbO and HbR for concentration calculation [19]	Standard in commercial systems (NirSmart-3000K) [19]
Sampling Rate	≥10 Hz	Adequately captures cardiac pulsation (~1 Hz) for filtering and signal quality assessment [16]	10.25 Hz in stroke rehabilitation study [16]
Trial Duration	3-5 seconds for event-related; 30-second blocks	Allows full HRF development while maintaining participant engagement	3-second trials in semantic categorization [8]
Participant Instructions	Standardized scripts for mental tasks	Minimizes inter-subject variability in cognitive strategy	Explicit instructions for visual, auditory, and tactile imagery [8]

Comprehensive Experimental Protocol for Semantic Decoding

Participant Preparation and fNIRS Setup

Materials and Equipment:

• fNIRS system (continuous-wave, frequency-domain, or time-domain)
• Optode cap or holder compatible with international 10-20 system
• Computer for stimulus presentation (e.g., E-Prime, PsychoPy, or MATLAB)
• Optional: Simultaneous EEG recording system
• Optional: Physiological monitoring (heart rate, respiration)

Procedure:

• Participant Screening and Consent: Recruit participants according to study criteria (e.g., right-handed, native language speakers for semantic tasks). Obtain informed consent following institutional ethical approval [8] [15].

• Cap Placement and Positioning:

  • Measure the participant's head according to the international 10-20 system, identifying key landmarks (nasion, inion, preauricular points).
  • Position the fNIRS cap ensuring Cz, C4, and T4 sites are properly located for right hemisphere coverage, with additional optodes for prefrontal regions as needed [20].
  • Ensure optodes make firm contact with the scalp, parting hair underneath to minimize signal loss.

• Signal Quality Check:

  • Verify signal-to-noise ratio for each channel before beginning the experiment.
  • Identify and mark channels with poor signal quality for exclusion from analysis [18].

Data Acquisition Protocol for Semantic Tasks

Stimulus Presentation:

• Implement a block or event-related design with counterbalanced conditions.
• For semantic category studies, use validated stimuli from standardized databases (e.g., 18 animals and 18 tools as in previous studies) [8].
• Present stimuli using specialized software that synchronizes with fNIRS recordings via triggers.

Experimental Conditions:

• Resting State: Record a 5-minute baseline with eyes closed or fixed on a crosshair.
• Task Conditions:
  • Silent Naming: Participants silently name displayed objects in their native language.
  • Visual Imagery: Participants visualize the object in their minds.
  • Auditory Imagery: Participants imagine sounds associated with the object.
  • Tactile Imagery: Participants imagine the feeling of touching the object [8].
• Control Conditions: Include appropriate low-level control tasks matched for perceptual and motor components.

Data Acquisition Parameters:

• Record both HbO and HbR concentrations at appropriate wavelengths (typically 730-850 nm).
• Maintain consistent sampling rate throughout acquisition (≥10 Hz recommended).
• Document any participant movements or protocol deviations for subsequent processing.

Data Processing and Analytical Framework

Preprocessing Pipeline

Raw fNIRS signals require extensive preprocessing to extract meaningful hemodynamic responses related to cognitive processes. A standardized pipeline includes:

• Signal Quality Assessment and Channel Rejection: Identify and exclude channels with poor signal-to-noise ratio using metrics such as coefficient of variation or signal-to-noise ratio thresholds [18].

• Motion Artifact Correction: Apply specialized algorithms (e.g., wavelet-based methods, spline interpolation, or correlation-based signal improvement) to detect and correct motion-induced signal distortions [18].

• Filtering and Drift Removal: Implement bandpass filtering (typically 0.01-0.5 Hz) to remove high-frequency physiological noise (cardiac, respiratory) and low-frequency drifts [18].

• Conversion to Hemoglobin Concentrations: Apply the modified Beer-Lambert law with appropriate differential pathlength factors to convert optical density measurements to HbO and HbR concentration changes [18] [15].

• Superficial Signal Regression: Utilize short-separation channels or principal component analysis to remove systemic physiological contamination originating from extracerebral layers [17].

Statistical Analysis and Semantic Decoding

For analyzing semantic category differences, several statistical approaches are available:

• General Linear Model (GLM): Model the hemodynamic response for each condition using canonical response functions and their temporal derivatives, including appropriate regressors for confounding effects [18] [14].

• Block Averaging: For simple designs, average responses across multiple trials of the same condition and compare mean activation across semantic categories.

• Machine Learning for Classification: Implement pattern classification algorithms (e.g., Support Vector Machines, Linear Discriminant Analysis) to decode semantic categories from multivariate fNIRS activation patterns [8] [15].

Advanced Analytical Approaches

For semantic decoding applications, more sophisticated analytical frameworks may be employed:

• Functional Connectivity: Assess interactions between brain regions during semantic processing using correlation-based or model-based approaches [19].
• Multimodal Data Fusion: Integrate fNIRS with simultaneous EEG to leverage the superior temporal resolution of electrophysiological measures with the spatial specificity of hemodynamic responses [8].
• Single-Trial Analysis: Extract features from individual trials for brain-computer interface applications or to study trial-by-trial variability in semantic processing [18].

Table 3: Research Reagent Solutions for fNIRS Semantic Decoding Studies

Category	Specific Tool/Resource	Function/Purpose	Implementation Example
fNIRS Hardware	Continuous-wave systems (NIRScout, NirSmart)	Measures light attenuation at cortical tissue	24 sources × 24 detectors configuration for whole-cortex coverage [16]
Stimulus Presentation	E-Prime, PsychoPy, MATLAB	Presents semantic stimuli with precise timing	Presentation of animal/tool images in randomized blocks [8] [19]
Data Processing	Homer2, NIRS-KIT, SPM-fNIRS	Preprocessing, visualization, and statistical analysis	Motion correction, filtering, GLM fitting [18]
Multimodal Integration	EEG-fNIRS co-registration platforms	Simultaneous electrophysiological and hemodynamic recording	12 participants performing semantic tasks with dual-modality recording [8]
Machine Learning	SVM, LDA, CNN classifiers	Semantic category decoding from hemodynamic patterns	Classification of animals vs. tools from prefrontal activation [8] [15]
Optode Localization	10-20 system measurement tools	Standardized positioning of sources and detectors	Positioning based on Cz, C4, and T4 landmarks [20]

Application in Semantic Decoding Research

The integration of fNIRS into semantic decoding research offers unique opportunities to study the neural basis of conceptual processing in more naturalistic settings than traditional neuroimaging methods allow. Studies have successfully differentiated between semantic categories such as animals and tools using fNIRS activation patterns from prefrontal and temporal regions [8]. The combination with EEG further enhances decoding accuracy by providing complementary information about rapid neural dynamics and slower hemodynamic changes.

Future applications in this domain may include:

• Development of fNIRS-based brain-computer interfaces for direct semantic communication
• Investigation of semantic processing in real-world social interactions
• Study of language and conceptual development in pediatric populations
• Assessment of semantic impairments in clinical populations (e.g., aphasia, dementia)

In conclusion, fNIRS provides a valuable methodological approach for investigating the hemodynamic correlates of semantic cognition. When implemented with careful attention to the principles outlined in this protocol, it offers a powerful tool for decoding conceptual representations and understanding the neural basis of human thought.

Neurovascular coupling (NVC) describes the fundamental physiological process that links neuronal electrical activity with subsequent changes in cerebral blood flow and oxygenation. This mechanism forms the critical bridge between the signals measured by electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). When neurons become active, they trigger a complex cascade of metabolic and vascular events that ultimately increase local blood flow, delivering oxygen and nutrients to meet metabolic demands. This results in a characteristic hemodynamic response function characterized by an increase in oxygenated hemoglobin and a decrease in deoxygenated hemoglobin in the activated brain region [21].

The integration of EEG and fNIRS has emerged as a powerful approach in cognitive neuroscience because it simultaneously captures both the electrophysiological activity from neurons and the hemodynamic response that supports this activity. This multimodal approach provides complementary information with high temporal resolution from EEG and improved spatial localization from fNIRS, offering a more comprehensive window into brain function than either modality alone [22]. Understanding neurovascular coupling is particularly crucial for semantic decoding research, as it enables researchers to correlate rapid electrical signatures of semantic processing with more localized hemodynamic responses that pinpoint the brain regions involved.

Physiological Mechanisms of Neurovascular Coupling

The Neurovascular Unit and Functional Hyperemia

The concept of the "neurovascular unit" is central to understanding NVC. This functional module consists of neurons, astrocytes, and vascular cells that work in concert to regulate cerebral blood flow. The widely accepted physiological sequence begins with synaptic activity triggering the release of neurotransmitters including glutamate. This activates postsynaptic neurons and surrounding astrocytes, which in turn release vasoactive agents that cause dilation of arterioles and capillaries [21].

This coordinated response leads to functional hyperemia - an increase in local cerebral blood flow that overcompensates for the oxygen consumption of active neurons. The resulting hemodynamic changes manifest as a local increase in oxyhemoglobin and decrease in deoxyhemoglobin, which fNIRS detects through its measurement of near-infrared light absorption [21]. This vascular response typically begins after a 2-second delay following neuronal activation, peaks around 6-10 seconds, and then gradually returns to baseline [21].

Correlated Electrical and Hemodynamic Signatures

Research has demonstrated consistent relationships between specific EEG components and hemodynamic responses measured by fNIRS. During auditory processing, for example, the N1 and P2 event-related potential components show amplitude increases that correlate with hemodynamic changes in the auditory cortex. Spearman correlation analysis has revealed significant relationships between left auditory cortex activation and N1 amplitude, and between right dorsolateral prefrontal cortex activation and P2 amplitude, particularly for deoxyhemoglobin concentrations [21].

These correlations are not merely observational but reflect tight physiological coupling. Studies employing simultaneous EEG-fNIRS recording during cognitive-motor tasks have found that decreased neurovascular coupling occurs during dual-task conditions compared to single tasks, reflecting the brain's divided attention resources [23]. This suggests that NVC strength itself may be a meaningful indicator of cognitive processing efficiency.

Quantitative Relationships Between EEG and fNIRS Signals

Table 1: Correlated EEG and fNIRS Parameters in Neurovascular Coupling Studies

EEG Parameter	fNIRS Parameter	Relationship	Brain Region	Experimental Context
N1 Amplitude	HbO Increase / HbR Decrease	Positive Correlation	Left Auditory Cortex	Auditory Intensity Processing [21]
P2 Amplitude	HbO Increase / HbR Decrease	Positive Correlation	Right Dorsolateral Cortex	Auditory Intensity Processing [21]
Alpha Band Power (8-12 Hz)	HbO Concentration	Negative Covariation	Sensorimotor Cortex	Motor Imagery [23]
Theta, Alpha, Beta Rhythms	HbO/HbR Changes	Decreased Coupling in Dual-Task	Prefrontal Cortex	Cognitive-Motor Interference [23]
Alpha and Low Beta Power	HbO/HbR Changes	Power Reduction with Activation	Motor Cortex (C3/C4)	Finger Tapping Task [24]

Table 2: Temporal and Spatial Characteristics of EEG and fNIRS Modalities

Characteristic	EEG	fNIRS
Temporal Resolution	Milliseconds [25]	Seconds [25]
Spatial Resolution	~2 cm [8]	1-3 cm [26]
Physiological Basis	Postsynaptic electrical potentials [25]	Hemodynamic response (HbO/HbR) [21]
Signal Delay After Neural Activity	Direct measure, virtually instantaneous [25]	~2 second delay, peaks at 6-10 seconds [21]
Depth Sensitivity	Primarily cortical surface [25]	Outer cortex (1-2.5 cm deep) [25]
Key Measured Parameters	Event-related potentials, spectral power [21]	HbO and HbR concentration changes [21]

Experimental Protocols for Investigating Neurovascular Coupling

Protocol 1: Auditory Intensity-Dependent Amplitude Changes

Purpose: To investigate neurovascular coupling during auditory processing using intensity-dependent amplitude changes (IDAP) [21].

Stimuli and Paradigm:

• Present auditory tones at multiple intensity levels (e.g., 70.9 dB, 77.9 dB, 84.5 dB, 89.5 dB, and 94.5 dB)
• Use either single tones (500 ms duration) or trains of tones (8 tones, each 70 ms duration)
• Employ random presentation with adequate inter-stimulus intervals (e.g., 54 trials per intensity for single tones; 20 train presentations)
• Ensure proper calibration of audio equipment and use of silent intervals between stimuli to capture the full hemodynamic response

Data Acquisition:

• Record EEG using standard scalp electrodes according to the 10-20 system, focusing on N1 and P2 components
• Simultaneously acquire fNIRS data from auditory, visual, and prefrontal cortices
• Synchronize EEG and fNIRS systems using shared trigger signals or unified processor
• Monitor and record systemic physiological parameters (heart rate, blood pressure, respiration) as controls

Analysis Methods:

• Extract N1, P2, and N1-P2 peak-to-peak amplitudes from EEG
• Calculate HbO and HbR concentration changes from fNIRS signals
• Perform Spearman correlation analysis between EEG components and fNIRS hemodynamic responses
• Conduct statistical comparisons across intensity levels and brain regions

Protocol 2: Semantic Category Decoding with Mental Imagery

Purpose: To examine neurovascular coupling during semantic processing of different conceptual categories [8].

Stimuli and Paradigm:

• Select visual images from distinct semantic categories (e.g., 18 animals and 18 tools)
• Implement a block or event-related design with randomized category presentation
• Instruct participants to perform four mental tasks after viewing each image:
  • Silent naming: Silently name the object in their native language
  • Visual imagery: Visualize the object in their mind
  • Auditory imagery: Imagine sounds associated with the object
  • Tactile imagery: Imagine the feeling of touching the object
• Use appropriate trial durations (3-5 seconds) with adequate rest intervals

Data Acquisition:

• Deploy high-density EEG systems focused on temporal and prefrontal regions
• Position fNIRS optodes over prefrontal, temporal, and parietal cortices
• Utilize integrated EEG-fNIRS caps with pre-defined compatible openings
• Implement precise synchronization between modalities via TTL pulses or shared clock systems

Analysis Methods:

• Apply task-related component analysis to extract reproducible neural patterns
• Compute within-class similarity and between-class distance metrics
• Analyze event-related spectral perturbations in EEG data
• Investigate correlation strengths between EEG spectral features and fNIRS hemodynamic responses
• Employ machine learning classifiers to decode semantic categories from combined features

Technical Implementation of Simultaneous EEG-fNIRS

Integrated System Design

Successful simultaneous EEG-fNIRS recording requires careful technical integration. Current systems typically utilize:

• Unified acquisition helmets that integrate EEG electrodes and fNIRS optodes on a shared substrate [22]
• Customized 3D-printed helmets or cryogenic thermoplastic sheets for optimal sensor placement and stability [22]
• Synchronization mechanisms ranging from simple trigger sharing to unified processors that simultaneously handle both signal types [22]
• Flexible electrode caps with pre-punctured openings for fNIRS probe integration, maintaining the international 10-20 system for consistent localization [22]

Figure 1: Integrated EEG-fNIRS System Architecture for Neurovascular Coupling Studies

Signal Processing and Data Fusion

The fundamentally different nature of EEG and fNIRS signals necessitates specialized processing pipelines before integration:

EEG Preprocessing:

• Filtering (e.g., 0.5-40 Hz bandpass, notch filters for line noise)
• Artifact removal (ocular, cardiac, muscular)
• Independent component analysis for source separation
• Time-frequency analysis for spectral feature extraction

fNIRS Preprocessing:

• Conversion of optical density to hemoglobin concentrations using modified Beer-Lambert law
• Motion artifact correction using wavelet or spline interpolation methods
• Bandpass filtering (0.01-0.5 Hz) to isolate hemodynamic responses
• Removal of systemic physiological confounds (cardiac, respiratory)

Multimodal Fusion Approaches:

• Joint Independent Component Analysis to identify coupled spatial patterns
• Canonical Correlation Analysis to find maximally correlated components
• Model-driven approaches that incorporate physiological constraints of neurovascular coupling
• Machine learning frameworks that combine features from both modalities for enhanced classification

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 3: Essential Equipment and Software for EEG-fNIRS Neurovascular Coupling Research

Category	Item	Specification/Function	Application Notes
Acquisition Hardware	EEG System	>64 channels, compatible with fNIRS integration	Prefer systems with high input impedance and wide dynamic range [22]
	fNIRS System	Multiple wavelength (e.g., 690nm, 830nm), continuous wave or frequency domain	Ensure adequate source-detector separation (2.5-4 cm) for cortical penetration [21]
	Integrated Cap	Compatible with international 10-20 system, customizable optode/electrode placement	3D-printed or thermoplastic materials provide optimal stability [22]
Synchronization	Trigger Box	TTL pulse generation for event marking	Critical for precise temporal alignment of multimodal data [22]
	Unified Processor	Simultaneous acquisition of EEG and fNIRS signals	Provides inherent synchronization but more complex implementation [22]
Software Tools	EEGLAB/FieldTrip	EEG processing and analysis	Open-source toolboxes with extensive plugin support [23]
	Homer2/NIRS-KIT	fNIRS data processing and visualization	Specialized tools for hemodynamic response quantification [21]
	Custom MATLAB/Python Scripts	Multimodal data fusion and NVC analysis	Essential for implementing joint analysis frameworks [23]
Calibration & Quality Assurance	Optical Phantoms	Tissue-simulating materials for fNIRS calibration	Verify light propagation models and system performance [26]
	Impedance Checker	EEG electrode-scalp contact quality assessment	Maintain impedance below 10 kΩ for optimal signal quality [22]
	Polhemus System	3D digitization for precise sensor localization	Enables co-registration with anatomical templates [26]

Application to Semantic Decoding Research

The investigation of neurovascular coupling has profound implications for semantic decoding research. Simultaneous EEG-fNIRS recordings during semantic tasks provide a unique opportunity to capture both the rapid temporal dynamics of semantic access (via EEG) and the spatially localized brain regions supporting semantic processing (via fNIRS) [8].

Recent studies have demonstrated that semantic category discrimination (e.g., animals vs. tools) is feasible using combined EEG-fNIRS features. The electrical signatures captured by EEG provide millisecond-resolution information about the timing of semantic processing, while fNIRS hemodynamic responses identify the contribution of specific cortical regions such as prefrontal, temporal, and parietal areas [8]. This multimodal approach significantly enhances decoding accuracy compared to unimodal approaches.

Figure 2: Semantic Processing Workflow Showing Temporal (EEG) and Spatial (fNIRS) Dynamics

The strength of neurovascular coupling itself may serve as a biomarker for semantic processing efficiency. Studies have shown that task-related component analysis can enhance the reproducibility and discriminability of neural patterns associated with different semantic categories [23]. Furthermore, the temporal synchrony between electrical and hemodynamic responses provides a validation mechanism for semantic decoding models, potentially leading to more robust brain-computer interfaces for direct semantic communication [8].

The integration of EEG and fNIRS through the framework of neurovascular coupling represents a powerful approach for advancing semantic decoding research. This multimodal methodology capitalizes on the complementary strengths of both techniques, providing both high temporal resolution and improved spatial localization. The physiological coupling between electrical neuronal activity and hemodynamic responses offers a natural mechanism for validating decoding models and enhancing classification accuracy.

Future research directions should focus on refining integrated hardware systems to improve user comfort and signal quality, developing more sophisticated data fusion algorithms that explicitly model neurovascular coupling, and establishing standardized protocols for semantic decoding across diverse populations. As these technical and methodological challenges are addressed, simultaneous EEG-fNIRS recording is poised to become an increasingly valuable tool for unlocking the neural basis of semantic cognition and developing more natural brain-computer interfaces for direct conceptual communication.

Semantic neural decoding represents a frontier in cognitive neuroscience, aiming to identify which semantic concepts an individual is processing based solely on recordings of their brain activity [8]. This capability is fundamental to developing a new class of brain-computer interface (BCI) that enables direct communication of semantic concepts, moving beyond the character-by-character spelling paradigms that limit current systems [8]. The differentiation of broad semantic categories such as animals and tools serves as a critical test case for these technologies. While previous breakthroughs in semantic decoding have relied heavily on functional magnetic resonance imaging (fMRI), the field is increasingly turning to multimodal approaches that combine electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) [8] [27]. These complementary modalities offer a unique combination of temporal and spatial resolution in a portable, cost-effective package suitable for real-world applications beyond the laboratory setting. This article frames these developments within the context of a broader thesis on semantic decoding using simultaneous EEG-fNIRS recordings, providing both application notes and detailed experimental protocols for researchers investigating the neural basis of conceptual representation.

Experimental Evidence: Differentiating Semantic Categories from Neural Signals

Key Studies on Animal vs. Tool Discrimination

Recent research has demonstrated the feasibility of distinguishing between semantic categories from non-invasive neural signals. The following table summarizes key experimental findings in differentiating animals from tools using various neuroimaging modalities.

Table 1: Experimental Evidence for Differentiating Animals and Tools from Brain Activity

Study Reference	Modality	Semantic Categories	Mental Tasks	Key Findings
Simultaneous EEG-fNIRS (2025) [8]	EEG + fNIRS	Animals (18) vs. Tools (18)	Silent naming, Visual imagery, Auditory imagery, Tactile imagery	Differentiation feasible during all four mental tasks using multimodal features.
fNIRS Neural Decoding (2017) [28]	fNIRS	Animals (4) vs. Body Parts (4)	Passive audiovisual viewing with semantic focus	Semantic representations encoded in fNIRS data, decodable using representational similarity analysis.
EEG-Based Image Generation (2025) [29]	EEG	Multiple object categories	Passive viewing of images	Neural Encoding Representation Vectorizer (NERV) achieved 94.8% accuracy in 2-way zero-shot classification.

The evidence confirms that distributed semantic representations are encoded in both electrophysiological (EEG) and hemodynamic (fNIRS) signals, with multimodal fusion offering a promising path toward improved decoding performance [8] [27] [30]. The selection of appropriate mental tasks—from silent naming to modality-specific imagery—proves crucial for activating distinct neural patterns that facilitate category discrimination [8].

Temporal and Spatial Characteristics of Semantic Processing

The differentiation of semantic categories leverages distinct temporal and spatial patterns in brain activity:

• Temporal Dynamics: EEG provides millisecond-level resolution of electrical potentials, capturing rapid neural events during semantic processing. Event-related potentials (ERPs) show enhanced amplitudes around 300 ms post-stimulus for intentional semantic processing, particularly in parietal and occipital regions [31].
• Spatial Organization: fNIRS reveals category-sensitive cortical regions, with posterior arrays (occipital lobe) effectively decoding visual-semantic information and lateral arrays (temporal lobe) contributing to broader semantic representations [28]. The mirror neuron system (premotor cortex, inferior frontal gyrus) and theory of mind networks (temporoparietal junction, medial prefrontal cortex) engage differentially during action-related semantic processing [32].

Technical Protocols for EEG-fNIRS Semantic Decoding

Experimental Design and Paradigm

Protocol 1: Multimodal Imagery Task for Category Discrimination

• Objective: To differentiate between semantic categories (animals vs. tools) during various mental imagery tasks using simultaneous EEG-fNIRS recordings.
• Stimuli Preparation:
  • Select 18 animals and 18 tools familiar to the participant population [8].
  • Use gray-scale photographic images (400×400 pixels) on white background to minimize low-level visual confounds [8].
  • Convert images to uniform format and conduct pre-experiment familiarization to ensure recognition.
• Task Paradigm:
  • Implement four mental tasks in randomized blocks:
    • Silent Naming: Participants silently name the displayed object in their native language [8].
    • Visual Imagery: Participants visualize the object in their mind [8].
    • Auditory Imagery: Participants imagine sounds associated with the object [8].
    • Tactile Imagery: Participants imagine the feeling of touching the object [8].
  • Each trial consists of:
    • Stimulus presentation: 3 seconds
    • Mental task period: 3-5 seconds (participants maintain engagement)
    • Inter-trial interval: jittered 6-9 seconds [8] [28]
• Participant Instructions:
  • Emphasize maintaining engagement throughout the mental task period.
  • Minimize physical movements (eye, face, head, body) to reduce artifacts [8].
  • Encourage naturalistic imagery strategies rather than constrained feature generation.

The experimental workflow for this protocol is standardized as follows:

Multimodal Data Acquisition Setup

Protocol 2: Simultaneous EEG-fNIRS Recording Configuration

• Equipment Specifications:

  • EEG System: 64-channel active electrode system with sampling rate ≥500 Hz [32].
  • fNIRS System: Continuous wave system with 24 posterior channels (occipital lobe) and 18-22 lateral channels (temporal lobe) [8] [28].
  • Synchronization: Hardware trigger synchronization between EEG and fNIRS systems [27].

• Optode Placement:

  • Posterior Array: Centered on the back of the head, most inferior row just over the inion [28].
  • Lateral Array: Positioned above left ear, centered anterior-to-posterior over ear, most anterior channel just beyond hairline [28].
  • EEG Electrodes: International 10-20 system placement with additional coverage over language and semantic regions [32].

• Data Quality Assurance:

  • Impedance checking for EEG (<10 kΩ) [32].
  • Signal quality inspection for fNIRS prior to experiment commencement.
  • Participant instruction to minimize movement artifacts [8].

Data Preprocessing Pipeline

Protocol 3: Multimodal Signal Processing and Feature Extraction

• EEG Preprocessing:

  • Bandpass filtering: 0.5-40 Hz [32].
  • Artifact removal: Independent Component Analysis (ICA) for ocular and muscle artifacts [27].
  • Re-referencing to average reference.
  • Epoch extraction: -0.5 to 5 seconds relative to stimulus onset.
  • Baseline correction: -0.5 to 0 seconds.

• fNIRS Preprocessing:

  • Conversion of raw intensity to optical density [28].
  • Principal Component Analysis to remove global physiological noise [28].
  • Bandpass filtering: 0.01-0.5 Hz [28].
  • Modified Beer-Lambert law application to compute oxygenated (HbO) and deoxygenated (HbR) hemoglobin concentrations [28] [32].
  • Motion artifact correction: 1-second window masking around spikes >5 standard deviations [28].

• Feature Extraction:

  • EEG Features: Event-related potentials (ERP), time-frequency features (theta, alpha power), and connectivity metrics [31] [32].
  • fNIRS Features: Mean HbO/HbR concentration during task periods, slope, peak values, and area under the curve [32].
  • Channel Stability Analysis: Correlation-based selection of reliable channels across blocks [28].

Data Fusion and Analytical Approaches

Multimodal Fusion Strategies

The integration of EEG and fNIRS data presents unique opportunities for leveraging complementary information. The following table compares predominant fusion methodologies in EEG-fNIRS research.

Table 2: EEG-fNIRS Data Fusion Strategies for Semantic Decoding

Fusion Level	Approach	Methodology	Advantages	Limitations
Early Fusion	Feature Concatenation	Direct combination of raw/preprocessed features from both modalities [30].	Simplicity; preserves low-level feature relationships.	Susceptible to modality-specific noise; curse of dimensionality.
Intermediate Fusion	Cross-Modal Attention (MBC-ATT)	Modality-guided attention mechanism dynamically weights features [30].	Automatically focuses on relevant signals; enhances complementary relationships.	Complex architecture; requires sufficient training data.
Model-Based Fusion	Source Decomposition	Joint blind source separation to identify shared latent components [27].	Reveals neurovascular coupling; models complex latent relationships.	Computationally intensive; requires precise temporal alignment.
Late Fusion	Decision Integration	Separate classification followed by voting or weighted decision integration [30].	Leverages modality-specific strengths; robust to single-modality failure.	May miss low-level cross-modal interactions.

Machine Learning and Neural Decoding

Representational Similarity Analysis (RSA):

• Abstract neural response patterns to similarity space and compare to semantic models [28].
• Use leave-one-out cross-validation to assess generalizability [28].
• Compare neural representational dissimilarity matrices to theoretical models [28].

Deep Learning Architectures:

• Implement multi-branch convolutional networks with attention (MBC-ATT) for cross-modality fusion [30].
• Employ recurrence plot-based CNN-LSTM frameworks for spatiotemporal pattern recognition [30].
• Utilize contrastive learning approaches (e.g., EEG-CLIP) to align neural and semantic spaces [29].

The relationship between data modalities and fusion strategies can be visualized as:

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Materials and Tools for EEG-fNIRS Semantic Decoding Research

Category	Item	Specification/Function	Example Applications
Recording Equipment	EEG System	64+ channels, sampling rate ≥500 Hz, synchronization capability [32]	Electrical potential measurement for temporal dynamics
	fNIRS System	Continuous wave, 20+ channels, 690-830 nm wavelengths [28]	Hemodynamic response measurement for spatial localization
Stimulus Presentation	Presentation Software	Precision timing, trigger output, experimental paradigm design [8]	Controlled delivery of semantic stimuli
Data Processing	EEGLAB/MNE-Python	EEG preprocessing, ICA, time-frequency analysis [27]	EEG signal processing and feature extraction
	Homer2	fNIRS preprocessing, optical data conversion, filtering [28]	fNIRS signal processing and hemodynamic feature extraction
Analysis & Decoding	Custom MATLAB/Python Scripts	Representational similarity analysis, machine learning pipelines [28]	Multimodal fusion and semantic classification
	Deep Learning Frameworks	PyTorch/TensorFlow with custom architectures (e.g., MBC-ATT) [30]	Neural network implementation for decoding
Experimental Materials	Stimulus Set	Standardized images of animals/tools (18+ per category) [8]	Controlled semantic category representation

The differentiation of semantic categories from brain activity using simultaneous EEG-fNIRS recordings represents a promising avenue for both basic cognitive neuroscience and applied BCI development. The protocols and applications detailed herein provide a foundation for investigating the neural representations underlying conceptual knowledge. Future research directions should focus on expanding the repertoire of decodable semantic categories, improving real-time decoding capabilities for BCI applications, and extending these approaches to clinical populations with communication impairments. The integration of advanced data-driven fusion methods, particularly cross-modal attention mechanisms and deep learning approaches, will be crucial for unlocking the full potential of multimodal neural recordings in semantic decoding research.

Semantic neural decoding aims to identify the specific semantic concepts an individual is focusing on based on their brain activity. This approach holds significant promise for developing a new type of Brain-Computer Interface (BCI) that enables the direct communication of semantic concepts, moving beyond the character-by-character spelling used in current systems [8]. Research consistently demonstrates that perceiving objects and imagining them elicit similar brain activity patterns, making mental imagery a core cognitive ability for probing semantic representations [8]. This protocol details the application of multimodal mental imagery paradigms—specifically visual, auditory, and tactile imagery—within a research framework utilizing simultaneous Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) recordings. The goal is to provide a standardized method for investigating the neural correlates of semantic categories, such as animals and tools, across different sensory modalities [8].

Experimental Design and Paradigm

The following structured paradigm is designed to differentiate between semantic categories during various mental imagery tasks.

Table 1: Core Experimental Tasks and Instructions

Task Name	Participant Instruction	Example for 'Cat' (Animal)	Example for 'Hammer' (Tool)
Silent Naming	Silently name the displayed object in your mind.	Silently think "cat".	Silently think "hammer".
Visual Imagery	Visualize the object in your mind.	Imagine what a cat looks like.	Imagine what a hammer looks like.
Auditory Imagery	Imagine the sounds associated with the object.	Imagine the sound of a cat meowing.	Imagine the sound of a hammer banging.
Tactile Imagery	Imagine the feeling of touching the object.	Imagine the feeling of petting a cat's fur.	Imagine the feeling of gripping a hammer's handle.

Stimuli Specification

• Semantic Categories: Animals and Tools.
• Concept Set: 18 animals and 18 tools (e.g., bear, cat, cow, crab, dog, elephant for animals; hammer, screwdriver, etc., for tools) [8].
• Stimulus Format: Photographic images converted to grayscale, cropped to 400x400 pixels, contrast-stretched, and presented on a white background [8].
• Procedure: Participants should be shown all images before the experiment to ensure recognition and avoid interpretation bias during the task [8].

Participant Preparation and Materials

Table 2: Research Reagent Solutions and Essential Materials

Item Category	Specific Item / Reagent	Function / Purpose
Neuroimaging Hardware	EEG Amplifier & Electrodes (e.g., 64-channel system)	Measures electrical brain activity with high temporal resolution.
	fNIRS System (Continuous Wave)	Measures hemodynamic responses (HbO/HbR) related to neuronal activity.
	EEG Cap (10-10 or 10-20 International System)	Standardized placement of EEG electrodes.
	fNIRS Optode Holder / Cap	Holds light sources and detectors at specified locations on the scalp.
Signal Quality & Conduction	Electrolyte Gel (e.g., NeuroPrep)	Optimizes conductivity and reduces impedance at the electrode-scalp interface for EEG [33].
	Abrasive Skin Prep Gel	Prepares the skin for better electrode contact.
Stimulus Presentation	Stimulus Presentation Software (e.g., PsychoPy)	Presents visual stimuli and records task timing with high precision [34].
Data Processing & Analysis	Data Analysis Suite (e.g., MATLAB, Python with MNE, Nilearn)	Processes and analyzes recorded EEG and fNIRS data.
	Statistical Software (e.g., R, SPSS)	Performs statistical tests on the extracted neural features.

Participant Criteria

• Dataset 1 (EEG + fNIRS): Native language speakers (e.g., English) to control for linguistic variability in semantic representation, particularly for the silent naming task. Participants should have normal or corrected-to-normal vision [8].
• Dataset 2 (EEG-only): Native language fluency is not required if the silent naming task is omitted [8].
• Exclusion Criteria: History of neurological or psychiatric disorders [35].

Protocol Approvals

• Obtain written informed consent from all participants.
• Secure approval from the local Institutional Review Board (IRB) or Ethics Committee before commencing the study [8] [18].

Data Acquisition Setup

Simultaneous EEG and fNIRS acquisition provides complementary information: EEG offers millisecond-level temporal resolution, while fNIRS provides better spatial resolution and is less susceptible to motion artifacts [8] [33].

EEG Acquisition Parameters

• Sampling Rate: ≥ 512 Hz is recommended [35].
• Electrode Placement: Use the 10-10 international system for high-density coverage [33].
• Impedance: Keep impedance below 10 kΩ to ensure high signal quality [33].

fNIRS Acquisition Parameters

• Technology: Continuous Wave (CW) fNIRS is commonly used [18].
• Wavelengths: Typically two wavelengths (e.g., 760 nm and 850 nm) to distinguish between oxygenated (HbO) and deoxygenated hemoglobin (HbR) [18].
• Source-Detector Distances: 3 cm is standard for measuring cortical activity; shorter distances (e.g., 1 cm) are recommended for measuring superficial, non-brain signals [18].
• Sampling Rate: Typically between 1-10 Hz.

The following workflow outlines the complete experimental procedure from participant preparation to data acquisition:

Experimental Protocol

A single experimental trial is structured as follows. The total duration of mental imagery can be adjusted; 3-second and 5-second durations have been used in previous studies [8].

Data Preprocessing and Analysis

Table 3: Preprocessing Pipeline for EEG and fNIRS Data

Step	EEG Processing	fNIRS Processing
Quality Check	Visual inspection; Channel rejection based on signal variance or amplitude.	Check signal-to-noise ratio; Reject channels with poor light intensity [18].
Filtering	Band-pass filter (e.g., 0.5-40 Hz) to remove slow drifts and line noise.	Band-pass filter (e.g., 0.01-0.5 Hz) to isolate hemodynamic signals and remove cardiac/pulse oscillations [18].
Artifact Removal	Apply algorithms (e.g., ICA) to remove ocular and muscle artifacts.	Use motion correction algorithms (e.g., wavelet-based, spline interpolation) [18].
Signal Conversion	Not Applicable.	Apply Modified Beer-Lambert Law (mBLL) to convert light intensity changes to HbO and HbR concentrations [18].
Epoching	Segment data into trials locked to the onset of the mental imagery period.	Segment data into trials locked to the onset of the mental imagery period.
Feature Extraction	Time-Frequency Power: Calculate power in frequency bands (Theta: 4-7 Hz, Alpha: 8-12 Hz, Beta: 13-30 Hz) [35].Event-Related Potential (ERP): Analyze amplitude and latency of components.	Hemodynamic Response: Average HbO/HbR concentrations across trials for block-related responses [18].

Statistical Analysis and Semantic Decoding

• General Linear Model (GLM): For fNIRS and EEG amplitude data, use a GLM to model the brain response during each task condition against baseline [18].
• Machine Learning Classification: Employ classifiers (e.g., Linear Discriminant Analysis, Support Vector Machines) to decode the semantic category (animal vs. tool) from neural features extracted during each imagery task. Performance is typically reported as cross-validated classification accuracy [8] [33].
• Hyperalignment: For group-level analysis that preserves individual-specific functional topographies, consider using hyperalignment techniques to map neural data into a common representational space before building between-subject decoding models [34].

The following chart summarizes the core analysis pathway for semantic decoding:

From Raw Data to Decoded Meaning: Methodologies and Clinical Applications

Simultaneous electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) recording provides a comprehensive picture of brain function by capturing complementary aspects of neural activity. This integrated approach is particularly valuable for semantic decoding research, where it can reveal both the rapid electrical signatures and the sustained hemodynamic correlates of language processing. EEG offers millisecond-level temporal resolution to track the rapid dynamics of neural electrical activity, while fNIRS provides centimeter-level spatial resolution to localize associated hemodynamic changes in the cortex [36] [37]. This multimodal integration enables researchers to investigate the complex neural processes underlying semantic processing, from initial word recognition to higher-level comprehension and integration.

The technological evolution of combined EEG-fNIRS systems has progressed from simply placing separate devices on the same subject to fully integrated, wearable systems that mitigate the challenges of mechanical interference, electrical crosstalk, and synchronization inaccuracy [36]. For semantic decoding studies, which often require ecological experimental setups with naturalistic stimuli, these integrated systems offer the practical advantage of allowing more natural participant movement while maintaining data quality.

Hardware Integration and System Design

Integrated System Architectures

Successfully integrating EEG and fNIRS technologies requires addressing mechanical, electrical, and synchronization challenges. Current approaches range from combining discrete commercial systems to using fully integrated platforms where both modalities share a common hardware architecture.

Table 1: Comparison of EEG-fNIRS Integration Approaches

Integration Type	Description	Advantages	Limitations	Representative Systems
Discrete Systems	Separate EEG and fNIRS devices combined via external synchronization [36].	Flexibility in device selection; utilizes established, optimized individual systems.	Prone to synchronization challenges; mechanical competition for headspace; potential electrical crosstalk [36].	NIRSport2 with g.tec g.Nautilus or Brain Products LiveAmp [38].
Electrically Integrated Systems	Shared control module, amplifier, and Analog-to-Digital Converter (ADC) for both modalities [36].	Eliminates time delay between signals; minimizes electrical crosstalk; simplifies setup.	Limited commercial availability; less flexibility in configuration.	g.tec g.HIamp with g.SENSOR fNIRS upgrade [37].
Fully Integrated Hybrid Systems	Mechanically and electrically integrated into a single, wearable unit with co-located sensors [36].	Optimal wearability and patient comfort; ideal for long-term monitoring and real-world applications.	Most complex development process; highest degree of technical integration required.	g.tec's g.Nautilus with NIRx [39].

Modern integrated systems, such as the g.Nautilus NIRx system, represent significant advancements by enabling wireless, simultaneous recording of up to 64 EEG channels and 32 fNIRS optodes housed in a comfortable cap (g.GAMMAcap), providing a powerful solution for advanced neuroscience research [39]. These systems are particularly suited for semantic decoding experiments that may involve paradigms where participants read, listen to narratives, or engage in conversation, as the reduced cabling minimizes movement artifacts and increases ecological validity.

Sensor Placement and Caps

The physical integration of EEG electrodes and fNIRS optodes is a critical consideration. Two primary sensor placement strategies exist:

• Adjacent Positioning: EEG electrodes and fNIRS optodes are placed next to each other on the scalp. This approach allows for the use of any type of EEG electrode and can reduce setup time [38].
• Co-located Positioning: EEG electrodes are positioned directly between fNIRS sources and detectors. This configuration ensures that both modalities are measuring activity from the same cortical area, which is crucial for correlating electrical and hemodynamic responses in semantic decoding [37].

Specialized caps are essential for successful integration. The cap must maintain fixed sensor positions to prevent artifacts and should be made of dark material to prevent ambient light from contaminating the fNIRS signal [37]. Systems like the g.GAMMAcap incorporate predefined holder rings for both EEG electrodes and fNIRS optodes, allowing for flexible arrangement of sources and detectors to achieve optimal channel configurations for specific research goals, such as focusing on language-related brain regions [39] [37].

Figure 1: Data acquisition workflow in a simultaneous EEG-fNIRS setup, showing the integration of complementary signals from cap to final analysis.

Synchronization Methodologies

Precise temporal synchronization between EEG and fNIRS data streams is fundamental for correlating the fast electrical events with slower hemodynamic responses in semantic processing. The synchronization strategy depends largely on whether discrete or integrated systems are being used.

Synchronization Solutions for Discrete Systems

When using separate EEG and fNIRS devices, external mechanisms are required to time-lock the acquired signals. The following methods are commonly employed:

• Hardware Synchronization: Dedicated hardware devices can generate marker signals that are recorded by both systems simultaneously. The PortaSync (Artinis) is a wireless handheld device that connects to fNIRS software via Bluetooth and has analog inputs/outputs to send synchronization signals to the EEG amplifier [40]. Alternatively, NIRx's Parallel Port Replicator splits a single trigger input to multiple outputs, sending the same trigger to both EEG and fNIRS systems [38].

• Software Synchronization: The Lab Streaming Layer (LSL), an open-source ecosystem for network-synchronized data streaming, can send and receive time-stamped data and event markers between software applications [40]. This allows stimulus presentation software to send simultaneous triggers to both EEG and fNIRS acquisition software via a local network.

• Stimulus-Locked Synchronization: In scenarios where both EEG and fNIRS systems can receive external triggers, the stimulus presentation software can be configured to send simultaneous triggers to both systems via LSL, DCOM, or parallel port [40].

Synchronization in Integrated Systems

Fully integrated systems significantly simplify synchronization by employing a shared architecture. In these systems, the EEG amplifier typically acts as the "master" device with the higher sampling rate, and both EEG and fNIRS data are sampled simultaneously using the same clock and ADC, ensuring inherent temporal alignment [37]. This architecture eliminates the need for complex post-hoc realignment and provides perfect sample-to-sample correspondence, which is particularly valuable for analyzing the precise timing relationships between neural events in semantic processing tasks.

Figure 2: Synchronization strategies for discrete EEG-fNIRS systems, emphasizing simultaneous trigger distribution to both modalities.

Table 2: Synchronization Methods for Combined EEG-fNIRS Recordings

Method	Principle	Implementation	Best For
Hardware Trigger	A physical device sends simultaneous electrical pulses to both systems [40] [38].	PortaSync, Parallel Port Replicator, LabStreamer.	Discrete systems with analog/digital inputs; environments with electrical noise.
Software Trigger (LSL)	Network-synchronized time-stamped events are sent over a local network [40].	Stimulus software (e.g., PsychoPy) configured with LSL output.	Flexible lab environments; setups where running cables is impractical.
Inherent Synchronization	Shared electronics (clock, ADC) in an integrated system [37].	Integrated systems like g.Nautilus with NIRx or g.HIamp with g.SENSOR fNIRS.	Studies requiring perfect sample alignment; real-time processing applications.

The Researcher's Toolkit

Establishing a simultaneous EEG-fNIRS laboratory for semantic decoding research requires specific hardware, software, and consumables. The following toolkit outlines essential components and their functions.

Table 3: Essential Research Toolkit for Simultaneous EEG-fNIRS

Category	Item	Specification/Function	Example Products/Tools
Core Hardware	Biosignal Amplifier	Acquires and digitizes EEG signals; may integrate fNIRS.	g.tec g.Nautilus, g.HIamp [37]
	fNIRS Module	Emits NIR light and detects reflected intensity.	g.SENSOR fNIRS, NIRSport2 [39] [37]
	Integrated Cap	Holds EEG electrodes and fNIRS optodes in fixed positions.	g.GAMMAcap [39] [37]
Synchronization	Trigger Device	Generates and distributes synchronization pulses.	PortaSync, NIRx Parallel Port Replicator [40] [38]
Electrodes & Optodes	EEG Electrodes	Active or passive; wet or dry. Ag/AgCl for optimal signal quality [36].	g.SCARABEO electrodes [37]
	fNIRS Optodes	Sources (LEDs/Lasers) and detectors in specific arrangements.	Configurable up to 32 optodes [39]
Software	Acquisition Software	Records synchronized data streams; real-time visualization.	g.HIsys, OxySoft [39] [40]
	Processing Toolbox	Preprocessing, analysis, and visualization of multimodal data.	MNE-Python, HOMER3, EEGlab [41] [42]
Consumables	Electrolyte Gel	Reduces impedance between EEG electrodes and scalp.	Standard EEG conductive gel
	Light-Blocking Material	Prevents ambient light from interfering with fNIRS signals.	Dark cap covers, opaque gauze

Experimental Protocol: A Practical Workflow for Semantic Decoding

This section provides a detailed step-by-step protocol for setting up and running a simultaneous EEG-fNIRS experiment, with a focus on semantic decoding paradigms.

Pre-Experimental Setup

• System Configuration: Power on all equipment. In the acquisition software (e.g., g.HIsys), configure the data stream settings. Set the EEG to a sampling rate of 250 Hz or higher and fNIRS to 10 Hz or higher [41] [42]. Define the channel names and locations according to the international 10-20 system, focusing on language-relevant regions (e.g., frontal, temporal, and parietal sites).
• Synchronization Check: Verify the trigger pathway. Send a test trigger from your stimulus presentation software (e.g., PsychoPy, Presentation) and confirm it is correctly recorded in both the EEG and fNIRS data streams. For LSL, use the LSL lab viewer to monitor the stream.
• Cap Preparation: Select the appropriate g.GAMMAcap size for the participant. Install the fNIRS optodes (sources and detectors) and EEG electrodes into the designated holders according to your experimental montage.

Participant Preparation and Data Acquisition

• Cap Placement: Measure the participant's head to locate the nasion, inion, and preaurical points. Mark the Cz position. Position the cap on the participant's head, aligning its reference points with the measured landmarks. Ensure the cap is snug but comfortable.
• EEG Setup: For wet electrodes, fill the electrodes with conductive gel and abrade the skin to achieve impedances below 20 kΩ for active electrode systems. This process typically takes about 10 minutes for a 32-channel setup with active electrodes [37].
• fNIRS Setup: Verify that all optodes have good contact with the scalp. Use the software's signal quality index to check light levels at each detector. Adjust optode positioning or add light-blocking material if signal quality is poor.
• Baseline Recording: Initiate data acquisition and record a 5-minute resting-state baseline. Instruct the participant to relax with their eyes open, fixating on a crosshair. This baseline is crucial for later data processing.
• Task Presentation: Start the experimental paradigm. For semantic decoding, this may involve word generation tasks, semantic decision tasks, or narrative listening. Ensure that every stimulus onset and participant response is marked with a trigger in the data.

Post-Recording Data Preprocessing

A standardized preprocessing pipeline is essential for ensuring data quality and interpretability.

EEG Preprocessing (using EEGLab or MNE-Python):

• Downsample data to 250 Hz [42].
• Apply a high-pass filter at 1 Hz and a low-pass filter at 40-70 Hz to remove slow drifts and high-frequency noise.
• Remove line noise (e.g., 50/60 Hz) using notch filtering or the cleanline function.
• Reject bad channels and interpolate them using spherical splines.
• Apply artifact subspace reconstruction (ASR) or Independent Component Analysis (ICA) to remove motion and ocular artifacts [42].
• Re-reference to the average reference.

fNIRS Preprocessing (using HOMER3 or MNE-Python):

• Convert raw intensity signals to optical density [41] [43].
• Identify and correct for motion artifacts using Savitzky-Golay filtering, wavelet-based methods, or moving average [43] [42].
• Apply a bandpass filter (e.g., 0.01 - 0.1 Hz) to isolate the hemodynamic response related to neurovascular coupling while suppressing physiological noise like heart rate and respiration [41] [43].
• Convert the filtered optical density data to concentration changes in oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) using the modified Beer-Lambert law [41] [43].

Data Integration: Following modality-specific preprocessing, the synchronized data streams are ready for analysis. Epoch the data around stimulus events, applying baseline correction. The resulting multimodal dataset allows for the investigation of how temporal EEG features (like N400 or P600 event-related potentials) correlate with spatial fNIRS features (like HbO increases in the left inferior frontal gyrus) during semantic processing.

The technical integration of EEG and fNIRS provides a powerful platform for semantic decoding research. By carefully selecting the appropriate level of hardware integration, implementing a robust synchronization strategy, and following a standardized experimental protocol, researchers can reliably capture the complementary neural signatures of language processing. The continued development of wearable, fully integrated systems and sophisticated analysis techniques, including deep learning approaches for artifact removal and data fusion [44] [45], promises to further enhance the utility of this multimodal approach. This will enable increasingly nuanced investigations into the neural basis of semantics in more naturalistic and clinically relevant contexts.

This document outlines detailed application notes and protocols for acquiring neural signals during silent naming and sensory imagery tasks, framed within a research paradigm focused on semantic decoding using simultaneous electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). The core objective of this research is to develop a new type of brain-computer interface (BCI) that enables direct communication of semantic concepts, bypassing the slower character-by-character spelling used in many current systems [46] [8]. These protocols are designed for researchers and scientists investigating the neural correlates of semantic concepts and mental imagery.

Simultaneous EEG-fNIRS is particularly suited for this research as it combines the high temporal resolution of EEG with the superior spatial resolution of fNIRS for cortical areas. While EEG directly measures the brain's electrical activity, fNIRS monitors hemodynamic responses by measuring changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR) [47] [48]. This multimodal approach provides a richer data source for decoding semantic representations from brain activity [49] [8].

Experimental Paradigms and Task Design

The following paradigms are designed to elicit distinct neural patterns for different semantic categories (e.g., animals vs. tools) across various mental tasks. The core structure involves cueing a participant with a specific semantic concept, followed by a period of mental execution without external stimulation [8].

Table 1: Core Mental Tasks for Semantic Decoding

Task Name	Participant Instruction	Example for "Cat" (Animal)	Example for "Hammer" (Tool)	Primary Neural System Engaged
Silent Naming	Silently name the displayed object in your mind.	Silently think "cat".	Silently think "hammer".	Language network [8]
Visual Imagery	Visualize the object in your mind.	Imagine what a cat looks like.	Imagine what a hammer looks like.	Visual association cortex [8]
Auditory Imagery	Imagine the sounds associated with the object.	Imagine a cat meowing.	Imagine the sound of a hammer banging.	Auditory association cortex [50] [8]
Tactile Imagery	Imagine the feeling of touching the object.	Imagine the feeling of petting a cat's fur.	Imagine the feeling of gripping a hammer's handle.	Somatosensory cortex [8]

Stimulus Selection

• Semantic Categories: Use well-defined categories such as animals and tools [8] [28].
• Exemplars: Select 18-20 familiar and easily recognizable items per category (e.g., bear, cat, cow for animals; hammer, screwdriver, saw for tools) [8] [28].
• Stimulus Presentation: Present images of the objects against a neutral background. Use a gray-scale, standardized size (e.g., 400x400 pixels) to minimize low-level visual confounds [8].

Participant Preparation and Materials

Participant Criteria

• Number: 12-20 participants per study is common [49] [8].
• Language: For tasks involving silent naming, recruit native speakers of the language used to ensure consistent neural representation of words [8].
• Health: Participants should have normal or corrected-to-normal vision and no history of neurological disorders [50] [8].

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 2: Essential Equipment and Materials for Simultaneous EEG-fNIRS

Item	Specification/Function	Example Use Case in Protocol
EEG Amplifier	High-quality, wearable amplifier (e.g., g.Nautilus, g.USBamp). Samples electrical activity at a high rate (≥250 Hz).	Acts as "master" device for synchronization; streams real-time data for closed-loop BCI experiments [47].
fNIRS System	Continuous-wave system with multiple sources/detectors (e.g., NIRSport2, Hitachi ETG-4000). Emits NIR light and measures reflected light.	Measures hemodynamic changes in the prefrontal, motor, and parietal cortices during sustained mental tasks [49] [8] [28].
Integrated Cap	EEG cap with pre-defined fNIRS-compatible openings (e.g., g.GAMMAcap). Uses the international 10-20 system for placement.	Ensures fixed, non-overlapping placement of EEG electrodes and fNIRS optodes, reducing artifacts [49] [47].
Active EEG Electrodes	Electrodes with integrated pre-amplification (e.g., g.SCARABEO).	Reduces preparation time to ~10 minutes and improves signal quality by mitigating noise [47].
Stimulus Presentation Software	Software capable of precise timing and triggering (e.g., Presentation, PsychToolbox).	Presents visual/auditory cues and sends synchronization triggers to the EEG-fNIRS acquisition systems [50] [8].
3D Digitizer	Magnetic space digitizer (e.g., Polhemus Fastrak).	Records the precise 3D locations of fNIRS optodes and EEG electrodes relative to anatomical landmarks [49].

Detailed Data Acquisition Protocol

Equipment Setup

• Cap Fitting: Select an appropriate-sized integrated EEG-fNIRS cap. Ensure a tight but comfortable fit to minimize movement artifacts [47] [48].
• Electrode/Optode Placement:
  • Place the fNIRS optodes over the regions of interest, typically covering prefrontal, sensorimotor, and parietal cortices, with an inter-optode distance of 20-30 mm [49] [47].
  • Insert EEG electrodes into the holders interspersed between the optodes. Use electrode gel to achieve good impedance (<10 kΩ).
• Synchronization: Connect the stimulus computer to the EEG amplifier to send TTL pulses or other triggers at the onset of each trial block and event. The EEG amplifier, with its higher sampling rate, typically acts as the master clock [47].

Experimental Procedure Workflow

Figure 1: Experimental workflow for a single trial block, illustrating the sequence from cue presentation to mental task execution and rest.

• Pre-Task:

  • Seat the participant comfortably in a chair, approximately 1 meter from the stimulus screen.
  • The experimenter provides detailed descriptions and examples of all four mental tasks (see Table 1).
  • Run a practice block with stimuli not used in the main experiment to ensure the participant understands the tasks.

• Task Execution (Per Trial):

  • Fixation Cross (2-4 s): The participant focuses on a crosshair to establish a baseline and minimize eye movements.
  • Stimulus Cue (1-2 s): An image of an object (e.g., a cat or a hammer) is displayed.
  • Mental Task (3-5 s): The screen turns blank. A written instruction (e.g., "Imagine Sound") prompts the participant to perform the designated mental task for the block silently and without moving [8].
  • Inter-Trial Interval (6-9 s, jittered): A rest period with a fixation cross allows the hemodynamic response to return to baseline [28].

• Post-Task:

  • Record participant feedback on task engagement and strategy.
  • Anonymize and securely store all data.

Data Processing and Analysis Workflow

The analysis pipeline involves pre-processing both EEG and fNIRS data separately before fusing them for joint analysis or neural decoding.

Figure 2: A simplified data processing and analysis workflow, from raw data to semantic classification.

fNIRS Data Pre-processing

• Convert Raw Intensity: Convert light intensity signals to optical density.
• Remove Physiological Noise: Use Principal Component Analysis (PCA) to remove global physiological signals like heartbeat and respiration [28].
• Filtering: Bandpass filter (e.g., 0.01 - 0.1 Hz) to isolate the task-related hemodynamic signal [28].
• Hemoglobin Conversion: Apply the modified Beer-Lambert law to convert optical density changes into concentration changes of oxygenated (HbO) and deoxygenated hemoglobin (HbR) [49] [28].
• Motion Artifact Correction: Use algorithms (e.g., wavelet transformation, spline interpolation) to identify and correct for motion artifacts [28].

EEG Data Pre-processing

• Filtering: Bandpass filter between 0.1 Hz (or 1.0 Hz) and 40 Hz to remove slow drifts and high-frequency noise.
• Bad Channel Removal: Identify and interpolate or discard channels with excessive noise.
• Artifact Removal: Use Independent Component Analysis (ICA) or regression techniques to remove artifacts from eye blinks, eye movements, and cardiac signals.
• Epoching: Segment the continuous data into epochs time-locked to the onset of the mental task period.
• Baseline Correction: Remove the mean baseline signal from the pre-stimulus period.

Multimodal Data Fusion and Analysis

• Structured Sparse Multiset Canonical Correlation Analysis (ssmCCA): This method fuses EEG and fNIRS data to identify brain regions where both modalities consistently detect task-related activity, improving the reliability of the findings [49].
• Multivariate Pattern Analysis (MVPA) / Machine Learning: Train classifiers (e.g., Support Vector Machines - SVM, Linear Discriminant Analysis - LDA) on features from both EEG (e.g., time-frequency power, event-related potentials) and fNIRS (e.g., HbO/HbR slopes) to distinguish between semantic categories (e.g., animals vs. tools) [8] [28].
• Representational Similarity Analysis (RSA): Abstract neural response patterns to a similarity space and compare them to theoretical models of semantic representation to decode the meaning of the concepts participants are focusing on [28].

These detailed protocols for signal acquisition during silent naming and sensory imagery tasks provide a robust foundation for research in semantic neural decoding. The combination of EEG and fNIRS leverages their complementary strengths, offering a powerful, portable, and ecologically valid method for probing semantic representations in the human brain. Adherence to these protocols, from careful participant instruction and equipment setup to rigorous data processing, will facilitate the generation of high-quality, reproducible data, accelerating the development of intuitive semantic brain-computer interfaces.

Simultaneous electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) recordings offer a powerful multimodal approach for semantic decoding research, combining EEG's millisecond-level temporal resolution with fNIRS's superior spatial localization of cortical hemodynamic activity. The effectiveness of this integrated approach hinges on robust preprocessing pipelines that address the distinct artifact profiles and signal properties of each modality. Carefully executed preprocessing is crucial for isolating neural correlates of semantic processing, such as the N400 event-related potential or hemodynamic responses in language-related networks, from confounding biological and non-biological noise. This application note details standardized protocols for preprocessing simultaneous EEG-fNIRS data, with particular emphasis on optimizing pipelines for semantic decoding experiments.

Unimodal Preprocessing Pipelines

EEG Preprocessing: Workflow and Algorithms

EEG preprocessing aims to isolate neural electrical activity from artifacts originating from ocular movements, muscle activity, cardiac signals, and environmental noise [51]. The pipeline must preserve the integrity of event-related potentials (ERPs) and oscillatory dynamics central to semantic processing.

Table 1: Core Steps in EEG Preprocessing for Semantic Decoding

Step	Purpose	Common Parameters & Methods	Impact on Semantic Decoding
Filtering	Remove non-neural frequency components	High-pass: 0.5-1 Hz; Low-pass: 30-40 Hz; Notch: 50/60 Hz [52]	Preserves N400/P600 components; reduces muscle & line noise
Bad Channel Removal	Identify malfunctioning electrodes	Abnormal variance, correlation, or spectral properties [53]	Ensures data quality for spatial analysis
Re-referencing	Mitigate reference electrode bias	Common Average Reference (CAR) or Mastoid reference [42]	Improves topographic mapping of language ERPs
Artifact Removal	Ocular, muscle, and cardiac correction	ICA (e.g., ICLabel), ASR, MWF, wICA [53] [51]	Critical for preventing confounds in single-trial decoding

The following diagram illustrates a standardized EEG preprocessing workflow, integrating both traditional and advanced cleaning methods:

Figure 1: EEG Preprocessing Workflow. This pipeline highlights key steps including filtering, bad channel handling, and multiple artifact removal strategies. CAR, Common Average Reference; ICA, Independent Component Analysis; ASR, Artifact Subspace Reconstruction; MWF, Multi-channel Wiener Filter; wICA, wavelet-enhanced ICA.

Impact of Preprocessing Choices on Decoding Performance

Preprocessing decisions significantly influence downstream decoding accuracy. A multiverse analysis demonstrated that high-pass filtering with a higher cutoff (e.g., 1 Hz) consistently improved decoding performance across multiple experiments, whereas artifact correction steps like ICA often decreased raw decoding accuracy by removing signal components that were systematically correlated with the task condition [52]. This underscores a critical trade-off: maximizing raw decoding performance may come at the cost of interpretability if the classifier leverages structured noise rather than neural activity. For semantic decoding, it is therefore essential to validate that the features driving classification align with established neural correlates of language processing.

fNIRS Preprocessing: Workflow and Algorithms

fNIRS preprocessing targets motion artifacts, physiological confounds (e.g., heart rate, respiration), and instrumental noise to isolate the hemodynamic response function (HRF) associated with neural activation [54] [18].

Table 2: Core Steps in fNIRS Preprocessing for Semantic Decoding

Step	Purpose	Common Parameters & Methods	Impact on Semantic Decoding
Conversion to OD	Convert raw light intensity	( OD = -\log{10}(I1 / I_0) ) [54]	Foundation for hemoglobin calculation
Motion Correction	Identify/correct movement artifacts	Savitzky-Golay, Wavelet, PCA, CBSI [54] [42]	Essential for block designs with prolonged stimuli
Bandpass Filtering	Isolate HRF from physiology	0.01 - 0.1 Hz [42] (Neurovascular coupling)	Removes cardiac (~1 Hz) & respiratory (~0.4 Hz) noise
Conversion to HbO/HbR	Calculate hemoglobin changes	Modified Beer-Lambert Law (DPF/PPF) [54]	Provides primary (HbO) and secondary (HbR) indicators
Signal Quality Index	Quantify channel quality	SQI algorithm (Scale 1-5) [55]	Informs channel rejection before statistical analysis

The following diagram outlines a standard fNIRS preprocessing pipeline:

Figure 2: fNIRS Preprocessing Workflow. Key steps include conversion to optical density, motion artifact handling, filtering to the neurovascular frequency band, and final conversion to hemoglobin concentrations. SQI, Signal Quality Index; OD, Optical Density; CBSI, Correlation-Based Signal Improvement.

fNIRS Signal Quality and Best Practices

Quantifying signal quality is a critical first step. The Signal Quality Index (SQI) algorithm provides an objective measure on a scale from 1 (very low) to 5 (very high) based on the strength of the cardiac component in the signal, which indicates good optode-scalp coupling [55]. Best practices recommend reporting detailed acquisition parameters (wavelengths, sample rate, source-detector distances), the optode array design and targeted brain regions, and all parameters for motion correction and filtering to ensure reproducibility [18]. For semantic decoding, special attention should be paid to covering classic language areas (e.g., left inferior frontal and temporal regions) with an appropriate optode montage.

Multimodal Integration and Fusion

In simultaneous EEG-fNIRS recordings, the preprocessing pipelines are applied in parallel, but the ultimate power lies in the fusion of the cleaned signals. Multimodal integration capitalizes on the complementary strengths of EEG and fNIRS: EEG provides millisecond-resolution tracking of electrical brain events, while fNIRS offers superior spatial localization of the ensuing hemodynamic response [5].

Advanced data fusion techniques, such as Structured Sparse Multiset Canonical Correlation Analysis (ssmCCA), can identify brain regions where both electrical and hemodynamic activities are consistently detected, strengthening the interpretation of the underlying neural events [4]. For semantic decoding, this could mean more robustly identifying the temporal sequence (from EEG) and spatial topography (from fNIRS) of brain activity as a subject processes words or sentences, providing a more complete picture of the neural basis of language.

Experimental Protocols

Protocol: Preprocessing Simultaneous EEG-fNIRS Data

This protocol outlines a standardized procedure for preprocessing data collected from a semantic categorization task.

Materials and Software

• Programming Environment: MATLAB (R2021a or higher) or Python (3.8 or higher).
• EEG Processing Toolboxes: EEGLAB (v2021.0) [53] with the ICLabel (v1.3) and RELAX (v1.1.0) [53] plugins.
• fNIRS Processing Toolboxes: HOMER3 (v1.80) [54] [42] or MNE-Python (v1.0).
• Computing Specs: Minimum 16 GB RAM, multi-core processor.

Procedure

Part A: EEG Preprocessing (Run in EEGLAB)

• Data Import: Load the raw .bdf or .set file. Store triggers marking stimulus onset (e.g., word presentation).
• Downsampling: Reduce the sampling rate to 250 Hz to minimize computational load [42].
• High-Pass Filtering: Apply a 1 Hz high-pass finite impulse response (FIR) filter to remove slow drifts [52] [42].
• Line Noise Removal: Use the cleanline function to attenuate 50 Hz/60 Hz line noise and its harmonics [42].
• Bad Channel & Epoch Rejection: Apply the clean_rawdata function to automatically identify and remove bad channels (later interpolated) and periods of extreme noise [53].
• Re-referencing: Re-reference the data to the common average reference.
• Artifact Removal: a. Run Independent Component Analysis (ICA). b. Automatically classify components using ICLabel. c. Apply the RELAX pipeline's Multi-channel Wiener Filter (MWF) and/or wavelet-enhanced ICA (wICA) to remove artifacts identified by ICLabel [53].
• Interpolation: Spherically interpolate any removed bad channels.
• Epoching: Segment the continuous data into epochs from -200 ms pre-stimulus to 1000 ms post-stimulus onset, relative to word presentation triggers. Save the cleaned, epoched data.

Part B: fNIRS Preprocessing (Run in HOMER3)

• Data Import: Load the raw .snirf file containing optical data and event markers.
• Convert to Optical Density: Transform raw light intensity signals for each wavelength to optical density (OD) units.
• Signal Quality Assessment: Run the SQI algorithm to rate each channel. Flag channels with a score below 3 for later rejection [55].
• Motion Artifact Detection & Correction: a. Detect motion artifacts using hmrMotionArtifactByChannel (parameters: tMotion=0.5, tMask=2, STDEVthresh=20, AMPthresh=0.5) [54]. b. Correct artifacts using the Savitzky-Golay filtering method (hmrR_MotionCorrectSpline) [42].
• Bandpass Filtering: Apply a 0.01 - 0.1 Hz bandpass filter (FIR) to the OD data to isolate the hemodynamic response [42].
• Convert to Hemoglobin: Use the Modified Beer-Lambert Law to convert filtered OD data to concentration changes of oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR). Assume a partial pathlength factor (PPF) of 6 for both wavelengths [54].
• Channel Rejection: Reject channels previously flagged by the SQI assessment.
• Epoching: Segment the continuous HbO/HbR data into blocks from -5 s pre-stimulus to 25 s post-stimulus onset. Perform baseline correction using the pre-stimulus period.

Part C: Data Fusion (Example using ssmCCA)

• Feature Extraction: For each modality and condition, create feature vectors (e.g., mean HbO from 5-15s post-stimulus for fNIRS; mean amplitude of the N400 window, 300-500ms, for EEG).
• Data Concatenation: Create multimodal data matrices for input into the ssmCCA algorithm, as described in [4].
• Fusion Analysis: Run ssmCCA to identify coupled patterns of activity across the EEG and fNIRS modalities that discriminate between experimental conditions (e.g., congruent vs. incongruent sentences).

The Scientist's Toolkit

Table 3: Essential Research Reagents and Software for EEG-fNIRS Preprocessing

Tool Name	Type	Primary Function	Key Features / Notes
EEGLAB [53]	Software Toolbox (MATLAB)	Interactive EEG data analysis	Extensible environment; supports ICA & many plugins.
RELAX [53]	Plugin (EEGLAB)	Fully automated EEG artifact cleaning	Combines MWF & wICA; reduces need for manual intervention.
HOMER3 [54] [42]	Software Toolbox (MATLAB)	fNIRS data processing & visualization	Standardized pipeline from raw intensity to hemoglobin.
MNE-Python	Software Library (Python)	Open-source Python-based EEG/MEG/fNIRS analysis	Supports EEG-fNIRS integration; includes TDDR motion correction.
ICLabel [53]	Plugin (EEGLAB)	Automated ICA component classification	Uses machine learning to label brain vs. non-brain components.
SQI Algorithm [55]	Algorithm / Tool	Quantitative fNIRS signal quality rating	Rates channels 1-5 based on cardiac component; objective quality control.

Multivariate Pattern Analysis (MVPA) represents a fundamental shift from traditional univariate analysis in neuroimaging. While univariate methods test hypotheses at each voxel or channel independently, MVPA is designed to identify distributed spatial and/or temporal patterns in the data that differentiate between cognitive tasks, stimulus categories, or subject groups [56]. This approach is particularly powerful because it captures information encoded in population activity that may be invisible to methods treating each spatial location separately. In the context of semantic decoding using simultaneous EEG-fNIRS, MVPA leverages the complementary strengths of both modalities: the excellent temporal resolution of EEG (millisecond precision) and the superior spatial localization of fNIRS (several centimeters depth sensitivity) [8] [5].

Machine learning (ML) provides the computational framework for implementing MVPA in neural decoding problems. Fundamentally, neural decoding is a regression or classification problem that uses brain signals to predict external variables or states [57]. Modern ML tools, including neural networks and ensemble methods, have demonstrated superior performance compared to traditional linear decoding methods like Wiener or Kalman filters, particularly for capturing nonlinear relationships in neural data [57]. The integration of MVPA and ML is especially valuable for semantic neural decoding, which aims to identify which semantic concepts an individual is processing based on their brain activity patterns [8] [58].

Table 1: Comparison of Neural Decoding Approaches

Feature	Univariate Analysis	Multivariate Pattern Analysis (MVPA)
Basic Unit of Analysis	Individual voxels/channels	Distributed patterns across multiple voxels/channels
Sensitivity	Localized activation	Network-level interactions and distributed representations
Information Capture	Magnitude of response	Pattern of response across regions
Typical Applications	Localizing function	Decoding states, content, and representations
Dimensionality	Single features	High-dimensional feature spaces

MVPA Fundamentals and Theoretical Framework

MVPA operates on the principle that information is distributed across populations of neurons rather than isolated in single units. This framework is particularly suited for studying higher-order cognitive functions like semantic processing, where concepts are represented across distributed cortical networks [56] [28]. The core mathematical foundation of MVPA involves latent-variable projection methods that handle the multicollinearity typical of neuroimaging data [59].

A key advantage of MVPA for semantic decoding is its ability to detect subtle population codes that differentiate between conceptual categories. For instance, distributed patterns can distinguish between animals and tools during silent naming or imagery tasks [8]. The method is sensitive to spatially covarying patterns of activity, making it intrinsically linked to functional connectivity analyses that seek to uncover functional networks in the brain [56].

MVPA can be implemented through various projection algorithms, with partial least squares (PLS) regression being particularly valuable for handling multicollinear data. The projection algorithm consists of four key steps: (1) selecting a normalized weight vector, (2) calculating score vectors, (3) calculating loading vectors, and (4) removing dimensions through orthogonalization [59]. For semantic decoding applications, target projection (TP) can be used to produce a single predictive latent variable that quantifies the association pattern between neural activity and semantic categories [59].

Machine Learning Approaches for Neural Decoding

Method Selection and Implementation

Selecting appropriate machine learning methods depends on the specific research aims. For engineering applications like brain-computer interfaces (BCIs), maximizing predictive accuracy is paramount, and modern ML methods generally provide significant benefits [57]. When the goal is understanding what information is contained in neural activity, ML can determine how much information a neural population contains about an external variable, though caution is needed in interpretation [57].

Neural networks and gradient boosting ensembles have demonstrated particularly strong performance in neural decoding tasks. Comparative studies on recordings from motor cortex, somatosensory cortex, and hippocampus show these methods significantly outperform traditional approaches [57]. For semantic decoding of categories like animals and tools, these methods can leverage the complex, nonlinear relationships in simultaneous EEG-fNIRS recordings.

Best Practices and Validation

Proper implementation of ML for neural decoding requires careful attention to data formatting, model testing, and hyperparameter optimization. Cross-validation is essential to avoid overfitting, particularly with high-dimensional neural data [57]. For semantic decoding tasks, it's crucial to separate the cue presentation period from the mental task period in the experimental design to properly validate the decoding approach [8].

A critical consideration is the interpretation limitations of ML models. While they excel at prediction, the mathematical transformations within most ML decoders are not directly interpretable as biological mechanisms. High decoding accuracy does not necessarily mean that a brain area is directly involved in processing the decoded information [57]. However, ML methods serve as valuable benchmarks for simpler models - if a hypothesis-driven decoder performs much worse than ML methods, it likely misses key aspects of the neural code [57].

Application to Semantic Decoding with EEG-fNIRS

Experimental Paradigms and Design

Semantic decoding research has employed various mental tasks to investigate the feasibility of distinguishing semantic categories. Silent naming tasks require participants to silently name displayed objects, while mental imagery tasks involve visualizing objects, imagining associated sounds, or imagining the feeling of touching objects [8]. These approaches leverage the principle that perceiving objects and imagining them elicit similar brain activity patterns [8].

Successful experimental designs for EEG-fNIRS semantic decoding typically involve:

• Blocked trials with randomized stimulus presentation across multiple blocks
• Jittered interstimulus intervals (e.g., 6-9 seconds) to allow hemodynamic responses to return to baseline
• Multiple trials per category (e.g., 12 blocks with randomized stimulus order)
• Precise timing synchronization between EEG and fNIRS recordings [8] [28]

Table 2: Mental Tasks for Semantic Decoding

Task Type	Description	Modality Relevance	Semantic Categories
Silent Naming	Silently naming displayed objects	Engages language networks	Animals, Tools
Visual Imagery	Visualizing objects in mind	Strong visual cortex activation	Animals, Tools
Auditory Imagery	Imagining sounds objects make	Auditory association cortex	Animals, Tools
Tactile Imagery	Imagining feeling of touching objects	Somatosensory cortex	Animals, Tools

Data Acquisition and Preprocessing

Simultaneous EEG-fNIRS acquisition requires careful hardware integration. The EEG amplifier typically acts as the "master device" with higher sampling frequency, while fNIRS provides complementary hemodynamic information [60]. Practical implementation uses EEG electrodes placed between fNIRS optodes, with a dark electrode cap material to prevent ambient light distortions [60].

fNIRS preprocessing typically includes:

• Principal components analysis on optical density data to remove nonneural physiological signals
• Bandpass filtering (e.g., high pass: 0.01 Hz, low pass: 1.0 Hz)
• Conversion to hemoglobin concentrations using the modified Beer-Lambert law
• Motion artifact removal by masking windows around observations exceeding standard deviation thresholds [28]

EEG preprocessing focuses on:

• Artifact removal for ocular activity (EOG) and muscle activity (EMG)
• Spatial filtering to improve signal-to-noise ratio
• Temporal alignment with fNIRS hemodynamic responses

Channel stability analysis adapted from fMRI decoding can determine which channels produce reliable responses across multiple blocks, independent of discriminability among stimulus classes. This procedure reduces dimensionality and increases signal-to-noise ratio by including consistently responsive channels while excluding noisy ones [28].

Analytical Protocols for EEG-fNIRS Semantic Decoding

Multimodal Fusion Strategies

Data fusion for EEG-fNIRS can be implemented at multiple levels:

1. Data Concatenation: Combining features from both modalities into a single input matrix for classification. This approach requires careful normalization to address the different scales and temporal characteristics of EEG and fNIRS data [27].

2. Model-Based Fusion: Using structured models that account for neurovascular coupling relationships between electrical and hemodynamic activity. These approaches can incorporate physiological priors about the expected timing relationships [27].

3. Decision-Level Fusion: Running separate decoders on each modality and combining their outputs through voting or weighted averaging schemes. This approach preserves modality-specific processing while leveraging complementary information [27].

4. Source Decomposition Techniques: Methods like joint independent component analysis (ICA) that identify latent components shared across modalities. These unsupervised symmetric techniques can reveal complex neurovascular coupling processes without requiring stimulus timing information [27].

Representational Similarity Analysis

Representational similarity analysis (RSA) provides a powerful framework for semantic decoding with EEG-fNIRS. This approach abstracts response patterns to similarity spaces and compares them to theoretical models of semantic representation [28]. The procedure involves:

• Creating neural representational similarity matrices (RSMs) based on correlation distances between response patterns for different stimuli
• Creating model RSMs based on theoretical semantic models (e.g., distributional semantic models)
• Comparing neural and model RSMs using cross-validation approaches (e.g., leave-one-subject-out)
• Statistical testing against chance-level accuracy using permutation tests [28]

This approach has successfully decoded semantic categories like animals and body parts from fNIRS data, demonstrating that semantic representations are encoded in fNIRS signals and preserved across subjects [28].

Practical Implementation and Protocols

Detailed Experimental Protocol for Semantic Category Decoding

Objective: To distinguish between semantic categories (animals vs. tools) during mental imagery tasks using simultaneous EEG-fNIRS.

Participants:

• 12+ right-handed native speakers (for language tasks)
• Normal or corrected-to-normal vision
• No neurological or psychiatric conditions

Stimuli:

• 18 animals and 18 tools selected for recognizability and suitability across sensory modalities
• Images converted to grayscale, cropped to 400×400 pixels, contrast stretched
• Presentation against white background [8]

Procedure:

• Equipment Setup: Apply EEG cap with integrated fNIRS optodes. For 32 EEG channels + fNIRS, preparation time is approximately 10 minutes with active electrode technology [60].
• Task Instruction: Participants perform four mental tasks in randomized blocks:
  • Silent Naming: Silently name the displayed object
  • Visual Imagery: Visualize the object in their mind
  • Auditory Imagery: Imagine sounds the object makes
  • Tactile Imagery: Imagine feeling of touching the object [8]
• Trial Structure:
  • Stimulus presentation: 3 seconds
  • Mental task period: 3-5 seconds
  • Inter-stimulus interval: 6-9 seconds (jittered) [8]
• Data Acquisition:
  • EEG sampling: ≥500 Hz
  • fNIRS sampling: ≥10 Hz
  • Precise synchronization of modalities

Data Analysis Pipeline:

• Preprocessing (separate for each modality)
• Feature Extraction: Temporal features for EEG, hemodynamic response features for fNIRS
• Data Fusion: Implement one or more fusion strategies
• MVPA Classification: Using neural networks or ensemble methods with cross-validation
• Statistical Evaluation: Compare accuracy against chance levels

The Scientist's Toolkit: Research Reagent Solutions

Table 3: Essential Materials and Software for EEG-fNIRS Semantic Decoding

Item	Function	Example Specifications
EEG-fNIRS Integrated System	Simultaneous acquisition of electrical and hemodynamic signals	Active EEG electrodes, fNIRS optodes with 20-30mm separation, dark cap material to prevent light contamination [60]
Stimulus Presentation Software	Controlled delivery of semantic stimuli	Precision timing for visual and auditory presentation, jittered ISI, block randomization
Homer2	fNIRS preprocessing pipeline	Optical density conversion, PCA-based noise removal, bandpass filtering, hemoglobin calculation [28]
MVPA R Package	Multivariate pattern analysis implementation	Handling multicollinear data, projection algorithms, visualization tools [59]
Neural Decoding Code Package	Machine learning for neural data	Github.com/kordinglab/neural_decoding with implementations of neural networks, gradient boosting [57]
NIRSport2 System	High-density fNIRS capability	16 sources, 16 detectors arrangement for enhanced spatial sampling [60]

Validation and Interpretation

Performance Metrics and Statistical Validation

Rigorous validation of semantic decoding performance requires:

• Cross-validation: Leave-one-subject-out or k-fold cross-validation to avoid overfitting
• Permutation testing: Comparing actual decoding accuracy to null distribution generated by randomly shuffling labels
• Channel stability analysis: Identifying channels with reliable responses across blocks [28]
• Temporal generalization: Testing whether models trained at one timepoint generalize to others

Successful semantic decoding typically achieves classification accuracy significantly above chance (e.g., >70% for binary classification where chance is 50%), with statistical significance determined through permutation tests (p < 0.05, corrected for multiple comparisons) [8] [28].

Interpretation and Limitations

When interpreting semantic decoding results, several considerations are crucial:

• High decoding accuracy does not necessarily indicate that a brain area's primary function is representing the decoded information [57]
• Causal interpretations require caution even when neural signals temporally precede responses [57]
• Stimulus confounds must be controlled (e.g., low-level visual features rather than semantic content) [57]
• Generalizability across participants and tasks should be tested rather than assumed

The spatial limitations of fNIRS (cortical coverage, depth sensitivity ~3cm) and volume conduction effects in EEG constrain the neural sources that can be successfully decoded. However, the complementary nature of these modalities provides a unique window into both rapid electrical dynamics and slower hemodynamic changes associated with semantic processing [5] [28].

MVPA and machine learning approaches for decoding semantic information from simultaneous EEG-fNIRS recordings represent a powerful toolkit for investigating how the brain represents conceptual knowledge. The integration of these analytical methods with multimodal neuroimaging enables researchers to capture both the rapid temporal dynamics and spatial distribution of semantic representations.

Future methodological developments will likely focus on:

• Advanced fusion algorithms that more effectively leverage neurovascular coupling
• Deep learning approaches tailored to the specific characteristics of EEG-fNIRS data
• Real-time decoding applications for brain-computer interfaces and neurofeedback
• Individual difference modeling to account for variability in semantic representations across people

As these methods continue to mature, they promise to advance both basic science understanding of semantic cognition and translational applications in clinical populations with communication impairments.

Application Notes

Semantic Brain-Computer Interfaces (BCIs) represent a paradigm shift from traditional character-based communication systems, aiming to decode conceptual meaning directly from neural activity. This approach bypasses the sequential character spelling used in current BCIs, potentially enabling more intuitive and efficient communication, particularly for individuals with severe motor impairments [8]. The core principle involves identifying which semantic concepts an individual is focusing on at a given moment by analyzing patterns in their brain signals [8].

The transition toward direct semantic decoding addresses critical limitations in communication rate (bit rate) and cognitive load associated with assistive communication devices. Research indicates that semantic BCIs can leverage distributed neural representations of concepts, which are preserved across subjects and can be identified using multivariate pattern analysis (MVPA) techniques [28]. Table 1 summarizes key performance metrics and technical specifications for semantic decoding approaches using different neural signals.

Table 1: Quantitative Performance Metrics for Semantic Neural Decoding

Metric / Specification	EEG-fNIRS Combination	fNIRS Alone	Implanted Microelectrode Arrays (Inner Speech)
Primary Signal Type	Electrical activity & hemodynamic changes [8]	Hemodynamic changes [28]	Neural spiking activity [61]
Spatial Resolution	~2 cm (EEG); centimeter scale (fNIRS) [8]	Centimeter scale [28] [62]	Single neuron level (local field potentials) [61]
Temporal Resolution	Millisecond (EEG); ~100 Hz (fNIRS) [8] [62]	~100 Hz (millisecond precision) [62]	High (direct neural recording) [61]
Invasiveness	Non-invasive	Non-invasive	Surgically implanted [61]
Portability	High (portable, cost-effective) [8]	High [63] [62]	Low (currently requires external cable) [61]
Reported Decoding Accuracy	Differentiates 2 semantic categories (animals vs. tools) [8]	Significant above-chance accuracy [28]	Proof-of-principle for inner speech [61]

Recent studies demonstrate the feasibility of differentiating between broad semantic categories, such as animals and tools, from non-invasive neural signals during various mental imagery tasks [8]. The integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) is particularly promising for developing practical semantic BCIs. This multimodal approach combines EEG's excellent temporal resolution with fNIRS's better spatial resolution and portability, creating a synergistic effect that enhances ecological validity and decoding potential [8] [63]. Furthermore, pioneering work with intracortical BCIs has begun to decode inner speech (imagined speech), which could provide a more rapid and comfortable communication channel than systems relying on attempted physical speech [61].

Experimental Protocols

Protocol: Semantic Category Decoding with Simultaneous EEG-fNIRS

This protocol details a validated methodology for acquiring a dataset to investigate the decoding of semantic concepts from simultaneous EEG and fNIRS recordings [8].

Participant Selection and Preparation

• Participants: Recruit right-handed native speakers (e.g., English for the silent naming task) to minimize variability in neural representation of semantic concepts due to language differences. Sample sizes of 12 participants have been used in prior studies [8].
• Screening: Ensure participants have normal or corrected-to-normal vision. Exclude individuals with a history of neurological or psychiatric disorders.
• Consent: Obtain written informed consent approved by an institutional review board or ethics committee.

Stimuli and Experimental Design

• Stimulus Set: Select images representing concepts from distinct semantic categories (e.g., 18 animals and 18 tools). Use gray-scale, high-contrast images on a white background [8].
• Mental Tasks: Implement a block-designed paradigm where participants perform the following tasks in randomized order upon viewing each image:
  • Silent Naming: Silently name the object in their mind.
  • Visual Imagery: Visualize the object in their mind.
  • Auditory Imagery: Imagine the sounds associated with the object.
  • Tactile Imagery: Imagine the feeling of touching the object [8].
• Trial Structure: Each trial begins with a visual stimulus presentation (e.g., 3 seconds), followed by the mental task period (e.g., 3-5 seconds), and a jittered inter-trial interval (e.g., 6-9 seconds) to allow the hemodynamic response to return to baseline [8] [28]. The sequence is repeated across multiple blocks.

Data Acquisition Parameters

• EEG Acquisition: Record using a standard cap system with appropriate electrode layout. Sampling rates typically exceed 200 Hz to capture event-related potentials.
• fNIRS Acquisition: Use a continuous-wave fNIRS system. Configure optode arrays to cover brain regions of interest, such as:
  • A posterior array over the occipital lobe for visual processing.
  • A lateral array over the temporal, parietal, and prefrontal lobes for language and semantic processing [28].
  • A source-detector distance of 3-4 cm is standard to ensure sensitivity to cortical brain tissue [62]. Record both oxygenated (HbO) and deoxygenated (HbR) hemoglobin concentrations.

Data Preprocessing and Analysis

• fNIRS Preprocessing (using tools like Homer2):
  • Convert raw light intensity to optical density.
  • Perform principle component analysis (PCA) to remove global physiological noise (e.g., heartbeat, blood pressure fluctuations) [28].
  • Band-pass filter (e.g., 0.01 - 0.5 Hz) to isolate the hemodynamic signal.
  • Convert to hemoglobin concentration changes using the modified Beer-Lambert law [28].
  • Perform channel stability analysis to identify and retain channels with reliable responses across blocks [28].
• EEG Preprocessing:
  • Apply band-pass filtering (e.g., 0.1 - 40 Hz).
  • Remove artifacts (e.g., ocular, muscle).
• Multivariate Pattern Analysis (MVPA):
  • Extract features from preprocessed EEG (e.g., spectral power) and fNIRS (e.g., HbO/HbR concentration) data.
  • Use machine learning classifiers (e.g., support vector machines, neural networks) to decode the semantic category (e.g., animal vs. tool) or the mental task type.
  • Validate model performance using leave-one-subject-out or k-fold cross-validation.

Figure 1: Experimental workflow for simultaneous EEG-fNIRS semantic decoding.

Protocol: Inner Speech Decoding with Intracortical BCIs

This protocol summarizes methods for investigating inner speech using microelectrode arrays, a key step toward restoring rapid communication [61].

Participant and Hardware Setup

• Participants: Individuals with severe speech and motor impairments (e.g., from amyotrophic lateral sclerosis or brainstem stroke) who are eligible for surgical implantation.
• Implantation: Surgically implant microelectrode arrays (e.g., smaller than a pea) into speech-related areas of the motor cortex. These arrays record neural activity at the level of individual neurons or local field potentials [61].

Experimental Paradigm

• Task: Participants are instructed to imagine speaking words or phrases (inner speech) without attempting any physical movement. For example, they might imagine saying the word "hello" or a short sentence.
• Stimuli: Cue the participant with text or an auditory prompt indicating what to imagine.
• Safety and Ethics: Given the sensitivity of decoding internal thought processes, implement privacy safeguards, such as password protection systems where the user must imagine a specific passphrase to activate decoding [61].

Data Analysis and Decoding

• Neural Feature Extraction: Extract firing rates or power features from recorded neural signals time-locked to the cue.
• Phoneme-Based Decoding:
  • Train machine learning models (e.g., recurrent neural networks) to recognize patterns of neural activity associated with phonemes—the smallest units of speech [61].
  • The decoding algorithm stitches the identified phonemes together into words and sentences.
• Privacy-Centric Training: For BCIs designed to decode attempted speech, include training data that allows the model to learn to ignore inner speech, preventing accidental "leakage" of private thoughts [61].

The Scientist's Toolkit: Research Reagent Solutions

Table 2: Essential Materials and Tools for Semantic BCI Research

Item Name	Function / Application	Technical Notes
High-Density EEG System	Records electrical brain activity from the scalp surface with high temporal resolution.	Critical for capturing millisecond-scale dynamics of semantic processing. Often integrated with fNIRS in hybrid systems [8].
fNIRS System (e.g., Hitachi ETG-4000)	Measures cortical hemodynamic activity (changes in HbO and HbR) via near-infrared light.	Provides better spatial resolution than EEG and is portable. Optode placement over language areas is crucial [8] [28].
Microelectrode Arrays (e.g., Utah Array)	Records neural spiking activity and local field potentials directly from the cortical surface.	Used in invasive BCIs for high-fidelity decoding, such as inner speech. Provides the highest signal quality [61].
Stimulus Presentation Software (e.g., Psychtoolbox, Presentation)	Prescribes the precise timing and delivery of visual/auditory cues to participants.	Ensures experimental rigor and reproducibility of the paradigm [8].
Homer2 Software	An open-source environment for fNIRS data preprocessing and analysis.	Standardized platform for filtering, converting optical density, and calculating hemoglobin concentrations [28].
Multivariate Pattern Analysis (MVPA) Toolboxes (e.g., Scikit-learn, PRoNTo)	Provides machine learning algorithms for decoding semantic information from neural patterns.	Enables classification of neural data into semantic categories (e.g., animal vs. tool) [8] [28].

Figure 2: Logical pathway of semantic concept decoding from multimodal signals.

Neurodegenerative diseases (NDDs), such as Alzheimer's disease (AD), Parkinson's disease (PD), and amyotrophic lateral sclerosis (ALS), represent a growing global health challenge, characterized by progressive loss of neuronal function and structure [64]. The clinical translation of advanced neuroimaging technologies is critical for addressing fundamental limitations in current diagnostic and therapeutic paradigms. The integration of functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) into a dual-modality imaging system presents a powerful platform for probing brain function by simultaneously capturing hemodynamic responses and electrophysiological activity [22]. This combination is particularly valuable for investigating the neurovascular coupling mechanism—the intimate relationship between neuronal energy demand and local cerebral blood flow—which is increasingly recognized as a key factor in the pathophysiology of various NDDs [64] [65].

Within the context of semantic decoding research, which aims to identify specific semantic concepts from brain activity patterns, simultaneous fNIRS-EEG recordings offer complementary data streams that may enhance decoding accuracy for developing more intuitive brain-computer interfaces (BCIs) [66]. This approach is now being extended to identify disease-specific biomarkers and track therapeutic interventions. The fNIRS-EEG platform offers several advantages for clinical translation, including portability, tolerance to movement artifacts, relatively low cost, and the ability to conduct repeated measurements in naturalistic settings, making it suitable for long-term monitoring of disease progression and treatment response [22] [64] [65]. This application note details specific protocols and analytical frameworks for applying fNIRS-EEG technology to neurodegenerative disease monitoring and CNS drug development.

Technical Foundation and Advantages of fNIRS-EEG Systems

The fNIRS-EEG dual-modality system technically complements each other by measuring different but related physiological phenomena. EEG records electrical potentials generated by synchronized postsynaptic neuronal activity on the millisecond timescale, providing excellent temporal resolution but limited spatial resolution [22] [65]. fNIRS measures hemodynamic changes associated with neural activity by detecting light attenuation in the near-infrared spectrum (650-950 nm) to quantify concentration changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR) [64]. This provides better spatial resolution than EEG and direct insight into brain metabolism, albeit with slower temporal response due to the hemodynamic nature of the signal.

Table 1: Comparison of Neuroimaging Modalities for Neurodegenerative Disease Applications

Feature	fNIRS	EEG	fMRI	PET
Temporal Resolution	~10 Hz [64]	256–1,024 Hz [64]	1-3 Hz [64]	Extremely Low [64]
Spatial Resolution	Low to Moderate [64]	Low [64]	Extremely High [64]	High [64]
Measured Biomarkers	HbO, HbR [64]	Electrical Potentials [65]	BOLD Signal [65]	Radioactive Tracers [67]
Cost	Low [64]	Low [64]	High [64]	High [64]
Portability & Tolerance to Motion	High [65]	High [65]	Low [65]	Low [67]
Key Strength in NDDs	Hemodynamic monitoring in natural settings [68]	Millisecond-level neural dynamics [69]	Whole-brain structural and functional mapping	Molecular and metabolic profiling

The synergistic integration of fNIRS and EEG allows for the extraction of neurovascular coupling-related features, which may provide more sensitive and accurate biomarkers for neurological dysfunction than either modality alone [65]. For instance, the relationship between EEG-derived power in specific frequency bands and fNIRS-measured HbO concentration can serve as a quantitative index of local metabolic demand relative to electrophysiological activity [67]. This is particularly relevant for neurodegenerative conditions where early neurovascular uncoupling has been observed [64].

Application Protocols for Neurodegenerative Disease Monitoring

Protocol 1: Resting-State Functional Connectivity Assessment

Objective: To quantify alterations in baseline brain network integrity in neurodegenerative diseases using resting-state fNIRS-EEG.

Background: Resting-state brain activity, measured in the absence of a task, yields valuable information about intrinsic functional network organization, which is disrupted in NDDs [68]. rsfNIRS has shown promise in identifying these alterations [68].

• Participant Preparation: Instruct the participant to remain awake, relaxed, and with their eyes open, fixed on a crosshair for a duration of 5-10 minutes. Minimize sensory input in the testing environment [68].
• Data Acquisition: Simultaneously collect fNIRS (HbO and HbR concentrations) and EEG (raw electrical potentials) data. A minimum of two channels is required, but a higher-density montage covering frontal, temporal, and parietal regions is recommended for comprehensive network analysis [68].
• Pre-processing:
  • fNIRS: Convert raw light intensity signals to optical density, then to HbO and HbR concentrations using the modified Beer-Lambert law. Apply band-pass filtering (0.01-0.1 Hz) to isolate low-frequency oscillations characteristic of resting-state networks. Remove motion artifacts using wavelet or principal component analysis-based methods [68].
  • EEG: Apply band-pass filtering (e.g., 0.5-70 Hz) and notch filtering (50/60 Hz). Remove artifacts from eye blinks, muscle movement, and cardiac activity using independent component analysis (ICA) [65].
• Data Analysis:
  • Functional Connectivity (fNIRS): Calculate the correlation between the pre-processed HbO time series from different brain regions to construct a connectivity matrix. Graph theory metrics (e.g., clustering coefficient, path length) can then be extracted to quantify network efficiency [68].
  • Functional Connectivity (EEG): Compute synchronization metrics such as the Phase Lag Index (PLI) or wavelet coherence within standard frequency bands (delta, theta, alpha, beta, gamma) to assess functional connectivity between electrode pairs [65].
  • Multimodal Integration: Correlate fNIRS-based and EEG-based connectivity maps to identify regions where hemodynamic and electrophysiological network integrity are jointly impaired.

Protocol 2: Semantic Decoding and Cognitive Task Paradigm

Objective: To differentiate between semantic categories (e.g., animals vs. tools) using brain activity patterns during mental imagery tasks, with applications for assessing cognitive deficits in NDDs.

Background: Semantic neural decoding identifies which concepts an individual focuses on based on brain activity, forming a basis for advanced BCIs and cognitive assessment tools [66]. This protocol is framed within semantic decoding research using simultaneous fNIRS-EEG.

• Participant Preparation: Familiarize the participant with the visual stimuli (images of animals and tools) before the experiment to ensure correct identification [66].
• Experimental Design: Employ a block or event-related design. In each trial, present an image for a few seconds, followed by a cue instructing the participant to perform a specific mental task for a 3-second period [66].
• Mental Tasks (executed in randomized blocks):
  • Silent Naming: Silently name the displayed object in one's mind [66].
  • Visual Imagery: Visualize the object and its appearance [66].
  • Auditory Imagery: Imagine the sounds associated with the object [66].
  • Tactile Imagery: Imagine the feeling of touching the object [66].
• Data Acquisition: Record simultaneous fNIRS-EEG data. A cap integrating both modalities is essential. For semantic tasks, coverage should include frontal, temporal, and parietal language and association areas [22] [66].
• Data Analysis:
  • Feature Extraction: For fNIRS, extract the mean HbO and HbR concentration changes during the task period relative to baseline. For EEG, extract power spectral densities in standard frequency bands or event-related potential (ERP) components like the P300 [66] [67].
  • Machine Learning: Train a classifier (e.g., Support Vector Machine, Linear Discriminant Analysis) using features from both modalities to distinguish between the two semantic categories (animals vs. tools). Cross-validate the results to ensure generalizability [66].

Diagram 1: Semantic decoding workflow for fNIRS-EEG.

Protocol 3: Motor Function Assessment in Stroke and Parkinson's Disease

Objective: To evaluate cortical reorganization and motor network connectivity during motor tasks in patients with motor deficits following stroke or PD.

Background: Stroke and PD lead to significant reorganization of the motor network. Combined fNIRS-EEG can track both the hemodynamic and electrical correlates of this plasticity, providing prognostic biomarkers and guiding rehabilitation [67] [65].

• Participant Preparation: Position the patient comfortably in front of a table. Attach sensors for electromyography (EMG) if measuring muscle activity.
• Task Paradigm: Utilize a block design alternating between rest and activity. Motor tasks can include:
  • Repetitive fist clenching (affected and unaffected hand).
  • Reaching and grasping objects.
  • Finger tapping.
• Data Acquisition: Ensure fNIRS optodes and EEG electrodes cover the primary motor cortex (M1), supplementary motor area (SMA), and premotor cortex bilaterally. A 3D digitizer should be used for accurate co-registration of sensor positions with brain anatomy [65].
• Data Analysis:
  • Quantitative EEG (qEEG): Calculate the Power Ratio Index (PRI: slow-wave/fast-wave power) and the Brain Symmetry Index (BSI). Higher PRI and BSI values post-stroke are correlated with poorer motor outcomes [65].
  • fNIRS Hemodynamics: Analyze the amplitude and latency of HbO responses in motor areas during movement of the affected versus unaffected limb.
  • Multimodal Correlation: Relate the lateralization of HbO activation in the motor cortex with the BSI derived from EEG to create a composite biomarker of motor recovery.

Application Protocols for CNS Drug Development

Protocol 4: Biomarker-Driven Target Engagement and Efficacy Testing

Objective: To utilize fNIRS-EEG biomarkers for confirming central target engagement and tracking functional outcomes in clinical trials for CNS therapeutics.

Background: EEG biomarkers are transforming CNS drug discovery by providing objective, quantifiable measures of brain activity to screen targets, confirm engagement, and track functional outcomes [69]. Integrating fNIRS adds a layer of metabolic validation.

• Study Design: A randomized, double-blind, placebo-controlled, crossover design is optimal. Include baseline (pre-dose), multiple post-dose measurements, and a follow-up.
• Data Acquisition: Conduct simultaneous fNIRS-EEG recordings during standardized tasks (e.g., cognitive tasks from Protocol 2 or motor tasks from Protocol 3) and at resting-state.
• Key Biomarkers for Analysis:
  • EEG Biomarkers: Quantitative EEG (qEEG) power spectra (Delta, Theta, Alpha, Beta, Gamma), event-related potentials (ERPs like P300), and functional connectivity metrics [69].
  • fNIRS Biomarkers: Task-evoked HbO/HbR response amplitude, spatial extent of activation, and resting-state functional connectivity [67].
• Data Analysis:
  • Target Engagement: Compare drug vs. placebo groups for significant changes in pre-specified EEG/fNIRS biomarkers at early post-dose time points. For example, a drug targeting cognitive enhancement should modulate alpha or gamma power and enhance the P300 amplitude [69].
  • Functional Outcome: Correlate changes in neuroimaging biomarkers from baseline to end-of-treatment with changes in clinical scale scores (e.g., ADAS-Cog for Alzheimer's, UPDRS for Parkinson's). This establishes a link between the physiological signal and clinical efficacy.

Table 2: Key fNIRS-EEG Biomarkers in CNS Drug Development for Neurodegenerative Diseases

Therapeutic Area	EEG Biomarkers	fNIRS Biomarkers	Clinical Correlates
Alzheimer's Disease	Reduction in Alpha & Beta power; Increase in Theta/Alpha ratio; Slowing of peak frequency [69]	Reduced prefrontal HbO during cognitive tasks; Altered resting-state connectivity [68]	Cognitive decline (MMSE, MoCA)
Parkinson's Disease	Abnormal Beta band synchronization in basal ganglia-cortical circuits [67]	Reduced HbO in motor cortex during movement; Altered neurovascular coupling [64]	Motor severity (UPDRS)
Depression	Frontal Alpha Asymmetry; altered reward-related theta activity [69]	Prefrontal cortex hyperactivity/ hypoactivity during emotional tasks	Mood scales (HAMD, MADRS)
Epilepsy	Interictal spike frequency; seizure pattern detection [69]	Ictal hyperperfusion/ post-ictal hypoperfusion mapped with fNIRS [22]	Seizure frequency and severity

Diagram 2: Drug trial biomarker assessment workflow.

The Scientist's Toolkit: Essential Research Reagent Solutions

Table 3: Key Research Reagents and Materials for fNIRS-EEG Experiments

Item	Function/Description	Application Note
Integrated fNIRS-EEG Cap	A helmet or cap with embedded EEG electrodes and fNIRS optodes allowing simultaneous data acquisition from co-registered brain areas.	Customizable 3D-printed or thermoplastic shells improve fit and reduce motion artifacts [22].
fNIRS Light Sources & Detectors	Emits near-infrared light (e.g., lasers/LEDs at 760 & 850 nm) and detects attenuated light after passing through tissue.	Enables calculation of HbO and HbR concentration changes [64].
EEG Amplifier	Amplifies microvolt-level electrical potentials from the scalp for digitization.	High-input impedance and high sampling rate (≥500 Hz) are essential for capturing neural signals [65].
Conductive EEG Gel/Paste	Reduces impedance between the scalp and EEG electrodes for high-quality signal acquisition.	Saline-based or water-based solutions offer a compromise between conductivity and ease of use [70].
3D Spatial Digitizer	A magnetic or optical system to record the precise 3D locations of fNIRS optodes and EEG electrodes on the head.	Critical for accurate co-registration of data with brain anatomy (e.g., MNI space) [22].
Physiological Monitors	Records electrocardiography (ECG), electromyography (EMG), and respiration.	Aids in identifying and removing physiological artifacts (e.g., heartbeat) from fNIRS and EEG signals [65].

The integration of fNIRS and EEG into a unified diagnostic and monitoring platform holds significant promise for advancing the clinical management of neurodegenerative diseases and accelerating CNS drug development. The protocols outlined herein provide a framework for employing this technology to extract robust, multimodal biomarkers of disease state and progression. These biomarkers, ranging from resting-state connectivity maps and semantic decoding accuracy to power ratio indices and neurovascular coupling parameters, offer a more nuanced and comprehensive view of brain health than traditional clinical assessments alone. As the field moves forward, the adoption of standardized reporting guidelines and the further development of analytical methods for multimodal data fusion will be crucial for validating these approaches and translating them from research tools into routine clinical and pharmaceutical practice.

Overcoming Technical Hurdles: A Guide to Optimizing EEG-fNIRS Signal Quality

The fusion of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) provides a powerful multimodal approach for investigating brain function, combining EEG's millisecond-level temporal resolution with fNIRS's centimeter-scale spatial localization [71]. This complementary nature is particularly valuable for semantic neural decoding, which aims to identify the specific semantic concepts an individual is processing based on their brain activity patterns [8]. However, the integrity of both EEG and fNIRS signals is notoriously vulnerable to motion artifacts (MA)—unwanted signal distortions caused by subject movement [72] [45]. These artifacts can severely compromise data quality, leading to erroneous interpretations and hindering the development of reliable brain-computer interfaces (BCIs) for direct semantic communication [8].

In mobile and clinical settings, the challenge of motion artifacts is particularly pronounced. Unlike controlled laboratory environments, these real-world scenarios involve naturalistic movements—from head adjustments during cognitive tasks to therapeutic movements in rehabilitation [73] [74]. For semantic decoding research, where the goal is to distinguish fine-grained neural patterns associated with different conceptual categories (e.g., animals vs. tools), even minor artifacts can significantly reduce classification accuracy [8]. Consequently, developing robust strategies for mitigating motion artifacts is not merely a technical concern but a fundamental prerequisite for advancing semantic BCI technologies from laboratory demonstrations to practical applications.

Motion artifacts manifest differently in EEG and fNIRS signals due to their distinct acquisition principles. In EEG signals, motion artifacts primarily arise from changes in the electrode-skin interface caused by head movements, muscle contractions, or cable sway. These disturbances introduce high-amplitude spikes, slow drifts, and baseline shifts that can obscure neural signals of interest, particularly the event-related potentials (ERPs) crucial for cognitive task analysis [72]. The problem is exacerbated by the fact that EEG measures microvolt-level electrical potentials, making it highly susceptible to electromagnetic interference from movement.

In fNIRS signals, motion artifacts occur when subject movement disrupts the optical coupling between optodes and the scalp. This disruption causes baseline shifts, high-frequency spikes, and signal dropouts in the measured hemodynamic responses (changes in oxygenated and deoxygenated hemoglobin) [45]. Unlike EEG, fNIRS relies on detecting light attenuation through cortical tissues, meaning that even minor changes in optode placement or pressure can significantly alter the optical path and introduce artifacts that mimic genuine hemodynamic responses associated with neural activation.

The table below summarizes the key characteristics and sources of motion artifacts in both modalities:

Table 1: Characteristics and Sources of Motion Artifacts in EEG and fNIRS

Aspect	EEG	fNIRS
Primary Sources	Electrode-skin interface disruption, cable movement, muscle activity [72]	Optode-skin interface disruption, pressure changes, hair obstruction [45]
Common Manifestations	High-amplitude spikes, slow drifts, baseline shifts [72]	Baseline shifts, high-frequency spikes, signal dropouts [45]
Impact on Signal	Obscures neural oscillations and ERPs [72]	Distorts hemodynamic response functions [45]
Typical Frequency Content	Broadband, often overlapping with neural signals [72]	Broadband, often overlapping with hemodynamic signals [45]

Integrated Framework for Motion Artifact Mitigation

A comprehensive approach to motion artifact management requires integrated strategies across three domains: preventive measures at the device-skin interface, hardware and material solutions, and advanced signal processing techniques. This multi-layered framework ensures maximum signal integrity from acquisition through analysis, which is particularly critical for semantic decoding applications where signal quality directly impacts classification performance.

Hardware and Material Innovations

Recent advancements in material science have introduced selectively damping materials specifically designed for skin-interfaced bioelectronics. These innovative materials absorb and dissipate mechanical vibrations, thereby enhancing stability during prolonged wear [75]. Two primary mechanical strategies have emerged:

The strain-compliance approach focuses on reducing the effective modulus of devices through structural designs like wavy geometries, serpentine interconnects, and Kirigami architectures [75]. These designs diffuse mechanical energy by allowing controlled deformation in non-critical regions, enabling intrinsically non-stretchable materials to accommodate movement without generating significant stress at the device-skin interface.

Complementarily, the strain-resistance strategy employs island-bridge geometries where rigid, small-footprint device islands (protecting critical electronic components) are connected through compliant, strain-absorbing interconnects [75]. This design ensures that most external strain is absorbed by the more flexible regions, minimizing dimensional changes and mechanical stress on sensitive components.

For EEG-fNIRS co-registration, proper montage design is crucial. Using caps with sufficient slits (e.g., 96 or 128) allows optimal placement of both EEG electrodes and fNIRS optodes according to the 10-20 system, with black fabric caps recommended for fNIRS to reduce unwanted optical reflection [71].

Advanced Signal Processing Techniques

Signal processing represents the most extensively researched domain for motion artifact correction, with techniques ranging from traditional algorithms to cutting-edge machine learning approaches.

Traditional and Hybrid Signal Processing Methods

Traditional motion artifact correction employs various signal decomposition and filtering techniques. Wavelet Packet Decomposition (WPD) has demonstrated exceptional performance for single-channel artifact removal, achieving a classification accuracy of 98.61% with only 4.61% valid signal loss in non-motion intervals when implemented in a hybrid model [76]. When combined with Canonical Correlation Analysis (CCA) in a two-stage approach (WPD-CCA), performance further improves, with reported ΔSNR values of 30.76 dB for EEG and 16.55 dB for fNIRS [72].

Other established methods include:

• Wavelet-based techniques: Particularly effective for fNIRS motion artifact correction [73]
• Independent Component Analysis (ICA): Useful for separating neural signals from artifacts in multi-channel EEG [73]
• Spline interpolation: Effective for correcting motion-induced baseline shifts in fNIRS [45]
• Kalman filtering: Adaptively estimates and removes motion artifacts [73]

Table 2: Performance Comparison of Motion Artifact Correction Methods

Method	Modality	Performance Metrics	Advantages	Limitations
WPD-CCA [72]	EEG	ΔSNR: 30.76 dB, η: 59.51%	Excellent for single-channel, high noise reduction	Computational complexity
WPD-CCA [72]	fNIRS	ΔSNR: 16.55 dB, η: 41.40%	Effective for hemodynamic signals	May oversmooth signals
Hybrid Model (BiGRU-FCN) [76]	BCG/General	Accuracy: 98.61%, Signal loss: 4.61%	Integrates deep learning with feature judgment	Requires specialized implementation
CNN/U-Net [45]	fNIRS	Lowest MSE for HRF estimation	Accurate HRF shape preservation	Requires large training datasets
Denoising Autoencoder [45]	fNIRS	Effective on experimental datasets	Generalizes well to real data	Dependent on synthetic training data quality

Machine Learning and Deep Learning Approaches

Learning-based methods have recently emerged as powerful alternatives for motion artifact processing:

Convolutional Neural Networks (CNNs), particularly U-Net architectures, have demonstrated remarkable effectiveness in reconstructing hemodynamic response functions (HRF) while reducing motion artifacts, producing the lowest mean squared error (MSE) and variance in HRF estimates compared to traditional methods [45].

Denoising Autoencoder (DAE) models trained on synthetic fNIRS datasets generated through auto-regressive models have shown promising results when validated on open-access experimental data, effectively removing artifacts while preserving neural signals [45].

Artificial Neural Networks (ANNs) and Back-Propagation Neural Networks (BPNNs) have been implemented for multi-channel fNIRS signal reconstruction, utilizing entropy cross-correlation to identify contaminated optodes [45]. However, these methods face limitations in handling extremely poor-quality data with prominent artifacts across multiple channels.

Traditional machine learning classifiers, including Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), and K-Nearest Neighbors (KNN), have been employed to detect vigilance levels during walking, though their performance degrades significantly in the presence of motion artifacts [45].

Experimental Protocols for Semantic Decoding Research

Protocol 1: Mobile EEG-fNIRS Setup for Semantic Imagery Tasks

Objective: To establish a robust experimental protocol for simultaneous EEG-fNIRS recording during semantic imagery tasks, minimizing motion artifacts while maintaining ecological validity.

Materials and Setup:

• EEG System: Mobile amplifier (e.g., LiveAmp) with active electrodes (e.g., actiCAP slim) [71]
• fNIRS System: Portable system (e.g., NIRSport 2) with appropriate optode configuration [71]
• Cap Integration: EasyCap with sufficient slits (96-128) to host both EEG electrodes and fNIRS optodes [71]
• Synchronization: LabStreamingLayer (LSL) for precise temporal alignment of EEG, fNIRS, and stimulus markers [71]
• Stimulus Presentation: Software capable of sending triggers via LSL (e.g., PsychoPy) [71]

Procedure:

• Montage Design: Design EEG-fNIRS montage using the 10-20 system, prioritizing placement of critical sensors over semantic processing regions (e.g., temporal and frontal areas) [8] [71].
• Cap Preparation: Fit cap with NIRx grommet bases first, followed by actiCAP snap holders for EEG electrodes [71].
• Participant Preparation: Apply conductive gel for EEG electrodes while ensuring proper optical contact for fNIRS optodes [71].
• Signal Quality Check: Verify EEG impedances (< 10 kΩ) and fNIRS signal quality before beginning experiment [71].
• Task Implementation: Present semantic categories (e.g., animals vs. tools) using standardize visual stimuli [8].
• Data Recording: Simultaneously record EEG and fNIRS via LSL, ensuring all data streams are synchronized with stimulus markers [71].

Protocol 2: Motion Artifact Correction Pipeline

Objective: To implement a comprehensive processing pipeline for identifying and correcting motion artifacts in simultaneous EEG-fNIRS data.

Processing Steps:

• Artifact Detection:

  • EEG: Identify segments with abnormally high amplitude (> ±100 μV) or abnormal trends [72]
  • fNIRS: Detect motion artifacts using moving standard deviation or machine learning classifiers [45]

• Artifact Correction:

  • Option A (Traditional): Apply WPD-CCA method using db1 wavelet packet for EEG and fk4/fk8 for fNIRS [72]
  • Option B (Learning-Based): Implement CNN/U-Net architecture trained on synthetic HRF data with motion noise [45]

• Validation:

  • Calculate ΔSNR and percentage reduction in motion artifacts (η) for quantitative assessment [72]
  • Verify preservation of neural signals by examining known response properties (e.g., ERP components for EEG, HRF shape for fNIRS) [45]

• Semantic Decoding Analysis:

  • Extract features from cleaned signals (e.g., ERP components for EEG, HbO/HbR concentrations for fNIRS)
  • Train classifiers (e.g., SVM, LDA) to distinguish between semantic categories [8]
  • Compare decoding accuracy before and after artifact correction to quantify improvement

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 3: Essential Materials for Mobile EEG-fNIRS Research

Category	Specific Product/Technology	Function/Purpose	Key Considerations
EEG Systems	LiveAmp with actiCAP slim electrodes [71]	Mobile EEG acquisition with active electrode technology	Wireless capability, high input impedance
fNIRS Systems	NIRSport 2 [71]	Portable fNIRS acquisition with flexible montage options	Number of sources/detectors, wireless operation
Integrated Caps	EasyCap with 96-128 slits [71]	Host both EEG electrodes and fNIRS optodes	Black fabric recommended for fNIRS light blocking
Synchronization	LabStreamingLayer (LSL) [71]	Temporal alignment of multimodal data streams	Network stability, trigger latency
Signal Processing	Homer2, nirsLAB, EEGLAB [71]	Analysis of fNIRS and EEG data respectively	Compatibility with multimodal data formats
Motion Correction	WPD-CCA algorithms [72]	artifact removal from single-channel signals	Wavelet packet selection (db1 for EEG, fk4/fk8 for fNIRS)
Advanced MA Processing	CNN/U-Net architectures [45]	Learning-based artifact removal	Requirement for training data, computational resources

Mitigating motion artifacts in simultaneous EEG-fNIRS recordings requires an integrated approach combining hardware innovations, careful experimental design, and advanced signal processing techniques. For semantic decoding research—where distinguishing fine-grained neural patterns associated with different conceptual categories is paramount—effective artifact management is not merely a technical consideration but a fundamental requirement for scientific validity.

Future developments in this field will likely focus on real-time artifact processing to enable more robust brain-computer interfaces, standardized evaluation metrics for comparing different correction methods, and improved material designs that further enhance signal stability during movement. As these technologies mature, they will accelerate the translation of semantic decoding research from laboratory demonstrations to practical applications in clinical assessment, communication aids for paralyzed patients, and fundamental cognitive neuroscience.

In the field of semantic decoding using simultaneous EEG-fNIRS, the precise placement of probes and electrodes is a critical determinant of data quality and interpretability. Signal cross-talk, the phenomenon where measurements from one channel are contaminated by signals from adjacent or underlying non-target regions, presents a significant challenge. This cross-talk can obscure the delicate neural signatures associated with semantic processing, such as those distinguishing between categories like animals and tools [8]. For researchers and drug development professionals investigating brain-computer interfaces and neurodegenerative diseases, optimizing probe placement is not merely a technical consideration but a fundamental requirement for obtaining reliable, reproducible neural data. This document provides detailed application notes and experimental protocols designed to address signal cross-talk through systematic approaches to scalp topography.

Quantitative Benchmarks for Placement Accuracy

The accuracy of probe and electrode placement directly impacts signal quality and the minimization of cross-talk. The following table summarizes key performance metrics achieved by advanced placement techniques, serving as benchmarks for protocol development.

Table 1: Performance Metrics of Advanced Probe Placement Techniques

Technique	Reported Positioning Error	Key Improvement Factors	Impact on Signal Quality
Augmented Reality Guidance (NeuroNavigatAR) [77]	1.52 cm (general atlas); 0.75 cm (subject-specific)	Use of age-matched atlas models and subject-specific head surfaces	Enhances consistency across longitudinal and group studies
Optimal Montage with Personalized Head Models [78]	Significantly improved spatial resolution vs. ultra-high-density montages	Mathematical optimization for target brain regions; reduced number of optodes	Improves quantitative accuracy of hemodynamic activity reconstruction
Integrated EEG/fNIRS Headgear [22]	Varies with design (elastic vs. 3D-printed)	Customized, rigid headgear materials; reduced cap movement	Minimizes motion artifacts and variations in probe-scalp contact pressure

Experimental Protocols for Optimal Placement

Protocol 1: AR-Guided Real-Time Placement for Consistent Targeting

This protocol utilizes augmented reality to guide the consistent placement of headgear across multiple sessions and operators, crucial for longitudinal semantic decoding studies [77].

1. Equipment and Software Setup

• Hardware: A standard laptop or tablet with a webcam capable of 15 frames per second.
• Software: Install the open-source NeuroNavigatAR (NNAR) software.
• Precomputed Models: Ensure access to age-matched or general population head atlases.

2. Subject-Specific Landmark Registration

• Position the subject comfortably in front of the camera.
• Run the NNAR software, which will use the MediaPipe facial recognition toolbox to extract and track 3D facial landmarks from the video stream.
• The software applies a precomputed robust linear transformation to estimate cranial landmarks (nasion, inion, preauricular points) from the facial landmarks.

3. Real-Time AR Overlay and Headgear Donning

• The software renders the estimated 10-20, 10-10, or 10-5 system landmarks as an overlay on the live video feed of the subject's head.
• While observing the AR overlay, don the EEG/fNIRS headgear, aligning the probes and electrodes with the projected landmarks.
• The system provides continuous feedback, allowing for micro-adjustments to ensure optimal placement before securing the headgear.

Protocol 2: Personalized fNIRS Optode Montage Using Computational Optimization

This protocol details the creation of a subject-specific optode montage designed to maximize sensitivity to target regions involved in semantic processing (e.g., prefrontal or temporal areas) while minimizing cross-talk [78] [79].

1. Anatomical Data Acquisition and Processing

• Acquire a high-resolution T1-weighted MRI scan of the subject.
• Segment the MRI to create detailed meshes of the scalp, skull, and cortical surface.
• Define the Target Region of Interest (ROI) on the cortical surface. This can be done manually based on a priori knowledge (e.g., Broca's area for language) or automatically using an anatomical atlas (e.g., Destrieux or Desikan-Killiany).

2. Sensitivity Profile and Optode Position Optimization

• Use software like NIRSTORM within Brainstorm with the MCXlab toolbox to simulate light propagation (fluence) for millions of photons through the subject's head model.
• Generate a sensitivity matrix that maps the sensitivity of every possible source-detector pair on the scalp to the target ROI.
• Formulate and solve an optimization problem (using a tool like IBM CPLEX) to select the set of optode positions that maximizes sensitivity to the target ROI. Key constraints include:
  • Number of sources and detectors.
  • Minimum and maximum source-detector separation (typically 3-4 cm).
  • Adjacency constraint (minimum number of detectors per source to ensure overlapping measurements).

3. Neuronavigation-Guided Montage Deployment

• Load the optimized optode coordinates into a 3D neuronavigation system.
• Use the neuronavigation probe to guide the precise attachment of each fNIRS optode to the subject's scalp at the predetermined locations.
• For prolonged recordings, secure optodes using a clinical adhesive like collodion to ensure stable optical coupling and signal quality for over six hours.

Protocol 3: Design and Application of Integrated EEG-fNIRS Headgear

This protocol addresses the physical integration of EEG electrodes and fNIRS optodes to minimize cross-talk from inconsistent probe pressure and positioning [22].

1. Headgear Substrate Selection

• Option A (Customized): Use 3D printing or cryogenic thermoplastic sheeting to create a rigid, subject-specific helmet. This offers the best fit and stability but at a higher cost [22].
• Option B (Standardized): Use a flexible EEG cap as a base, but reinforce the areas where fNIRS optodes will be mounted to prevent stretching and movement.

2. Co-registration of Modalities

• Plan the layout of EEG electrodes and fNIRS optodes on the headgear substrate based on the international 10-20 system or a task-specific optimal montage.
• Ensure that the fNIRS source-detector pairs are positioned to form channels that are co-registered with the EEG electrodes. This spatial alignment is essential for correlating electrical and hemodynamic activity during semantic tasks [22].
• For fNIRS, maintain a standard source-detector separation of 3 cm to ensure sufficient cortical penetration while maintaining signal strength [80].

3. Signal Quality Verification

• With the headgear fitted on the subject, perform a signal quality check.
• For EEG, verify impedance values are below a predefined threshold (e.g., 5-10 kΩ for active electrodes).
• For fNIRS, inspect the raw light intensity levels and signal-to-noise ratio for each channel to ensure proper scalp coupling.
• Use a short baseline recording or a simple functional task (e.g., finger tapping) to confirm the expected activation patterns are detectable.

Workflow Visualization

The following diagrams illustrate the core workflows for the protocols described above, highlighting the logical sequence of steps to achieve optimal placement.

Figure 1: AR-guided real-time placement workflow for consistent headgear targeting.

Figure 2: Computational optimization workflow for personalized fNIRS montage design and deployment.

The Scientist's Toolkit

The following table catalogues essential reagents, software, and hardware solutions for implementing the described protocols.

Table 2: Essential Research Toolkit for Optimal EEG/fNIRS Placement

Tool / Solution	Function / Application	Example / Specification
NeuroNavigatAR [77]	Open-source AR software for real-time cranial landmark visualization.	Operates at 15 fps on a laptop; integrates MediaPipe for facial recognition.
NIRSTORM with Optimal Montage [79]	Brainstorm plugin for computational optode positioning.	Requires IBM CPLEX for optimization; uses MCXlab for light simulation.
3D Neuronavigation System [78]	Precise spatial guidance for deploying pre-planned optode positions.	Used with subject-specific MRI; ensures millimeter accuracy.
Collodion Adhesive [78]	Water-resistant medical adhesive for securing optodes/electrodes.	Enables >6 hours of stable recording; requires ventilated room.
Integrated EEG/fNIRS Caps [22] [80]	Physical headgear for simultaneous multimodal recording.	Custom 3D-printed or thermoplastic for rigidity; standard 3 cm source-detector separation for fNIRS.
Hybrid Amplifier Systems [80] [81]	Synchronized data acquisition hardware for EEG and fNIRS.	g.HIamp amplifier (EEG) + NirScan (fNIRS); synchronized via event markers from e-Prime.

Functional near-infrared spectroscopy (fNIRS) has emerged as a vital neuroimaging tool, particularly for developing semantic decoding systems in brain-computer interfaces (BCIs). Its portability, cost-effectiveness, and compatibility with electroencephalography (EEG) make it ideal for studying naturalistic cognitive processes like semantic categorization of imagined concepts [8]. However, fNIRS signal quality is critically dependent on optimal optical contact with the scalp, a significant challenge in hair-bearing regions where hair obstructs light pathways, increases signal attenuation, and introduces systemic physiological noise [82] [83]. This application note details evidence-based techniques and protocols to enhance the signal-to-noise ratio (SNR) in fNIRS studies, with a specific focus on semantic decoding research using simultaneous EEG-fNIRS recordings.

Hardware and Probe Design Solutions

Innovations in physical probe design are the first line of defense against signal quality degradation caused by hair.

Brush Optodes for Enhanced Optical Contact

Traditional flat-faced fiber bundle optodes often fail to make consistent contact with the scalp through hair. Brush optodes, implemented as attachments to conventional optodes, consist of loose bundles of individual optical fibers that thread through hair like a hairbrush, dramatically improving light coupling [83].

Table 1: Performance Comparison of Flat vs. Brush Optodes

Metric	Flat-Faced Optodes	Brush Optodes	Experimental Context
Study Success Rate	Significantly lower	~100%	Sensorimotor measurement during finger tapping [83]
Setup Time	Baseline	Reduced by a factor of 3	17 subjects with varying hair density [83]
Activation SNR	Baseline	Improved by up to 10x	17 subjects with varying hair density [83]
Detected Area of Activation	Baseline	Significant increase (p < 0.05)	17 subjects with varying hair density [83]

High-Density (HD) Arrays with Multidistance Channels

Sparse fNIRS arrays with 30 mm channel spacing suffer from limited spatial resolution and sensitivity. High-density (HD) arrays with overlapping, multidistance channels improve depth sensitivity, spatial resolution, and localization of brain activity [84].

• Superior Localization: HD arrays provide significantly better localization and sensitivity in image space compared to sparse arrays, especially for tasks with lower cognitive load [84].
• Improved Signal Strength: HD layouts capture stronger hemodynamic responses and demonstrate greater inter-subject consistency [84].

Signal Processing and Analytical Techniques

Advanced processing methods are crucial for distinguishing cortical activity from superficial noise.

Short-Channel Regression

A 'gold standard' method for removing systemic physiological noise from superficial tissues is the use of short-separation channels (e.g., 8 mm). These channels are sensitive primarily to extracerebral hemodynamics, which can be regressed out from the standard long-channel signals [85].

• GLM-PCA Method: A highly effective approach involves performing Principal Component Analysis (PCA) on multiple short channels to extract the global components of superficial noise, which are then regressed out of the long-channel data using a General Linear Model (GLM) [85].
• Performance: The GLM-PCA method has been shown to outperform the anti-correlation method, which relies on the assumed anti-correlation between neuronal HbO and HbR without using short channels [85].

Non-Stationary Filtering and Preprocessing

fNIRS data are non-stationary, meaning their statistical properties change over time. Specialized preprocessing can automatically identify noisy channels and improve SNR.

• Noisy Channel Detection: Statistical methods like Multivariate Entropy (MVE) can accurately detect (97.56%) noisy channels for exclusion or correction [86].
• CCFA Filtering: The Cumulative Curve Fitting Approximation (CCFA) algorithm is a non-stationary filtering procedure for detrending and removing high-frequency contamination. It can produce a greater SNR improvement compared to conventional Discrete Cosine Transform (DCT) or Band-Pass Filtering (BPF) methods [86].

Experimental Protocol for fNIRS in Semantic Decoding

The following protocol is adapted from semantic decoding research and integrates the aforementioned techniques to optimize data quality [8].

Pre-Experimental Setup and Cap Configuration

• Cap Selection: Choose a cap that accommodates a high-density array of optodes. Ensure it allows for precise positioning according to the international 10-20 system for co-registration with EEG.
• Optode Selection and Preparation: Attach brush optode extensions to all detector fiber bundles. For systems supporting it, integrate short-separation channels (e.g., 8 mm) within the probe layout, positioned adjacent to key long-channels (e.g., over prefrontal or lateral cortical areas targeted for semantic decoding).
• Hair Management: Part the hair carefully under each optode location. For subjects with dense hair, use a blunt-ended needle to create a clear path to the scalp. Document hair color and density for post-hoc quality control [82].
• Signal Quality Check: Before beginning the experiment, verify the signal quality on all channels (long and short). Adjust or re-seat any optodes showing poor optical contact or low signal strength.

Data Acquisition During Semantic Tasks

This protocol is designed for a study aiming to differentiate between semantic categories (e.g., animals vs. tools) during mental imagery.

• Task Design: Implement a block or event-related design. In each trial:
  • Cue Presentation (1-2 s): Display an image or word representing a concept from the target categories (e.g., "cat" or "hammer").
  • Mental Task Period (3-5 s): Instruct participants to perform a specific mental task based on the cue, such as:
    • Silent Naming: Silently name the object.
    • Visual Imagery: Visualize the object in their mind.
    • Auditory Imagery: Imagine the sound the object makes.
    • Tactile Imagery: Imagine the feeling of touching the object [8].
  • Rest/Inter-stimulus Interval (10-15 s): Allow hemodynamic responses to return to baseline.
• Simultaneous Recording: Acquire fNIRS and EEG data continuously throughout the session. Ensure synchronization pulses are sent between all recording devices.

Data Processing Workflow

The following workflow integrates best practices for handling data from hair-bearing regions.

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 2: Key Materials for High-Quality fNIRS Research

Item	Function/Description	Considerations for Hair-Bearing Regions
fNIRS System	A continuous-wave, frequency-domain, or time-domain system for measuring hemodynamics.	Ensure compatibility with high-density probe arrays and short-separation channels.
EEG System	A synchronous EEG system for recording electrophysiological activity.	Integrated caps or separate systems should be designed for minimal interference.
Brush Optode Attachments	Fiber bundles that thread through hair to improve scalp contact for detectors [83].	Can be 3D-printed. Most critical for detector optodes in hairy areas.
High-Density Probe Cap	A cap holding sources and detectors in a dense, overlapping layout.	Covers targeted regions (e.g., prefrontal, temporal, parietal). Must be adjustable for head size.
Short-Separation Optodes	Optodes placed 8-15 mm from sources to measure superficial signals [85].	Essential for GLM-PCA regression of systemic physiological noise.
Blunt-Ended Needle / Probe	A tool for parting hair and creating a path for optodes to reach the scalp.	Reduces setup time and improves comfort and signal consistency [82].
Optical Gel	A clear gel used to improve optical coupling between optode and scalp.	Use sparingly to avoid matting hair and creating bridges between channels.

Achieving a high SNR in fNIRS recordings from hair-bearing regions is a multi-faceted challenge requiring both hardware and software solutions. The combination of brush optodes to ensure physical contact, high-density arrays for improved sensitivity, short-channel regression to remove physiological confounds, and robust preprocessing pipelines forms a comprehensive strategy for enhancing data quality. By implementing these techniques and protocols, researchers can significantly improve the reliability of fNIRS data, thereby advancing the development of robust semantic decoding BCIs and other cognitive neuroscience applications.

The integration of high-temporal-resolution electroencephalography (EEG) and high-spatial-resolution functional near-infrared spectroscopy (fNIRS) presents a transformative approach for advancing neuroscience research, particularly in the domain of semantic decoding. This integration aims to overcome the inherent limitations of each standalone modality: while EEG provides millisecond-level temporal resolution crucial for tracking the rapid dynamics of neural electrical activity, it suffers from limited spatial resolution and sensitivity to subcortical processes. Conversely, fNIRS measures hemodynamic responses with superior spatial localization but experiences a characteristic 5-10 second delay due to neurovascular coupling [87]. The fusion of these complementary signals enables researchers to investigate complex cognitive processes, including semantic representation and decoding, with unprecedented spatiotemporal precision [22] [8].

The burgeoning field of semantic decoding using simultaneous EEG-fNIRS recordings seeks to identify specific semantic concepts an individual focuses on based on their brain activity patterns [8]. This research has significant implications for developing advanced brain-computer interfaces (BCIs) that enable direct communication of conceptual meaning, bypassing the character-by-character spelling approach used in current systems [8]. However, effectively integrating these heterogeneous neurophysiological signals presents substantial technical challenges that must be addressed to advance the field.

Fundamental Technical Challenges in EEG-fNIRS Data Fusion

Spatial Alignment and Co-registration Difficulties

The effective integration of EEG and fNIRS data requires precise spatial co-registration of measurement channels with corresponding brain regions, which presents considerable practical challenges. EEG electrodes and fNIRS optodes must be positioned on the scalp to maximize signal quality while ensuring accurate mapping to underlying cortical structures. Current approaches often integrate both modalities into a single headcap, but existing elastic fabric caps frequently result in uncontrollable variations in optode placement due to individual head shape differences [22]. This variability leads to inconsistent probe-to-scalp contact pressure, particularly during movement or long-duration experiments, ultimately compromising data quality [22].

Advanced solutions have emerged to address these spatial alignment challenges. Customized joint-acquisition helmets crafted using 3D printing technology offer flexible positioning of both EEG electrodes and NIR probes, accommodating head-size variations across subjects [22]. Alternatively, composite polymer cryogenic thermoplastic sheets provide a cost-effective solution—these materials can be softened and shaped at approximately 60°C, retaining form stability upon cooling to create customized helmet configurations [22]. The fNIRS-guided Spatial Alignment (FGSA) approach represents a computational solution that calculates spatial attention maps from fNIRS data to identify sensitive brain regions and spatially aligns EEG with fNIRS through strategic weighting of EEG channels [87].

Temporal Synchronization and Hemodynamic Delay

The inherent temporal mismatch between EEG and fNIRS signals constitutes a fundamental challenge for data fusion. EEG records electrical activity with millisecond resolution, capturing neural events almost instantaneously, while fNIRS measures hemodynamic responses characterized by a delayed peak response typically occurring 5-10 seconds after neural activation [87]. This neurovascular coupling delay varies across individuals, brain regions, and task types, complicating temporal alignment [87]. For instance, research has demonstrated that fNIRS response times during motor imagery tasks are significantly shorter than those observed during mental arithmetic and word generation tasks [87].

Traditional approaches to address temporal misalignment have applied fixed temporal offsets relative to task onset, but these methods fail to account for inter-individual and inter-task variability [87]. Advanced computational solutions like the EEG-guided Temporal Alignment (EGTA) layer have been developed to dynamically align fNIRS with EEG signals by generating temporal attention maps based on cross-attention mechanisms [87]. This approach produces fNIRS signals that are temporally aligned with EEG, resolving the issue of temporal mismatch more effectively than fixed-offset methods.

Signal Characteristics and Dimensionality

EEG and fNIRS signals differ fundamentally in their physiological origins, statistical properties, and dimensional characteristics, creating challenges for integrated analysis. EEG signals represent electrical potentials resulting from neuronal activity, typically recorded from 20-256 channels with sampling rates of 100-1000 Hz, producing high-dimensional time-series data [22] [88]. fNIRS measures concentration changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR) using near-infrared light, typically sampling at 1-100 Hz from fewer channels but generating multiple hemodynamic parameters [22] [89].

The signal-to-noise ratio profiles also differ substantially between modalities. EEG is susceptible to various artifacts including muscle activity, eye movements, and environmental interference, while fNIRS signals contain physiological noise from cardiac pulsation, respiration, and blood pressure fluctuations [28]. The multivariate pattern analysis (MVPA) techniques commonly employed for semantic decoding must accommodate these divergent signal characteristics while extracting meaningful neural representations from the integrated data [90].

Table 1: Comparative Characteristics of EEG and fNIRS Modalities

Parameter	EEG	fNIRS
Temporal Resolution	Millisecond level (~1-100 ms)	~0.1-1.0 seconds
Spatial Resolution	Low (~2 cm)	Moderate (~1 cm)
Measured Signal	Electrical potentials from neuronal activity	Hemodynamic response (HbO, HbR)
Depth Sensitivity	Cortical and subcortical (with volume conduction)	Superficial cortex (2-3 cm depth)
Main Artifacts	Muscle activity, eye movements, line noise	Cardiac pulsation, respiration, blood pressure
Typical Channels	20-256	10-100
Portability	High	High

Fusion Strategies and Methodological Approaches

Fusion Stage Frameworks

The integration of EEG and fNIRS data can be implemented at different processing stages, each with distinct advantages and limitations. Research comparing these fusion stages has demonstrated that early-stage fusion, where raw or preprocessed signals are combined before feature extraction, significantly outperforms middle-stage and late-stage fusion approaches in classification accuracy for motor imagery tasks [89].

Early-stage fusion involves combining raw or minimally processed EEG and fNIRS data before feature extraction. This approach preserves the maximum information content from both modalities but requires addressing dimensional mismatch and temporal alignment challenges upfront. Early fusion has demonstrated superior performance in multiple studies, with one investigation reporting significantly higher classification accuracy compared to middle and late fusion approaches (N = 57, P < 0.05) [89].

Middle-stage (feature-level) fusion extracts features separately from each modality before combining them into a unified feature vector. This approach offers flexibility in selecting modality-specific feature extraction techniques but risks information loss if features are not optimally selected. Methods include Common Spatial Pattern (CSP) features for EEG combined with mean and slope values for fNIRS [89], or more advanced approaches like cross-attention mechanisms to investigate mutual information between EEG and fNIRS features [87].

Late-stage (decision-level) fusion processes each modality independently through separate classification pipelines before combining their decisions. This approach preserves modality-specific characteristics and allows for specialized processing but may fail to capture fine-grained interactions between modalities. Techniques include weighted decision fusion based on prediction scores [87] and evidence theory approaches using Dirichlet distribution parameter estimation to model uncertainty followed by Dempster-Shafer Theory for evidence fusion [91].

Table 2: Comparison of EEG-fNIRS Fusion Strategies

Fusion Stage	Description	Advantages	Limitations
Early Fusion	Combining raw/preprocessed signals before feature extraction	Preserves maximum information; Higher performance potential [89]	Requires solving temporal alignment first; High dimensionality
Middle Fusion	Combining extracted features from each modality	Flexible feature selection; Modality-specific processing	Potential information loss; Feature selection critical
Late Fusion	Combining decisions from separate classifiers	Preserves modality specificity; Robust to missing data	May miss fine-grained interactions; Complex implementation
Hybrid Fusion	Combining multiple fusion stages	Leverages advantages of different approaches; Enhanced performance [87]	Increased complexity; Potential overfitting

Advanced Computational Frameworks

Recent advances in deep learning have produced sophisticated architectures specifically designed for EEG-fNIRS fusion. The Spatial-Temporal Alignment Network (STA-Net) represents an end-to-end framework that addresses both spatial and temporal alignment challenges simultaneously [87]. This architecture comprises two specialized sub-layers: the fNIRS-guided Spatial Alignment (FGSA) layer, which identifies sensitive brain regions from fNIRS data to spatially align EEG channels, and the EEG-guided Temporal Alignment (EGTA) layer, which generates temporally aligned fNIRS signals using cross-attention mechanisms [87].

Y-shaped neural networks have emerged as effective architectures for multimodal fusion, typically featuring separate encoder pathways for each modality that merge at various stages depending on the fusion strategy [89]. These networks have demonstrated strong performance in motor imagery classification tasks, with one study reporting average accuracy of 76.21% in left-versus-right hand motor imagery discrimination using leave-one-out cross-validation [89].

Evidence theory approaches incorporating Dempster-Shafer Theory (DST) provide robust frameworks for decision-level fusion by explicitly modeling and managing uncertainty [91]. These methods first quantify decision outputs using Dirichlet distribution parameter estimation, followed by a two-layer reasoning process that fuses evidence from basic belief assignment methods and both modalities [91]. This approach has achieved state-of-the-art performance, with one implementation reporting 83.26% average accuracy in motor imagery classification, representing a 3.78% improvement over previous methods [91].

Experimental Protocols for Semantic Decoding

Protocol 1: Semantic Category Discrimination

This protocol outlines a comprehensive procedure for discriminating between semantic categories (e.g., animals vs. tools) using simultaneous EEG-fNIRS recordings, adapted from established paradigms in the literature [8].

Experimental Setup:

• Participants: 12-29 right-handed native English speakers (or language appropriate to stimulus set)
• Stimuli: 18 animals and 18 tools selected for recognizability and suitability across mental tasks
• Presentation: Images converted to grayscale, cropped to 400×400 pixels, contrast stretched, objects presented against white background
• Equipment: Simultaneous EEG-fNIRS recording system with integrated cap ensuring precise co-registration

Procedure:

• Preparation (30 minutes): Apply EEG-fNIRS integrated cap according to 10-20 system, verify impedance and signal quality for both modalities, perform co-registration using anatomical landmarks or 3D digitization.
• Baseline Recording (5 minutes): Collect resting-state data with eyes open for both modalities to establish baseline signals.
• Task Block Sequence (60 minutes total):
  • Silent Naming Task: Participants silently name displayed objects in their mind without articulation.
  • Visual Imagery Task: Participants visualize the object in their minds, focusing on mental representation.
  • Auditory Imagery Task: Participants imagine sounds associated with the object.
  • Tactile Imagery Task: Participants imagine the feeling of touching the object.
• Trial Structure: Each trial begins with 2-second stimulus presentation, followed by 3-5 second mental task period, followed by 10-12 second rest period. Task order should be randomized across blocks.
• Data Quality Verification: Continuous monitoring of both EEG (artifact detection) and fNIRS (signal-to-noise ratio) throughout recording.

Data Processing Pipeline:

• Preprocessing:
  • EEG: Downsample to 128-250 Hz, bandpass filter 0.5-45 Hz, remove EOG artifacts, re-reference to common average.
  • fNIRS: Convert raw intensity to optical density, remove motion artifacts, bandpass filter 0.01-0.5 Hz, convert to HbO/HbR concentrations via modified Beer-Lambert law.
• Temporal Alignment: Apply adaptive temporal alignment methods (e.g., EGTA layer) to address hemodynamic delay, avoiding fixed offsets when possible.
• Feature Extraction:
  • EEG: Time-frequency features (ERD/ERS), event-related potentials, multivariate pattern components.
  • fNIRS: Mean, slope, and variance of HbO/HbR during task periods, spatial pattern features.
• Fusion and Classification: Implement early fusion using Y-shaped network architecture or similar framework, train classifiers using cross-validation approaches.

Protocol 2: Representational Similarity Analysis for Semantic Representations

This protocol employs representational similarity analysis (RSA) to decode semantic representations from fNIRS-EEG data, building on established methods [28].

Stimuli and Paradigm:

• Stimuli: 8 audiovisual word-picture pairs from two semantic categories (e.g., animals and body parts)
• Design: Passive viewing task with 3-second stimulus presentations, jittered 6-9 second inter-stimulus intervals
• Block Structure: 12 blocks with randomized stimulus order within each block

Data Acquisition Parameters:

• fNIRS Configuration: Dual-array design with posterior array (24 channels) covering occipital lobe and lateral array (18-22 channels) over left temporal, parietal, and prefrontal lobes
• EEG Configuration: 20-64 channels according to 10-20 system, focusing on posterior and left-lateral regions
• Synchronization: Hardware synchronization between EEG and fNIRS systems with precision <10 ms

Analytical Procedure:

• Representational Similarity Matrices: Compute neural representational dissimilarity matrices (RDMs) for both EEG and fNIRS separately based on response patterns to each stimulus.
• Model Comparison: Compare neural RDMs with theoretical model RDMs based on distributional semantics or feature-based representations.
• Cross-validation: Use leave-one-subject-out cross-validation to assess generalizability of semantic representations.
• Integrated Analysis: Combine EEG and fNIRS RDMs using weighted integration based on signal reliability metrics.

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 3: Essential Research Materials for EEG-fNIRS Semantic Decoding Studies

Item	Specification	Function/Purpose
Integrated EEG-fNIRS Cap	Customizable with 3D-printed or thermoplastic mount	Ensures precise spatial co-registration of modalities; Maintains consistent optode-electrode positioning [22]
fNIRS System	14-16 sources, 16-20 detectors; 700-850 nm wavelengths	Measures hemodynamic response via HbO/HbR concentration changes; Provides spatial localization [28]
EEG Amplifier	20-64 channels; High-input impedance; Referential recording	Captures electrical neural activity with millisecond resolution; Provides temporal dynamics [8]
Stimulus Presentation Software	MATLAB Psychtoolbox, PsychoPy, or E-Prime	Prescribes precise timing of semantic stimuli; Synchronizes with neural recordings
3D Digitization System	Polhemus or similar electromagnetic digitizer	Records precise electrode/optode positions; Enables accurate spatial co-registration with brain anatomy
Data Synchronization Unit	Hardware trigger box or software synchronization	Ensures precise temporal alignment of EEG and fNIRS data streams; Critical for fusion analysis
Preprocessing Software	Homer2, EEGLAB, FieldTrip, MNE-Python	Performs modality-specific preprocessing; Artifact removal; Signal quality verification
Multivariate Analysis Tools	Custom MATLAB/Python scripts with scikit-learn, PyTorch/TensorFlow	Implements fusion algorithms; Classification; Representational similarity analysis [90]

Visualization of EEG-fNIRS Fusion Framework

EEG-fNIRS Fusion Framework: This diagram illustrates the comprehensive framework for integrating EEG and fNIRS data, highlighting the key challenges (red), alignment solutions (yellow), and fusion strategies (blue/green) that enable semantic decoding applications.

The integration of high-temporal-resolution EEG and high-spatial-resolution fNIRS represents a powerful approach for advancing semantic decoding research, offering complementary insights into both the rapid electrical dynamics and localized hemodynamic correlates of neural information processing. While significant challenges remain in spatial co-registration, temporal alignment, and signal characteristic disparities, emerging computational frameworks and experimental protocols provide robust solutions for effective data fusion. The continued refinement of these integration methodologies will undoubtedly accelerate progress in understanding the neural basis of semantic representation and advance the development of sophisticated brain-computer interfaces for direct semantic communication.

Integrating electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) into a single, reliable recording platform presents significant hardware challenges for researchers, particularly in the emerging field of semantic neural decoding. Semantic decoding research aims to identify which semantic concepts an individual is focusing on based on brain activity patterns, with the goal of developing more intuitive brain-computer interfaces (BCIs) that communicate conceptual meaning directly, bypassing character-by-character spelling systems [8]. Simultaneous EEG-fNIRS recordings are particularly valuable for this research as they capture complementary information: EEG provides millisecond-level temporal resolution of electrical neural activity, while fNIRS tracks hemodynamic responses with better spatial localization than EEG alone [92].

The development of custom helmets and 3D-printed mounting solutions addresses critical integration challenges including optode and electrode co-registration, motion artifact minimization, and subject comfort during extended recording sessions. This application note provides detailed methodologies and protocols for designing, manufacturing, and validating integrated EEG-fNIRS headgear optimized for semantic decoding research.

Technical Background and Specifications

Comparative Analysis of EEG and fNIRS Modalities

Table 1: Technical specification comparison between EEG and fNIRS

Feature	EEG (Electroencephalography)	fNIRS (Functional Near-Infrared Spectroscopy)
What It Measures	Electrical activity of neurons	Hemodynamic response (blood oxygenation levels)
Signal Source	Postsynaptic potentials in cortical neurons	Changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR)
Temporal Resolution	High (milliseconds)	Low (seconds)
Spatial Resolution	Low (centimeter-level)	Moderate (better than EEG, but limited to cortex)
Depth of Measurement	Cortical surface	Outer cortex (~1–2.5 cm deep)
Sensitivity to Motion	High – susceptible to movement artifacts	Low – more tolerant to subject movement
Best Use Cases	Fast cognitive tasks, ERP studies	Naturalistic studies, sustained cognitive states
Portability	High – lightweight and wireless systems available	High – often used in mobile and wearable formats

EEG and fNIRS capture complementary physiological phenomena. EEG records electrical potentials generated by synchronized neuronal firing, providing direct measurement of neural activity with millisecond temporal precision – ideal for capturing rapid cognitive processes involved in semantic categorization [92]. Conversely, fNIRS measures hemodynamic responses indirectly through neurovascular coupling, detecting changes in cerebral blood oxygenation in response to neural activity with better spatial localization than EEG, though with slower temporal resolution (2-6 second delay) [92].

For semantic decoding research, this complementarity enables investigators to correlate immediate electrical signatures of semantic processing with more localized hemodynamic responses in language-relevant cortical areas. Studies have successfully differentiated between semantic categories such as animals and tools using these combined modalities during various mental imagery tasks [8].

Key Integration Challenges

Simultaneous EEG-fNIRS recording presents several hardware integration challenges that custom helmet solutions must address:

• Spatial interference: EEG electrodes and fNIRS optodes compete for scalp surface area, particularly in regions of interest for semantic processing such as prefrontal, temporal, and parietal cortices [93].
• Motion artifacts: Even small movements can disrupt electrode-scalp contact for EEG or optode-scalp coupling for fNIRS, creating noise in the recorded signals [92].
• Comfort during extended protocols: Semantic decoding experiments often require multiple trials across different task conditions (e.g., visual imagery, auditory imagery, tactile imagery), necessitating comfortable headgear that maintains signal quality throughout the session [8].
• Cross-modal interference: fNIRS optodes can physically interfere with EEG electrode placement, and in some cases, the materials used in one system can affect the signals of the other [92].

Research Reagent Solutions and Materials

Table 2: Essential materials for integrated EEG-fNIRS helmet systems

Item	Function	Technical Considerations
Medical-Grade ABS	3D printing of structural helmet components and rigid mounts	Biocompatibility, structural integrity, low particle shedding under continuous airflow [94]
Flexible TPU-95A	3D printing of comfort layers, customizable padding, and conformal mounts	Shore A hardness of 95 provides optimal balance between flexibility and support; enables vibration welding to helmet shell [94]
PA 2200 (Nylon)	Selective laser sintering for complex mounting geometries	High detail resolution; requires thorough post-processing to remove unsintered particles [94]
EEG Electrodes (Ag/AgCl)	Electrical signal acquisition from scalp	Standard 10-20 system placement; compatibility with 3D-printed mounts to maintain position
fNIRS Optodes	Light emission and detection for hemodynamic monitoring	Source-detector distances of 25-35 mm optimal for cortical penetration; requires precise alignment [95]
Vibration Welding Interface	Joining 3D-printed sockets to helmet shell	Creates uniform, airtight, and mechanically durable bonds without chemical adhesives [94]
Conductive EEG Gel	Ensuring electrical connectivity between scalp and electrodes	Hydrogel composition must not interfere with fNIRS optical signals

Design and Manufacturing Protocols

Helmet Geometry and Component Integration

The design process begins with creating a unified socket geometry compatible with the EN ISO 5356-1:2015 standard for respiratory equipment connectors, adapted for neuroimaging applications [94]. This standardization ensures interoperability and eliminates the need for intermediary components.

Key design specifications:

• Unified socket design for both inlet and outlet assemblies to streamline manufacturing
• Press-fit functionality designed with tolerances that account for additive manufacturing variations
• Ergonomic contouring to maximize subject comfort during extended semantic decoding protocols, which may involve multiple 3-second mental imagery trials [8]
• Strategic placement of mounting points prioritizing coverage over semantic networks (typically including prefrontal, temporal, and parietal regions)

For semantic decoding research, the helmet design must accommodate optimal optode and electrode placement over brain regions implicated in semantic processing, including inferior frontal, temporal, and parietal areas based on previous fMRI studies of semantic categorization [8].

Additive Manufacturing Workflow

Figure 1: 3D Printing and Validation Workflow

The manufacturing protocol follows a structured workflow from digital design to physical validation. Fused Filament Fabrication (FFF) using medical-grade ABS is recommended for components in the direct oxygen pathway, as testing has confirmed no measurable release of microplastic particles under high-flow air conditions [94]. For complex mounting geometries requiring fine details, Selective Laser Sintering (SLS) with PA 2200 (nylon) may be employed, though thorough post-processing is essential to remove unsintered powder particles that could pose inhalation risks [94].

Post-processing protocols must include:

• Comprehensive cleaning to remove all support structures and manufacturing residues
• Surface smoothing to ensure comfortable subject contact
• Validation of dimensional accuracy, particularly for press-fit components
• Particle detachment testing under simulated use conditions

Signal Quality Assurance Protocol

Component Integration and Signal Validation:

• Pre-assembly Testing: Verify individual component dimensional accuracy and material integrity
• Subassembly Integration: Join 3D-printed sockets to helmet shell using vibration welding for TPU components [94]
• Electrode/Optode Mounting: Secure EEG electrodes and fNIRS optodes in designated mounting points
• Signal Baseline Recording: Collect resting-state data to establish signal quality benchmarks
• Task Activation Validation: Implement standardized semantic categorization tasks (e.g., animal vs. tool imagery) to verify functional sensitivity [8]

Experimental Validation Protocols

Mechanical and Material Testing

Table 3: Material and mechanical testing protocol

Test	Methodology	Acceptance Criteria
Particle Detachment	Expose 3D-printed components to high-flow air (similar to respiratory support equipment); measure particulate release	No measurable release of microplastic particles [94]
Mechanical Durability	Cyclical loading simulating extended use; vibration testing	No visible cracking or deformation after 1000 cycles
Material Biocompatibility	Standard skin sensitivity tests; chemical leaching analysis	Compliance with ISO 10993-10 for skin contact
Joint Integrity	Tensile testing of vibration-welded seams	Failure force exceeds 2x expected operational load

Rigorous material testing is essential, particularly for components near the respiratory pathway. Experimental data confirms that ABS Medical printed without active cooling exhibits no measurable particle release, making it the preferred material for these applications [94]. TPU-95A demonstrates excellent elastic recovery, maintaining secure connections through repeated donning and doffing cycles.

Neuroimaging Validation Experiment

Protocol for validating integrated helmet performance in semantic decoding tasks:

• Participant Preparation: 12 right-handed native speakers (or more as required by statistical power analysis), normal or corrected-to-normal vision [8]
• Helmet Fitting: Don integrated EEG-fNIRS helmet with proper optode-scalp and electrode-scalp contact verification
• Experimental Task: Present visual stimuli from two semantic categories (animals and tools) across multiple mental tasks:
  • Silent naming of displayed objects
  • Visual imagery of the objects
  • Auditory imagery of sounds associated with objects
  • Tactile imagery of touching the objects [8]
• Data Collection: Simultaneous EEG (30 electrodes, 1000 Hz sampling) and fNIRS (36 channels, 12.5 Hz sampling, 760 nm and 850 nm wavelengths) recording [93]
• Signal Quality Metrics:
  • EEG: Signal-to-noise ratio, artifact incidence
  • fNIRS: Scalp Coupling Index (SCI) > 0.7, physiological noise reduction

Figure 2: Signal Processing and Data Fusion Pathway

This validation protocol tests the integrated system under realistic semantic decoding conditions. Successful differentiation between semantic categories (animals vs. tools) across multiple mental imagery conditions demonstrates both signal quality and functional utility for the intended research application [8].

Discussion and Implementation Guidelines

Custom 3D-printed helmet solutions effectively address the primary hardware integration challenges in simultaneous EEG-fNIRS research. The modular design approach allows researchers to tailor mounting configurations to specific experimental needs, particularly valuable for semantic decoding studies that require comprehensive coverage of language and semantic networks.

Key implementation considerations:

• Material Selection Balance: Prioritize ABS for structural components in oxygen pathways, TPU for comfort elements, and nylon for complex geometries requiring high resolution
• Manufacturing Method Alignment: Use FFF for most components, reserving SLS for highly complex mounts with thorough post-processing
• Experimental Design Optimization: Leverage fNIRS's motion tolerance for naturalistic paradigms while utilizing EEG's temporal precision for rapid semantic processing
• Signal Quality Prioritization: Implement short-separation fNIRS channels and motion-resistant EEG mounting to maintain data integrity throughout extended semantic categorization protocols

Clinical trials involving 120 participants have demonstrated good ergonomic performance and high user comfort with similar 3D-printed medical helmet systems [94], supporting the feasibility of this approach for extended neuroimaging research sessions.

The integration framework presented here enables more robust and reliable simultaneous EEG-fNIRS recordings, accelerating research in semantic neural decoding and contributing to the development of more intuitive brain-computer interfaces that communicate conceptual meaning directly rather than through character-by-character spelling.

The integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) has emerged as a powerful multimodal approach for brain-computer interfaces (BCIs), particularly in the domain of semantic neural decoding. This integration leverages the complementary strengths of EEG's millisecond-level temporal resolution and fNIRS's superior spatial localization capabilities [8]. However, the effective fusion and processing of these heterogeneous data streams present significant computational challenges. This application note provides a comprehensive overview of state-of-the-art algorithmic optimization techniques that substantially enhance both classification accuracy and real-time processing speed for simultaneous EEG-fNIRS systems. We detail specific methodologies, provide experimental protocols, and present performance benchmarks to guide researchers in developing more efficient and practical BCI systems for semantic decoding applications.

Quantitative Performance Comparison of Advanced Algorithms

Table 1: Classification Performance of EEG-fNIRS Fusion Algorithms

Algorithm	Feature Fusion Strategy	Classification Task	Accuracy	Key Advantage
Multi-Domain Features + Multi-Level Progressive Learning [96]	Feature-level fusion with ASO-based selection	Motor Imagery vs. Mental Arithmetic	96.74% (MI), 98.42% (MA)	Optimal feature complementarity
Dynamic Graph Convolutional-Capsule Networks [97]	Cross-attention capsule fusion	Motor Imagery/Execution	92.60% ± 4.49%	Preserves hierarchical spatial relationships
Multimodal DenseNet Fusion (MDNF) [98]	2D EEG spectrograms + fNIRS spectral entropy	Cognitive & Motor Tasks	>87% (across multiple tasks)	Leverages transfer learning
Enhanced Whale Optimization Algorithm (E-WOA) [99]	Wrapper-based feature selection	Motor Imagery	94.22% ± 5.39%	Optimal fused feature subset selection
Common Spatial Pattern + SVM/LDA [100]	Spatial filtering + feature reduction	Hand Motion & Motor Imagery	84.19% ± 3.18% (with CSP)	Significant dimensionality reduction

Core Methodologies and Experimental Protocols

Multi-Domain Feature Fusion with Atomic Search Optimization

This approach systematically extracts complementary information from both temporal and spectral domains of EEG and fNIRS signals, addressing the inherent limitations of single-domain features [96].

Experimental Protocol:

• Data Acquisition: Record simultaneous EEG (30 channels, 1000Hz) and fNIRS (36 channels, 2.5Hz) during motor imagery and mental arithmetic tasks using a standardized paradigm [99].
• Multi-Domain Feature Extraction:
  • EEG Features: Compute power spectral density (PSD), differential entropy (DE), and wavelet transform coefficients across standard frequency bands.
  • fNIRS Features: Extract mean, variance, slope, skewness, and kurtosis of hemoglobin concentration changes (ΔHbO and ΔHbR).
• Feature Selection: Implement Atomic Search Optimization (ASO) to identify the most discriminative feature subset while eliminating redundancy.
• Progressive Classification: Employ a multi-level classifier that hierarchically combines feature-level and decision-level fusion strategies.

Table 2: Research Reagent Solutions for EEG-fNIRS Experiments

Item Category	Specific Examples	Function/Purpose
Recording Equipment	Nihon Kohden EEG-1100, Hitachi ETG-4000 fNIRS	Simultaneous signal acquisition with synchronization
Electrode/Cap Systems	32-channel EEG cap with integrated fNIRS optodes	Ensures co-registration of electrophysiological and hemodynamic signals
Software Toolkits	Homer2, EEGLab, BCILAB	Preprocessing, artifact removal, and initial feature extraction
Stimulus Presentation	PsychToolbox, Presentation, E-Prime	Precise control of experimental paradigms and timing
Computational Frameworks	TensorFlow, PyTorch, scikit-learn	Implementation of custom deep learning and machine learning models

Dynamic Graph Convolutional-Capsule Networks for Spatial-Temporal Decoding

This architecture captures the dynamic functional connectivity between brain regions while preserving hierarchical relationships in the data, making it particularly suitable for complex semantic decoding tasks [97].

Experimental Protocol:

• Data Preprocessing:
  • EEG: Apply bandpass filtering (0.5-50 Hz), re-referencing to common average, and artifact removal using hybrid ICA-regression [99].
  • fNIRS: Convert raw light intensities to hemoglobin concentrations using the modified Beer-Lambert law, followed by bandpass filtering (0.01-0.5 Hz) to remove physiological noise.
• Graph Construction: Represent EEG electrodes and fNIRS channels as nodes in a graph, with edges weighted by dynamic functional connectivity metrics.
• Temporal Modeling: Apply temporal convolution blocks to extract time-varying features from both modalities.
• Spatial Modeling: Implement Dynamic Graph Convolution (DGCN) to learn endogenous relationships between channels.
• Capsule Fusion: Generate primary capsules from both modalities and fuse them using cross-attention mechanisms before final classification.

Enhanced Whale Optimization for Feature Selection

The Enhanced Whale Optimization Algorithm (E-WOA) addresses the critical challenge of high-dimensional feature spaces in multimodal BCIs by systematically selecting the most discriminative feature subset [99].

Experimental Protocol:

• Feature Extraction: Compute temporal statistical features (mean, variance, slope) from both EEG and fNIRS signals using sliding windows.
• Feature Concatenation: Create a combined feature vector from both modalities.
• Optimization Process: Implement binary E-WOA with SVM-based cost function to select optimal feature subset.
• Validation: Evaluate selected features using cross-validation and compute performance metrics (accuracy, precision, F1-score).

Diagram 1: EEG-fNIRS Processing Pipeline with Algorithmic Optimization Points. The workflow illustrates the sequential stages of multimodal data processing, highlighting key points where optimization algorithms can be integrated to enhance performance.

Semantic Decoding Applications and Protocols

Semantic Category Decoding Protocol

Semantic neural decoding aims to identify which semantic concepts an individual is processing based on brain activity patterns, with significant implications for BCIs that communicate conceptual meaning directly rather than character-by-character [8].

Experimental Protocol for Semantic Category Discrimination:

• Stimulus Selection: Curate images representing distinct semantic categories (e.g., 18 animals vs. 18 tools) with standardized visual properties [8].
• Task Paradigm: Implement a cued experimental design with separate presentation and mental task periods:
  • Stimulus Presentation: Display category exemplars for 2-3 seconds.
  • Mental Imagery Period: Instruct participants to perform specific mental tasks for 3-5 seconds:
    • Silent naming of the object
    • Visual imagery of the object
    • Auditory imagery of associated sounds
    • Tactile imagery of touching the object
• Data Collection: Record simultaneous EEG (focusing on temporal regions for semantic processing) and fNIRS (covering prefrontal and parietal regions).
• Decoding Approach: Apply representational similarity analysis to compare neural response patterns to theoretical semantic models [28].

Implementation Considerations for Real-Time Processing

Channel Optimization and Computational Efficiency

Achieving real-time processing requires careful balance between comprehensive spatial coverage and computational demands.

Optimization Strategies:

• Strategic Channel Selection: Neuroanatomically-guided channel selection (e.g., focusing on FCz and P3 for motivation effects) can reduce data dimensionality while maintaining classification performance [31].
• Progressive Processing: Implement multi-level processing where computationally inexpensive features provide initial classification, with more complex analyses activated only when needed.
• Hardware Considerations: Utilize GPU acceleration for deep learning models and optimize memory usage through efficient data structures.

Algorithmic optimization for simultaneous EEG-fNIRS systems has demonstrated remarkable progress in improving both classification accuracy and processing speed. The integration of advanced feature selection techniques like E-WOA and ASO with sophisticated deep learning architectures such as dynamic graph convolutional-capsule networks has pushed classification performance above 95% for certain tasks while maintaining computational efficiency. These advancements are particularly relevant for semantic decoding applications, where the complementary nature of EEG and fNIRS provides a unique window into the neural representations of conceptual knowledge.

Future development should focus on adaptive algorithms that can self-optimize based on individual user characteristics, transfer learning approaches to reduce calibration time, and lightweight models suitable for embedded BCI systems. The continued refinement of these algorithmic strategies will be essential for translating laboratory demonstrations of semantic decoding into practical BCI applications that enable direct communication of conceptual meaning.

Benchmarking Performance: Validating EEG-fNIRS Against Other Neuroimaging Modalities

Understanding the intricate functions of the human brain requires multimodal approaches that integrate complementary neuroimaging techniques. Electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS), and functional magnetic resonance imaging (fMRI) represent three pillars of non-invasive brain imaging, each with distinct spatiotemporal resolution characteristics. For researchers investigating semantic neural decoding—identifying which semantic concepts an individual focuses on based on brain activity recordings—understanding these trade-offs is crucial for experimental design [8]. Semantic decoding aims to develop brain-computer interfaces (BCIs) that allow direct communication of semantic concepts, bypassing the character-by-character spelling used in current systems [8]. This application note provides a detailed comparison of these modalities, with specific protocols for semantic decoding research using simultaneous EEG-fNIRS, which offers an optimal balance of portability, cost, and complementary resolution profiles for developing practical semantic BCIs.

Technical Specifications Comparison

The table below summarizes the fundamental characteristics of EEG, fNIRS, and fMRI across key parameters relevant to semantic decoding research.

Table 1: Technical comparison of EEG, fNIRS, and fMRI

Feature	EEG	fNIRS	fMRI
Spatial Resolution	Low (centimeter-level) [101]	Moderate (better than EEG, limited to cortex) [101]	High (millimeter-level) [102]
Temporal Resolution	High (milliseconds) [101]	Low (seconds) [101]	Low (seconds) [102]
Depth of Measurement	Cortical surface [101]	Outer cortex (~1-2.5 cm deep) [101]	Whole brain (cortical and subcortical) [102]
What It Measures	Electrical activity of neurons [101]	Hemodynamic response (blood oxygenation) [101]	Hemodynamic response (BOLD signal) [102]
Portability	High (lightweight, wireless systems) [101]	High (portable, wearable formats) [101]	None (immobile equipment) [102]
Cost	Generally lower [101]	Generally higher than EEG [101]	Very high [8]
Tolerance to Motion Artifacts	Low (susceptible to movement) [101]	High (more tolerant to movement) [101]	Low (sensitive to motion) [102]

Neurovascular Coupling and Signal Generation

The following diagram illustrates the fundamental relationship between neural activity and the signals detected by EEG, fNIRS, and fMRI. This neurovascular coupling is foundational to interpreting multimodal data.

Diagram 1: Signal Generation Pathways

This relationship demonstrates why EEG and fNIRS/fMRI provide complementary information: EEG captures the direct electrical activity with high temporal precision, while fNIRS and fMRI measure the subsequent hemodynamic response with better spatial localization [101]. The neurovascular coupling process typically creates a 4-6 second delay between neural activity and hemodynamic response [102].

Simultaneous EEG-fNIRS for Semantic Decoding: Experimental Protocol

Background and Rationale

While fMRI provides excellent spatial resolution for semantic decoding [8], its practical limitations including high cost, lack of portability, and restrictive scanning environment make it unsuitable for real-world BCI applications [8]. Simultaneous EEG-fNIRS presents an ideal alternative as these techniques complement each other—EEG provides excellent temporal resolution while fNIRS addresses EEG's poor spatial resolution [8]. Both are portable, cost-effective, and better suited to real-world applications [8].

Protocol for Semantic Category Decoding

Table 2: Research reagents and materials for simultaneous EEG-fNIRS

Item	Specification	Function
EEG System	30+ channels, 1000 Hz sampling rate [93]	Records electrical brain activity with high temporal resolution
fNIRS System	36 channels (14 sources, 16 detectors), 760nm & 850nm wavelengths [93]	Measures hemodynamic changes in cortical regions
Integrated Cap	International 10-20/10-5 system placement [93]	Ensures proper spatial alignment of EEG electrodes and fNIRS optodes
Stimulus Presentation	Software for image/word presentation with precise timing	Preserves semantic stimuli with accurate event markers
Synchronization Interface	Hardware TTL pulses or shared clock system [101]	Aligns EEG and fNIRS data streams temporally
Data Processing Software	MATLAB, Python, MNE, Brainstorm [93]	Preprocesses and fuses multimodal data

Experimental Workflow:

Diagram 2: Experimental Workflow

Step-by-Step Procedure:

• Participant Preparation (30-45 minutes)

  • Fit participants with integrated EEG-fNIRS cap following the international 10-20 system [93]
  • For EEG: Apply electrolyte gel to achieve impedance <10 kΩ
  • For fNIRS: Ensure optode-scalp contact with appropriate pressure
  • Digitize electrode and optode positions relative to anatomical landmarks (nasion, inion, preauricular points) using a 3D digitizer [4]

• Stimulus Presentation and Mental Tasks (60-90 minutes)

  • Present visual stimuli representing target semantic categories (e.g., animals vs. tools) [8]
  • Implement multiple trial types with randomized presentation:
    • Silent Naming: Participants silently name the displayed object [8]
    • Visual Imagery: Visualize the object in their minds [8]
    • Auditory Imagery: Imagine sounds associated with the object [8]
    • Tactile Imagery: Imagine the feeling of touching the object [8]
  • Use trial structure: Fixation (2s) → Stimulus (3-5s) → Mental task (3-5s) → Rest (10-15s) [8]
  • Record precise event markers synchronized with both EEG and fNIRS

• Simultaneous Data Recording

  • Record EEG from 30+ channels at 1000 Hz sampling rate [93]
  • Record fNIRS from 36 channels at 10+ Hz sampling rate using dual wavelengths (760nm & 850nm) [93]
  • Maintain synchronization between systems using hardware triggers (TTL pulses) or shared clock [101]

• Data Preprocessing (Separate pipelines for each modality)

  • EEG Processing:
    • Apply bandpass filtering (e.g., 0.5-40 Hz)
    • Remove ocular, cardiac, and muscle artifacts using ICA or regression
    • Re-reference to average reference
  • fNIRS Processing:
    • Convert raw intensity to optical density
    • Apply bandpass filtering (0.01-0.1 Hz) to remove physiological noise [93]
    • Convert to hemoglobin concentrations (HbO, HbR) using Modified Beer-Lambert Law
    • Remove motion artifacts using wavelet or PCA-based methods [93]

• Multimodal Data Fusion and Analysis

  • Temporally align EEG and fNIRS data using recorded synchronization markers
  • Address inherent fNIRS hemodynamic delay (typically 4-6s) using temporal alignment algorithms [87]
  • Extract task-related features from both modalities:
    • EEG: Time-frequency features (ERD/ERS), event-related potentials
    • fNIRS: HbO/HbR concentration changes during tasks
  • Apply machine learning classifiers (SVM, deep learning) to decode semantic categories from fused features [87]

Advanced Integration Methodologies

Addressing Spatiotemporal Alignment Challenges

A significant challenge in EEG-fNIRS integration is the temporal mismatch caused by the inherent delay of the hemodynamic response measured by fNIRS relative to the immediate electrical activity captured by EEG [87]. Advanced methods like the Spatial-Temporal Alignment Network (STA-Net) have been developed to address this issue [87]. STA-Net comprises two sub-layers: the fNIRS-guided Spatial Alignment (FGSA) layer, which identifies sensitive brain regions from fNIRS to spatially align EEG channels, and the EEG-guided Temporal Alignment (EGTA) layer, which generates temporally aligned fNIRS signals with EEG through cross-attention mechanisms [87].

Joint Source Reconstruction

For enhanced spatial localization, joint EEG-fNIRS source reconstruction algorithms utilize the high spatial precision of fNIRS to constrain the EEG inverse problem [103]. This approach enables reconstruction of neuronal sources with higher spatiotemporal resolution than either modality alone, allowing resolution of sources separated by 2.3-3.3 cm and 50 ms [103]. The restricted maximum likelihood (ReML) framework incorporates DOT reconstruction as spatial priors for EEG reconstruction, significantly improving localization accuracy [103].

Application in Semantic Brain-Computer Interfaces

The complementary nature of EEG and fNIRS is particularly advantageous for developing semantic BCIs. EEG's millisecond-scale temporal resolution captures rapid neural dynamics during semantic processing, while fNIRS provides better spatial localization of semantic representations in cortical regions like the prefrontal and parietal areas [8] [101]. Research demonstrates that simultaneous EEG-fNIRS can differentiate between semantic categories (animals vs. tools) during various mental imagery tasks, providing a foundation for more intuitive BCIs that communicate conceptual meaning directly rather than character-by-character [8].

This multimodal approach significantly enhances classification accuracy for mental tasks compared to unimodal systems. For instance, one study achieved average accuracies of 69.65% for motor imagery, 85.14% for mental arithmetic, and 79.03% for word generation using advanced fusion methods [87]. These performance improvements demonstrate the practical benefit of combining temporal and spatial information for decoding cognitive states.

For drug development professionals, this technology offers opportunities for evaluating neurotherapeutics targeting cognitive functions, with the advantage of assessing both rapid neural dynamics and localized cortical activation during semantic processing tasks in more naturalistic settings than traditional fMRI paradigms.

Functional near-infrared spectroscopy (fNIRS) and functional magnetic resonance imaging (fMRI) represent two cornerstone neuroimaging techniques for investigating human brain function through hemodynamic correlates of neural activity. While fMRI is considered the gold standard for in vivo imaging with high spatial resolution, fNIRS has gained popularity due to its portability, cost-effectiveness, and superior tolerance for motion artifacts [104] [102]. Understanding the correlation between these modalities and their trade-offs is particularly relevant for advancing semantic decoding research, where fNIRS can be combined with EEG in portable settings to study higher-order cognitive processes [8].

Both techniques leverage the neurovascular coupling response but differ fundamentally in their physical principles and operational constraints. This application note provides a comprehensive technical comparison, detailing quantitative correlations, experimental protocols for multimodal validation, and practical guidance for implementing these technologies in semantic decoding research.

Technical Comparison: fNIRS vs. fMRI

Fundamental Operational Principles

fMRI measures the Blood Oxygen Level Dependent (BOLD) contrast, which reflects differences in magnetic susceptibility dependent on the concentration of paramagnetic deoxy-hemoglobin (deoxy-Hb) [104] [105]. This technique provides indirect measurement of neural activity through associated hemodynamic changes, offering whole-brain coverage including deep subcortical structures with high spatial resolution (millimeter-level) [102].

fNIRS utilizes near-infrared light (650-1000 nm) to measure changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR) concentrations in surface cortical regions [106] [105]. The technique relies on the relative transparency of biological tissues to near-infrared light and the differential absorption properties of hemoglobin species, providing an indirect measure of neural activity confined to superficial cortical regions [102].

Quantitative Performance Comparison

Table 1: Technical specifications and performance comparison between fNIRS and fMRI

Parameter	fNIRS	fMRI
Spatial Resolution	1-3 cm [102]	Millimeter level [102]
Temporal Resolution	Millisecond-level precision [102]	Limited by hemodynamic response (4-6 s lag) [102]
Penetration Depth	Superficial cortical regions (up to ~2 cm) [8] [102]	Whole-brain, including subcortical structures [102]
Measured Parameters	HbO, HbR concentration changes [104]	BOLD signal (primarily reflects deoxy-Hb changes) [105]
Signal-to-Noise Ratio	Significantly weaker [104] [107]	Higher [104]
Portability	High - suitable for field studies [106] [8]	Low - requires dedicated scanner facilities [102]
Tolerance to Motion Artifacts	High [105] [102]	Low - requires participant immobility [102]
Operational Costs	Relatively low [104] [8]	High [104] [102]
Typical Correlation Between Modalities	Wide variability (0-0.8) depending on multiple factors [105]	Reference standard

Table 2: Factors affecting fNIRS-fMRI correlation and practical implications

Factor	Impact on Correlation	Practical Recommendation
Scalp-Brain Distance	Negative correlation - greater distance reduces signal quality [104] [107]	Focus on cortical regions with minimal CSF separation
Channel Signal-to-Noise	Positive correlation - higher SNR improves correspondence [104]	Implement rigorous quality control metrics during data collection
Chromophore Type	Variable - HbO often shows higher correlation with BOLD [105]	Analyze both HbO and HbR for comprehensive assessment
Brain Region	Variable - depending on cortical anatomy [104]	Consider anatomical variations during experimental design
Task Paradigm	Higher cognitive demands may show different correlation patterns [104]	Validate paradigms for each modality specifically

Hemodynamic Correlation and Spatial Correspondence

Temporal Correlation Patterns

Multimodal studies reveal a complex relationship between fNIRS chromophores and the fMRI BOLD signal. While early models suggested a primary relationship between BOLD and deoxygenated hemoglobin, empirical evidence shows significant variability in these correlations [105]. Research demonstrates correlation values ranging from 0 to 0.8 between fNIRS signals and BOLD responses, with substantial variation across brain regions, individuals, and task paradigms [105] [102].

The spatial correspondence between modalities has been systematically investigated using subject-specific fNIRS data to model previously acquired fMRI data during motor tasks. This approach revealed that both HbO and HbR can identify motor-related activation clusters in fMRI data, with no statistically significant differences observed in multimodal spatial correspondence between HbO, HbR, and total hemoglobin (HbT) for motor imagery and execution tasks [105].

Signaling Pathway and Neurovascular Coupling

The following diagram illustrates the neurovascular coupling mechanism and measurement approaches common to both fNIRS and fMRI:

Figure 1: Neurovascular coupling pathway and measurement principles common to fNIRS and fMRI. Both techniques measure hemodynamic responses secondary to neural activity but differ in their physical detection methods.

Experimental Protocols for Multimodal Validation

Simultaneous fNIRS-fMRI Acquisition Protocol

Purpose: To validate fNIRS signals against the fMRI gold standard and investigate spatial-temporal correspondence between modalities [104] [105].

Equipment:

• MRI scanner (3T recommended) with compatible head coil
• MRI-compatible fNIRS system (e.g., NIRSport2) with fiber-optic probes
• Stimulus presentation system compatible with MRI environment
• Synchronization device to align fNIRS and fMRI temporal data

Procedure:

• Participant Preparation: Screen for MRI contraindications. Position fNIRS optodes according to the international 10-5 system, focusing on target regions (e.g., prefrontal cortex, motor areas). Ensure optode holders are MRI-compatible and securely attached [104].
• System Setup: Place fNIRS sources and detectors with inter-optode distance of 30 mm for standard channels and 8 mm for short-distance channels to correct for extracerebral confounds [105]. Verify no metal components are in the MRI bore.
• Anatomical Scan: Acquire high-resolution T1-weighted structural image (e.g., MPRAGE sequence) for co-registration of fNIRS and fMRI data [105].
• Functional Acquisition: Implement block-design or event-related paradigm with simultaneous recording:
  • fMRI parameters: TR=1500ms, TE=30ms, voxel size=3×3×3.5mm [105]
  • fNIRS parameters: Sampling rate=5.08Hz, wavelengths=760nm and 850nm [105]
• Task Paradigm: Implement cognitive tasks such as:
  • Motor tasks (finger tapping) [104] [105]
  • Cognitive tasks (n-back, go/no-go) [104]
  • Semantic tasks (silent naming, category identification) [8]
• Synchronization: Record precise timing of stimulus presentation onset relative to both fNIRS and fMRI clocks.

Semantic Decoding Protocol with Combined EEG-fNIRS

Purpose: To decode semantic categories (e.g., animals vs. tools) during mental imagery tasks using portable multimodal imaging [8].

Equipment:

• Portable fNIRS system with EEG integration capability
• Simultaneous EEG recording system
• Stimulus presentation software
• Comfortable chair in a quiet, controlled environment

Procedure:

• Participant Preparation: Apply EEG cap according to international 10-20 system. Position fNIRS optodes over language-related regions (inferior frontal gyrus, temporal cortex). Measure and record precise optode locations using 3D digitization [8].
• Stimulus Presentation: Display images representing target semantic categories (e.g., animals and tools) in randomized, blocked design.
• Task Instructions: For each trial, instruct participants to perform one of four mental tasks:
  • Silent naming: Silently name the displayed object [8]
  • Visual imagery: Visualize the object in their mind [8]
  • Auditory imagery: Imagine sounds associated with the object [8]
  • Tactile imagery: Imagine the feeling of touching the object [8]
• Data Acquisition:
  • Record fNIRS signals (HbO, HbR) throughout the task
  • Simultaneously record EEG at sufficient sampling rate (≥500Hz)
  • Maintain each trial for 3-5 seconds with adequate inter-trial intervals [8]
• Data Quality Assessment: Implement real-time quality metrics (e.g., scalp-coupling index for fNIRS, impedance checks for EEG) [108].

Data Processing and Analysis Workflow

The following diagram outlines the core processing workflow for multimodal fNIRS-fMRI data:

Figure 2: Multimodal fNIRS-fMRI data processing workflow. Parallel preprocessing streams converge at general linear model (GLM) analysis and correlation assessment.

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 3: Essential equipment and materials for multimodal fNIRS-fMRI research

Item	Specification/Example	Primary Function
fNIRS System	NIRSport2 (NIRx)	Continuous wave fNIRS data acquisition with 16 sources, 15 detectors [105]
fMRI Scanner	3T Siemens Magnetom TimTrio	High-resolution BOLD fMRI acquisition with whole-brain coverage [105]
MRI-Compatible fNIRS Probes	Fiber-optic bundles with MRI-safe fixtures	Enable simultaneous fNIRS-fMRI without compromising safety or signal quality [105]
Short-Distance Detectors	8mm separation optodes	Measure and correct for extracerebral confounds in fNIRS signals [105]
3D Digitizer	Polhemus Patriot	Precise localization of fNIRS optodes on scalp for co-registration with anatomical MRI [8]
Stimulus Presentation Software	E-Prime, PsychoPy, Presentation	Controlled delivery of experimental paradigms with precise timing [104] [8]
Data Processing Tools	Homer3, BrainVoyager, SPM, NIRS-KIT	Preprocessing and statistical analysis of multimodal neuroimaging data [105]
Synchronization Hardware	TTL pulse generator, NI-DAQ	Temporal alignment of fNIRS, fMRI, and stimulus presentation systems [105]

Application in Semantic Decoding Research

For semantic decoding research, the portability of fNIRS enables investigation of naturalistic language processing and conceptual representation that would be challenging in the restrictive fMRI environment. Recent studies have successfully differentiated between semantic categories (animals vs. tools) during silent naming and sensory-based imagery tasks using combined EEG-fNIRS, demonstrating the feasibility of this approach for developing more intuitive brain-computer interfaces [8].

The combination of fNIRS with EEG is particularly promising for semantic decoding applications, as it provides complementary information: EEG offers millisecond temporal resolution to capture rapid neural dynamics during language processing, while fNIRS provides more localized hemodynamic information about cortical regions involved in semantic processing [8]. This multimodal approach addresses the spatial limitations of EEG and the temporal limitations of standalone fNIRS.

When implementing fNIRS for semantic research, particular attention should be paid to optode placement over language-relevant cortical regions (inferior frontal gyrus, superior temporal cortex, angular gyrus) and careful task design that controls for non-semantic cognitive processes like attention and working memory.

fNIRS demonstrates significant yet variable correlation with fMRI hemodynamic responses across different brain regions, tasks, and individuals. While challenges remain in spatial resolution and depth penetration, the portability, cost-effectiveness, and motion tolerance of fNIRS make it a valuable tool for semantic decoding research, particularly when combined with EEG in multimodal approaches. Researchers should carefully consider their specific research questions, target brain regions, and participant populations when selecting between these modalities or implementing combined approaches. As hardware compatibility and analysis techniques continue to advance, multimodal integration of fNIRS with fMRI and EEG will further enhance our ability to investigate complex semantic processes in both laboratory and real-world settings.

Electroencephalography (EEG) and magnetoencephalography (MEG) are paramount non-invasive techniques for measuring human brain activity with millisecond temporal resolution, enabling the study of fast-scale neural dynamics underlying cognitive functions. Within the specific research context of semantic decoding using simultaneous EEG and functional near-infrared spectroscopy (fNIRS), understanding the capabilities and limitations of EEG and MEG for electrical source reconstruction is critical. Both modalities record signals generated by the same neuronal currents, yet they possess fundamentally different sensitivity profiles, physical foundations, and practical constraints [109] [110]. This application note provides a detailed comparison of EEG and MEG in reconstructing electrical brain sources, supplemented with structured quantitative data and experimental protocols, to guide researchers in making informed methodological choices.

Technical Comparison: EEG vs. MEG

Fundamental Principles and Sensitivity Profiles

EEG measures the electrical potential differences on the scalp surface resulting from primarily postsynaptic currents in the pyramidal neurons of the cortex. These potentials are strongly influenced by the conductive properties of the intervening tissues, especially the low conductivity of the skull, which smears and attenuates the electrical signals [109] [110].

MEG records the weak magnetic fields outside the head that are produced by the same neuronal currents. These magnetic fields are largely unaffected by the skull and other extracerebral tissues, as the permeability of biological tissue is nearly equal to that of free space [111] [110].

Their distinct biophysical properties lead to complementary sensitivity profiles:

• Source Orientation: MEG is predominantly sensitive to currents that are tangential to the scalp surface. In contrast, EEG is sensitive to sources of all orientations, including tangential, radial, and oblique [109] [110].
• Source Depth: The sensitivity of MEG to a source decays sharply with its depth, making it less sensitive to deep brain sources. EEG, while also suffering from sensitivity decay with depth, maintains a better ability to detect activity from deeper and subcortical structures due to its sensitivity to radial components [109].
• Impact of Volume Conduction: The EEG signal is significantly spatially smeared due to volume conduction through the cerebrospinal fluid, skull, and scalp. MEG is less susceptible to this effect, generally providing a higher spatial resolution for superficial, tangential sources [112] [110].

Quantitative Performance Comparison

The table below summarizes key performance metrics and practical considerations for EEG and MEG, drawing from recent literature and direct comparative studies.

Table 1: Comprehensive Comparison of EEG and MEG Characteristics

Feature	Electroencephalography (EEG)	Magnetoencephalography (MEG)
Spatial Resolution	~2-3 cm on the scalp; limited by skull smearing [8] [112]	~3-5 mm for superficial cortical sources; less distorted by skull [111]
Temporal Resolution	Millisecond level [5]	Millisecond level [111]
Signal Origin	Post-synaptic potentials (primarily) [5]	Intracellular currents (primarily tangential) [111] [110]
Sensitivity to Source Orientation	Tangential, radial, and oblique [110]	Primarily tangential to the scalp [109] [110]
Sensitivity to Deep Sources	Moderate (better than MEG) [109]	Poor [109]
Impact of Skull Conductivity	High; model accuracy is critical [109] [110]	Negligible [110]
Typical Clinical Use Cases	Epilepsy monitoring, sleep studies, brain-computer interfaces (BCIs) [111] [113]	Pre-surgical mapping, epilepsy focus localization [111] [110]
Instrumentation Cost	Relatively low [5]	Very high (requires SQUIDs, magnetic shielding) [113]
Portability	High (wearable systems available) [5]	Very low (fixed, cryogenic system)
Environmental Noise	Sensitive to muscle and eye movement artifacts	Sensitive to external magnetic interference (e.g., from metals, electronics)
Subject Preparation	Conductive gel application; longer setup time	No gel; generally quicker setup [113]

Performance in source reconstruction and real-world applications further highlights their differences. A study on presurgical epilepsy diagnosis demonstrated that combined EEG/MEG (EMEG) source analysis provided more accurate reconstructions at the onset of epileptic spikes than either modality alone. EMEG was also better at revealing the complete propagation pathway of epileptic activity, which was only partially captured by single modalities [110]. In the context of brain-computer interfaces (BCIs) for auditory attention, classification accuracy was highest for whole-scalp MEG (73.2% on average), while 64-channel EEG achieved 69% accuracy. Notably, the performance of a low-channel-count EEG (3 channels) dropped significantly when training data was limited, falling to chance level in some conditions [113].

Integrated Experimental Protocols

Protocol 1: Combined EEG-MEG Source Reconstruction for Cognitive Tasks

This protocol is designed for studies requiring high-fidelity spatiotemporal localization of neural activity, such as pinpointing the core nodes of the semantic network during language tasks.

1. Objective: To accurately reconstruct the cortical sources underlying evoked or induced brain activity by leveraging the complementary information from simultaneous EEG and MEG recordings.

2. Materials and Reagents:

• Neuroimaging Systems: Simultaneous EEG-MEG system housed in a magnetically shielded room.
• EEG Cap: High-density cap (e.g., 64-128 electrodes) integrated with the MEG helmet.
• Head Localization Coils: Coils attached to the EEG cap for continuous head position tracking relative to the MEG sensors.
• Structural MRI Scanner: For obtaining individual T1-weighted anatomical images.
• Conductivity Gel: Electrolyte gel for ensuring proper electrode-scalp contact.
• 3D Digitizer: For precise co-registration of electrode/fiducial positions with the MRI.
• Analysis Software: Software capable of multimodal source reconstruction (e.g., Brainstorm, MNE-Python, FieldTrip) [112].

3. Procedure:

• Step 1: Subject Preparation. Explain the procedure and obtain informed consent. Attach the EEG cap with integrated head localization coils. Prepare the scalp with light abrasion and fill electrodes with conductivity gel to achieve impedances below 10 kΩ. Measure the 3D positions of the electrodes, fiducial points (nasion, left/right pre-auricular points), and head shape using the digitizer.
• Step 2: Data Acquisition Co-registration. Position the subject in the MEG dewar. Record the head position within the MEG helmet. Acquire a high-resolution structural MRI with fiducial markers visible in both MRI and digitizer space. In the analysis software, co-register the EEG/MEG sensor positions and the head shape with the subject's MRI to create an accurate head model [111] [110].
• Step 3: Experimental Paradigm. Present the cognitive task (e.g., a picture naming or word reading task for semantic decoding). Record simultaneous EEG and MEG data. For semantic paradigms, ensure sufficient trial counts (e.g., >50 per condition) and randomized inter-stimulus intervals to avoid prediction.
• Step 4: Head Model and Forward Solution. Construct a realistic head model from the structural MRI. For combined EMEG, a six-compartment Finite Element Model (FEM) incorporating anisotropic white matter conductivity is recommended for highest accuracy. Use this model to compute the forward solution, which defines how sources in the brain generate signals at the EEG and MEG sensors [110].
• Step 5: Data Preprocessing. Filter the data (e.g., 0.1-100 Hz for EEG, 0.1-330 Hz for MEG). Apply artifact correction routines (e.g., Signal Space Separation (SSS) for MEG, Independent Component Analysis (ICA) for EEG) to remove environmental and biological artifacts (eye blinks, heartbeats) [111] [113].
• Step 6: Combined Source Inversion. Perform source reconstruction using the combined EEG and MEG data. A normalized noise-weighting approach is critical to balance the influence of each modality based on their individual noise levels [109]. Sparse source imaging methods like Variation-Based Sparse Cortical Current Density (VB-SCCD) have been shown to effectively handle the combined data and reconstruct complex activation patterns [109].

4. Data Analysis:

• The output is a time-varying estimate of current density across the cortical surface.
• Compare source activity between conditions (e.g., animals vs. tools) using statistical parametric mapping.
• Identify the centroids and spatial extent of significantly activated regions.

The following workflow diagram illustrates the key steps in this protocol:

Diagram 1: Combined EEG-MEG source analysis workflow.

Protocol 2: Semantic Decoding with Simultaneous EEG-fNIRS

This protocol is framed within the user's thesis context, detailing how to acquire a multimodal dataset for decoding semantic categories.

1. Objective: To differentiate between semantic categories (e.g., animals vs. tools) during various mental imagery tasks using simultaneously recorded EEG and fNIRS signals.

2. Materials and Reagents:

• EEG System: High-impedance amplifier with 64+ channels.
• fNIRS System: Continuous-wave fNIRS system with multiple source-detector pairs (e.g., 8x10 layout forming 24 channels) [4] [8].
• Integrated Cap: An EEG cap with embedded fNIRS optodes.
• EEG Electrolyte Gel: Standard electrolyte gel.
• fNIRS Interface: Optical gel or rubber holders to ensure optode-scalp coupling.
• 3D Digitizer: For mapping EEG electrode and fNIRS optode locations.
• Stimulus Presentation Software: e.g., PsychoPy [113].

3. Procedure:

• Step 1: Setup and Co-registration. Fit the integrated EEG-fNIRS cap on the participant. For EEG, prepare the scalp and apply gel to all electrodes. For fNIRS, ensure all optodes have good contact with the scalp. Use the 3D digitizer to record the positions of all EEG electrodes and fNIRS optodes.
• Step 2: Paradigm Design. Implement a block or event-related design. In each trial:
  • Present a visual cue (an image of an animal or tool) for 1-2 seconds.
  • This is followed by a mental task period (3-5 seconds) where the participant performs a specified imagery task without moving [8].
• Step 3: Mental Tasks. The following tasks can be used to probe semantic representations from different angles [8]:
  • Silent Naming: The participant silently names the object in their mind.
  • Visual Imagery: The participant visualizes the object in their mind.
  • Auditory Imagery: The participant imagines the sound associated with the object.
  • Tactile Imagery: The participant imagines the feeling of touching the object.
• Step 4: Simultaneous Recording. Record continuous EEG and fNIRS data throughout the experiment. For fNIRS, record light intensity at (typically) two wavelengths (e.g., 695 nm and 830 nm) [5].
• Step 5: Data Preprocessing.
  • EEG: Downsample, band-pass filter (e.g., 0.5-45 Hz), re-reference, and remove artifacts using ICA. Epoch data relative to stimulus onset.
  • fNIRS: Convert raw light intensities to optical density. Filter to remove physiological noise (e.g., cardiac, respiratory). Use the Modified Beer-Lambert Law to calculate concentration changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR). Epoch the hemodynamic data.

4. Data Fusion and Analysis:

• Unimodal Analysis: Train machine learning classifiers (e.g., Support Vector Machines, Linear Discriminant Analysis) on EEG features (e.g., time-frequency power) and fNIRS features (e.g., HbO/HbR concentration changes) separately to decode semantic category.
• Multimodal Fusion: Employ advanced fusion techniques like structured sparse multiset Canonical Correlation Analysis (ssmCCA) to identify components in the data that are consistently represented in both the electrical (EEG) and hemodynamic (fNIRS) responses, thereby pinpointing robust neural correlates of semantic processing [4].

Diagram 2: Trial structure and multimodal data flow for semantic decoding.

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 2: Key Materials for Multimodal Neuroimaging Studies

Item Name	Function/Application	Key Considerations
High-Density EEG System	Records scalp electrical potentials with high temporal resolution.	64+ channels recommended for adequate spatial sampling; integrated amplifiers reduce noise.
Whole-Scalp MEG System	Records magnetic fields generated by neuronal currents.	Requires superconducting quantum interference devices (SQUIDs) and a magnetically shielded room.
Continuous-Wave fNIRS System	Measures hemodynamic changes by detecting back-scattered near-infrared light.	Look for systems compatible with simultaneous EEG recording. Configurations with multiple wavelengths (e.g., 695 & 830 nm) are standard [5].
Integrated EEG-fNIRS Caps	Holds both EEG electrodes and fNIRS optodes in a stable, co-registered configuration.	Custom layouts are often needed to optimize coverage for a specific cognitive domain (e.g., language network).
Electrolyte Gel	Ensures conductive connection between EEG electrodes and the scalp.	Low-chloride gels are preferred to prevent electrode corrosion.
3D Magnetic Space Digitizer	Precisely records the 3D locations of EEG electrodes, fNIRS optodes, and head landmarks.	Critical for accurate co-registration with structural MRI and for building realistic head models [4].
Structural MRI Dataset	Provides individual anatomical context for source reconstruction and optode placement.	T1-weighted scans are essential. Can be substituted with a template brain (e.g., ICBM152) if unavailable [112].

EEG and MEG offer complementary strengths for electrical source reconstruction. While MEG provides superior spatial accuracy for superficial, tangentially oriented sources, EEG offers unique sensitivity to radial and deeper sources. The choice between them, or the decision to use them in combination, hinges on the specific neural sources of interest and practical constraints like cost and portability. For the growing field of semantic decoding, EEG's compatibility with simultaneous fNIRS presents a powerful, practical, and cost-effective multimodal approach. This combination leverages EEG's millisecond temporal resolution to track the rapid dynamics of semantic access and fNIRS's superior spatial localization to anchor these processes in specific cortical regions, offering a comprehensive window into the neural basis of meaning.

Within the field of semantic neural decoding, quantifying the performance of classification algorithms is paramount for evaluating the efficacy of Brain-Computer Interfaces (BCIs). Decoding accuracy serves as the primary metric for determining how well a system can distinguish between different semantic categories, such as animals versus tools, based on neural signals [8]. This document, framed within broader thesis research on simultaneous EEG-fNIRS recordings, outlines standard metrics, presents benchmark performance data, and provides detailed protocols for quantifying decoding accuracy in semantic category classification.

Core Metrics for Quantifying Decoding Accuracy

The performance of a semantic decoding model is typically evaluated using a standard set of metrics derived from the classification confusion matrix. The most fundamental metric is Classification Accuracy, which measures the proportion of total correct predictions (both positive and negative) among the total number of cases examined [114]. For binary classification, such as distinguishing between the semantic categories of "animals" and "tools," this is calculated as (True Positives + True Negatives) / Total Predictions.

Additional metrics offer nuanced insights into model performance, which is particularly valuable for addressing issues like class imbalance [115]. The table below summarizes these key metrics.

Table 1: Key Metrics for Evaluating Semantic Decoding Classification Performance

Metric	Formula	Interpretation in Semantic Context
Accuracy	(TP + TN) / (TP + TN + FP + FN)	Overall ability to correctly identify both target and non-target semantic categories.
Precision	TP / (TP + FP)	When a category is predicted, the probability that it is correct. Measures reliability.
Recall (Sensitivity)	TP / (TP + FN)	Ability to correctly identify all instances of a specific semantic category.
F1-Score	2 * (Precision * Recall) / (Precision + Recall)	Harmonic mean of precision and recall; provides a single balanced metric.
Area Under the Curve (AUC)	Area under the ROC curve	Measures the model's ability to distinguish between classes across all classification thresholds.

Beyond singular metric values, the statistical significance of results must be assessed. Performance is typically compared against chance-level accuracy, which is 50% for binary classification and 1/N for an N-class problem [114]. Statistical validation, often using paired t-tests or non-parametric equivalents across multiple cross-validation folds or participants, is essential to confirm that decoding performance is significantly above chance [116].

Benchmark Performance Data from Literature

Reported decoding accuracies vary significantly based on the neuroimaging modality, the specific semantic categories, the mental tasks employed, and the analytical approach. The following table synthesizes quantitative results from recent studies to provide a benchmark for expected performance.

Table 2: Reported Accuracies in Semantic and Cognitive Decoding Studies

Study (Source)	Modality	Classification Task	Methodology	Reported Accuracy
Simultaneous EEG/fNIRS [8]	EEG + fNIRS	Animals vs. Tools	Silent naming & sensory imagery tasks	Data provided; accuracy results are study-dependent.
Word Category Decoding [114]	EEG + MEG	Living vs. Non-living objects	Support Vector Machine (SVM)	76% (chance = 50%)
Action Observation [117]	EEG + fNIRS	Three action intentions	Feature fusion from complex brain networks	72.7%
Inner Speech Recognition [115]	EEG	Eight imagined words	Spectro-temporal Transformer	82.4%
Hybrid BCI for Movement [118]	NIRS-EEG	Four movement directions	Linear Discriminant Analysis (LDA)	Over 80%
Multimodal Fusion (MBC-ATT) [119]	EEG + fNIRS	Cognitive tasks (n-back, WG)	Cross-modal attention fusion	Outperformed conventional approaches

The data indicates that multimodal fusion of EEG and fNIRS consistently enhances classification performance compared to single-modality approaches by leveraging their complementary strengths [119] [117]. For instance, one study on emotion recognition found that a multimodal EEG-fNIRS system achieved notable accuracy improvements of 7.5%, 3%, and 6.5% across different emotional dimensions compared to single-modality systems [120]. Furthermore, the choice of machine learning algorithms, from traditional SVMs [114] and LDAs [118] to advanced deep learning models like Cross-modal Attention [119] and Transformers [115], plays a critical role in achieving high decoding accuracy.

Detailed Experimental Protocol for Semantic Decoding

This protocol details the methodology for a simultaneous EEG-fNIRS experiment aimed at decoding semantic categories, based on established paradigms [8].

Participant Preparation and Stimuli

• Participants: Recruit right-handed, native-speaking participants (or as required by the task) with normal or corrected-to-normal vision. Secure written informed consent and ethical approval from the relevant institutional review board.
• Stimuli Selection: Prepare a set of images or words representing the target semantic categories (e.g., 18 animals and 18 tools). Images should be converted to grayscale, cropped, and normalized for size and contrast to minimize low-level visual confounds [8].

Data Acquisition Setup

• EEG Setup: Position a high-density EEG cap (e.g., 64-channel) according to the international 10-20 system. Use a mastoid reference and ensure all electrode impedances are maintained below 10 kΩ. Set the sampling rate to a minimum of 500 Hz [8] [114].
• fNIRS Setup: Place fNIRS optodes over the brain regions of interest (e.g., prefrontal cortex, bilateral temporal lobes). Configure the system to record continuous-wave signals at multiple wavelengths (e.g., 760 nm and 850 nm) to compute concentration changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR). A typical sampling rate is 10 Hz [8] [118].
• Synchronization: Use a common trigger signal from the stimulus presentation software to synchronize the onset of each trial across both EEG and fNIRS systems.

Experimental Paradigm

The experiment should follow a block or event-related design. The following workflow diagram illustrates the structure of a single trial.

Mental Tasks (during the Mental Task Period):

• Silent Naming: Participants silently name the displayed object in their mind [8].
• Visual Imagery: Participants visualize the object in their minds.
• Auditory Imagery: Participants imagine the sounds associated with the object.
• Tactile Imagery: Participants imagine the feeling of touching the object [8]. Participants should be instructed to minimize physical movements throughout the task period.

Data Preprocessing and Feature Extraction

• EEG Preprocessing:

  • Bandpass filter (e.g., 1-30 Hz) to remove drift and line noise.
  • Independent Component Analysis (ICA) to remove artifacts from eye movements and cardiac activity.
  • Segment data into epochs (e.g., -1 s to +5 s relative to stimulus onset).
  • Baseline correction and automated artifact rejection [114] [115].
  • Features: Time-domain features (mean amplitude, variance), frequency-domain features (band power in theta, alpha, beta bands), or time-frequency representations.

• fNIRS Preprocessing:

  • Convert raw light intensity to optical density and then to HbO and HbR concentrations using the Modified Beer-Lambert Law.
  • Apply a bandpass filter (e.g., 0.01 - 0.2 Hz) to remove physiological noise (heart rate, respiration) and slow drifts.
  • Segment hemodynamic responses into epochs aligned with trial onset.
  • Features: Mean HbO/HbR concentration during the task window, slope of the signal, or area under the curve [8] [118].

Model Training and Accuracy Quantification

• Feature Fusion and Normalization: Concatenate features from EEG and fNIRS modalities (feature-level fusion) or train separate classifiers for later decision-level fusion. Normalize features (e.g., z-scoring) across the dataset.
• Classifier Training: Employ a classifier such as Support Vector Machine (SVM) [114], Linear Discriminant Analysis (LDA) [118], or a deep learning model (e.g., CNN, Transformer) [119] [115]. Use a nested cross-validation strategy:
  • Outer Loop: For estimating generalized performance (e.g., 10-fold cross-validation or Leave-One-Subject-Out).
  • Inner Loop: For hyperparameter tuning within the training set of each outer fold to avoid data leakage and overoptimism.
• Accuracy Quantification: For each test fold in the outer loop, calculate the metrics listed in Table 1 (Accuracy, Precision, Recall, F1-Score). Report the mean and standard deviation of these metrics across all folds or participants.

The Scientist's Toolkit: Research Reagent Solutions

The following table outlines essential materials and tools for conducting semantic decoding research with simultaneous EEG-fNIRS.

Table 3: Essential Research Materials and Tools for EEG-fNIRS Semantic Decoding

Item	Specification / Example	Function in Research
Multimodal Amplifier	A system capable of simultaneous, synchronized EEG and fNIRS recording (e.g., BrainAmp for EEG, NIRScout for fNIRS).	Acquires raw neural and hemodynamic data. Synchronization is critical for trial alignment.
EEG Electrode Cap	64-channel Ag/AgCl electrodes, configured in the 10-20 system.	Measures electrical potentials from the scalp surface. High density improves spatial resolution.
fNIRS Optodes	Sources and detectors covering prefrontal and temporal lobes. Configurations can include 12+ channels.	Emits and detects near-infrared light to measure cortical hemodynamic responses.
Electrolyte Gel	Standard conductive EEG gel.	Ensures low impedance connection between EEG electrodes and the scalp.
Stimulus Presentation Software	Presentation, PsychToolbox, or E-Prime.	Prescribes the experimental paradigm, displays visual stimuli, and sends synchronization triggers.
Data Analysis Suite	MATLAB with EEGLAB, FieldTrip, MNE-Python, or custom scripts in Python/R.	Performs preprocessing, feature extraction, machine learning, and statistical analysis.
Machine Learning Library	Scikit-learn, TensorFlow, or PyTorch.	Provides algorithms for feature classification (e.g., SVM, LDA, CNN) and accuracy calculation.

Semantic decoding—the process of identifying which semantic concepts an individual is processing based on their brain activity—holds transformative potential for brain-computer interfaces (BCIs) and cognitive neuroscience [8]. While functional magnetic resonance imaging (fMRI) has demonstrated remarkable success in semantic decoding, its practical applications are limited by high costs, lack of portability, and restrictive scanning environments [8]. Functional near-infrared spectroscopy (fNIRS) offers a promising alternative with its portability, tolerance for movement, and lower operational costs [28] [121].

This case study examines the validation of fNIRS-based semantic decoding using fMRI as a ground truth benchmark. We present a framework for establishing the validity of fNIRS measurements for detecting semantic representations, enabling future research into more ecological and clinically viable neural decoding systems, particularly those leveraging simultaneous EEG-fNIRS recordings [8].

Theoretical Framework and Rationale

Neurovascular Correlates of Semantic Processing

Both fMRI and fNIRS measure brain activity indirectly by detecting hemodynamic changes related to neuronal firing through the mechanism of neurovascular coupling. fMRI measures the blood-oxygen-level-dependent (BOLD) signal, while fNIRS quantifies concentration changes in oxygenated hemoglobin (Δ[HbO]) and deoxygenated hemoglobin (Δ[HbR]) in cortical tissues [122] [123]. Despite technological differences, both modalities capture the same fundamental physiological processes, providing a biological basis for cross-modal validation.

Semantic processing of words and concepts engages a distributed cortical network. Studies have consistently identified the temporal lobe, particularly regions associated with language processing, and occipital areas, involved in visual semantic representations, as key hubs for semantic information encoding [28]. The supplementary motor area (SMA) also shows involvement during imagined semantic tasks [123].

fNIRS Advantages and Limitations for Semantic Decoding

fNIRS offers several distinct advantages for semantic decoding research:

• Portability and tolerance for movement enable studies of naturalistic behaviors, social interactions, and populations unsuitable for fMRI [28] [121]
• Silent operation prevents interference with auditory semantic stimuli [124]
• Higher temporal sampling (typically 10+ Hz) provides more observations of the hemodynamic response curve [28]

However, fNIRS also presents significant challenges:

• Limited spatial resolution with channel regions covering several centimeters of cortical surface [28]
• Superficial sensitivity restricts measurement to cortical areas, with no access to subcortical structures [123]
• Lower signal-to-noise ratio compared to fMRI, requiring sophisticated processing to extract neural signals [28] [122]

These limitations necessitate rigorous validation against established modalities like fMRI to establish fNIRS as a reliable tool for semantic decoding.

Experimental Protocol for Cross-Modal Validation

Participant Recruitment and Stimuli Design

Participants: Recruit 20-30 healthy adult participants with normal or corrected-to-normal vision and no history of neurological conditions. The study should obtain ethical approval and written informed consent [122].

Stimuli Selection: Select stimuli from distinct semantic categories to maximize neural discriminability. The established categories of animals (e.g., bear, cat, dog) and tools (e.g., hammer, scissors, knife) have proven effective for semantic decoding research [8]. Use standardized grayscale images (e.g., 400×400 pixels) presented against a neutral background to minimize low-level visual confounds [8].

Paradigm Design: Implement a block design with the following structure:

• Stimulus Presentation: 3-second audiovisual presentation (picture + spoken word) [28]
• Mental Task Period: 3-5 second period for silent naming, visual imagery, auditory imagery, or tactile imagery of the stimulus [8]
• Inter-stimulus Interval: 6-9 second jittered rest period with neutral stimuli (e.g., fireworks display with music) to establish baseline [28]
• Block Repetition: 12 blocks with randomized stimulus order within each block [28]

Simultaneous fNIRS-fMRI Data Acquisition

fMRI Acquisition Parameters:

• Use a 3T Philips Achieva scanner with 32-channel head coil [122]
• Acquire T1-weighted structural images (TR=7ms, TE=3.2ms) [122]
• Collect functional BOLD images (180 volumes, TR=2s, TI=900ms) [122]
• Implement eye tracking and physiological monitoring (cardiac, respiration)

fNIRS Acquisition Parameters:

• Use a NIRScout or NIRSport system (NIRx) with 16 sources and 32 detectors creating 64 channels [122]
• Employ wavelengths of 760nm and 850nm with a sampling rate of 7.8Hz [122]
• Configure source-detector distances of 2.8-3.5cm to optimize cortical sensitivity [122]
• Arrange probes in two arrays: posterior (occipital lobe, 4×4 grid) and left lateral (temporal lobe, 3×5 grid) [28]
• Digitize optode positions using a Fastrak system (Polhemus) and coregister to the Colin27 atlas using AtlasViewer [122]

Table 1: Data Acquisition Parameters

Parameter	fMRI	fNIRS
Temporal Resolution	2s (TR)	7.8Hz (128ms)
Spatial Resolution	2-3mm isotropic	2-3cm channel diameter
Measurement	BOLD signal	Δ[HbO], Δ[HbR]
Coverage	Whole brain	Superficial cortex
Stimulus Synchronization	Trigger pulses	Event markers via parallel port

Data Preprocessing Pipeline

fMRI Preprocessing:

• Preprocess with SPM12 and UF2C toolbox [122]
• Perform motion correction (FD threshold 0.5mm, DVARS 5%) [122]
• Apply band-pass filtering (0.009-0.08Hz) with 50dB stopband attenuation [122]
• Regress out white matter, cerebral spinal fluid, and global signal [122]
• Normalize to standard space and parcel into 94 regions of interest (AAL atlas) [122]

fNIRS Preprocessing:

• Convert raw intensity to optical density [28] [122]
• Prune low-SNR channels (SNR<8) and exclude runs with >50% bad channels [122]
• Correct motion artifacts using hybrid spline interpolation and wavelet decomposition [122]
• Remove first principal component to reduce non-neural physiological signals [28]
• Apply bandpass filtering (0.01-1.0Hz) [28]
• Convert to hemoglobin concentrations using modified Beer-Lambert law [28]
• Perform channel stability analysis to identify reliable channels [28]

Analytical Approach for Validation

Representational Similarity Analysis

Representational similarity analysis (RSA) provides a powerful framework for cross-modal validation of semantic representations:

• For each modality, compute neural representational dissimilarity matrices (RDMs) by correlating response patterns across all stimulus pairs and conditions [28]

• Create model RDMs based on semantic models (e.g., distributional semantic models from word co-occurrence statistics) [28]

• Compare neural RDMs between fMRI and fNIRS using Spearman correlation or related similarity metrics [28]

• Test statistical significance using cross-validation approaches (e.g., leave-one-subject-out) and permutation tests [28]

Multivariate Pattern Classification

Implement decoding analyses to directly assess information content:

• Extract features from both modalities: BOLD activation patterns from fMRI regions of interest; Δ[HbO] and Δ[HbR] time series from fNIRS channels

• Train classifiers (e.g., linear SVM, LDA) to discriminate between semantic categories (animals vs. tools) using cross-validation

• Compare decoding accuracies between modalities to quantify information preservation

• Perform cross-modal generalization tests where classifiers trained on fMRI data are tested on fNIRS data, and vice versa

Brain Fingerprinting Validation

Leverage recent advances in fNIRS-based brain fingerprinting to establish individual-level reliability [122]:

• Extract resting-state functional connectivity matrices from both modalities

• Calculate similarity between sessions using Pearson correlation or geodesic distance [122]

• Perform subject identification with simple linear classifiers [122]

• Compare identification accuracy between modalities (fMRI: ~99.9%; fNIRS: 75-98% depending on regions and runs) [122]

Key Validation Metrics and Expected Results

Table 2: Expected Performance Metrics for fNIRS Semantic Decoding

Validation Metric	Expected Performance	Benchmark (fMRI)
Category Decoding Accuracy	70-85% (depending on mental task)	85-95% [8]
Between-Subject Generalization	Significant above chance (p<.05) [28]	Significant above chance
Semantic Model Correlation	Significant above chance (p<.05) [28]	Significant above chance
Brain Fingerprinting Accuracy	75-98% (depending on coverage) [122]	99.9% [122]
Spatial Specificity Correlation	Significant for motor/visual tasks [123]	High
Temporal Correlation	Moderate (physiological noise impact)	High

Technical Considerations and Optimization

fNIRS System Selection and Configuration

System Requirements for Semantic Decoding:

• High-density arrays with 20+ channels per region of interest [122]
• Dual-wavelength sources (760nm, 850nm) for HbO/HbR separation [122]
• Spring-loaded grommets to ensure adequate scalp coupling, particularly through hair [125]
• Real-time quality metrics to monitor signal integrity during acquisition [126]

Optode Placement Strategy:

• Target left temporal regions for language-related semantic processing [28]
• Include occipital coverage for visual semantic representations [28]
• Consider prefrontal coverage for working memory components of semantic tasks [121]
• Use individual anatomical guidance (e.g., fOLD, AtlasViewer) when possible [123]

Signal Quality Optimization

Minimizing Physiological Noise:

• Implement short-separation channels (<1cm) to regress out systemic physiological noise [122]
• Collect auxiliary physiological measurements (heart rate, respiration, blood pressure) for noise modeling [122]
• Apply global signal regression techniques to remove widespread physiological artifacts [28]

Motion Artifact Mitigation:

• Use hybrid motion correction algorithms combining spline interpolation and wavelet decomposition [122]
• Instruct participants to minimize head movements during critical task periods [8]
• Implement accelerometer-based motion tracking for artifact detection [126]

Application to Simultaneous EEG-fNIRS Research

The validation of fNIRS-based semantic decoding directly enables more powerful multimodal approaches combining EEG and fNIRS:

Complementary Strengths:

• EEG provides millisecond temporal resolution to track rapid semantic access processes [8]
• fNIRS offers improved spatial localization of semantic representations compared to EEG alone [8]
• Combined systems allow investigation of both electrophysiological and hemodynamic aspects of semantic processing [8]

Experimental Design Considerations for EEG-fNIRS:

• Use carbon electrodes or specialized fNIRS-compatible EEG caps to minimize optical interference
• Implement careful light shielding to prevent contamination of EEG signals by NIR light
• Employ synchronized data acquisition systems with shared trigger channels [8]
• Plan for extended setup time (typically 60-90 minutes) for proper placement of both modalities

This validation protocol establishes a rigorous framework for verifying fNIRS-based semantic decoding against fMRI ground truth. The convergence of evidence across multiple analytical approaches—representational similarity analysis, multivariate classification, and brain fingerprinting—provides a comprehensive assessment of fNIRS capabilities for detecting semantic representations.

Once validated, fNIRS semantic decoding can be deployed in more naturalistic settings, including studies of social communication, developmental populations, and clinical applications. The integration with EEG further enhances temporal resolution, opening new possibilities for tracking the dynamic time course of semantic processing.

Future work should focus on improving spatial resolution through high-density arrays, enhancing signal processing to isolate semantic-specific components, and developing standardized protocols for cross-laboratory replication. As fNIRS technology continues to advance, semantic decoding approaches will become increasingly viable for real-world BCI applications and clinical translation.

Research Reagent Solutions

Table 3: Essential Research Materials and Equipment

Item	Specifications	Function
fNIRS System	NIRScout/NIRSport (NIRx); 16+ sources, 32+ detectors; 760 & 850nm wavelengths [122]	Measures cortical hemodynamic responses via near-infrared light
fMRI System	3T Philips Achieva with 32-channel head coil [122]	Provides high-spatial-resolution BOLD signal as ground truth
Stimulus Presentation Software	MATLAB Psychtoolbox, Presentation, or similar	Controls precise timing of audiovisual stimulus delivery
EEG System	fNIRS-compatible with carbon electrodes or specialized cap	Captures electrophysiological activity with high temporal resolution
Data Analysis Platform	Homer2, SPM12, AtlasViewer, custom MATLAB/Python scripts [28] [122]	Processes and analyzes multimodal neuroimaging data
Optode Positioning Tool	fOLD, AtlasViewer with 10-20 digitization [122] [123]	Guides accurate placement of fNIRS optodes on cortical regions of interest
Physiological Monitoring	NIRxWINGS2, pulse oximeter, respiration belt [126]	Records systemic physiological signals for noise regression

The integration of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) presents a compelling multimodal approach for probing brain function. This cost-benefit analysis examines the practical considerations for deploying this technology in both research and clinical settings, with a specific focus on the emerging field of semantic decoding. Semantic neural decoding, which aims to identify the specific concepts an individual is focusing on by analyzing their brain activity, has significant implications for developing a new class of brain-computer interfaces (BCIs) that enable direct communication of semantic meaning [8]. While EEG-fNIRS integration offers complementary advantages, researchers and clinicians must carefully evaluate the trade-offs between the enhanced information gain and the increased technical complexity, operational costs, and data processing challenges.

Core Cost-Benefit Analysis of EEG-fNIRS Technology

The decision to implement a hybrid EEG-fNIRS system involves balancing significant advantages against notable challenges. The table below summarizes the key factors influencing this cost-benefit analysis.

Table 1: Comprehensive Cost-Benefit Analysis of EEG-fNIRS Integration

Analysis Factor	Benefits & Advantages	Costs & Challenges
Technical Performance	Complementary spatiotemporal resolution: EEG provides millisecond temporal resolution, while fNIRS offers ~2 cm spatial resolution [8] [127].	Inherent physiological differences: Electrical (EEG) and hemodynamic (fNIRS) signals have different temporal dynamics and origins [5] [127].
	Provides dual insight into electrical brain activity and metabolic oxygen consumption [128].	Signal decoupling challenges: Requires sophisticated algorithms to separate modality-specific and shared features [129].
Financial Considerations	Lower hardware costs compared to fMRI, PET, and MEG systems [22] [5].	Significant initial investment for integrated systems or discrete components.
	Growing commercial availability of compatible systems from various manufacturers [130].	Ongoing maintenance and potential need for specialized technical staff.
Operational & Clinical Utility	Portability enables use in natural environments, bedside monitoring, and real-world applications [22] [130].	Mechanical integration challenges: Competition for scalp space between electrodes and optodes [130].
	Suitable for diverse populations (infants to elderly) and long-term monitoring [22] [5].	Sensitivity to motion artifacts and physiological noise, complicating data acquisition during movement [131].
Data Processing & Analysis	Enhanced classification accuracy for BCI applications compared to single modalities [131].	Complex data synchronization requirements between systems [130].
	Enables study of neurovascular coupling mechanisms [5].	Advanced computational needs for processing high-dimensional multimodal datasets [129].

Experimental Protocols for Semantic Decoding Research

Semantic decoding research using EEG-fNIRS aims to distinguish between different semantic categories (e.g., animals vs. tools) based on brain activity patterns. The following protocol provides a detailed methodology for conducting such studies.

Protocol: Semantic Category Discrimination Using Simultaneous EEG-fNIRS

Objective: To differentiate between neural representations of semantic categories (animals and tools) during silent naming and mental imagery tasks using simultaneous EEG-fNIRS recordings.

Experimental Setup and Reagent Solutions:

Table 2: Research Reagent Solutions and Essential Materials

Item	Function/Application
EEG System (e.g., BrainAMP)	Records electrical brain activity with high temporal resolution. Essential for capturing event-related potentials (ERPs) [8].
fNIRS System (e.g., NIRScout)	Measures hemodynamic responses (changes in HbO and HbR) in the cortex [22] [8].
Integrated Cap/Holder	Custom-designed helmet (e.g., 3D-printed or cryogenic thermoplastic) that houses both EEG electrodes and fNIRS optodes, ensuring proper placement and scalp coupling [22].
Stimulus Presentation Software	Displays visual cues (images of animals and tools) and task instructions with precise timing [8].
Synchronization Interface	Hardware or software solution to time-lock EEG and fNIRS recordings, ensuring temporal alignment of multimodal data [130].

Participant Preparation:

• Recruit right-handed native speakers (for language-dependent tasks) with normal or corrected-to-normal vision [8].
• Obtain informed consent following institutional ethical guidelines.
• Measure participant's head circumference and select appropriate integrated EEG-fNIRS cap size.
• Prepare scalp sites: Apply electrolyte gel for EEG electrodes following standard protocols. Ensure fNIRS optodes have proper scalp contact by checking signal quality.

Stimuli and Task Design:

• Stimulus Selection: Use grayscale images of 18 animals and 18 tools, standardized for size and contrast [8].
• Task Conditions:
  • Silent Naming: Participants silently name the displayed object in their mind.
  • Visual Imagery: Participants visualize the object in their minds.
  • Auditory Imagery: Participants imagine sounds associated with the object.
  • Tactile Imagery: Participants imagine the feeling of touching the object.
• Trial Structure:
  • Visual cue presentation (2-3 seconds)
  • Mental task period (3-5 seconds)
  • Inter-trial interval (randomized 10-15 seconds)
• Experimental Design: Implement a block design with randomized task conditions and stimulus categories across multiple runs.

Data Acquisition Parameters:

• EEG: Record from at least 32 channels including frontal, central, parietal, and occipital sites based on 10-20 system. Sampling rate ≥ 500 Hz.
• fNIRS: Configure source-detector pairs with distances of 3 cm for cortical sensitivity and 1.5 cm for superficial signal correction [5]. Use dual wavelengths (e.g., 760 nm and 850 nm) to distinguish HbO and HbR.
• Synchronization: Implement hardware synchronization or use trigger signals shared between EEG and fNIRS systems at stimulus onset.

Data Processing Workflow:

Diagram 1: Data processing workflow for simultaneous EEG-fNIRS in semantic decoding.

Analysis Methods:

• EEG Analysis: Extract event-related potentials (ERPs) focusing on components like N400 and P300 that are associated with semantic processing. Perform time-frequency analysis to examine power in theta (4-7 Hz) and alpha (8-12 Hz) bands [132].
• fNIRS Analysis: Calculate oxygenated (HbO) and deoxygenated hemoglobin (HbR) concentration changes. Perform general linear model (GLM) analysis to identify channels with significant hemodynamic responses to different semantic categories.
• Multimodal Fusion: Implement feature-level or decision-level fusion approaches. Apply advanced machine learning classifiers (e.g., support vector machines, deep neural networks) to distinguish between animal and tool categories based on combined EEG and fNIRS features [129].

Technical Implementation and Integration Challenges

System Integration Considerations

Successful deployment of EEG-fNIRS technology requires careful attention to system integration challenges. Mechanical integration poses significant hurdles as EEG electrodes and fNIRS optodes compete for limited scalp space [130]. This is particularly challenging for semantic decoding studies that require coverage of language-related regions (e.g., temporal and frontal areas). Custom-designed helmets using 3D printing or thermoplastic materials can optimize probe placement for specific experimental needs, though at increased cost [22].

Electrical integration must address potential crosstalk between systems. fNIRS source modulation frequencies should be set above the EEG frequency band of interest (typically >40 Hz) to prevent interference with neural signals [130]. Implementing shared analog-to-digital converter (ADC) architecture between fNIRS and EEG acquisitions can eliminate timing synchronization issues, though this typically requires purpose-built integrated systems rather than combining discrete commercial systems.

Advanced Data Fusion Techniques

Effective integration of EEG and fNIRS data requires sophisticated fusion approaches that address the inherent differences in temporal dynamics and physiological origins of the signals. Recent advances in machine learning offer promising directions:

Dual-Decoder Architecture: This approach employs separate decoders to extract modality-general features (shared between EEG and fNIRS) and modality-specific features (unique to each modality). The complementary information enhances decoding performance for semantic classification tasks [129].

Gradient Rebalancing Strategy: To prevent one modality from dominating the learning process, this technique monitors and adjusts gradient contributions during model training, ensuring balanced feature extraction from both EEG and fNIRS signals [129].

Multilayer Network Analysis: This method models brain connectivity across modalities, capturing both fast electrical processes (via EEG) and slower hemodynamic responses (via fNIRS). This approach has shown particular utility in characterizing the complex brain network dynamics underlying cognitive processes like semantic reasoning [127].

Clinical Translation and Research Applications

Clinical Implementation Framework

The path to clinical deployment of EEG-fNIRS for semantic decoding faces both opportunities and challenges. For patients with severe communication impairments (e.g., locked-in syndrome), semantic BCIs could potentially restore direct communication capabilities, bypassing the character-by-character spelling used in current systems [8]. However, translation to clinical practice requires addressing several practical considerations:

Regulatory Compliance: Systems must meet medical device regulations (e.g., FDA, CE marking), requiring rigorous validation studies and quality management systems.

Clinical Workflow Integration: Successful deployment requires minimizing setup time, simplifying operation, and ensuring compatibility with clinical environments.

Reimbursement Strategy: Demonstrating clear clinical utility and cost-effectiveness is essential for insurance coverage and healthcare adoption.

Emerging Research Applications

Beyond semantic decoding, integrated EEG-fNIRS shows promise across multiple research domains:

Motor Rehabilitation: Combined monitoring of electrical and hemodynamic activity provides comprehensive assessment of cortical reorganization during stroke recovery [65] [131].

Cognitive Monitoring: The hybrid approach enables investigation of neural correlates of mental workload, fatigue, and cognitive states in real-world environments [128] [132].

Developmental Neuroscience: The portability and minimal constraints of EEG-fNIRS make it particularly suitable for studying neurodevelopment in infants and children [22] [130].

Signaling Pathways and Neurophysiological Basis

The complementary nature of EEG and fNIRS stems from their sensitivity to different aspects of neural activity linked through neurovascular coupling. The following diagram illustrates the relationship between electrical and hemodynamic signals in response to neural activation.

Diagram 2: Neurovascular coupling linking neural activity to EEG and fNIRS signals.

This neurophysiological relationship forms the foundation for multimodal semantic decoding. EEG captures the rapid electrical activity associated with immediate neural processing of semantic information, while fNIRS reflects the slower metabolic support required for sustained cognitive processing. The combination provides a more complete picture of brain activity during semantic tasks than either modality alone.

The integration of EEG and fNIRS technologies offers a powerful approach for semantic decoding research and clinical applications. The cost-benefit analysis reveals that while significant challenges exist in system integration, data processing, and clinical translation, the complementary information provided by these modalities delivers unique insights into brain function that cannot be obtained with single-modality approaches. As technical advancements continue to address current limitations and computational methods improve for multimodal data fusion, EEG-fNIRS is poised to become an increasingly valuable tool for understanding semantic processing and developing practical BCIs for communication restoration. Researchers and clinicians should carefully consider the specific requirements of their applications when evaluating the implementation of this promising multimodal technology.

Conclusion

Simultaneous EEG-fNIRS recording presents a powerfully synergistic and clinically viable approach for semantic neural decoding, successfully bridging the critical gap between temporal and spatial resolution that limits individual modalities. The integration of EEG's millisecond-level electrical tracking with fNIRS's localized hemodynamic monitoring provides a more complete picture of brain activity during semantic processing, underpinned by neurovascular coupling. Future directions should focus on standardizing experimental protocols, advancing real-time data fusion algorithms powered by machine learning, and developing compact, wearable integrated systems. For biomedical and clinical research, these advancements promise not only more robust brain-computer interfaces for communication but also novel, non-invasive biomarkers for tracking cognitive decline in neurodegenerative diseases and objectively evaluating the efficacy of therapeutic interventions in clinical trials.

References

• 1. Semantic reconstruction of continuous language from non- ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11304553/]
• 2. of Imagined Speech and Visual Imagery as Intuitive... Neural Decoding [https://pubmed.ncbi.nlm.nih.gov/33232243/]
• 3. Towards decoding individual words from non-invasive ... [https://www.nature.com/articles/s41467-025-65499-0]
• 4. Simultaneous multimodal fNIRS-EEG recordings reveal ... [https://www.nature.com/articles/s41598-023-31609-5]
• 5. Simultaneous functional near-infrared spectroscopy and ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5566595/]
• 6. Systematic review: progress in EEG-based speech imagery ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12453656/]
• 7. for Direct Communication in Brain-Computer Interface Neural Decoding [https://www.frontiersin.org/research-topics/46709/neural-decoding-for-direct-communication-in-brain-computer-interface/magazine]
• 8. Simultaneous EEG and fNIRS recordings for semantic ... [https://www.nature.com/articles/s41597-025-04967-0]
• 9. Concurrent fNIRS and EEG for Brain Function Investigation [https://pmc.ncbi.nlm.nih.gov/articles/PMC9371171/]
• 10. Introduction - Electroencephalography (EEG) - NCBI - NIH [https://www.ncbi.nlm.nih.gov/books/NBK390346/]
• 11. Brain Electrical Oscillation Signature Profiling [https://en.wikipedia.org/wiki/Brain_Electrical_Oscillation_Signature_Profiling]
• 12. Evaluating the effects of multimodal EEG-fNIRS ... [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0331177]
• 13. Millisecond-Timescale Optical Control of Neural Dynamics ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC2830644/]
• 14. How to design real-world functional near-infrared ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12618016/]
• 15. Prefrontal fNIRS-based clinical data analysis of brain functions ... [https://jbiomedsem.biomedcentral.com/articles/10.1186/s13326-021-00256-y]
• 16. Use of cortical hemodynamic responses in digital therapeutics ... [https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-024-01404-y]
• 17. Lock-in functional near-infrared spectroscopy for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9045899/]
• 18. Best practices for fNIRS publications - PMC - PubMed Central [https://pmc.ncbi.nlm.nih.gov/articles/PMC7793571/]
• 19. Prefrontal cortex activity during binocular color fusion and ... [https://www.frontiersin.org/journals/neurology/articles/10.3389/fneur.2025.1527434/full]
• 20. Identification of the human cerebral cortical hemodynamic ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10751349/]
• 21. Neurovascular coupling during auditory stimulation: event-related potentials and fNIRS hemodynamic - PMC [https://pmc.ncbi.nlm.nih.gov/articles/PMC10517045/]
• 22. A Cross-Disciplinary Review of the fNIRS-EEG Dual-Modality ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11506513/]
• 23. An EEG-fNIRS neurovascular coupling analysis method to investigate cognitive-motor interference - ScienceDirect [https://www.sciencedirect.com/science/article/abs/pii/S001048252300433X]
• 24. Quantifying neurovascular coupling through a concurrent assessment of arterial, capillary, and neuronal activation in humans: A multimodal EEG-fNIRS-TCD investigation - ScienceDirect [https://www.sciencedirect.com/science/article/pii/S1053811924004075]
• 25. fNIRS vs EEG: Comparing Non-Invasive Brain Imaging ... [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/?srsltid=AfmBOoradWaxfMrjbpjp75N1yZ5McSwTpNgR_TSw-RmOlOx9YXsmMn1K]
• 26. Quantitative comparison of resting-state functional ... [https://www.sciencedirect.com/science/article/abs/pii/S1053811912001917]
• 27. Multimodal fNIRS–EEG sensor fusion: Review of data-driven ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12592382/]
• 28. Decoding semantic representations from functional near- ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5568915/]
• 29. Exploring the Potential of Electroencephalography Signal ... [https://medinform.jmir.org/2025/1/e72027]
• 30. Frontiers | Multimodal MBC-ATT: cross-modality attentional fusion of... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1660532/full]
• 31. Simultaneous EEG - fNIRS study of visual cognitive processing: ERP... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12157042/]
• 32. Neural Activity and Decoding of Action Observation Using ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6803538/]
• 33. Exploring Imagined Movement for Brain–Computer Interface ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12468679/]
• 34. Modeling Semantic Encoding in a Common Neural ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6048235/]
• 35. Brain activity characteristics of RGB stimulus: an EEG study [https://www.nature.com/articles/s41598-023-46450-z]
• 36. Wearable, Integrated EEG–fNIRS Technologies: A Review [https://pmc.ncbi.nlm.nih.gov/articles/PMC8469799/]
• 37. A Practical Guide for Combined fNIRS & EEG Recordings [https://www.gtec.at/2021/04/12/combined-fnirs-eeg-recordings/?srsltid=AfmBOoopL-YgMz5E37dhOM4dhUv2FoZMp7sZxc2EGJMFbdkKQp6tKFXk]
• 38. fNIRS - EEG | fNIRS Systems | NIRS Devices | NIRx [https://nirx.net/fnirs-eeg]
• 39. Introducing our new products and events in 2025 [https://www.gtec.at/2025/01/14/introducing-new-products-and-events-in-2025/?srsltid=AfmBOoo6Y-ci5dH_4RgR63_n0S-ZxD0UyNEXcr2ffL_eds8GWZctvQPa]
• 40. Multimodal fNIRS - EEG measurements — Staying in sync — Artinis... [https://www.artinis.com/blogpost-all/2023/multimodal-fnirs-eeg-measurements-staying-in-sync]
• 41. Preprocessing functional near-infrared spectroscopy ... [https://mne.tools/stable/auto_tutorials/preprocessing/70_fnirs_processing.html]
• 42. fNIRS–EEG data preprocessing [https://bio-protocol.org/exchange/minidetail?id=18156217&type=30]
• 43. Data Processing in Functional Near-Infrared Spectroscopy ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8151801/]
• 44. Deep learning in fNIRS: a review - PMC - PubMed Central [https://pmc.ncbi.nlm.nih.gov/articles/PMC9301871/]
• 45. Learning based motion artifacts processing in fNIRS: a mini ... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2023.1280590/full]
• 46. Simultaneous EEG and fNIRS recordings for semantic decoding of... [https://pubmed.ncbi.nlm.nih.gov/40221457/]
• 47. A Practical Guide for Combined fNIRS & EEG Recordings [https://www.gtec.at/2021/04/12/combined-fnirs-eeg-recordings/]
• 48. vs fNIRS : Comparing Non-Invasive Brain EEG ... - iMotions Imaging [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/]
• 49. multimodal Simultaneous - fNIRS reveal new insights... EEG recordings [https://www.nature.com/articles/s41598-023-31609-5?error=cookies_not_supported&code=827d7122-9221-4f13-bf81-d1f41398d37c]
• 50. The Music of Silence: Part II: Music Listening Induces Imagery ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8412992/]
• 51. Advanced Signal Processing and Machine/Deep Learning Approaches on a Preprocessing Block for EEG Artifact Removal: A Comprehensive Review | Circuits, Systems, and Signal Processing [https://link.springer.com/article/10.1007/s00034-024-02936-3]
• 52. How EEG preprocessing shapes decoding performance [https://www.nature.com/articles/s42003-025-08464-3]
• 53. Introducing RELAX: An automated pre-processing pipeline for cleaning EEG data - Part 1: Algorithm and application to oscillations - ScienceDirect [https://www.sciencedirect.com/science/article/abs/pii/S1388245723000275]
• 54. Preprocessing functional Near Infrared Spectroscopy ... [https://nimrobotics.com/blog/2025/fnirs-preprocessing/]
• 55. Signal quality index: an algorithm for quantitative ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7687963/]
• 56. Applications of Multivariate Pattern Classification Analyses ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC2773173/]
• 57. Machine Learning for Neural Decoding [https://www.eneuro.org/content/7/4/ENEURO.0506-19.2020]
• 58. Decoding the brain: From neural representations to ... [https://www.sciencedirect.com/science/article/pii/S0092867424009802]
• 59. Multivariate pattern analysis: a method and software to reveal ... [https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-024-05660-6]
• 60. A Practical Guide for Combined fNIRS & EEG Recordings [https://www.gtec.at/2021/04/12/combined-fnirs-eeg-recordings/?srsltid=AfmBOorr4zH9qliMssrZP0wf5_ZgCzl_DMn3VRLDSbGNl7Y40iz6IxEA]
• 61. Study of promising speech-enabling interface offers hope for ... [https://med.stanford.edu/news/all-news/2025/08/brain-computer-interface.html]
• 62. fNIRS - OpenNeurosurgery.com [https://openneurosurgery.com/fnirs/]
• 63. Functional Magnetic Resonance Imaging and Functional Near ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC5563305/]
• 64. Functional Near-Infrared Spectroscopy in neurodegenerative ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11479976/]
• 65. Current implications of EEG and fNIRS as functional ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11629311/]
• 66. and Simultaneous recordings for EEG of... fNIRS semantic decoding [https://pmc.ncbi.nlm.nih.gov/articles/PMC11993746/]
• 67. Systemic Review on Transcranial Electrical Stimulation ... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2021.629323/full]
• 68. Resting-state functional near-infrared spectroscopy in ... [https://www.sciencedirect.com/science/article/pii/S2213158225000038]
• 69. EEG Biomarkers: Transforming CNS Drug Discovery [https://www.neuroelectrics.com/blog/eeg-biomarkers-transforming-cns-drug-discovery]
• 70. Editorial: Translational brain-computer interfaces [https://pmc.ncbi.nlm.nih.gov/articles/PMC9969149/]
• 71. Brain Products & NIRx: Introducing a mobile , multimodal EEG - fNIRS ... [https://pressrelease.brainproducts.com/eeg-fnirs-setup/?seq_no=3]
• 72. Motion Artifacts Correction from Single-Channel EEG and ... [https://www.mdpi.com/1424-8220/22/9/3169]
• 73. Frontiers | Current State and Future Prospects of EEG and fNIRS in... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2019.00172/full]
• 74. A review of combined functional neuroimaging and motion capture for... [https://jneuroengrehab.biomedcentral.com/articles/10.1186/s12984-023-01294-6]
• 75. Selectively damping materials for next-generation motion ... [https://pubs.rsc.org/en/content/articlehtml/2025/mh/d5mh00700c]
• 76. A hybrid model for detecting motion artifacts in ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12288294/]
• 77. Improving neuroimaging headgear placement robustness ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12546995/]
• 78. Optimal positioning of optodes on the scalp for ... [https://www.sciencedirect.com/science/article/abs/pii/S0165027018302425]
• 79. Tutorials/NIRS_Optimal_montage - Brainstorm [https://neuroimage.usc.edu/brainstorm/Tutorials/NIRS_Optimal_montage]
• 80. HEFMI-ICH: a hybrid EEG-fNIRS motor imagery dataset for ... [https://www.nature.com/articles/s41597-025-06100-7]
• 81. Development of an Integrated EEG/fNIRS Brain Function ... [https://www.mdpi.com/1424-8220/21/22/7703]
• 82. Quantifying the impact of hair and skin characteristics on fNIRS signal quality for enhanced inclusivity - PubMed [https://pubmed.ncbi.nlm.nih.gov/40897802/]
• 83. Improving optical contact for functional near‑infrared brain spectroscopy and imaging with brush optodes - PMC [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3342194/]
• 84. High-density multidistance fNIRS enhances detection of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12412631/]
• 85. Comparing fNIRS signal qualities between approaches with and without short channels | PLOS One [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0244186]
• 86. fNIRS: Non-stationary preprocessing methods [https://www.sciencedirect.com/science/article/pii/S1746809422005675]
• 87. STA-Net: Spatial–temporal alignment network for hybrid ... [https://www.sciencedirect.com/science/article/abs/pii/S156625352500096X]
• 88. EEG Decoding Reveals the Strength and Temporal Dynamics ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6588723/]
• 89. Early-stage fusion of EEG and fNIRS improves ... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.1062889/full]
• 90. Decoding cognition in neurodevelopmental, psychiatric ... [https://www.sciencedirect.com/science/article/pii/S0149763424002641]
• 91. EEG–fNIRS signal integration for motor imagery ... [https://www.sciencedirect.com/science/article/pii/S2772528625000299]
• 92. fNIRS vs EEG: Comparing Non-Invasive Brain Imaging ... [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/?srsltid=AfmBOopwB1g7rFys9MJCgM1Uuw7Abo9hZsBFfjspaPHynhN_uhhPp_Ku]
• 93. Comparing structure–function relationships in brain ... [https://www.nature.com/articles/s41598-024-79817-x]
• 94. Design, manufacturing, and testing of 3D-printed fittings for ... [https://www.nature.com/articles/s41598-025-25851-2]
• 95. Quantitative comparison of correction techniques for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7511246/]
• 96. Improved classification performance of EEG-fNIRS ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2022.973959/full]
• 97. Hybrid EEG-fNIRS decoding with dynamic graph ... [https://www.sciencedirect.com/science/article/abs/pii/S1746809425000813]
• 98. Simultaneous EEG-fNIRS Data Classification Through ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11379445/]
• 99. OptEF-BCI: An Optimization-Based Hybrid EEG and fNIRS ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC10215946/]
• 100. Improvement of classification accuracy of functional near- ... [https://www.sciencedirect.com/science/article/pii/S2667102625000245]
• 101. fNIRS vs EEG: Comparing Non-Invasive Brain Imaging ... [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/?srsltid=AfmBOoqIlMGmHM1EkZn7mTSzjg3rNdb5PR98PexIW1a7DOKPAnSBXnXX]
• 102. Applications and advances of combined fMRI-fNIRs ... [https://www.frontiersin.org/journals/neurology/articles/10.3389/fneur.2025.1542075/full]
• 103. Enhanced spatiotemporal resolution imaging of neuronal ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7778454/]
• 104. A quantitative comparison of NIRS and fMRI across multiple ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC3021967/]
• 105. Multimodal assessment of the spatial correspondence ... [https://www.nature.com/articles/s41598-023-29123-9]
• 106. The application of fNIRS in studies on occupational workload [https://pmc.ncbi.nlm.nih.gov/articles/PMC12053328/]
• 107. A quantitative comparison of NIRS and fMRI across ... [https://www.sciencedirect.com/science/article/abs/pii/S1053811910013777]
• 108. The effects of protocol factors and participant ... [https://www.sciencedirect.com/science/article/pii/S2666956025000443]
• 109. Simultaneous EEG and MEG source reconstruction in sparse ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6870179/]
• 110. Combined EEG/MEG Can Outperform Single Modality EEG or ... [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0118753]
• 111. IFCN-endorsed practical guidelines for clinical ... [https://www.sciencedirect.com/science/article/pii/S1388245718306576]
• 112. A comparison between scalp- and source-reconstructed ... [https://www.nature.com/articles/s41598-018-30869-w]
• 113. Comparing MEG and EEG measurement set-ups for a brain ... [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0319328]
• 114. Decoding word and category-specific spatiotemporal ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC3020243/]
• 115. Deep learning for inner speech recognition [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1668935/full]
• 116. Decoding Semantics from Dynamic Brain Activation Patterns [https://pmc.ncbi.nlm.nih.gov/articles/PMC10913025/]
• 117. Neural Activity and Decoding of Action Observation Using ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2019.00357/full]
• 118. Decoding of four movement directions using hybrid NIRS ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC4009438/]
• 119. Multimodal MBC-ATT: cross-modality attentional fusion of ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12504385/]
• 120. Decoding emotions through personalized multi-modal ... [https://www.sciencedirect.com/science/article/pii/S1746809425001430]
• 121. Validation of a mobile fNIRS device for measuring working ... [https://pubmed.ncbi.nlm.nih.gov/38049074/]
• 122. a simultaneous fNIRS-fMRI study [https://pmc.ncbi.nlm.nih.gov/articles/PMC9896013/]
• 123. fMRI-based validation of continuous-wave fNIRS ... [https://www.nature.com/articles/s41598-022-06519-7]
• 124. NIRS Publications | Published fNIRS Research with ... [https://nirx.net/publications]
• 125. NIRx | fNIRS Systems | NIRS Devices [https://nirx.net/]
• 126. NIRx fNIRS Webinars | fNIRS Systems | NIRS Devices [https://nirx.net/webinars]
• 127. Investigating the interaction between EEG and fNIRS [https://www.sciencedirect.com/science/article/pii/S1877750324002096]
• 128. The Advantages of Combining fNIRS and EEG for Brain ... [https://www.artinis.com/blogpost-all/advantages-of-combining-fnirs-and-eeg]
• 129. Towards effective joint learning for EEG-fNIRS integration [https://www.sciencedirect.com/science/article/abs/pii/S0925231225008008]
• 130. Wearable, Integrated EEG–fNIRS Technologies: A Review [https://www.mdpi.com/1424-8220/21/18/6106]
• 131. Analysis of Human Gait Using Hybrid EEG-fNIRS-Based ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2020.613254/full]
• 132. Simultaneous EEG-fNIRS study of visual cognitive processing [https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0325017]