A Comprehensive Guide to fNIRS-EEG Data Preprocessing Pipelines for Robust Multimodal Fusion

Abstract

This article provides a comprehensive overview of data preprocessing pipelines for the fusion of functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG), tailored for researchers and drug development professionals. It covers the foundational principles of both modalities, explores methodological approaches for integrated analysis—from early fusion to deep learning strategies—and addresses critical troubleshooting steps for artifact correction and data quality control. Furthermore, it outlines validation frameworks and comparative analyses of fusion techniques, highlighting their applications in brain-computer interfaces and clinical neurology. The goal is to serve as a technical guideline for implementing reproducible and effective fNIRS-EEG fusion to advance multimodal brain imaging research.

Understanding the Core Principles of fNIRS and EEG for Multimodal Integration

Fundamental Concepts of EEG

What is the neurophysiological basis of EEG signals?

Electroencephalography (EEG) measures electrical activity generated by the synchronized firing of neuronal populations in the brain. These signals represent the summation of post-synaptic potentials from pyramidal cells that are oriented in parallel, creating electrical fields strong enough to be detected at the scalp surface. The basis of EEG lies in the rhythmic, synchronized oscillations of these neural populations, which produce distinct brain wave patterns classified by their frequency ranges: delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), and gamma (>30 Hz) [1] [2].

What makes the alpha rhythm particularly significant in EEG research?

The alpha rhythm (8-13 Hz), first discovered by Hans Berger in 1929, remains one of the most studied EEG oscillations due to its prominent role in brain function [3]. This rhythm is most evident during wakeful relaxation with closed eyes and is thought to play an inhibitory function by actively suppressing irrelevant brain regions during cognitive processing [3]. Recent research suggests the alpha rhythm may have played a pivotal role in the cognitive evolution of nocturnal mammals, potentially serving to maintain wakefulness during nighttime hours [3]. Studies have shown that alpha oscillations exhibit characteristic patterns across different states: reduced alpha power during cognitive engagement, increased alpha synchronization during internal processing, and specific spatial distributions that vary with age and cognitive status [3].

Technical Troubleshooting Guide & FAQs

EEG Recording Issues

Q: Why are my reference or ground electrodes showing persistent high impedance or oversaturation?

A: This issue commonly arises from improper electrode-skin contact or individual physiological factors [4].

• Step-by-Step Troubleshooting:
  • Check Electrode Connections: Ensure all electrodes are properly plugged in, reapplied with proper skin preparation (cleaning, abrasion, and conductive paste), and check for "bridging" between electrodes due to excess gel [4].
  • Test Alternative Placements: If issues persist, try alternative ground placements such as the participant's hand, collarbone area, or sternum [4].
  • Isolate Hardware Issues: Restart the amplifier unit and acquisition software. If available, test with a different headbox or complete system in another room to rule out hardware failure [4].
  • Remove Metal Objects: Ask the participant to remove all metal accessories, including necklaces and bracelets [4].
  • Consider Individual Factors: Some individuals may carry more static electricity or have unique skin properties that cause oversaturation. In such cases, placing the ground further away (e.g., on the hand) may resolve the issue [4].

Q: How can I minimize jitter and latency when synchronizing EEG with other devices like fNIRS?

A: Precise temporal alignment is crucial for multimodal research [5] [6].

• Solution: Implement a Lab Streaming Layer (LSL)-based system for data acquisition [5] [6]. This open-source platform helps overcome two common issues:
  • Jitter: Millisecond-order temporal variability that readily affects the signal-to-noise ratio of electrophysiological outcomes.
  • Latency: Constant delays between different data streams. LSL ensures precise time-alignment of datasets, which is particularly critical for detecting stimulus-induced transient neural responses or testing hypotheses about temporal relationships between different functional aspects [5] [6].

Data Preprocessing & Fusion Challenges

Q: Which pre-processing steps have the most significant impact on EEG data quality?

A: Research indicates that signal segmentation and re-referencing methods are particularly critical [7].

• Segmentation: The approach to segmenting continuous data (into trials/epochs) significantly affects subsequent cleaning procedures [7].
• Re-referencing: Four common approaches show different impacts [7]:
  • Common Averaged Reference (CAR)
  • robust Common Averaged Reference (rCAR)
  • Reference Electrode Standardization Technique (REST)
  • Reference Electrode Standardization and Interpolation Technique (RESIT) Studies found similar topographical representations after CAR, REST, and RESIT, while rCAR showed the most different event-related spectral perturbation (ERSP) topographical pattern [7].
• Artifact Removal: Interestingly, the choice of Independent Component Analysis (ICA) algorithm (e.g., SOBI vs. Extended Infomax) had relatively small effects on the cleaning procedure compared to segmentation and re-referencing [7].

Q: What are the primary challenges in fusing EEG with fNIRS data?

A: The fusion of EEG and fNIRS is complicated by their fundamentally different signal origins and artifact profiles [8].

• Complementary Properties: EEG captures fast electrical neural signals (millisecond resolution) with limited spatial precision, while fNIRS measures slower hemodynamic responses (reflecting blood oxygenation changes) with better spatial localization [8] [9].
• Artifact Challenges: Both modalities are contaminated by physiological artifacts, but these manifest differently [8]:
  • EEG is susceptible to ocular (EOG) and muscle (EMG) artifacts.
  • fNIRS is contaminated by systemic physiology (cardiac, respiratory, blood pressure) that affects hemodynamics in the scalp and brain.
• Fusion Complexity: The different temporal resolutions and physiological origins make fusion non-trivial. Most current methods rely on data concatenation, model-based, or decision-level strategies, while more advanced source-decomposition techniques that could reveal complex neurovascular coupling processes remain underrepresented [8].

Experimental Protocols for Multimodal Research

Protocol 1: Motor Imagery and Mental Arithmetic Task Classification

This protocol outlines the methodology for acquiring a simultaneous EEG-fNIRS dataset for brain-computer interface applications, adapted from a publicly available benchmark dataset [9].

• Participants: 29 subjects (28 right-handed, 1 left-handed; 14 males, 15 females; average age 28.5 ± 3.7 years) [9].
• Experimental Tasks:
  • Motor Imagery (MI): Imagination of left-hand versus right-hand movements without physical execution. Each trial consists of a rest period followed by an imagination period cued by visual stimuli [9].
  • Mental Arithmetic (MA): Performing serial subtractions of two numbers from a given starting number (e.g., subtracting 7 from 1000 repeatedly). Each trial includes a rest period and a task period [9].
• Data Acquisition:
  • EEG: Recorded using a specific electrode cap following the international 10-10 or 10-20 system.
  • fNIRS: Recorded using optodes placed over relevant cortical areas (e.g., motor cortex for MI, prefrontal cortex for MA).
• Data Analysis (DeepSyncNet Framework):
  • Preprocessing: Standard filtering and artifact removal for both modalities.
  • Feature Extraction: 1D EEG and fNIRS signals are converted into 3D tensors to capture spatiotemporal information. A Receptive Field Block (RFB) is used for multi-scale feature extraction [9].
  • Fusion: An Attentional Fusion (AF) mechanism with residual connections adaptively integrates EEG and fNIRS features at early network layers. Feature Attention Mechanisms (FAM) and Spatiotemporal Attention (STA) dynamically refine the fused representations [9].
  • Classification: A learnable weighted fusion mechanism optimizes the contribution of each modality for final task classification [9].

Protocol 2: Investigating Emotional Processing with EEG Microstates

This protocol examines alpha rhythm dynamics during emotional experiences using EEG microstate analysis [3].

• Stimuli: Presentation of emotionally charged music videos categorized as "happy" or "sad" [3].
• EEG Recording: Standard high-density EEG recording from 64+ channels.
• Analysis Pipeline:
  • Preprocessing: Standard filtering, artifact removal, and re-referencing.
  • Microstate Analysis: Identification of prototypical topographic maps (classes A, B, C, D) that remain stable for ~60-120ms before rapidly transitioning to another map.
  • Source Localization: Use of eLORETA (exact Low Resolution Brain Electromagnetic Tomography) to estimate cortical sources of activity.
  • Statistical Comparison: Compare microstate occurrence, duration, and functional connectivity between "happy" and "sad" conditions.
• Expected Outcomes: Increased class D microstate occurrence and current source density in the central parietal region during happy music (indicating enhanced attention), and elevated class C microstate occurrence and functional connectivity in the precuneus during sad music (associated with mind-wandering) [3].

Table 1: Impact of Different Re-referencing Methods on EEG Data Quality

Re-referencing Method	Acronym	Key Characteristics	Effect on ERSP Topography
Common Averaged Reference [7]	CAR	Rereferences to the average of all electrodes	Similar to REST and RESIT
robust Common Averaged Reference [7]	rCAR	A variant of CAR less sensitive to outliers	Shows most different pattern
Reference Electrode Standardization Technique [7]	REST	Estimates reference at infinity using a head model	Similar to CAR and RESIT
Reference Electrode Standardization and Interpolation Technique [7]	RESIT	Combines standardization with interpolation	Similar to CAR and REST

Table 2: Comparative Characteristics of EEG and fNIRS Neuroimaging Techniques

Characteristic	EEG	fNIRS
Measured Signal	Electrical activity from synchronized neuronal firing [8] [9]	Hemodynamic response (blood oxygenation) [8] [9]
Temporal Resolution	Millisecond level [8] [9]	Slower (seconds) [8] [9]
Spatial Resolution	Limited [8] [9]	Better than EEG [8] [9]
Primary Artifacts	Ocular (EOG), muscle (EMG) [8]	Systemic physiology (cardiac, respiratory, blood pressure) [8]
Main Strength	Direct measure of neural electrical activity with high temporal precision [9]	Better spatial localization and less susceptible to movement artifacts [8] [9]

Signaling Pathways and Workflows

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 3: Essential Materials for EEG-fNIRS Multimodal Research

Item	Function & Application
EEG Electrode Cap	Holds electrodes in standardized positions (10-10/10-20 system) for consistent scalp coverage [9] [10].
Conductive Electrode Gel/Paste	Improves electrical contact between scalp and electrodes, reducing impedance and signal noise [4].
fNIRS Optodes	Sources emit near-infrared light into the head; detectors measure light intensity after tissue absorption [8].
Abrasive Skin Prep Gel	Gently removes dead skin cells and oils to significantly reduce skin-electrode impedance [4].
Lab Streaming Layer (LSL)	Open-source platform for synchronized multimodal data acquisition, critical for temporal alignment of EEG and fNIRS [5] [6].
Reference & Ground Electrodes	Essential for creating a stable electrical reference point; often placed on mastoids or other locations [4] [10].

Core Theoretical Foundations

The Hemodynamic Response in fNIRS

What is the hemodynamic response and how does fNIRS measure it?

Functional near-infrared spectroscopy (fNIRS) is a non-invasive optical neuroimaging technique that measures brain activity by detecting hemodynamic changes associated with neuronal activation. This process relies on neurovascular coupling, where active neuronal tissue triggers a rapid delivery of blood, resulting in localized changes in blood oxygenation [11] [12].

During brain activation, a complex physiological sequence occurs: First, neuronal activity increases local energy demands, initially depleting oxygen and causing a brief rise in deoxygenated hemoglobin. This triggers a subsequent oversupply of cerebral blood flow through local arterial vasodilation. The resulting hemodynamic response typically shows an increase in oxygenated hemoglobin and a decrease in deoxygenated hemoglobin as oxygenated blood flushes through the active region [12] [13].

fNIRS Hemodynamic Response Pathway:

The typical adult hemodynamic response follows a characteristic pattern, often modeled using a canonical hemodynamic response function composed of two Gamma functions to characterize the positive response and undershoot [14]. However, this response can vary between brain regions, across trial repetitions, and among individuals. In newborn populations, for instance, studies have demonstrated a more mixed hemodynamic response compared to adults, potentially due to developing neurovascular coupling mechanisms [13].

The Modified Beer-Lambert Law

How does the Modified Beer-Lambert Law convert light attenuation into hemoglobin concentrations?

The Modified Beer-Lambert Law (MBLL) is the fundamental principle enabling fNIRS to quantify changes in hemoglobin concentrations from light attenuation measurements. This approach generalizes the traditional Beer-Lambert law to account for light scattering in biological tissues [15] [16].

Principles of the Modified Beer-Lambert Law

The MBLL is implemented through these key equations:

• Optical Density: ( OD = -\log\left(\frac{I}{I0}\right) = \varepsilon c d DPF + G ) [15] Where ( I0 ) is incident light intensity, ( I ) is detected light intensity, ( \varepsilon ) is the molar extinction coefficient, ( c ) is chromophore concentration, ( d ) is source-detector separation, ( DPF ) is the differential pathlength factor, and ( G ) accounts for light loss due to scattering.

• Differential Form: ( \Delta OD = -\log\left(\frac{I(t)}{I0}\right) \approx \langle L \rangle \Delta\mua(t) ) [16] This differential form relates changes in optical density to changes in absorption coefficients, where ( \langle L \rangle ) represents the mean photon pathlength.

For practical calculation of hemoglobin concentrations, the system of equations becomes:

[ \begin{bmatrix} \Delta OD{\lambda1} \ \Delta OD{\lambda2}

\end{bmatrix}

d \cdot DPF \cdot \begin{bmatrix} \varepsilon{HbO2}^{\lambda1} & \varepsilon{HbR}^{\lambda1} \ \varepsilon{HbO2}^{\lambda2} & \varepsilon{HbR}^{\lambda2} \end{bmatrix} \cdot \begin{bmatrix} \Delta [HbO_2] \ \Delta [HbR] \end{bmatrix} ]

This allows researchers to solve for the concentration changes of oxyhemoglobin (( \Delta [HbO_2] )) and deoxyhemoglobin (( \Delta [HbR] )) by measuring optical density changes at multiple wavelengths [15] [12].

Technical Support & Troubleshooting

Frequently Asked Questions

FAQ 1: Why is my fNIRS signal showing an inverted hemodynamic response?

An inverted hemodynamic response (decrease in HbO₂ instead of increase) can result from several factors:

• Physiological Variability: Newborns and infants often exhibit mixed hemodynamic responses due to developing neurovascular coupling. Approximately 46% of studies in newborns report atypical responses [13].
• Improper Probe Placement: Ensure optodes are positioned over the cortical region of interest with adequate pressure without causing discomfort.
• Systemic Confounds: Physiological processes like blood pressure changes (Mayer waves), respiration, or cardiac pulsation can contaminate the signal. Implement short-separation channels to regress out systemic influences [12] [17].
• Task Design Issues: Overly complex paradigms or insufficient rest periods may cause atypical responses. Review your block/event-related design timing.

FAQ 2: How can I distinguish true neural activation from physiological noise?

Physiological noise is a common challenge in fNIRS experiments. Implement these strategies:

• Frequency Filtering: Apply bandpass filters (typically 0.01-0.5 Hz) to remove cardiac (~1 Hz) and respiratory (~0.3 Hz) oscillations [18] [12].
• Short-Separation Regression: Use short-distance channels (<1 cm) to capture superficial contaminants that can be regressed from long-distance channels [17].
• Signal Quality Metrics: Calculate the Scalp Coupling Index (SCI) to identify poorly coupled channels. Remove channels with SCI <0.5 [18].
• Adaptive Filtering: Employ Kalman filters or principal component analysis to separate physiological noise from neural signals [14].

FAQ 3: What are the optimal parameters for the canonical hemodynamic response function in fNIRS?

The canonical HRF in fNIRS is typically modeled using two Gamma functions with these key parameters [14]:

Parameter	Typical Value	Description
Response Delay	2-4 seconds	Time to peak response after stimulus onset
Undershoot Delay	8-12 seconds	Time to undershoot minimum
Response Dispersion	1.0-1.5	Width of the positive response
Undershoot Dispersion	1.5-2.0	Width of the undershoot
Response-to-Undershoot Ratio	6:1	Amplitude ratio of response to undershoot

Optimal parameters vary by brain region, task paradigm, and population. For motor tasks, the peak HbO₂ response typically occurs around 6 seconds post-stimulus [18].

Troubleshooting Common Experimental Issues

Problem: Poor signal quality across multiple channels

• Check optode-scalp coupling: Ensure adequate pressure and use appropriate amounts of gel if using electro-optical systems.
• Verify source-detector distance: Maintain 3-4 cm separation for adult cortical measurements. Distances <1 cm primarily sample extracerebral tissue [18].
• Assess ambient light contamination: Use opaque head caps and shield measurement environment from external light sources.
• Evaluate signal-to-noise ratio: Calculate coefficient of variation for each channel; remove channels with excessive noise (>15-20%).

Problem: Inconsistent responses across subjects

• Standardize preprocessing pipeline: Apply identical filtering, motion correction, and quality thresholds across all subjects [12].
• Account for anatomical variability: Use 3D digitizer or MRI co-registration when possible to verify optode placement.
• Control physiological states: Standardize instructions regarding caffeine, food intake, and physical activity before experiments.
• Implement quality control metrics: Reject epochs with excessive motion artifacts (>80-100 μM amplitude) or poor scalp coupling [18].

Problem: Difficulty interpreting HbO₂ and HbR responses

• Expect canonical response pattern: Typically, HbO₂ increases while HbR decreases during neural activation.
• Check for cross-talk: Significant positive correlation between HbO₂ and HbR may indicate systemic contamination rather than neural activity.
• Validate with control conditions: Compare activation during task periods versus baseline or control conditions.
• Consider population-specific responses: Infant and clinical populations may show atypical response patterns [13].

Experimental Protocols & Methodologies

Standard fNIRS Processing Pipeline

fNIRS Data Processing Workflow:

Step-by-Step Processing Protocol

Step 1: Convert Raw Intensity to Optical Density

• Calculate optical density as: ( OD = -\log{10}\left(\frac{I}{I0}\right) )
• Where ( I ) is measured intensity and ( I_0 ) is reference intensity
• Perform this conversion for each wavelength [18]

Step 2: Quality Assessment and Channel Exclusion

• Calculate Scalp Coupling Index (SCI) for each channel
• Exclude channels with SCI <0.5
• Remove channels with source-detector distance <1 cm (short channels) or >4.5 cm (excessive attenuation) [18]

Step 3: Convert to Hemoglobin Concentrations

• Apply Modified Beer-Lambert Law: ( \begin{bmatrix} \Delta [HbO2] \ \Delta [HbR] \end{bmatrix} = \frac{1}{d \cdot DPF} \begin{bmatrix} \varepsilon{HbO2}^{\lambda1} & \varepsilon{HbR}^{\lambda1} \ \varepsilon{HbO2}^{\lambda2} & \varepsilon{HbR}^{\lambda2} \end{bmatrix}^{-1} \begin{bmatrix} \Delta OD{\lambda1} \ \Delta OD{\lambda_2} \end{bmatrix} )
• Typical DPF values: 5-7 for adults at 700-850 nm wavelengths [15] [12]

Step 4: Filtering and Artifact Removal

• Apply bandpass filter (0.01-0.5 Hz) to remove physiological noise
• Implement motion artifact correction (e.g., wavelet-based, spline interpolation)
• Use short-separation regression if available [12]

Step 5: Epoch Extraction and Analysis

• Extract epochs aligned to stimulus onset (typically -5 to +15 seconds)
• Apply baseline correction (pre-stimulus interval)
• Perform statistical analysis (GLM, t-tests) to identify significant responses [18]

Common Preprocessing Techniques

Table: Frequency Filters for Physiological Noise Removal

Noise Source	Frequency Range	Filter Type	Recommended Cutoff
Cardiac Pulsation	0.8-2.0 Hz	Low-pass	0.5-0.7 Hz
Respiratory Rate	0.2-0.5 Hz	Band-stop	0.2-0.5 Hz
Mayer Waves	0.07-0.13 Hz	High-pass	0.01-0.05 Hz
Very Low Frequency Drift	<0.01 Hz	High-pass	0.01 Hz

Research Reagent Solutions & Materials

Essential fNIRS Research Components

Table: Key Research Materials for fNIRS Experiments

Component	Function	Specifications & Considerations
fNIRS Instrument	Measures light attenuation	CW (continuous wave) most common; FD (frequency domain) and TR (time-resolved) offer additional information [14] [12]
Optodes	Light emission and detection	Source-detector separation: 3-4 cm for adults; Material should ensure proper scalp coupling [18]
Wavelengths	Chromophore differentiation	Typically 760 nm (sensitive to HbR) and 830-850 nm (sensitive to HbO₂) [14]
Head Cap	Optode positioning	Should provide stable positioning while maintaining comfort; Various sizes for population-specific fit
Coupling Gel	Improves light transmission	Optional for some systems; Electro-optical gels improve signal quality
Digitization System	Spatial registration	3D digitizers for co-registration with anatomical images; Essential for source localization
Quality Metrics	Signal validation	Scalp Coupling Index (SCI), coefficient of variation, signal-to-noise ratio [18]

Advanced Methodologies for fNIRS-EEG Fusion

For researchers integrating fNIRS with EEG in multimodal studies:

Temporal Alignment

• Synchronize fNIRS and EEG clocks at experiment start
• Use common trigger pulses for stimulus presentation
• Account for inherent hemodynamic delay (4-6 seconds) in fNIRS compared to EEG [17]

Artifact Handling

• EEG: Apply robust artifact removal (ICA, template subtraction)
• fNIRS: Implement motion correction and short-separation regression
• Joint: Develop common artifact rejection criteria [17]

Data Fusion Approaches

• Concatenation-based: Simple feature concatenation before classification
• Model-based: Incorporate neurovascular coupling models
• Source-decomposition: Identify latent components across modalities [17]

Experimental Design Considerations

• Include resting-state blocks for baseline signal characterization
• Implement control conditions to validate specific neural responses
• Balance task complexity with signal interpretability [12] [13]

This technical support guide provides the fundamental principles and practical methodologies essential for successful fNIRS research, with particular attention to integration with EEG in multimodal studies. The troubleshooting recommendations address the most common challenges encountered during fNIRS experimentation and data analysis.

Functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) are two non-invasive neuroimaging techniques that, when combined, create a powerful tool for neuroscience research. The primary rationale for their fusion lies in their complementary spatiotemporal resolution profiles. EEG measures the electrical activity of neurons with a millisecond-scale temporal resolution, allowing it to capture fast neural dynamics. However, its spatial resolution is poor, on the order of centimeters, due to the blurring effect of the skull and scalp [19]. In contrast, fNIRS (or its high-density version, Diffuse Optical Tomography - DOT) measures hemodynamic changes related to neural activity. It offers a relatively high spatial resolution (millimeter-scale) but suffers from a fundamentally limited temporal resolution because the hemodynamic response it measures evolves over several seconds [19] [8].

This complementarity is crucial for neuroscientific investigations. For instance, if two spatially close neuronal sources are activated sequentially with only a small temporal separation (e.g., 50 ms), using either EEG or fNIRS alone would fail to resolve them correctly. EEG would blur them spatially, while fNIRS would smooth them temporally [19]. Multimodal fusion aims to overcome the inherent limitations of each standalone modality.

Technical FAQs and Troubleshooting

FAQ 1: Why can't I resolve two finger-tapping events that are close in time and space with a single modality?

• EEG Alone: The centimeter-scale point spread of EEG reconstruction makes the sources spatially indistinguishable.
• fNIRS Alone: The slow nature of hemodynamics (lasting a few seconds) smooths out the responses, making them temporally indistinguishable if they occur with a short separation (e.g., 1 second or less) [19].
• Recommended Solution: Perform joint EEG-fNIRS source reconstruction. The high spatial precision from fNIRS can be used as a spatial prior to constrain the high-temporal-resolution EEG inversion, yielding a reconstruction with enhanced spatiotemporal resolution [19].

FAQ 2: How do I handle the inherent temporal delay of fNIRS signals relative to EEG?

The hemodynamic response measured by fNIRS has an inherent delay of several seconds compared to the electrical activity captured by EEG. A fixed temporal offset (e.g., 2-8 seconds) is sometimes applied, but this is suboptimal as the delay can vary by subject and task [20].

• Advanced Solution: Implement a dynamic temporal alignment strategy. For example, use an EEG-guided Temporal Alignment (EGTA) layer, which employs a cross-attention mechanism to generate fNIRS signals that are temporally aligned with EEG, resolving the issue of temporal mismatch [20].

FAQ 3: What are the best practices for filtering my simultaneous EEG-fNIRS data to maximize fusion quality?

Both signals contain physiological noise, but they manifest differently and require specific filtering approaches. The table below summarizes recommended parameters based on the signal type.

Table 1: Standard Filtering Parameters for EEG-fNIRS Fusion

Modality	Filter Type	Typical Frequency Bands	Primary Purpose
fNIRS	Band-Pass Filter	0.01 - 0.1 Hz [21] or 0.05 - 0.7 Hz [22]	Preserve the hemodynamic response while removing cardiac (~1 Hz), respiratory (~0.3 Hz), and very low-frequency drifts.
EEG	Band-Pass Filter	1 Hz (High-Pass) and above [21]	Remove slow drifts and line noise; specific frequency bands (e.g., alpha, beta) are often extracted for analysis.

FAQ 4: My fNIRS signals are contaminated by strong systemic physiological noise. How can fusion with EEG help?

Cardiac activity, blood pressure changes, and respiration can create noise in fNIRS that masks neural activation. While EEG is also susceptible to physiological artifacts (like ECG and EMG), the same physiological source manifests with distinct characteristics in each modality. Data-driven, unsupervised symmetric fusion methods can exploit these differences to robustly model and reject shared physiological confounders, thereby enhancing the signal-to-noise ratio of the neurally-evoked activity in both modalities [8].

Key Experimental Protocols and Methodologies

Protocol: Joint EEG and DOT Source Reconstruction

This protocol uses fNIRS/DOT reconstruction as a spatial prior for EEG source localization to resolve spatiotemporally close neural events [19].

• Mesh Generation: A segmented brain atlas (e.g., ICBM152) is used to create a tetrahedral mesh of the head, typically comprising four tissue types: scalp, skull, cerebrospinal fluid (CSF), and brain.
• Forward Modeling:
  • EEG: Calculate a leadfield matrix that models how electrical currents from neural sources propagate to electrodes placed on the scalp (e.g., according to the 10-20 system).
  • DOT: Calculate a forward model that describes how light propagates through tissue from optical sources to detectors.
• Inverse Problem: Solve the joint inverse problem using a framework like Restricted Maximum Likelihood (ReML). The DOT reconstruction provides the spatial prior, which is then used to constrain the EEG reconstruction.
• Validation: The method can be validated with simulated data where two activation spots (e.g., ~8 mm in diameter, mimicking digit representations in the somatosensory cortex) are activated sequentially with a known temporal separation (e.g., 50 ms). The joint algorithm should accurately recover these sources, which neither modality could resolve in isolation [19].

Diagram 1: Workflow for Joint EEG-DOT Source Reconstruction

Protocol: Spatial-Temporal Alignment Network (STA-Net) for BCI Decoding

This protocol is an end-to-end deep learning approach for fusing EEG and fNIRS for brain-computer interface (BCI) tasks, explicitly addressing spatial and temporal misalignment [20].

• Signal Preprocessing: Independently preprocess EEG and fNIRS signals according to standard pipelines (e.g., filtering, artifact removal).
• 3D Feature Extraction: Use 3D convolution to extract spatio-temporal features from both modalities.
• Spatial Alignment (FGSA Layer): The fNIRS-guided Spatial Alignment (FGSA) layer calculates spatial attention maps from fNIRS to identify sensitive brain regions. These maps are used to weight the corresponding EEG channels, spatially aligning EEG with fNIRS.
• Temporal Alignment (EGTA Layer): The EEG-guided Temporal Alignment (EGTA) layer generates temporal attention maps based on a cross-attention mechanism. This produces fNIRS signals that are dynamically aligned with the EEG, correcting for the variable hemodynamic delay.
• Classification: The spatio-temporally aligned features are fused and fed into a classifier for tasks like motor imagery (MI), mental arithmetic (MA), and word generation (WG) [20].

Diagram 2: STA-Net Architecture for Spatiotemporal Alignment

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 2: Key Materials and Tools for fNIRS-EEG Fusion Research

Item / Solution	Function / Explanation
High-Density DOT (HD-DOT)	An advanced fNIRS setup using multiple source-detector separations with overlapping sensitivity profiles to enable 3D image reconstruction of functional activation with spatial resolution comparable to fMRI [8].
Open-Source Analysis Toolboxes (HOMER3, MNE-Python, EEGlab)	Software packages providing standardized pipelines for data preprocessing, including conversion to optical density/chromophore concentration (fNIRS) and artifact removal/rerferencing (EEG) [22] [21].
SimBio/FieldTrip Toolbox	An environment for advanced EEG (and MEG) forward modeling, used for calculating the leadfield matrix in a realistic head model, which is critical for source reconstruction [19].
ICBM152 Brain Atlas	A standardized, non-linear asymmetric brain template used for generating realistic head models for both EEG and DOT forward modeling in simulation studies [19].
Short-Separation Channels	fNIRS source-detector pairs placed with a small separation (e.g., < 1 cm) to selectively measure systemic physiological noise from the scalp. These signals can be used as regressors to improve the recovery of cerebral signals [8] [23].
Cross-Modal Attention Mechanisms	A deep learning component that allows a model to dynamically focus on the most relevant features from one modality based on the context provided by the other modality, enhancing fusion performance [20] [24].

Neurovascular coupling (NVC) describes the fundamental physiological process whereby neural activity triggers subsequent changes in local cerebral blood flow and hemodynamics [25]. This relationship forms the critical link between the electrical brain activity measured by electroencephalography (EEG) and the hemodynamic responses measured by functional near-infrared spectroscopy (fNIRS). In combined EEG-fNIRS studies, understanding NVC is paramount, as it allows researchers to interpret the two distinct signals not as separate phenomena, but as interconnected aspects of the same underlying brain activity. The EEG signal captures the direct, millisecond-scale electrical discharges of neurons, primarily from the cortical surface [26]. Conversely, fNIRS measures the slower, second-scale hemodynamic response—changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR) concentrations—that serves as an indirect marker of this neural activity [27] [25]. This complementary relationship is the cornerstone of multimodal fusion research, enabling a more complete picture of brain function by combining excellent temporal resolution (EEG) with improved spatial localization (fNIRS) [8] [27].

Technical FAQs: Resolving Common Experimental Challenges

Q1: What are the most effective strategies to minimize motion artifacts in simultaneous EEG-fNIRS recordings?

Motion artifacts present a significant challenge, but their impact can be mitigated through a combination of hardware choices and signal processing techniques. fNIRS is generally more robust to movement than EEG [26]. Key strategies include:

• Secure Cap Fit: Use a tight but comfortable electrode cap to keep both EEG electrodes and fNIRS optodes in a fixed position, which is crucial for artifact-free data [28].
• Accelerometer Integration: For protocols involving significant movement (e.g., walking or treadmill running), attaching an accelerometer to the subject's head is recommended. The recorded motion data can then be used with advanced signal processing, like adaptive filtering, to clean the fNIRS signal [29].
• Active EEG Electrodes: Employ active EEG electrodes, which provide a higher signal-to-noise ratio and are less susceptible to environmental interference [28].
• Post-Processing Algorithms: Utilize motion correction algorithms during the preprocessing stage for both modalities. While many robust methods exist for EEG, confounder correction in fNIRS often relies on filtering or dedicated motion artifact removal techniques [8].

Q2: How can we reliably synchronize EEG and fNIRS data streams from separate devices?

Precise temporal synchronization is essential for valid NVC analysis. The recommended approach is:

• Master-Slave Configuration: Typically, the EEG amplifier, which usually has a higher sampling frequency, acts as the "master" device. The fNIRS system is then synchronized to it [28].
• Hardware Triggers: Use transistor-transistor logic (TTL) pulses sent via a parallel port or other digital I/O line to mark specific events in both data streams simultaneously. The delay for such triggers is generally very low (e.g., not exceeding 5 msec) [29].
• Software Synchronization: Employ protocols like the Lab Streaming Layer (LSL), which allows for synchronized data collection from multiple devices by sharing a common clock signal across a network [27].

Q3: Our fNIRS signals are contaminated by strong systemic physiology (e.g., heart rate, respiration). How can this be addressed in the context of NVC analysis?

Systemic physiological interference is a common confounder in fNIRS, as the technique is sensitive to cardiac pulsation, respiratory fluctuations, and blood pressure changes [8] [30]. Several data-driven approaches can help isolate the NVC-related signal:

• Filtering: Applying band-pass filters can remove high-frequency cardiac noise and very low-frequency drift.
• Advanced Decomposition: Techniques like Principal Component Analysis (PCA) can be used to identify and remove components of the signal that correlate with systemic physiology [29].
• Short-Separation Channels: Incorporating fNIRS channels with a very short source-detector distance (e.g., < 1 cm) is a highly effective method. These channels are predominantly sensitive to systemic artifacts in the scalp and not to cerebral hemodynamics. Their signals can be used as regressors to clean the standard, long-separation channels [8]. Note that this powerful technique remains underutilized.

Q4: What is the optimal sensor placement strategy to avoid interference between EEG electrodes and fNIRS optodes?

The goal is to achieve co-registration without physical or signal interference.

• Integrated Caps: The most straightforward solution is to use a high-density EEG cap that has pre-defined, fNIRS-compatible openings or holder rings, allowing for interleaved placement [28] [26].
• Interleaving Pattern: A standard method is to "put the EEG electrodes in between the optodes" since EEG electrodes are typically smaller [28].
• Material Considerations: The cap material should be dark to prevent ambient light from contaminating the fNIRS signal [28]. It is also vital to ensure that the combined weight of the sensors is lightweight to reduce the risk of movement artifacts and ensure subject comfort.

Q5: Why might we observe a decoupling between EEG and fNIRS signals, and what does it signify?

Observing a decoupling—where the expected correlation between electrical and hemodynamic activity breaks down—is not always an artifact; it can be a significant physiological finding. For example, a study on cognitive-motor interference found that divided attention during a dual-task led to a decreased neurovascular coupling across theta, alpha, and beta EEG rhythms [31]. Furthermore, research on retired athletes with a history of mild traumatic brain injury (mTBI) showed a reduced hemodynamic response compared to controls, suggesting altered cerebral metabolic demands and potentially impaired NVC due to past injuries [25]. Before concluding a physiological decoupling, however, technical causes like those addressed in FAQs Q1-Q3 must be rigorously excluded.

Table 1: Troubleshooting Common Artifacts in EEG-fNIRS Fusion Studies

Artifact Type	Primary Affected Modality	Root Cause	Preventive Solutions	Corrective Processing Methods
Motion Artifacts	Both (EEG more susceptible) [26]	Head movement, loose cap fit	Secure, lightweight cap; accelerometer use [28] [29]	Adaptive filtering, motion correction algorithms [8] [29]
Systemic Physiology	fNIRS [8]	Cardiac, respiratory, blood pressure cycles	Controlled environment, subject relaxation	Filtering, PCA/ICA, short-separation regression [8] [29]
Scalp Hemodynamics	fNIRS	Blood flow changes in skin/scalp	Proper optode pressure & coupling	Use of short-separation channels [8]
Ocular/Muscle Artifacts	EEG [8]	Eye blinks (EOG), head/neck muscle (EMG)	Instruct subject to minimize movement	Blind source separation (e.g., ICA), regression [8]
Synchronization Errors	Data Fusion	Separate device clocks, software lag	Hardware TTL triggers, Lab Streaming Layer (LSL) [27] [28]	Post-hoc alignment using event markers

Essential Experimental Protocols for NVC Investigation

Protocol 1: The Cognitive-Motor Interference (CMI) Task

This protocol is designed to study how the brain allocates resources when cognitive and motor tasks are performed simultaneously, a paradigm known to modulate NVC [31].

• Objective: To investigate the neurovascular correlates of divided attention and cognitive-motor interference.
• Task Design:
  • Single Motor Task: Participants perform an isolated upper limb motor task.
  • Single Cognitive Task: Participants perform an isolated cognitive task.
  • Cognitive-Motor Dual Task: Participants perform both the motor and cognitive tasks simultaneously [31].
• Measured Signals: Simultaneous recording of EEG and fNIRS bimodal signals.
• Key NVC Analysis: The correlation between task-related EEG components (in theta, alpha, and beta rhythms) and the concurrent fNIRS hemodynamic responses is computed. A decrease in this correlation during the dual-task condition indicates CMI-induced decoupling of neurovascular activity [31].

Protocol 2: The "Where's Wally" Neurovascular Coupling Test

This protocol is a validated method for assessing the integrity of the NVC response itself and has been used to study populations with suspected NVC impairment, such as those with a history of concussion [25].

• Objective: To elicit and measure a standardized hemodynamic response to a visual cognitive task.
• Task Design:
  • Baseline: The participant sits quietly for 5 minutes, breathing normally with eyes open.
  • Stimulation: The participant performs five cycles of a visual search task. Each cycle consists of 20 seconds with eyes closed, followed by 40 seconds with eyes open searching for the character "Wally" (or "Waldo") in a complex image. If found quickly, the image is advanced [25].
• Measured Signals: fNIRS over the prefrontal cortex (covering dorsolateral and orbitofrontal cortices) is essential. Simultaneous EEG can add valuable electrical correlates.
• Key NVC Analysis: In healthy controls, the task should induce a relative increase in O2Hb and a decrease in HHb in the prefrontal cortex. A blunted or altered response, such as a reduction in O2Hb increase, suggests impaired NVC [25].

Table 2: Core Experimental Protocols for NVC Research

Protocol Name	Primary Research Application	Task Paradigm	Key NVC Metrics	Typical Participant Groups
Cognitive-Motor Interference (CMI) [31]	Divided attention, dual-task cost	Sequential single and dual tasks	EEG-fNIRS correlation in theta, alpha, beta bands	Healthy young adults, elderly, clinical populations with attention deficits
"Where's Wally" NVC Test [25]	NVC integrity, metabolic demand	Repeated cycles of visual search/rest	Prefrontal O2Hb increase, HHb decrease	Populations with suspected NVC impairment (e.g., mTBI, concussion)
n-Back Working Memory	Cognitive workload, executive function	Continuous performance task with varying memory load	Prefrontal HbO amplitude, latency; EEG theta/gamma power	Broad cognitive neuroscience, neuroergonomics, clinical studies

The Data Preprocessing Pipeline for Robust NVC Analysis

A rigorous preprocessing pipeline is critical for cleaning the data and enabling a valid analysis of the relationship between EEG and fNIRS signals. The following workflow outlines the key steps for each modality before data fusion.

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 3: Essential Materials for Combined EEG-fNIRS Experiments

Item Name	Function/Application	Technical Specifications
Integrated EEG-fNIRS Cap	Holds sensors in co-registered positions for simultaneous measurement.	Compatible with 10-20 system; openings for EEG electrodes & fNIRS optodes; dark-colored material to block ambient light [28] [26].
Active EEG Electrodes	Measure electrical brain activity with high signal-to-noise ratio.	Active electrodes (e.g., g.SCARABEO) for reduced preparation time and motion resilience [28].
fNIRS Optodes	Emit near-infrared light and detect reflected light to measure hemodynamics.	Sources (LEDs/lasers) and detectors; typical source-detector separation of 20-30 mm for cerebral measurement [28] [29].
Short-Separation Channels	A specialized type of fNIRS channel for measuring and removing scalp hemodynamics.	Source-detector separation of < 1 cm; critical for robust artifact removal in data-driven analysis [8].
Conductive Electrolyte Gel	Ensures low impedance electrical contact for EEG electrodes.	Saline-based or abrasive gel for wet EEG systems; not required for dry electrodes.
Accelerometer	Records head movement to assist in motion artifact correction.	Small, lightweight sensor attached to the cap; provides reference signal for adaptive filtering [29].
Synchronization Hardware/Software	Temporally aligns EEG and fNIRS data streams from the start of recording.	TTL pulse generator, parallel port, or software platform like Lab Streaming Layer (LSL) [27] [28].

Advantages of Integrated fNIRS-EEG over Single-Modality Neuroimaging

Technical FAQs & Troubleshooting Guide

FAQ 1: What are the primary technical advantages of integrating fNIRS with EEG? The integration of fNIRS and EEG creates a synergistic system that overcomes the inherent limitations of each modality when used alone. Electroencephalography (EEG) records electrical activity from neuronal firing, providing excellent temporal resolution on the order of milliseconds, but suffers from relatively low spatial resolution and sensitivity to electrical noise and motion artifacts [32] [33]. Conversely, functional near-infrared spectroscopy (fNIRS) measures hemodynamic changes (changes in oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) concentrations), providing good spatial resolution and being less susceptible to motion artifacts, but has lower temporal resolution due to the slow nature of the hemodynamic response [32] [34]. By combining them, the dual-modal system provides simultaneous information on both the electrical neural activity and the hemodynamic metabolic response without electromagnetic interference, offering a more complete picture of brain function [32] [35].

FAQ 2: How do I address motion artifacts in my fNIRS-EEG data? Motion artifacts (MAs) are a common challenge that can significantly degrade signal quality. The table below summarizes various correction techniques.

Table 1: Motion Artifact Correction Techniques for fNIRS and EEG

Technique Category	Specific Methods	Description	Applicability
Algorithmic (fNIRS & EEG)	Wavelet Packet Decomposition (WPD) [36]	Decomposes signals using wavelet packets; effective for single-channel artifact correction.	fNIRS EEG
	WPD with Canonical Correlation Analysis (WPD-CCA) [36]	A two-stage method; shown to improve ΔSNR by 11.28% (EEG) and 56.82% (fNIRS) over WPD alone [36].	fNIRS EEG
Hardware-Based (fNIRS)	Accelerometer-based Active Noise Cancelation (ANC) [37]	Uses accelerometer data as a noise reference for an adaptive filter to clean the fNIRS signal.	fNIRS EEG
Data Handling	Channel Rejection [37]	Discarding data segments or entire channels that are heavily corrupted by motion artifacts.	fNIRS EEG

FAQ 3: What are the main strategies for fusing fNIRS and EEG data? Data fusion can be implemented at three primary levels, each with its own advantages.

Table 2: Data Fusion Strategies for fNIRS-EEG

Fusion Level	Description	Advantages	Examples
Data-Level Fusion	Direct combination of raw or preprocessed data from both modalities [33].	Potentially retains the most complete information.	-
Feature-Level Fusion	Extracting features from each modality (e.g., EEG band powers, fNIRS HbO/HbR slopes) and concatenating them into a combined feature vector [33] [38].	Often provides high classification accuracy; widely used and effective.	Combining EEG band powers and fNIRS signal peaks/means for BCI [38].
Decision-Level Fusion	Each modality is processed and classified independently, and the final results are combined (e.g., by voting or weighted averaging) [33].	Can eliminate redundant information; provides robustness.	Combining SVM classifier outputs for EEG and fNIRS for mental stress detection [33].

FAQ 4: What are the key design considerations for an fNIRS-EEG acquisition helmet? The helmet design is critical for signal quality and co-registration. Key considerations include:

• Probe Integration: EEG electrodes and fNIRS optodes can be integrated on a shared substrate or arranged separately. A common approach is to use a flexible EEG cap as a base and create punctures for fNIRS probe fixtures [32].
• Customization: Standard elastic caps can lead to inconsistent probe-scalp contact pressure due to head shape variations. 3D-printed custom helmets or those made from cryogenic thermoplastic sheets offer a better, customized fit, improving signal quality and spatial alignment, though at a higher cost or potential comfort trade-off [32].
• Co-registration: Precise spatial localization is essential. The arrangement should allow for co-registering fNIRS channels and EEG electrodes to standard brain atlas coordinates (e.g., Montreal Neurological Institute - MNI space) for accurate interpretation [32] [35].

FAQ 5: My synchronized data shows temporal misalignment. How can I improve synchronization? Precise synchronization is challenging. There are two primary methods:

• Separate Systems with Software Sync: Using separate commercial systems (e.g., NIRScout for fNIRS and BrainAMP for EEG) and synchronizing them via host computer software. This is simpler but may lack the microsecond precision needed for some EEG analyses [32].
• Unified Hardware Processor: Using a single, unified hardware processor to acquire and process both EEG and fNIRS signals simultaneously. This method is more complex but achieves highly precise synchronization, streamlining subsequent analysis [32]. Always check and report the synchronization error, which should ideally be less than 100 ms [35].

Essential Experimental Protocols & Methodologies

Protocol 1: Mental Arithmetic (MA) and Motor Imagery (MI) Task for BCI Classification

This is a common paradigm for testing hybrid BCI systems [33].

• Participants: Healthy subjects.
• Task Design:
  • MA Task: Subjects perform silent arithmetic operations (e.g., sequential subtraction) without vocalizing or moving [33].
  • MI Task: Subjects imagine performing a specific motor action (e.g., squeezing a ball with the right or left hand) without any actual movement [33].
• Data Acquisition: Simultaneously record EEG (e.g., 64-channel system) and fNIRS (e.g., system with multiple sources and detectors over the prefrontal and motor cortices) [33].
• Data Processing:
  • EEG: Preprocess (bandpass filter 1-40 Hz, notch filter), then extract features like band powers (δ, θ, α, β, γ) from specific channels [35] [33].
  • fNIRS: Convert raw light intensity to HbO and HbR concentrations, then extract features like mean, peak, or slope of the signals [33] [34].
• Fusion & Classification: Fuse EEG and fNIRS features at the feature-level (e.g., by simple concatenation) and feed into a classifier (e.g., Support Vector Machine - SVM, Linear Discriminant Analysis - LDA). This approach has achieved classification accuracies over 96% for discriminating between tasks [33].

Protocol 2: Neurovascular Coupling (NVC) Analysis in Substance Use Disorder

This protocol investigates the relationship between electrical and hemodynamic brain activity [35].

• Participants: Two groups: individuals with a specific substance use disorder (e.g., etomidate) and healthy controls [35].
• Task Design: Resting-state measurement with eyes closed for 5 minutes [35].
• Data Acquisition: Simultaneous high-density EEG and fNIRS recording, with precise co-registration of channels [35].
• Data Processing & Fusion:
  • EEG Source Localization: Reconstruct the source of EEG signals to match the locations of fNIRS channels [35].
  • Multi-Band Local Neurovascular Coupling (MBLNVC): Analyze the coupling between EEG rhythms (δ, θ, α, β, γ) and the fNIRS HbO signal at specific brain locations [35].
  • Network-Based Feature Fusion: Map the multi-modal features to well-established brain networks (e.g., Yeo 7 networks) to identify network-specific coupling alterations [35].

The following diagram illustrates a generalized workflow for a multimodal fNIRS-EEG data preprocessing pipeline, integrating the key steps from the protocols above.

The Scientist's Toolkit: Key Research Reagents & Materials

Table 3: Essential Materials and Software for fNIRS-EEG Research

Item Name	Type	Primary Function	Key Details
Multi-modal Acquisition Helmet	Hardware	Holds EEG electrodes and fNIRS optodes in stable, co-registered positions on the scalp.	Custom 3D-printed or thermoplastic designs are superior to standard caps for ensuring consistent probe contact [32].
Unified Data Acquisition System	Hardware	Simultaneously acquires and time-stamps EEG and fNIRS data streams.	Critical for minimizing synchronization error; preferred over loosely coupling two separate systems [32].
Accelerometer	Hardware	Records head movement data concurrently with brain signals.	Serves as a reference signal for hardware-based motion artifact correction algorithms in fNIRS [37].
NIRS Toolbox	Software	A MATLAB-based suite for fNIRS data analysis.	Supports GLM, functional connectivity, and multi-modal analysis; compatible with NIRx data for automatic 3D probe import [39].
Homer2 / Homer3	Software	Widely used fNIRS analysis packages.	Provide GUI and script-level processing streams; compatible with data in the common *.nirs file format [39].
EEGLAB	Software	A MATLAB toolbox for processing EEG data.	Used for standard preprocessing (filtering, ICA-based artifact removal) and rhythm extraction [35] [40].
Turbo-Satori	Software	Real-time fNIRS analysis software.	Optimized for brain-computer interface (BCI) and neurofeedback research; integrates with NIRx acquisition systems [39].

The logical relationships between the core components of an integrated fNIRS-EEG system and the synergistic advantages they create are summarized below.

Building and Applying fNIRS-EEG Preprocessing and Fusion Pipelines

Functional Near-Infrared Spectroscopy (fNIRS) is a non-invasive neuroimaging technique that measures changes in cerebral blood oxygenation by shining near-infrared light through the skull and into the brain tissue. This light is absorbed differently by oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR), allowing researchers to infer changes in blood flow and oxygenation in specific brain regions. Preprocessing raw fNIRS data is crucial for cleaning the signal and ensuring that subsequent analyses accurately reflect underlying neural activity rather than physiological noise or motion artifacts. This guide outlines standardized steps for converting raw light intensity signals to meaningful HbO and HbR concentrations, with particular attention to the context of multimodal fNIRS-EEG fusion research.

Frequently Asked Questions (FAQs)

Q1: What is the purpose of converting raw light intensity to optical density? The initial raw light intensity measurements are influenced by various factors unrelated to brain activity, including instrument properties and ambient light. Converting to optical density (OD) standardizes the signal and provides a more stable baseline for subsequent calculations. The conversion uses the formula: OD = -log10(I/I0), where I is the detected light intensity and I0 is the emitted light intensity [41]. This step is a prerequisite for applying the Modified Beer-Lambert Law.

Q2: Why must I apply a bandpass filter to my fNIRS data? Bandpass filtering is essential for isolating the hemodynamic response related to neural activity by removing unwanted physiological noise. The useful fNIRS signal is typically confined to frequencies below 0.5 Hz, while common physiological noises occur at higher frequencies: heartbeat (~1-2 Hz), respiration (~0.4 Hz), and Mayer waves related to blood pressure (~0.1 Hz) [41]. A typical bandpass filter with cutoffs of 0.01 Hz (or 0.05 Hz) to 0.5 Hz effectively removes these noise components while preserving the signal of interest [22] [41].

Q3: What are motion artifacts, and how can I correct for them? Motion artifacts are sudden, large shifts in the signal caused by head movements, muscle contractions, or other physical activities during recording [41]. They are a major source of noise and can obscure the true hemodynamic response. Several correction methods are available, and the choice depends on your data and software. Common algorithms include:

• CBSI (Correlation-Based Signal Improvement): A method that leverages the negative correlation between HbO and HbR signals to correct for motion [41].
• PCA (Principal Component Analysis): Identifies and removes components of the signal that correlate with motion [41].
• Spline Correction: Models and interpolates over motion-corrupted segments [41].
• Wavelet-Based Correction: Identifies and removes artifacts in the wavelet domain [41]. It is recommended to visualize your data before and after applying any motion correction method to ensure its effectiveness.

Q4: How does the Modified Beer-Lambert Law work? The Modified Beer-Lambert Law (MBLL) relates changes in optical density to changes in the concentration of chromophores (HbO and HbR) in the tissue. It modifies the classic law to account for light scattering in biological tissues. The formula for the change in optical density at a given wavelength is: ΔOD(λ) = α(λ) * Δc * l * DPF(λ) Where:

• ΔOD(λ) is the change in optical density at wavelength λ.
• α(λ) is the molar extinction coefficient of the chromophore at wavelength λ.
• Δc is the change in chromophore concentration.
• l is the source-detector separation (physical distance).
• DPF(λ) is the Differential Pathlength Factor, which accounts for the increased path length due to scattering [42] [41].

Q5: Why are short source-detector channels important? Short-distance channels (typically with a separation of less than 1 cm) are primarily sensitive to physiological noise in the scalp and skull rather than brain activity [22]. By measuring this superficial noise, they provide a regressor that can be used to remove it from the standard channels (which contain a mixture of brain signal and superficial noise), thereby enhancing the brain-specific signal [43]. This is a key step in improving the quality of fNIRS data.

Q6: What specific considerations exist for fNIRS-EEG fusion? Successful fusion of fNIRS and EEG data requires careful preprocessing to account for the different nature of the signals. fNIRS measures slow hemodynamic changes (requiring high-pass filtering around 0.01-0.05 Hz), while EEG measures fast electrical potentials (often filtered between 0.5-70 Hz) [17] [24]. A major challenge is that many studies incorporate robust artifact handling for EEG, but confounder correction in fNIRS remains limited to basic filtering or motion removal [17]. Furthermore, short-separation measurements for fNIRS are still underutilized in fusion studies [17]. Fusion methods themselves can be categorized as:

• Early Fusion: Combining raw data or low-level features before analysis.
• Late Fusion: Integrating the outputs or decisions of separate analyses [24]. Advanced methods like cross-modal attention mechanisms are also being developed to dynamically weight the importance of each modality [24].

Standardized Preprocessing Workflow

The following diagram illustrates the complete, standardized workflow for preprocessing fNIRS data, from raw measurements to analysis-ready hemoglobin concentrations.

Step-by-Step Protocol

• Raw Data Input: Begin with raw light intensity data for (typically) two wavelengths (e.g., 730 nm and 850 nm) [29].
• Convert to Optical Density (OD): Transform the raw intensity to optical density to create a stable baseline for analysis using the formula: OD = -log10(I/I0) [22] [41].
• Signal Cleaning: This critical step involves several sub-procedures to remove noise.
  • Short Channel Regression: Use signals from short source-detector pairs (<1 cm) to regress out superficial, non-cerebral physiological noise [22] [43].
  • Motion Artifact Correction: Apply algorithms like CBSI, PCA, Spline, or Wavelet to identify and correct for motion-induced signal shifts [41].
  • Bandpass Filtering: Use a zero-phase bandpass filter (e.g., 0.05 - 0.7 Hz or 0.01 - 0.5 Hz) to remove high-frequency noise (heartbeat, respiration) and low-frequency drift [22] [41].
• Convert to Hemoglobin: Apply the Modified Beer-Lambert Law to the cleaned OD data to calculate relative concentration changes of HbO and HbR [22] [42].
• Epoch and Analyze: Segment the continuous HbO/HbR data into epochs time-locked to experimental events for statistical analysis and visualization [22].

Technical Specifications & Parameters

Key Parameters for the Modified Beer-Lambert Law

Table 1: Parameters and typical values for converting optical density to hemoglobin concentration.

Parameter	Symbol	Description	Typical Value / Formula
Molar Extinction Coefficient	ε	Wavelength-specific absorption property of HbO and HbR.	Look-up tables (e.g., for 730 nm & 850 nm) [42].
Source-Detector Separation	l	Physical distance between light source and detector on the scalp.	2.5 - 3.0 cm [29].
Differential Pathlength Factor	DPF	Factor correcting for increased photon pathlength due to scattering.	Wavelength- and age-dependent (e.g., ~6 for adults) [42] [41].
Partial Pathlength Factor	PPF	Combined factor (DPF * PVF) accounting for scattering and the fraction of path in brain tissue.	Used in some advanced models [41].

Physiological Noise Characteristics

Table 2: Frequency ranges of common physiological noise sources in fNIRS signals.

Noise Source	Frequency Range	Notes
Heartbeat	~1 - 2 Hz	Can be suppressed by low-pass filtering [41].
Respiration	~0.4 Hz	Can be suppressed by low-pass filtering [41].
Mayer Waves (Blood Pressure)	~0.1 Hz	Very close to the signal of interest; may require careful filtering or source separation [41].
Hemodynamic Response	< 0.1 Hz	The target signal for functional brain activation studies [41].

Table 3: A selection of key software tools for fNIRS data preprocessing and analysis.

Tool Name	Primary Function	Key Feature	URL/Location
HOMER2 / HOMER3	Comprehensive fNIRS analysis	GUI and scripting; extensive processing stream including MBLL, motion correction, and filtering.	homer-fnirs.org [44]
MNE-Python	Multimodal neuroimaging (EEG/MEG/fNIRS)	Python-based; integrates fNIRS preprocessing with EEG analysis, ideal for fusion research.	mne.tools [22] [44]
NIRSLab	Complete fNIRS data analysis	Modules for registration, preprocessing, 3D projection, and GLM analysis.	nirs-lab.com [44]
NIRS-SPM	Statistical parametric mapping	SPM-based toolbox for statistical analysis of fNIRS signals.	bisp.kaist.ac.kr [44]
fnirsSOFT (BIOPAC)	Process, analyze, and visualize fNIRS	Stand-alone software with a graphical user interface.	nirx.net/fnirssoft [44]

How do I establish a robust EEG re-referencing procedure for my preprocessing pipeline?

A statistically robust re-referencing procedure is crucial for mitigating the effect of reference electrode activity, which can contaminate all EEG channels. The common average reference (CAR) is widely used but can be biased by neural activity present at the reference site. A robust maximum-likelihood type estimator can be adapted to mitigate this issue.

Methodology for Robust Re-referencing:

• Model the observed voltage dt,k at channel k and time t as: dt,k = st,k - rt + nt,k, where st,k is the ideal silent-reference signal, rt is the unknown reference voltage, and nt,k is sensor noise [45].
• Apply a robust estimator, such as the median or a trimmed mean, to the data from all channels at each time point to calculate the reference estimate řt. This approach reduces the influence of channels with high-amplitude neural activity, which act as outliers in the reference estimation [45].
• Add the estimated reference back to each channel to obtain the re-referenced signal: št,k = mt,k + řt [45].

This procedure is simple, fast, and avoids the substantial bias that can occur with traditional methods like CAR, especially when working with low-density EEG setups [45].

What are the standard parameters for filtering continuous EEG data?

Filtering is essential for removing unwanted biological and line noise artifacts from the EEG signal. The table below summarizes standard parameters for a basic preprocessing pipeline, with examples from recent research.

Table 1: Standard EEG Filtering Parameters and Applications

Filter Type	Standard Frequency Bands	Purpose	Example from Literature	Key Considerations
High-Pass Filter	≥ 0.5 Hz or 1 Hz [46] [47]	Removes slow drifts and DC offset; improves ICA decomposition quality [46].	A 1 Hz high-pass filter is recommended before ICA [46].	Overly aggressive high-pass filtering (e.g., >0.5 Hz) can distort ERPs [47].
Low-Pass Filter	≤ 30 Hz to 40 Hz [35] [48]	Attenuates high-frequency muscle noise and other high-frequency artifacts.	A 40 Hz low-pass filter was used in an etomidate study to focus on classic EEG rhythms [35].	The cutoff should be above the highest frequency of interest for your analysis.
Band-Stop (Notch) Filter	50 Hz or 60 Hz (region-dependent)	Removes mains line noise.	A 50 Hz notch filter was applied in a visual evoked potential study [48].	As an alternative, consider adaptive methods like the CleanLine plugin for line noise removal [46].
Band-Pass for Rhythms	δ (1-3 Hz), θ (3-8 Hz), α (8-13 Hz), β (13-30 Hz), γ (30-40 Hz) [35]	Isolates specific neural oscillatory rhythms for analysis.	Rhythms were extracted using band-pass filters in a neurovascular coupling study [35].	Use basic FIR filters and avoid causal filters if phase preservation is critical [35] [46].

Experimental Protocol: A typical filtering sequence for continuous data, as implemented in EEGLAB, involves:

• High-pass filter at 1 Hz using a basic FIR filter [46].
• Low-pass filter at 40 Hz in a separate step to avoid unnecessarily steep filter slopes [46].
• Notch filter at 50 Hz (or 60 Hz) [48]. It is recommended to filter continuous data before epoching to minimize artifacts at epoch boundaries [46].

What steps should I take to ensure proper epoch extraction for Event-Related Potentials (ERPs)?

Epoch extraction involves segmenting the continuous EEG signal into time-locked windows around events of interest. The key is to balance sufficient baseline and post-stimulus periods while managing data dimensionality.

Detailed Methodology for Epoch Extraction:

• Define Epoch Boundaries: A typical epoch might span from -200 ms before the event to +800 ms after, but this depends on the cognitive component under study. The baseline period (pre-event) is used for baseline correction [49] [47].
• Perform Baseline Correction: This process removes the DC offset from each epoch by subtracting the average amplitude of the baseline period. For example, a baseline window of -190 ms to -10 ms might be used [47]. Note: If a high-pass filter (e.g., 0.5 Hz) has been applied, additional baseline correction may be redundant or even distort the signal [47].
• Address Dimensionality: ERP data can be conceptualized as a hypercube with dimensions: sensors × time × conditions × subjects × trials. No single plot can visualize all dimensions, so researchers must strategically slice or average across dimensions [49]. Common practices include:
  • Plotting a single channel or region of interest (ROI) across time for one condition, averaging over subjects and trials.
  • Creating a butterfly plot to show all channels for a single condition and time window.
  • Using a topoplot to visualize spatial voltage distribution across the scalp at a specific time point [49].

Table 2: Common ERP Visualization Types and Their Uses

Plot Type	Dimensions Visualized	Best Use Case	Common Tools / Functions
ERP Plot	Time (sliced), Condition (sliced)	Showing the amplitude time-course of a specific component at a selected channel.	Standard in all ERP toolboxes.
Butterfly Plot	Sensors (all), Time (sliced)	Overview of all channel activities for a single condition; identifying widespread artifacts.	epochs.plot() in MNE-Python.
Topoplot	Sensors (all, spatial), Time (sliced)	Visualizing the spatial distribution of voltage at a specific latency.	topoplot in EEGLAB, plot_topomap in MNE.
Channel Image	Time, Sensors	Depicting trial-by-trial and channel-by-channel activity as a heatmap; useful for identifying consistent patterns.	Used in the LIMO toolbox [49].

How can I troubleshoot inconsistent results after applying my preprocessing pipeline?

Inconsistencies often arise from subtle differences in parameter choices, software tools, or the order of operations.

Troubleshooting Guide:

• Problem: Inconsistent PSDs or peaks after filtering.

  • Potential Cause: Different software libraries (e.g., PyLSL, BrainFlow vs. native hardware software) may apply filters or handle data scaling differently, even with identical nominal parameters [48].
  • Solution: Verify the filter functions and all input parameters (e.g., filter type, order, roll-off) are identical across software. Check for built-in preprocessing in the hardware's native software that isn't replicated in your custom script [48].

• Problem: Poor baseline in epoched data after high-pass filtering.

  • Potential Cause: The high-pass filter may not have been applied effectively, or an additional baseline correction (DC offset removal) might be causing a conflict [47].
  • Solution: Visually inspect the power spectrum before and after filtering to confirm the filter worked. If a high-pass filter (≥0.5 Hz) was successfully applied, avoid applying a separate baseline correction, as the filter should have already centered the data [47].

• Problem: Low classification accuracy or signal quality after artifact removal.

  • Potential Cause: Overly aggressive artifact rejection can remove valuable neural signal along with the artifact, biasing your final dataset [50].
  • Solution: Assess the impact of artifact removal on your final results (e.g., classification accuracy or correlation with an external variable). Compare results with and without stringent artifact rejection. It may be better to use a more conservative threshold or a method like ICA to preserve more of the true neural signal [50].

What essential materials and reagents are required for a synchronous fNIRS-EEG experiment?

Integrating fNIRS and EEG requires specific hardware and software to enable synchronous data acquisition and analysis.

Table 3: Research Reagent Solutions for fNIRS-EEG Fusion

Item	Function / Description	Example from Literature
EEG Acquisition System	Records electrical activity from the scalp with high temporal resolution.	64-channel EEG system (e.g., NeuSen W) with sampling frequency ≥ 250 Hz [35].
fNIRS Acquisition System	Measures hemodynamic changes by detecting near-infrared light attenuation, providing spatial resolution.	System with multiple emitters and detectors (e.g., 23 emitters, 16 detectors) using dual-wavelength lasers (730 & 850 nm) [35].
Integrated Probe Cap	A custom helmet or cap that holds both EEG electrodes and fNIRS optodes in a co-registered spatial arrangement.	Flexible EEG cap with punctures for fNIRS fixtures; 3D-printed custom helmet for better fit and stability [32].
Synchronization Hardware/Software	Ensures precise temporal alignment of EEG and fNIRS data streams.	Unified processor for simultaneous acquisition; or synchronization of separate systems via host computer [32].
Software for Multimodal Analysis	Tools for preprocessing, feature fusion, and joint analysis of the two data modalities.	Custom scripts in MATLAB/Python; toolboxes like EEGLAB for EEG and Homer2 for fNIRS [35] [32].

Figure 1: A standardized preprocessing pipeline for EEG data within an fNIRS-EEG fusion framework. Dashed lines indicate the integration point for synchronized fNIRS data prior to multimodal analysis.

In functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) fusion research, artifacts represent non-neural signals that can significantly corrupt data quality and interpretation. These unwanted signals originate from multiple sources: motion artifacts from subject movement, physiological artifacts from cardiac pulsation, respiration, and blood pressure changes, and environmental artifacts from instrumental and external interference [12] [8]. Effective artifact handling is particularly crucial in fNIRS-EEG studies because both modalities are susceptible to different artifact types with distinct characteristics, and the fusion process can amplify artifacts if not properly addressed [8] [51]. The portability of fNIRS and EEG systems enables brain monitoring in naturalistic settings, but this advantage comes with increased vulnerability to artifacts, making robust preprocessing pipelines essential for data validity [8] [52].

Troubleshooting Guides

Motion Artifacts

Q: What are the primary causes and characteristics of motion artifacts?

Motion artifacts (MAs) arise from imperfect contact between sensors and the scalp during subject movement. In fNIRS, optode displacement causes sudden, high-amplitude signal shifts due to changes in light coupling efficiency [37]. Specific movements causing MAs include head movements (nodding, shaking, tilting), facial muscle movements (raising eyebrows), body movements (limb movements causing head motion), and jaw movements (talking, eating) [37]. In EEG, motion creates similar signal disruptions but manifests as electrical potential changes from electrode movement relative to skin [52] [36]. Motion artifacts typically exhibit higher amplitude and different frequency characteristics compared to the underlying neural signals in both modalities.

Q: What methods effectively correct for motion artifacts?

Multiple algorithmic approaches exist for motion artifact correction, each with distinct advantages:

Table 1: Motion Artifact Correction Methods for fNIRS and EEG

Method	Modality	Principle	Performance Metrics	Limitations
Wavelet Packet Decomposition (WPD)	fNIRS & EEG	Signal decomposition into wavelet packets with artifact component identification and removal [36]	ΔSNR: 29.44 dB (EEG), 16.11 dB (fNIRS); η: 53.48% (EEG), 26.40% (fNIRS) [36]	Wavelet selection affects performance
WPD with Canonical Correlation Analysis (WPD-CCA)	fNIRS & EEG	Two-stage approach: WPD followed by CCA for enhanced artifact separation [36]	ΔSNR: 30.76 dB (EEG), 16.55 dB (fNIRS); η: 59.51% (EEG), 41.40% (fNIRS) [36]	Increased computational complexity
Accelerometer-Based Methods (ABAMAR/ABMARA)	fNIRS	Use accelerometer data as noise reference for adaptive filtering [37]	Enables real-time rejection; improves classification accuracy [37]	Requires additional hardware; placement affects performance
Moving Average & Spline Interpolation	fNIRS	Identifies artifact periods and interpolates using clean data segments [37]	Simple implementation; effective for isolated artifacts [37]	Can distort signal morphology near artifacts

The following workflow illustrates a systematic approach to motion artifact management:

Motion Artifact Management Workflow

Physiological Artifacts

Q: What physiological processes cause artifacts and how do they differ between modalities?

Physiological artifacts originate from various bodily functions with distinct manifestations in fNIRS and EEG:

Table 2: Physiological Artifacts in fNIRS and EEG

Source	fNIRS Manifestation	EEG Manifestation	Frequency Characteristics
Cardiac Pulsation	Low-frequency oscillations from blood volume changes	Electrical spikes from heart muscle activity (ECG) [8]	fNIRS: ~1-1.5 Hz; EEG: ~1-1.5 Hz with sharper peaks
Respiration	Slow oscillations from blood pressure and volume changes	Minimal direct impact	fNIRS: ~0.2-0.3 Hz; EEG: Less prominent
Mayer Waves	Very low-frequency oscillations from blood pressure regulation	Not typically detectable	fNIRS: ~0.1 Hz; EEG: Not applicable
Blood Pressure Changes	Systemic hemodynamic fluctuations	Not typically detectable	fNIRS: <0.1 Hz; EEG: Not applicable

The fundamental difference lies in how these physiological processes affect each modality: fNIRS captures the hemodynamic consequences (blood volume/oxygenation changes), while EEG records the bioelectrical activity directly [8]. This distinction is crucial for designing effective artifact removal strategies.

Q: What filtering approaches effectively remove physiological artifacts?

Frequency-domain filtering is the primary approach for physiological artifact removal:

• High-pass filters effectively remove very low-frequency drifts (<0.01 Hz) in fNIRS caused by slow blood pressure changes [12]
• Low-pass filters remove high-frequency cardiac pulsation in fNIRS (typically >0.5-1 Hz cutoff) [12]
• Bandpass filters (0.01-0.5 Hz) are commonly used in fNIRS to preserve the hemodynamic response while removing cardiac and respiratory noise [12]
• Notch filters (45-55 Hz or 55-65 Hz depending on region) effectively remove power line interference in EEG [52]

For optimal results, filter selection should consider both the artifact characteristics and the target neural signal properties. Finite impulse response (FIR) filters are generally preferred over infinite impulse response (IIR) filters for neurophysiological data due to their linear phase characteristics and stability [12].

Environmental and Instrumental Artifacts

Q: What environmental factors most commonly corrupt fNIRS-EEG signals?

Environmental artifacts stem from external sources rather than subject physiology:

• Power line interference: 50/60 Hz noise and harmonics in EEG signals [52]
• Instrumental noise: Electronic noise from amplifiers, analog-to-digital converters, and optical components [8] [52]
• Ambient light leakage: Contamination of fNIRS signals by external light sources [37]
• Electromagnetic interference: Noise from nearby electronic equipment affecting both modalities [52]

These artifacts are particularly problematic in naturalistic settings where laboratory-level environmental control is impossible [8] [52]. The move toward wearable fNIRS and EEG systems for ecological monitoring has amplified the significance of environmental artifact management.

Q: What strategies minimize environmental and instrumental artifacts?

A multi-layered approach is most effective for environmental artifact management:

• Shielding and grounding: Proper electrical shielding of cables and equipment, secure grounding of all systems [52]
• Environmental control: Conduct experiments in electrically shielded rooms when possible, minimize ambient light [37]
• Hardware solutions: Use optical filters for fNIRS, high-quality amplifiers with high common-mode rejection for EEG [52]
• Signal processing approaches:
  • Adaptive filtering for periodic interference
  • Blind source separation methods (ICA, PCA) for instrumental noise [52]
  • Short-separation channels in fNIRS to regress out superficial contaminants [8]

Experimental Protocols for Artifact Handling

Protocol: Motion Artifact Correction Using WPD-CCA

Purpose: To effectively remove motion artifacts from single-channel fNIRS and EEG signals using the WPD-CCA method [36].

Materials: Raw fNIRS/EEG data with motion artifacts, signal processing software (MATLAB, Python), WPD-CCA implementation.

Procedure:

• Signal Decomposition: Apply wavelet packet decomposition to the contaminated signal using an appropriate wavelet family (db1 for EEG, fk4 for fNIRS recommended) [36]
• Threshold Identification: Calculate appropriate thresholds for each wavelet packet coefficient based on statistical properties of the signal
• Denoising: Apply thresholding to wavelet coefficients to remove artifact components while preserving neural signals
• CCA Processing: Perform canonical correlation analysis on the denoised components to identify and remove residual artifact sources
• Signal Reconstruction: Reconstruct the cleaned signal from the processed wavelet coefficients
• Validation: Calculate performance metrics (ΔSNR, η) to quantify artifact reduction [36]

Expected Outcomes: Significant improvement in signal-to-noise ratio (average ΔSNR: 30.76 dB for EEG, 16.55 dB for fNIRS) and motion artifact reduction (average η: 59.51% for EEG, 41.40% for fNIRS) [36].

Protocol: Physiological Noise Removal Using Filtering Techniques

Purpose: To remove physiological noise (cardiac, respiratory, Mayer waves) from fNIRS signals using frequency-domain filtering [12].

Materials: Raw fNIRS data, digital filter implementation (FIR/IIR), spectral analysis tool.

Procedure:

• Spectral Analysis: Perform Fourier analysis of raw fNIRS signals to identify noise frequency components
• Filter Design:
  • For cardiac noise: Design low-pass filter with cutoff frequency 1.5-2 Hz [12]
  • For respiratory/Mayer waves: Design bandpass filter 0.01-0.5 Hz to preserve hemodynamic response
  • Select FIR filter type for linear phase response
• Filter Application: Apply designed filters to raw fNIRS signals
• Validation: Verify preservation of hemodynamic response function while removing physiological noise components

Expected Outcomes: Cleaned fNIRS signals with preserved task-related hemodynamic responses and significantly reduced physiological noise components.

The Scientist's Toolkit: Essential Research Reagents and Materials

Table 3: Essential Materials for fNIRS-EEG Artifact Handling Research

Item	Function	Application Notes
Accelerometers	Motion detection and reference signal for artifact correction [37]	Place near optodes/electrodes; sample rate ≥100 Hz
Inertial Measurement Units (IMUs)	Multi-axis motion detection (acceleration, rotation) [37]	Provides comprehensive movement data for motion artifact correction
Short-Separation fNIRS Detectors	Measures superficial signals for global noise regression [8]	Place 0.8-1.5 cm from source; critical for separating cerebral and extracerebral signals
Electrode/Gel Compatibility Testing Kit	Ensures optimal electrode-skin interface for EEG [52]	Reduces impedance-related artifacts; essential for wearable EEG
Optode Stabilization Systems	Minimizes optode movement relative to scalp [37]	Headbands, custom caps; critical for motion-prone paradigms
Optical Shielding Materials	Prevents ambient light contamination in fNIRS [37]	Black cloth, opaque caps; essential for valid fNIRS measurements
Reference Noise Recording Electrodes	Records environmental noise for adaptive filtering [52]	Place away from scalp; provides noise reference for advanced filtering

FAQ: Addressing Common Researcher Questions

Q: How do I determine whether to use wavelet-based methods versus filter-based approaches for artifact removal?

The choice depends on artifact characteristics and research objectives. Wavelet-based methods (WPD, WPD-CCA) are particularly effective for non-stationary, transient artifacts like motion artifacts, as they provide both time and frequency information [36]. Filter-based approaches are more suitable for periodic, stationary physiological artifacts with consistent frequency characteristics (cardiac, respiratory noise) [12]. For comprehensive artifact handling, a combined approach often yields best results: wavelet methods for motion artifacts followed by frequency filtering for physiological noise.

Q: What are the most critical metrics for evaluating artifact removal effectiveness?

The optimal metrics depend on data availability and artifact type:

• When clean reference is available: Signal-to-noise ratio improvement (ΔSNR) and percentage artifact reduction (η) provide direct performance measures [36]
• Without clean reference: Visual inspection, signal quality indices (SQI), and task-related functional sensitivity (e.g., activation effect size) serve as practical alternatives [37] [53]
• For comparative studies: Contrast-to-noise ratio (CNR) and classification accuracy in brain-computer interface applications validate functional preservation [37]

Q: Why do artifact handling methods sometimes produce different results across research teams?

Methodological variability in artifact handling stems from multiple factors: different parameter selections in algorithms, varying quality thresholds for data inclusion, diverse approaches to handling borderline cases, and researcher experience levels [53]. The FRESH initiative found that nearly 80% of research teams agreed on group-level results when hypotheses were strongly literature-supported, but individual-level analyses showed greater variability [53]. Teams with higher self-reported analysis confidence (correlated with fNIRS experience) demonstrated greater inter-team agreement, highlighting the importance of methodological expertise and standardized reporting.

Q: How can I optimize my artifact handling pipeline for real-time applications?

Real-time artifact handling requires balancing computational efficiency with effectiveness:

• For motion artifacts: Accelerometer-based methods (ABAMAR) or simplified WPD implementations enable online correction [37]
• For physiological noise: Fixed filter coefficients (rather than adaptive approaches) reduce computational load
• Implementation considerations: Pre-optimize algorithms, use efficient programming practices, and consider embedded system deployment for wearable applications [8] [52]
• Validation: Test pipeline timing with representative data before deployment

The following diagram illustrates the relationship between various artifacts and corresponding correction methods:

Artifact-Method Relationship Diagram

The integration of Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) has emerged as a powerful approach in brain-computer interface (BCI) and cognitive neuroscience research. EEG measures neuronal electrical activity with millisecond temporal resolution, while fNIRS measures hemodynamic responses with better spatial localization [8]. These complementary properties make them ideal for multimodal fusion, which can be strategically implemented at three primary stages: early, middle, and late fusion. Understanding these approaches is essential for designing effective data preprocessing pipelines in fNIRS-EEG research.

Early-stage fusion involves combining raw or minimally processed data from both modalities before feature extraction. This approach preserves the richest information but requires handling different temporal resolutions and dimensionalities [54] [9].

Middle-stage fusion (also called feature-level fusion) integrates extracted features from each modality before classification. This allows for specialized processing for each signal type while capturing cross-modal relationships [55] [56].

Late-stage fusion (decision-level fusion) processes each modality through separate pipelines and combines the results at the decision level. This provides robustness when modalities have different reliability patterns but may underutilize complementary information [24] [57].

Experimental Protocols and Implementation

Early Fusion Protocol

The Y-shaped neural network architecture provides a validated protocol for early fusion implementation [54]:

• Data Preprocessing: Downsample EEG to 128 Hz, apply band-pass filter (8-25 Hz), and select 8 electrodes around the sensorimotor cortex. Normalize fNIRS HbO/HbR signals and temporally align with EEG.

• Network Architecture: Implement independent encoders for each modality in the initial layers, then merge the streams for joint processing.

• Training Configuration: Use leave-one-out cross-validation with 29 participants performing left/right hand motor imagery tasks.

This approach achieved 76.21% classification accuracy, significantly outperforming middle and late fusion in the same study (N=57, p<0.05) [54].

Middle Fusion Protocol with AR-TSNET

For feature-level fusion in drug addiction detection [55]:

• Modality-Specific Processing: Use Tception module for EEG temporal features and Sception module for fNIRS spatial features.

• Attention Mechanisms: Incorporate separate attention modules to weight informative features and reduce redundancy.

• Feature Integration: Concatenate attended features from both modalities before the final classification layer.

This protocol achieved 92.6% accuracy in classifying healthy individuals versus those with drug addiction using six-fold cross-validation [55].

Late Fusion Protocol with Evidence Theory

For decision-level integration in motor imagery classification [57]:

• Independent Processing Paths:

  • EEG: Extract spatiotemporal features using dual-scale temporal convolution and depthwise separable convolution
  • fNIRS: Employ spatial convolution across channels and GRU for temporal dynamics

• Uncertainty Modeling: Quantify decisions using Dirichlet distribution parameter estimation

• Evidence Fusion: Apply Dempster-Shafer Theory for two-layer reasoning with basic belief assignment

This method achieved 83.26% accuracy on motor imagery tasks, a 3.78% improvement over previous benchmarks [57].

Performance Comparison Table

Table 1: Comparative Performance of Fusion Strategies Across Applications

Fusion Approach	Application Domain	Classification Accuracy	Key Advantages
Early Fusion [54]	Motor Imagery	76.21%	Maximizes complementary information utilization
Middle Fusion [55]	Drug Addiction Detection	92.6%	Balanced processing with attention mechanisms
Late Fusion [57]	Motor Imagery	83.26%	Robust to modality-specific noise and artifacts
DeepSyncNet (Early) [9]	Motor Imagery/Mental Arithmetic	Superior to late fusion	Effective cross-modal interaction
MBC-ATT (Late) [24]	Cognitive Task (n-back)	Competitive performance	Dynamic dependency modeling

Technical Support Center

Troubleshooting Guides

Issue: Poor Fusion Performance Despite High Single-Modality Accuracy

Symptoms: Combined model performs worse than individual modalities; validation loss fluctuates excessively.

Diagnosis: Temporal misalignment between EEG and fNIRS signals, or inadequate cross-modal feature interaction.

Solutions:

• Implement neurovascular coupling-informed alignment using EEG alpha/beta power correlation with HbO signals [58]
• For early fusion: Add cross-modal attention mechanisms like in DeepSyncNet [9]
• For middle fusion: Incorporate contrastive learning to align feature spaces as in EFRM [56]

Issue: Model Overfitting with Limited Labeled Data

Symptoms: High training accuracy with poor test performance; frequent oscillations in test loss.

Diagnosis: Insufficient regularization for high-dimensional multimodal features.

Solutions:

• Apply transfer learning with pre-trained representations (e.g., EFRM model) [56]
• Use dropout and L2 regularization as implemented in AR-TSNET [55]
• Incorporate self-supervised pre-training on unlabeled multimodal data

Issue: Handling Different Temporal resolutions

Symptoms: Information loss; failure to capture complementary timing relationships.

Diagnosis: Inadequate temporal alignment strategy between fast EEG and slow fNIRS responses.

Solutions:

• Implement sliding window approach (3s windows with 1s steps) as validated [54]
• Use fNIRS-guided temporal alignment and EEG-guided spatial alignment [56]
• For decision fusion, employ uncertainty modeling with Dempster-Shafer theory [57]

Frequently Asked Questions

Q: When should I choose early fusion over late fusion?

A: Early fusion is preferable when you have high-quality, temporally aligned data and want to maximize information exchange between modalities. It's particularly effective for learning complex cross-modal relationships, as demonstrated in motor imagery tasks where it outperformed other approaches [54] [9]. Late fusion is better when modalities have different reliability patterns or when computational efficiency is prioritized.

Q: How can I address the spatial mismatch between EEG and fNIRS?

A: Implement coordinate mapping to common reference space (e.g., MNI coordinates) as shown in Table 1 of the error-related brain state study [58]. For deep learning approaches, convert 1D signals to 3D tensors based on sensor positions to align spatial representations before fusion [9].

Q: What attention mechanisms work best for EEG-fNIRS fusion?

A: Cross-modal attention with residual integration effectively balances modality contributions [9]. Modality-guided attention selectively emphasizes relevant features based on task context [24]. For feature-level fusion, separate attention mechanisms for each modality (like in AR-TSNET) reduce redundant features [55].

Research Reagent Solutions

Table 2: Essential Materials and Computational Tools for EEG-fNIRS Fusion Research

Resource Type	Specific Tool/Method	Function/Purpose
Public Datasets	Shin et al. Dataset [54]	Motor imagery and mental arithmetic tasks with simultaneous EEG-fNIRS
Deep Learning Frameworks	Y-shaped Network Architecture [54]	Early fusion implementation with modality-specific encoders
Fusion Algorithms	Attentional Fusion (AF) [9]	Adaptive integration of EEG and fNIRS features
Pre-trained Models	EFRM [56]	Transfer learning for limited data scenarios
Signal Processing	Regularized Canonical Correlation Analysis [58]	Joint analysis of EEG band power and HbO changes

Methodological Workflows

Diagram 1: Early fusion workflow with raw data integration

Diagram 2: Late fusion with decision-level integration

The selection of appropriate fusion strategies fundamentally shapes the effectiveness of EEG-fNIRS integration in research pipelines. Early fusion maximizes information exchange but demands careful temporal alignment. Middle fusion balances specialized processing with cross-modal learning, while late fusion offers robustness against modality-specific artifacts. Current evidence suggests early fusion generally provides superior performance for motor imagery and cognitive tasks, though optimal implementation requires attention to neurovascular coupling principles and modality-specific characteristics. As multimodal research advances, emerging techniques like cross-modal attention and representation learning will further enhance our ability to leverage the complementary strengths of EEG and fNIRS.

Frequently Asked Questions (FAQs) & Troubleshooting Guides

This technical support center addresses common challenges in fNIRS-EEG fusion research, framed within a broader thesis on data preprocessing pipelines.

FAQ 1: What is the optimal stage to fuse EEG and fNIRS data: early, middle, or late fusion?

Answer: The choice of fusion stage involves a trade-off between model performance and computational complexity. Recent evidence suggests that for tasks like motor imagery, early-stage fusion can yield superior classification accuracy.

• Experimental Evidence: A study investigating a Y-shaped neural network for left/right hand motor imagery classification found that early-stage fusion of EEG and fNIRS significantly outperformed middle and late-stage fusion (N = 57, P < 0.05), achieving an average accuracy of 76.21% [59].
• Comparison of Fusion Strategies: The table below summarizes the core characteristics of each fusion stage.

Table 1: Comparison of EEG-fNIRS Fusion Strategies

Fusion Stage	Description	Key Advantage	Reported Challenge
Early Fusion	Combining raw or pre-processed signals before feature extraction [59].	Higher performance in some tasks (e.g., motor imagery) [59].	Requires careful handling of temporal and spatial misalignment [60].
Late Fusion	Combining decisions or high-level features from unimodal classifiers [24].	Leverages modality-specific expertise; more flexible.	May fail to exploit deep cross-modal correlations [24].
Cross-Modal Attention	A middle-stage fusion that dynamically weights features based on inter-modal relationships [24] [60].	Dynamically focuses on relevant signals and modalities; improves interpretability [24].	Increases model complexity and requires more data for training.

FAQ 2: How can I handle the significant differences in temporal resolution and latency between EEG and fNIRS signals?

Answer: This is a fundamental challenge, as EEG captures millisecond-level electrical activity while fNIRS measures hemodynamic responses over seconds. Advanced deep learning models are designed to explicitly address this.

• Troubleshooting Guide:
  • Recommended Approach: Use models with bidirectional cross-modal alignment. For instance, the TSMMF framework uses a Bidirectional Cross-Modal Transformer (BCMT) to project EEG and fNIRS features into a joint representation space, mitigating inter-modal differences [60].
  • Model-Based Solution: Other architectures, like STA-Net, introduce specialized layers for fNIRS-Guided Spatial Alignment and EEG-Guided Temporal Alignment to harmonize the spatiotemporal discrepancies [56].
  • Common Pitfall: Simply concatenating raw temporal data from both modalities without any alignment strategy often leads to poor model performance due to the inherent misalignment.

FAQ 3: Our hybrid fNIRS-EEG helmet has poor probe-scalp contact, leading to motion artifacts and data loss. What are the best hardware and preprocessing practices?

Answer: Poor probe contact is a major source of noise. Solutions span both hardware design and signal processing.

• Hardware & Setup Checklist:
  • Helmet Design: Standard elastic EEG caps can lead to inconsistent optode placement and pressure. Consider 3D-printed custom helmets or those made from cryogenic thermoplastic sheets for a more secure, subject-specific fit [32].
  • Optode Placement: Ensure source-detector distances are optimized (e.g., ~30mm for long fNIRS channels, ~8.5mm for short channels) and consistent across subjects [61].
• Preprocessing Protocol for Motion Artifacts:
  • Identify Bad Channels: Check fNIRS signal quality using metrics like signal-to-noise ratio and reject channels that do not meet a quality threshold [62].
  • Apply Motion Correction: Use algorithms like wavelet-based filtering or correlation-based methods to detect and correct motion artifacts [62] [61].
  • Implement Short-Channel Regression (SCR): This is a best practice for removing physiological noise from scalp layers. If physical short-separation detectors are unavailable, transformer-based deep learning models can now predict virtual short-channel signals from long-separation data for effective denoising [61].

FAQ 4: How can I validate that my fusion model is learning genuine neurovascular coupling and not just dataset-specific noise?

Answer: Ensuring model interpretability and biological plausibility is crucial for scientific validity.

• Methodology:
  • Incorporate Interpretability Tools: Use models with built-in attention mechanisms. The cross-attention weights in a Cross-Modal Transformer can be visualized to show which EEG and fNIRS channels are most influential for a decision, allowing you to check if they align with known brain regions (e.g., the inferior parietal lobe during motor tasks) [63] [60].
  • Leverage Multimodal Datasets: Train and test your models on publicly available, high-quality simultaneous fNIRS-EEG datasets. This provides a benchmark and helps ensure your model learns generalizable neural patterns rather than artifacts [24] [63] [59].
  • Synthetic Data Validation: As reviewed by Codina et al., using synthetic fNIRS-EEG datasets with a known ground truth is a powerful method for validating fusion algorithms and their ability to recover true underlying neural activity [17].

Experimental Protocols for Key Fusion Models

Protocol 1: Implementing a Cross-Modal Attention Fusion (MBC-ATT) Framework

This protocol is based on the Multimodal MBC-ATT model for cognitive state decoding [24].

• Dataset: Utilize a publicly available multimodal dataset (e.g., the dataset from Shin et al.) containing simultaneous EEG and fNIRS recordings during cognitive tasks like the n-back and Word Generation (WG) tasks [24].
• Preprocessing Pipeline:
  • EEG: Apply band-pass filtering (e.g., 0.5-40 Hz), re-reference to the common average, and remove EOG artifacts.
  • fNIRS: Convert raw light intensity to oxygenated (HbO) and deoxygenated (HbR) hemoglobin concentrations using the Modified Beer-Lambert Law. Apply a band-pass filter (e.g., 0.01-0.2 Hz) to remove physiological noise [62].
• Model Architecture & Workflow:

Diagram 1: MBC-ATT Fusion Workflow

• Key Steps:
  • Feature Extraction: Process EEG and fNIRS through separate convolutional network branches to extract high-level, modality-specific features.
  • Cross-Modal Attention: The attention mechanism computes dynamic weights to determine the importance of features from one modality for reinforcing or complementing the other.
  • Fusion & Classification: The attended features are combined and passed to a classifier (e.g., a fully connected layer) for final cognitive state decoding [24].

Protocol 2: A Bidirectional Cross-Modal Transformer for Affective Brain-Computer Interfaces

This protocol outlines the TSMMF model for cross-subject emotion recognition [60].

• Core Innovation: This model moves beyond simple feature concatenation by using a Bidirectional Cross-Modal Transformer (BCMT) to achieve deep temporal-spatial fusion.
• Experimental Workflow:

Diagram 2: BCMT Fusion Process

• Quantitative Performance: The model achieved state-of-the-art cross-subject emotion recognition accuracy of 76.15%, which was 6.06% and 12.44% higher than using EEG or fNIRS alone, respectively [60].
• Interpretability Analysis: The model's cross-attention mechanism allows researchers to identify which brain regions (via fNIRS) and electrical activity patterns (via EEG) were most critical for the emotion classification, providing neuroscientific insights [60].

The Scientist's Toolkit: Research Reagent Solutions

Table 2: Essential Components for an EEG-fNIRS Fusion Research Pipeline

Item / Reagent	Function & Explanation	Technical Specifications / Notes
Simultaneous fNIRS-EEG System	Core hardware for data acquisition. Integration is key to avoid synchronization issues.	Prefer systems with a unified processor for acquisition [32]. Ensure compatibility between EEG amplifiers and fNIRS hardware.
Custom Hybrid Helmet	Ensures stable and consistent placement of EEG electrodes and fNIRS optodes.	3D-printed or thermoplastic custom helmets are superior to modified elastic caps for probe stability [32].
Short-Separation fNIRS Channels	Critical "reagent" for signal preprocessing. Used to regress out scalp hemodynamics and improve brain signal specificity [61].	Optimal source-detector distance: ~8.4 mm for adults. If physically unavailable, a transformer-based virtual short-channel generator can be used [61].
Public Multimodal Datasets	Serves as a benchmark and training resource for developing and validating new algorithms.	Examples include the dataset by Shin et al. for motor imagery and mental arithmetic [59], and others used for n-back and word generation tasks [24].
Structured Sparse Multiset CCA (ssmCCA)	A data fusion analysis "reagent" used to identify latent variables that are maximally correlated across EEG and fNIRS modalities.	Useful for identifying brain regions consistently activated in both electrical and hemodynamic domains (e.g., the left inferior parietal lobe during motor tasks) [63].

Frequently Asked Questions (FAQs)

Q1: My MNE-Python installer fails with an error on macOS. What should I do? A common issue, particularly on macOS, is that the installer may show an error message, yet the installation can still be successful. After running the installer, verify the installation by opening a prompt and executing python -c "import mne; mne.sys_info()". This command will generate a report detailing the versions of MNE-Python and all optional dependencies. As long as this report shows no errors and lists mne as installed, your installation is likely functional despite the installer error [64]. If problems persist, try using the latest version of the installer.

Q2: How can I handle errors gracefully in an automated MNE-BIDS pipeline? When running automated processing with the MNE-BIDS-Pipeline, you can control its behavior upon encountering an error via the on_error configuration setting. This can be set to:

• 'abort': Stop processing immediately (default) [65].
• 'continue': Attempt to continue with other processing steps [65].
• 'debug': Drop into a debugger to investigate the error (note: this deactivates parallel processing) [65].

Q3: What is a typical fNIRS preprocessing workflow in MNE-Python? A standard pipeline to convert raw fNIRS data to analyzable haemoglobin concentrations involves several key stages [22] [66]:

• Read Data: Import data from various formats (e.g., NIRx, BIDS) into an MNE Raw object.
• Optical Density: Convert raw intensity signals to optical density (OD) using optical_density.
• Beer-Lambert Law: Convert OD to relative oxyhaemoglobin (HbO) and deoxyhaemoglobin (HbR) concentration changes with beer_lambert_law.
• Filtering: Apply a band-pass filter (e.g., 0.05 - 0.7 Hz) to remove high-frequency noise (like heart rate) and slow drifts [22].
• Epoching: Segment the continuous data into trials (epochs) time-locked to experimental events.

Q4: How does BrainFusion support multimodal data fusion, and how does it compare to MNE-Python? BrainFusion is a unified, low-code framework designed specifically to simplify the complexity of multimodal data analysis. It provides standardized data containers and automated pipelines for integrating EEG, fNIRS, EMG, and ECG signals. Its key advantage for multimodal research is its focus on cross-modal feature engineering, coupling analysis, and an application generator that allows you to export workflows as standalone executable tools [67]. While MNE-Python is a powerful script-based toolkit that can analyze EEG and fNIRS in a unified environment, BrainFusion aims to make these advanced analyses more accessible and deployable with less coding required [67].

Troubleshooting Guides

Issue 1: Poor Signal Quality or Excessive Noise in fNIRS Data

Problem: Processed fNIRS data contains unexpected noise, making it difficult to observe the haemodynamic response.

Solutions:

• Inspect and Remove Bad Channels: Before processing, visually inspect the raw data using raw_intensity.plot() and mark consistently poor-quality channels as "bad" [22] [66].
• Check Source-Detector Distances: Ensure you are using channels appropriate for detecting neural responses. Short-distance channels (typically less than 1 cm) are primarily sensitive to scalp blood flow and should be excluded from neural analysis. You can automatically pick channels with appropriate distances [22].
• Apply Filtering: A common cause of noise is cardiac signal interference (around 1 Hz). Apply a low-pass filter with a cutoff of 0.7 Hz to remove this. Also, use a high-pass filter (e.g., 0.05 Hz) to eliminate slow drifts [22].
• Evaluate Scalp Coupling: Use the Scalp Coupling Index (SCI) to quantify optode-scalp contact. While the example dataset was clean, consider marking channels with an SCI less than 0.5 as bad for noisier data [22].
• Reject Bad Epochs: During epoching, set rejection criteria to automatically discard trials with excessive motion artifacts or signal amplitude beyond a plausible physiological range (e.g., reject_criteria = dict(hbo=80e-6)) [22].

Issue 2: Errors During Epoching or Epochs Being Dropped

Problem: The mne.Epochs function throws an error, or the log shows that a large number of epochs were dropped.

Solutions:

• Validate Events and Annotations: Ensure the event timings and IDs extracted from the raw data align with your experimental design. You can visualize them with mne.viz.plot_events [66].
• Check Rejection Criteria: If many epochs are being dropped, your rejection threshold might be too strict. Review the reject parameter and adjust the tolerance (e.g., 80e-6 for HbO) based on your signal's amplitude [22].
• Inspect Annotations: Use reject_by_annotation=True to automatically exclude data segments you have manually marked as bad during raw data inspection [22] [68].
• Verify Time Range: Ensure the epoch time window (tmin, tmax) is defined correctly and does not extend beyond the available data, especially for trials near the start or end of the recording.

Issue 3: Challenges Integrating fNIRS with EEG

Problem: Difficulty in synchronizing, aligning, and jointly analyzing fNIRS and EEG data streams.

Solutions:

• Use a Standardized Framework: Consider using a tool like BrainFusion, which is explicitly designed for this purpose. It provides a standardized data container for heterogeneous signals and includes built-in strategies for time-point or event-based alignment of multimodal data [67].
• Leverage BIDS Format: Using the Brain Imaging Data Structure (BIDS) format for your data ensures all necessary metadata is available and simplifies the synchronization of different modalities, as both MNE-BIDS and BrainFusion support BIDS [67] [68].
• Synchronize at Acquisition: The most reliable method is to use a hardware sync trigger at the start of the recording and record all modalities on a system with a unified clock, minimizing post-hoc alignment issues.

Experimental Protocols & Workflows

Protocol 1: Basic fNIRS Processing Pipeline with MNE-Python

This protocol outlines the key steps for converting raw fNIRS data to haemoglobin changes and extracting event-related responses [22] [66].

1. Data Loading and Inspection:

• Use mne.io.read_raw_nirx() or BIDS-compatible functions to load data.
• Visually inspect the raw signal with raw_intensity.plot() to identify obvious artifacts or bad channels.

2. Pre-processing:

• Convert raw intensity to optical density: raw_od = mne.preprocessing.nirs.optical_density(raw_intensity).
• Convert optical density to haemoglobin: raw_haemo = mne.preprocessing.nirs.beer_lambert_law(raw_od, ppf=0.1).
• Apply a band-pass filter (e.g., 0.05 - 0.7 Hz) to remove unwanted physiological noise: raw_haemo.filter(0.05, 0.7, ...).

3. Epoching and Averaging:

• Extract events from annotations: events, event_dict = events_from_annotations(raw_haemo).
• Create epochs around stimulus events: epochs = mne.Epochs(raw_haemo, events, tmin=-5, tmax=15, baseline=(None, 0), reject=reject_criteria).
• Average epochs to create evoked responses: evoked = epochs['Condition'].average().

The following diagram illustrates this workflow.

Protocol 2: Multimodal EEG-fNIRS Fusion with BrainFusion

This protocol describes a high-level workflow for conducting a joint analysis of EEG and fNIRS data within the BrainFusion environment for a motor imagery task [67].

1. Data Ingestion and Standardization:

• Import EEG and fNIRS data into BrainFusion's standardized multimodal data container.
• The platform will automatically handle signal alignment using either time-point or event-based strategies.

2. Automated Pre-processing:

• Run the built-in, modality-specific preprocessing pipelines for both EEG and fNIRS data. This includes standard steps like filtering and artifact removal tailored to each signal type.

3. Feature Engineering and Modeling:

• Use the cross-domain feature extraction library to compute features from both EEG and fNIRS streams.
• Employ the integrated machine learning modules (e.g., ensemble models, deep learning) to classify conditions (e.g., left vs. right motor imagery) using the fused features.

4. Deployment:

• Utilize the application generator to export the validated workflow as a standalone executable, enabling easy sharing and application to new data without the original software environment.

The workflow for this multimodal fusion is shown below.

Key Experimental Parameters and Reagents

The tables below summarize critical parameters and software tools used in typical fNIRS and multimodal experiments.

Table 1: Key Pre-processing Parameters in MNE-Python for fNIRS

Parameter	Typical Value / Function	Purpose
ppf (Partial pathlength factor)	0.1 [22] or 6 [66]	Factor for modified Beer-Lambert law conversion.
reject (Epoch rejection)	dict(hbo=80e-6) [22]	Threshold for automatic rejection of noisy epochs.
filter (Band-pass)	0.05 - 0.7 Hz [22]	Removes cardiac noise (high-pass) and slow drifts (low-pass).
baseline	(None, 0) [22]	Defines the time period for baseline correction of epochs.
source_detector_dist	> 0.01 m (1 cm) [22]	Minimum distance to pick channels sensitive to brain activity.

Table 2: Essential Software Tools for the Research Pipeline

Tool / "Reagent"	Function	Application Context
MNE-Python	A powerful, script-based toolbox for electrophysiology and fNIRS data analysis.	Core processing of individual modalities (EEG or fNIRS); building custom analysis pipelines [22] [69].
BrainFusion	A low-code, unified framework for multimodal BCI and brain-body interaction research.	Simplifying EEG-fNIRS-ECG-EMG fusion; feature engineering; deploying models as executables [67].
Cedalion	An open-source Python toolbox for fNIRS and DOT analysis, supporting image reconstruction.	An alternative for channel- and image-space analysis, GLM, and optode co-registration [70].
NeuroDOT	A MATLAB-based software package for diffuse optical tomography and image reconstruction.	Volumetric imaging of functional brain activations from fNIRS data [70].
BIDS Format	A standardized file system and metadata structure for brain data.	Ensuring reproducibility and simplifying data sharing and reading across tools like MNE-BIDS and BrainFusion [67] [68].

Solving Common Challenges and Optimizing Fusion Pipeline Performance

The fusion of functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) data presents a significant dimensionality challenge for researchers. This multimodality integration results in high-dimensional data spaces with a vast number of potential features extracted from both electrical and hemodynamic responses [8]. The curse of dimensionality is particularly severe in neuroimaging data, where large sets of potential neural features (e.g., responses from voxels, electrodes, temporal windows, and frequency bands) are often recorded across a limited set of stimuli and samples [71]. Without proper regularization and feature selection techniques, researchers risk developing models that overfit the training data and fail to generalize to new datasets, ultimately compromising the validity and reliability of study findings.

This technical guide addresses the most common dimensionality-related challenges encountered in fNIRS-EEG research pipelines and provides evidence-based solutions to enhance the quality of multimodal data analysis.

Troubleshooting Guides & FAQs

FAQ 1: What is the practical impact of dimensionality on my fNIRS-EEG analysis?

Answer: High dimensionality without proper regularization negatively impacts analysis in several measurable ways:

• Reduced Classification Performance: Models may achieve perfect training performance while completely failing to generalize to new data due to memorization of training data involving redundancy [72].
• Increased Computational Demands: Processing high-dimensional feature spaces requires greater computational resources and time, which can be particularly problematic for real-time applications like brain-computer interfaces [73].
• Interpretation Challenges: As dimensionality increases, it becomes increasingly difficult to identify which neural features are truly meaningful for the research question [71].

FAQ 2: How can I determine if my dataset has insufficient samples for the feature space?

Answer: The sample-to-feature ratio is a critical consideration. These warning signs indicate insufficient samples:

• Model Instability: Small changes in training data lead to significant changes in model parameters or feature importance rankings.
• Perfect Training Performance with Poor Testing Performance: This classic sign of overfitting suggests the model has memorized noise rather than learned generalizable patterns [72].
• Feature Importance Inconsistency: Different feature subsets are selected as most important across different cross-validation folds.

As a general guideline, the number of samples should substantially exceed the number of features, with some studies recommending ratios of 10:1 or higher for stable model estimation [71].

FAQ 3: What are the most effective feature selection methods for fNIRS-EEG data?

Answer: Research indicates several effective approaches:

Table 1: Feature Selection Methods for fNIRS-EEG Data

Method	Mechanism	Best Use Cases	Performance Evidence
Mutual Information-Based Selection	Maximizes relevance and minimizes redundancy between features	Optimal for identifying complementary features across modalities [72]	Up to 5% improvement in hybrid classification accuracy [72]
Correlation Stability	Ranks features based on stability across stimulus repetitions	Establishing reliable neural signatures for stimulus classification [71]	Successfully applied across multiple semantic representation studies [71]
Attribute/Feature Correlation	Selects features based on correlation with semantic attributes	Zero-shot learning and neural-semantic mapping [71]	Achieves similar accuracy with far fewer features than stability methods [71]
Wrapper Methods with Search Optimization	Uses prediction performance to guide feature selection	Maximizing classification accuracy for specific tasks [74]	Achieved >96% accuracy in MI and MA tasks with optimized feature subsets [74]

FAQ 4: Which regularization techniques work best with high-dimensional fNIRS-EEG data?

Answer: The choice of regularization depends on your analysis goals:

• Ridge Regression: Effective for encoding models predicting neural features from semantic attributes, particularly when using ridge regression as a coefficient shrinkage method to safeguard against overfitting [71].
• Cross-Validation: Essential for tuning regularization parameters; ensures that the chosen regularization strength generalizes to unseen data.
• Embedded Regularization: Modern machine learning methods like LightGBM include built-in regularization; one study demonstrated remarkable efficiency with <110 ms for training and <1 ms for inference [73].

FAQ 5: How can I effectively combine feature selection and regularization?

Answer: These techniques should be implemented as sequential steps in your pipeline:

• First, apply feature selection to reduce the dimensionality to a manageable set of potentially relevant features.
• Then, apply regularization to further prevent overfitting within the reduced feature space.
• Always validate the combined approach using nested cross-validation to obtain unbiased performance estimates.

Research shows that focusing on highly informative features before model training enhances performance significantly. One study found that utilizing only the most informative features revealed differential encoding patterns, with accelerometry jerk primarily decoded through local spectral power while bar press rate was decoded via inter-regional connectivity [73].

Experimental Protocols & Methodologies

Mutual Information-Based Feature Selection Protocol

This protocol follows the methodology demonstrated to improve EEG-fNIRS classification performance by optimizing complementarity, redundancy, and relevance between multimodal features [72].

Materials Needed:

• Preprocessed fNIRS (HbO, HbR) and EEG data (time-series or spectral features)
• Computing environment with mutual information calculation capabilities
• Feature ranking and selection infrastructure

Step-by-Step Procedure:

• Feature Extraction:

  • Extract multiple spectral and temporal features from both EEG and fNIRS modalities
  • Create a combined feature matrix representing the high-dimensional space

• Mutual Information Calculation:

  • Calculate mutual information between each feature and class labels (relevance)
  • Compute mutual information between feature pairs (redundancy)
  • Evaluate complementarity between features from different modalities

• Feature Subset Selection:

  • Apply optimization criterion to maximize relevance while minimizing redundancy
  • Select the optimized subset of features through a cross-validation process
  • Validate selection stability across different data segments

• Classification Performance Evaluation:

  • Train classifiers using the selected feature subset
  • Compare performance against full feature set and single modalities
  • Assess generalization using held-out test data

Table 2: Research Reagent Solutions for fNIRS-EEG Fusion Studies

Reagent/Resource	Function/Purpose	Example Application	Implementation Notes
Modified Beer-Lambert Law (mBLL)	Converts raw fNIRS intensity to hemoglobin concentrations	Deriving relative HbO and HbR concentration changes [18] [75]	Requires appropriate pathlength factor (PPF) correction [18]
Scalp Coupling Index (SCI)	Quantifies optode-scalp coupling quality	Identifying and rejecting poor-quality fNIRS channels [18]	Channels with SCI <0.5 are typically marked as bad [18]
Motion Artifact Correction Algorithms	Identifies and removes motion-induced signal components	Improving signal quality in movement-prone experiments	Multiple methods available; must be documented transparently [62]
Joint Independent Component Analysis (jICA)	Projects multimodal features to new feature space	Mental stress detection from fused EEG-fNIRS [74]	Can improve detection rates (91%→98% in one study) [74]
Atomic Search Optimization	Nature-inspired algorithm for feature selection	Identifying optimal feature subsets in high-dimensional data [74]	Part of a larger multi-level progressive learning framework [74]

Regularized Encoding Model Implementation

This protocol implements the ridge regression-based encoding model effective for zero-shot learning applications, which can be adapted for various fNIRS-EEG fusion contexts [71].

Procedure:

• Data Preparation:

  • Arrange neural data into matrix Y (samples × features)
  • Arrange semantic attributes into matrix X (samples × attributes)
  • Normalize X rows to unit length and Y columns to zero mean, unit variance

• Model Training:

  • For each neural feature m, compute optimal encoding weights:
    • wm(en) = (XᵀX + λ(en)I)⁻¹Xᵀym
  • Set regularization parameter λ(en) through cross-validation

• Regularization Optimization:

  • Perform k-fold cross-validation across a range of λ values
  • Select λ that maximizes prediction performance on validation folds
  • Balance bias-variance tradeoff based on research objectives

• Model Validation:

  • Evaluate encoding performance on held-out test data
  • Assess generalization across different stimulus conditions
  • Compare against unregularized baseline models

Visualization of Workflows

Dimensionality Reduction Pipeline for fNIRS-EEG Data

Dimensionality Reduction Pipeline for fNIRS-EEG Data

Multimodal Fusion Strategies for EEG-fNIRS

Multimodal Fusion Strategies for EEG-fNIRS

Table 3: Software Tools for fNIRS-EEG Dimensionality Analysis

Tool/Platform	Primary Function	Dimensionality Features	Implementation Considerations
MNE-Python	Full fNIRS and EEG processing pipeline	Signal filtering, HRF estimation, feature extraction	Provides comprehensive tutorial for fNIRS processing [18]
FieldTrip	Preprocessing and averaging of NIRS data	Optical density conversion, motion artifact handling	Supports single-channel and multi-channel analysis [75]
LightGBM	Gradient boosting framework for decoding	Built-in feature selection and regularization	Demonstrated <110 ms training time for neural decoding [73]
Custom MATLAB/Python Scripts	Implementing specialized algorithms	Mutual information calculation, custom regularization	Flexibility for research-specific dimensionality needs [72] [71]

Functional Near-Infrared Spectroscopy (fNIRS) signals contain components originating from both neurovascular coupling and systemic physiological sources, creating a significant challenge for accurate data interpretation. The fNIRS signal comprises six components: neuronal evoked changes in the cerebral compartment (the signal of interest), systemic evoked changes in the cerebral compartment, systemic evoked changes in the extracerebral compartment, vascular evoked changes in both cerebral and extracerebral compartments, and muscular evoked changes in the extracerebral compartment [76]. Systemic physiological confounders—including cardiorespiratory activity (heart rate, respiration), blood pressure changes (Mayer waves), and changes in arterial carbon dioxide concentration (PaCO2)—can mimic true hemodynamic responses, potentially causing false positives or masking genuine neuronal activity (false negatives) [76]. Proper identification and correction of these confounders is therefore essential for valid fNIRS research, particularly in fNIRS-EEG fusion studies where accurate hemodynamic information complements electrophysiological data.

Troubleshooting Guide: Common fNIRS Issues and Solutions

FAQ 1: Why does my fNIRS data show hemodynamic-like responses during rest periods?

Problem: Unexpected hemodynamic response morphology during rest or control conditions.

Root Cause: Systemic physiological changes can mimic the hemodynamic response function (HRF). For example, an increase in PaCO2 during a task can cause a large increase in HbO and a slight decrease in HbR, closely resembling a normal neurovascular response [76]. These systemic changes are not adequately filtered out by standard bandpass filters.

Solution: Implement Systemic Physiology Augmented fNIRS (SPA-fNIRS)

• Simultaneously monitor: Heart rate (HR), mean arterial pressure (MAP), end-tidal CO2 (EtCO2), and respiration [76] [77].
• Use short-channel measurements: Incorporate source-detector separations of ~0.8 cm for adults to regress out superficial scalp contributions [77].
• Apply advanced processing: Use Principal Component Analysis (PCA) or General Linear Model (GLM) with physiological regressors to remove systemic components [12] [77].

FAQ 2: Why is my functional connectivity (rsFC) overestimated?

Problem: Artificially high correlation between fNIRS channels during resting-state measurements.

Root Cause: Systemic physiological noise (e.g., blood pressure oscillations, respiration) represents a global source of variance that synchronizes signals across channels, leading to spurious correlations [77]. Approximately 94% of the signal measured by a regular fNIRS channel (source-detector distances ~3 cm) reflects systemic hemodynamic changes from extracerebral tissue [77].

Solution:

• Short-channel regression: Include 1 cm source-detector distance measurements to capture superficial confounds [77].
• Global Average Signal Removal: Apply with caution as it may introduce spurious negative correlations [77].
• PCA-based approaches: Remove first n-principal components containing global physiological variance [77].

Table 1: Comparison of Physiological Confounder Correction Methods

Method	Key Principle	Effectiveness	Limitations
Short-Channel Regression	Uses <1cm separation channels to measure & regress out superficial signals	High for scalp contributions	Reduces number of available neural channels; requires specific hardware [77]
PCA/ICA	Removes components with highest variance (typically physiological)	High for global systemic noise	Number of components to remove is arbitrary; may remove neural signal [77]
SPA-fNIRS	Directly measures & regresses physiological variables (HR, MAP, CO2)	Comprehensive for multiple confounders	Requires additional equipment and synchronization [76]
Bandpass Filtering	Removes frequency content outside expected HRF range (e.g., >0.7 Hz)	Limited to specific frequency noise	Cannot remove confounders in same frequency band as HRF [12] [22]

FAQ 3: Why is my group-level HRF inconsistent or unexpected?

Problem: Lack of clear HRF shape, multiple signal dips, or inconsistent responses across subjects [78].

Root Cause: Variability in data quality, analysis pipelines, and researcher choices significantly impacts results. A recent reproducibility study found that nearly 80% of research teams agreed on group-level results only when hypotheses were strongly supported by literature, with agreement improving with better data quality and researcher experience [53].

Solution:

• Standardize preprocessing pipelines: Establish laboratory-wide protocols for filtering, motion correction, and quality metrics [53].
• Check data quality metrics: Use Scalp Coupling Index (SCI) to verify optode-scalp contact quality [22].
• Implement quality thresholds: Reject channels with SCI <0.5 and epochs with excessive noise [22].

FAQ 4: Why do I get different results with different analysis pipelines?

Problem: Variability in statistical significance and effect sizes when using different processing approaches.

Root Cause: fNIRS analysis involves multiple stages with numerous valid choices at each step, including data selection criteria, preprocessing options, region of interest selection, and statistical modeling [53]. Teams with higher self-reported analysis confidence (correlated with fNIRS experience) showed greater agreement [53].

Solution:

• Document analysis choices thoroughly: Maintain detailed records of all processing parameters.
• Use established software platforms: Leverage community-vetted tools like HOMER3, NIRSLab, or MNE-NIRS [44].
• Perform sensitivity analyses: Test key results across multiple processing approaches.

Experimental Protocols for Confounder Identification

Protocol 1: Comprehensive Physiological Monitoring (SPA-fNIRS)

Purpose: To simultaneously measure systemic physiological variables alongside fNIRS for complete confounder identification [76].

Materials:

• fNIRS system with short-channel capability
• Electrocardiogram (ECG) for heart rate monitoring
• Finger photoplethysmograph for blood pressure
• Capnograph for end-tidal CO2
• Respiratory belt or spirometer for respiration
• Synchronization hardware/software

Procedure:

• Apply fNIRS optodes with integrated short-separation channels (0.8-1.0 cm for adults).
• Attach physiological sensors according to manufacturer specifications.
• Start simultaneous recording of all devices with precise time synchronization.
• Perform a 5-minute resting-state baseline recording.
• Conduct experimental paradigm with task and control conditions.
• Continue monitoring throughout the entire session.

Analysis:

• Preprocess fNIRS data (optical density conversion, Beer-Lambert law).
• Extract physiological time-series (heart rate, blood pressure, respiration, CO2).
• Include physiological regressors in GLM or use signal decomposition methods.

Protocol 2: Resting-State Functional Connectivity with Confounder Control

Purpose: To obtain accurate resting-state networks by controlling for systemic physiological influences [77].

Materials: fNIRS system with short-separation channels, physiological monitors.

Procedure:

• Acquire 12 minutes of resting-state data from each participant.
• Instruct participants to remain still with eyes open or closed.
• Ensure stable environmental conditions to minimize physiological variations.

Processing Pipeline:

• Convert raw intensity to optical density then to hemoglobin concentrations [22].
• Apply bandpass filter (0.01-0.15 Hz) to focus on resting-state fluctuations [77].
• Remove motion artifacts using wavelet or spline interpolation methods.
• Regress out short-separation channel signals.
• Apply PCA to remove global physiological components.
• Calculate correlation matrices between cleaned channels.
• Perform statistical analysis on connectivity measures.

Table 2: Essential Research Reagents and Tools for fNIRS Confounder Management

Item	Function	Example Products/Formats
Short-Separation Optodes	Measures superficial scalp hemodynamics for signal regression	Custom designs; integrated in commercial systems like NIRScout, NIRSport [77]
fNIRS Analysis Software	Data processing, visualization, and statistical analysis	HOMER3, NIRSLab, MNE-NIRS, NIRS-SPM, ICNNA [44]
Physiological Monitors	Measures heart rate, blood pressure, respiration, CO2	ECG, finger PPG, capnograph, respiratory belt [76]
3D Digitization System	Records precise optode locations for accurate spatial registration	Polhemus, Structure Sensor, photogrammetry systems [32]
Synchronization Hardware	Aligns fNIRS, EEG, and physiological data streams	Lab Streaming Layer (LSL), trigger boxes, network synchronization [32]
Custom Headgear	Maintains stable optode placement with integrated EEG	3D-printed helmets, cryogenic thermoplastic sheets [32]

Advanced Integration: fNIRS-EEG Fusion Considerations

The combination of fNIRS and EEG presents unique opportunities and challenges for confounder management. EEG provides excellent temporal resolution but suffers from poor spatial localization, while fNIRS offers better spatial resolution but is contaminated by systemic physiology [32] [27]. In fused systems:

• Use fNIRS to localize EEG sources: Hemodynamic information can constrain EEG source modeling [27].
• Leverage EEG for temporal precision: Identify physiological events (heartbeats, movement) in EEG that correlate with fNIRS artifacts [32].
• Implement integrated hardware: Custom headgear with co-registered fNIRS optodes and EEG electrodes minimizes spatial uncertainty [32].
• Synchronize data streams precisely: Use unified processors or precise timing protocols to align multimodal data [32].

Effective identification and correction of systemic physiological confounders is essential for robust fNIRS research, particularly in fused fNIRS-EEG studies. By implementing SPA-fNIRS approaches, incorporating short-channel measurements, and using appropriate processing pipelines, researchers can significantly improve the validity and reproducibility of their findings. Standardized protocols and comprehensive reporting of methodological choices will further enhance the reliability of fNIRS across the research community.

In electroencephalography (EEG) research, motion and muscle artifacts pose a significant challenge for data interpretation, particularly in naturalistic study designs and mobile brain-imaging scenarios. These artifacts introduce non-neural signals that can obscure genuine brain activity, complicating both unimodal analysis and the growing field of multimodal fNIRS-EEG fusion research [8]. Motion artifacts typically arise from head movements, electrode displacement, or cable sway, often producing high-amplitude, low-frequency signals that can mask event-related potentials. Muscle artifacts, primarily from jaw clenching, forehead tension, or neck strain, introduce high-frequency, non-stationary noise that contaminates the EEG spectrum [79] [80]. Effectively removing these contaminants is a critical preprocessing step to ensure the validity and reliability of neural signatures, especially when correlating electrical activity with the hemodynamic responses measured by fNIRS [8] [63].

Frequently Asked Questions (FAQs)

Q1: What are the most effective automated techniques for removing motion artifacts from high-density EEG during running or walking?

For high-motion scenarios like running, Artifact Subspace Reconstruction (ASR) and iCanClean are currently the most effective automated techniques [80]. ASR uses a sliding-window principal component analysis (PCA) to identify and remove high-variance signal components that deviate significantly from a calibrated baseline period. A key parameter is the standard deviation cutoff (k), with values between 10-20 recommended for locomotion studies to avoid over-cleaning [80]. iCanClean leverages canonical correlation analysis (CCA) to identify and subtract noise subspaces that are highly correlated with pseudo-reference noise signals derived from the EEG data itself (e.g., very low-frequency content below 3 Hz). Studies show that using iCanClean with an R² threshold of 0.65 and a 4-second sliding window optimally preserves brain signals while removing gait-related motion artifacts [80].

Q2: How can I distinguish muscle artifact components from neural components in an Independent Component Analysis (ICA) decomposition?

Distinguishing muscle from neural components in ICA relies on spatial, temporal, and spectral heuristics [81]. The table below summarizes key distinguishing characteristics:

Table: Distinguishing ICA Components

Feature	Muscle Artifact Component	Neural (Brain) Component
Spatial Topography	Focal projections over temporal, frontal, or neck muscles; non-dipolar [81].	Spatially smooth, dipolar maps consistent with cortical generators [80].
Spectral Power	High-frequency content (> 20 Hz); broad spectral profile [79] [81].	Peak power in standard bands (Delta, Theta, Alpha, Beta); rhythmic activity [79].
Time Course	Bursty, high-frequency, non-stationary activations [81].	More continuous, oscillatory dynamics time-locked to tasks or events.

Q3: My research involves simultaneous fNIRS-EEG. Will motion artifact removal in the EEG affect the fidelity of the fused data?

Robust artifact removal in EEG is essential for high-quality data fusion and generally does not negatively impact fused data fidelity if performed correctly [8] [63]. Since fNIRS and EEG capture different physiological signals (hemodynamic vs. electrical), they are susceptible to different artifact types and require separate, optimized preprocessing pipelines before fusion [8] [82]. Effectively cleaning the EEG ensures that the shared latent neural variables discovered through fusion algorithms like structured sparse multiset Canonical Correlation Analysis (ssmCCA) genuinely reflect coupled neurovascular activity, rather than being driven by residual EEG artifacts [63]. The complementary nature of the signals means proper cleaning enhances fusion outcomes [51].

Q4: Why does my ICA decomposition perform poorly on data from participants with excessive motion?

Excessive motion creates widespread, high-amplitude artifacts that violate ICA's core assumption of statistical independence among sources [80]. When artifacts dominate the signal, ICA cannot effectively separate brain activity from noise, leading to components that represent mixed sources. To address this, apply a robust preprocessing pipeline before ICA. This includes using ASR or iCanClean to reduce large motion artifacts, followed by bad channel detection and interpolation. This process "conditions" the data, allowing ICA to subsequently find more physiologically plausible and dipolar brain components [83] [80].

Q5: Are deep learning methods viable for muscle and motion artifact removal in experimental pipelines?

Yes, deep learning (DL) is an emerging and highly viable approach. Models like AnEEG (an LSTM-based Generative Adversarial Network) show promise in generating artifact-free EEG signals by learning complex, non-linear noise patterns from training data [84]. These models can be trained in a supervised manner to map noisy EEG inputs to clean outputs. The primary advantage is their ability to adapt to specific artifact types without requiring manual parameter tuning for each dataset. However, their effectiveness depends on large, diverse, and well-labeled training datasets, which can be a limitation for some experimental paradigms [84].

Troubleshooting Guides

Troubleshooting Guide: Poor ERP Recovery After Motion Artifact Removal

Problem: Expected Event-Related Potential (ERP) components (e.g., P300) are absent or severely attenuated after cleaning data from a mobile experiment.

Solution: This indicates potential over-cleaning, where neural signals of interest have been removed along with the artifacts.

• Adjust ASR Parameters: If using ASR, the k parameter might be set too low. Increase the k value (e.g., from 10 to 20 or 30) to make the algorithm less aggressive, thereby preserving more of the neural signal variance [80].
• Validate with a Ground Truth Task: Always include a simple, well-established paradigm (e.g., an oddball task for P300) in a stationary, seated position. Use this data to confirm that your analysis pipeline can recover the expected ERP when motion is not a factor. This provides a benchmark for tuning parameters in the mobile condition [80].
• Inspect Intermediate Steps: Before and after applying artifact removal, check the power spectral density. If the cleaning process has removed an excessive amount of signal in the frequency bands of interest (e.g., Delta/Theta for P300), it's a clear sign of over-cleaning.

Troubleshooting Guide: Residual High-Frequency Muscle Noise

Problem: After standard preprocessing and ICA, high-frequency muscle noise (EMG) remains visible in the temporal or frontal electrodes.

Solution: Muscle artifacts are often persistent and may require a targeted approach.

• Re-run ICA with Corrected Data: If muscle artifacts are severe, the initial ICA may have failed. First, apply a high-pass filter at 1 Hz to remove slow drifts, then use ASR to remove large, transient artifacts. Re-running ICA on this cleaned data often yields a better decomposition where muscle artifacts are isolated into fewer, more clearly defined components [80].
• Incorporate Spectral Features: Use an algorithm like ICLabel which automatically classifies ICA components based on their spectral and spatial features. This tool can help identify components that have a significant "Muscle" probability, aiding in the selection of components for rejection [80].
• Targeted Filtering (Last Resort): As a final step, consider applying a notch filter (e.g., at 50/60 Hz) to remove line noise, followed by a low-pass filter below 40-45 Hz to attenuate the bulk of high-frequency EMG. Use this cautiously, as it will also remove genuine high-frequency neural activity (Gamma waves) [79].

Quantitative Comparison of Removal Techniques

The following table provides a structured comparison of the primary artifact removal techniques discussed, based on recent literature.

Table: Comparison of Motion and Muscle Artifact Removal Techniques

Technique	Underlying Principle	Best For	Key Parameters	Performance Metrics	Key Advantages	Key Limitations
ICA [79] [81]	Blind Source Separation (BSS) to isolate statistically independent sources.	Removing stereotyped artifacts: eye blinks, lateral eye movements, heart signals.	Number of components; heuristics for component rejection.	High component dipolarity indicates good separation [80].	Does not require reference channels; provides intuitive component topographies.	Fails with high-amplitude, non-stereotyped motion; requires manual component inspection.
ASR [83] [80]	Real-time PCA to identify and remove high-variance components.	Continuous data with large, non-stereotyped motion artifacts (e.g., walking, running).	Standard deviation cutoff (k); calibration data.	Reduces power at gait frequency & harmonics; improves ICA dipolarity [80].	Fast, automated, good for mobile EEG; handles large-amplitude artifacts.	Risk of over-cleaning with low k; performance depends on quality of calibration data.
iCanClean [80]	Canonical Correlation Analysis (CCA) to subtract noise subspaces.	High-motion environments; ideal when dual-layer EEG sensors are available.	R² correlation threshold (e.g., 0.65); sliding window size.	Outperforms ASR in producing dipolar components & recovering ERPs during running [80].	Highly effective for motion artifact; can use pseudo-reference signals.	Can be computationally intensive; requires parameter tuning.
Deep Learning (e.g., AnEEG) [84]	Trained neural network (e.g., GAN with LSTM) to map noisy EEG to clean EEG.	Scenarios with large, labeled datasets for training.	Model architecture; loss functions; training dataset size/quality.	Lower NMSE & RMSE; higher CC with ground truth signals [84].	Can model complex, non-linear artifacts; minimal manual intervention after training.	Requires large, diverse training datasets; "black box" nature; potential for overfitting.

Essential Experimental Protocols

Protocol 1: Preprocessing for Mobile EEG with Robust Motion Removal

This protocol is designed for experiments involving significant participant movement, such as walking or running [80].

• Data Import and Channel Location: Import the raw data into an analysis environment (e.g., EEGLAB). Assign correct channel locations based on the standard 10-20 system.
• High-Pass Filtering: Apply a high-pass filter at 1 Hz to remove slow drifts and DC offsets, which can interfere with subsequent artifact removal algorithms.
• Bad Channel Detection and Interpolation: Use a robust method like the Local Outlier Factor (LOF) to automatically identify channels with excessive noise or flat signals [83]. Interpolate the identified bad channels using data from surrounding good channels.
• Line Noise Removal: Use the cleanline function or similar to adaptively estimate and remove 50/60 Hz line noise and its harmonics.
• Motion Artifact Removal with ASR/iCanClean: This is the core step.
  • For ASR: Use the clean_artifacts function with a k parameter of 15 (start with this value and adjust based on data). The algorithm will use an initial clean segment of the data for calibration [80].
  • For iCanClean: Use the software with an R² threshold of 0.65 and a 4-second sliding window to identify and remove motion-correlated subspaces [80].
• Re-referencing: Re-reference the data to the average of all channels or a specific reference (e.g., linked mastoids).
• ICA Decomposition: Run ICA (e.g., using the runica algorithm) on the preprocessed data. The data is now suitable for ICA, which will effectively isolate remaining artifacts like eye blinks and residual muscle activity.
• Component Classification and Removal: Use ICLabel to automatically classify brain and non-brain components. Manually confirm the automatic labels and reject components clearly identified as artifact.
• Final Filtering: Apply a low-pass filter suitable for your analysis (e.g., 40 Hz for ERPs, 80 Hz for time-frequency analysis).

The workflow for this protocol is visualized below.

Protocol 2: A Targeted Pipeline for Muscle Artifact Reduction

This protocol focuses specifically on mitigating the effects of electromyogenic (EMG) contamination.

• Standard Preprocessing: Perform steps 1-4 from Protocol 1 (Import, 1Hz High-Pass, Bad Channel Interpolation, Line Noise Removal).
• Muscle-Specific Filtering (Optional): If high-frequency neural activity is not critical for your analysis, apply a low-pass filter with a 45 Hz cutoff to broadly attenuate EMG, which typically has power above 20 Hz [79].
• ICA Decomposition: Run ICA on the filtered data.
• Targeted Component Rejection: This is the key step.
  • Use ICLabel to get probability estimates for each component.
  • Sort components by their "Muscle" probability.
  • Manually inspect high-probability muscle components. Confirm they have: a) Spectral power peaking above 20 Hz, and b) a focal scalp map projecting to temporal, frontal, or neck muscles [81].
  • Reject the confirmed muscle components.
• Reconstruct the Data: Project the remaining components back to the sensor space to obtain the clean EEG.

The Scientist's Toolkit: Key Research Reagents & Solutions

Table: Essential Tools for a Robust EEG Artifact Removal Pipeline

Tool Name	Type	Primary Function	Application Context
EEGLAB [83] [81]	Software Environment	Interactive MATLAB toolbox for processing EEG data.	The foundational platform for most preprocessing workflows, including ICA analysis and plugin integration.
ICLabel	EEGLAB Plugin	Automated classification of ICA components into categories (Brain, Eye, Muscle, Heart, Line Noise, Channel Noise, Other).	Rapid, objective initial assessment of components to guide manual cleaning decisions [80].
RELAX	EEGLAB Plugin	Pipeline for targeted artifact reduction, cleaning specific periods of eye components and frequencies of muscle components.	Prevents effect size inflation and source localization bias; ideal for Go/No-Go and N400 tasks [85].
NEAR	EEGLAB Plugin	Newborn EEG Artifact Removal pipeline automating bad channel detection (via LOF) and ASR.	Specifically designed for the unique challenges of noisy, non-stereotyped artifacts in newborn and infant EEG [83].
Artifacts Subspace Reconstruction (ASR)	Algorithm/Plugin	Removes high-amplitude, non-stereotyped artifacts from continuous data via PCA and calibration.	Essential preprocessing for mobile EEG studies involving walking, running, or significant movement [80].
iCanClean	Algorithm/Software	Uses CCA and reference noise signals to detect and correct motion artifact subspaces in the EEG.	Superior motion removal in high-mobility studies, especially with dual-layer sensor setups [80].
Structured Sparse Multiset CCA (ssmCCA)	Data Fusion Algorithm	Fuses multimodal data (e.g., fNIRS-EEG) to find shared latent variables.	Identifies brain regions with consistent activity across modalities after unimodal preprocessing [63].

Temporal and Spatial Alignment of Multimodal Data Streams

Core Concepts of Multimodal Alignment

What are the fundamental principles behind temporal and spatial alignment for fNIRS-EEG integration?

Temporal alignment ensures that the electrophysiological activity captured by EEG and the hemodynamic responses measured by fNIRS are synchronized in time. Spatial alignment involves precisely co-registering EEG electrode locations with fNIRS optode positions on the scalp to enable accurate mapping of brain activity [32].

The neurovascular coupling mechanism forms the theoretical basis for this integration, where neural electrical activity (measured by EEG) is inherently accompanied by hemodynamic and metabolic responses (measured by fNIRS) [86]. This relationship allows researchers to study both the direct electrical neural activity and the indirect metabolic responses simultaneously.

Why is precise alignment critical for valid research outcomes?

Without proper alignment, researchers cannot confidently attribute signals to specific neural events or brain regions, potentially leading to erroneous conclusions about brain function and connectivity [87] [88].

Table 1: Consequences of Poor Alignment in fNIRS-EEG Research

Alignment Type	Potential Research Impact	Data Quality Issues
Temporal Misalignment	Incorrect interpretation of neurovascular coupling dynamics; flawed event-related analysis	Cannot correlate fast EEG responses with slower fNIRS hemodynamic responses
Spatial Misalignment	Misattribution of brain activity to incorrect anatomical regions; reduced spatial accuracy	Poor co-registration between electrical and hemodynamic activity maps
Semantic Misalignment	Failure to identify meaningful cross-modal relationships; incomplete data interpretation	Inability to connect physiological patterns across modalities meaningfully

Technical Implementation Guides

What are the primary methodologies for temporal alignment?

Two main technical approaches exist for temporal synchronization:

• Separate System Synchronization: fNIRS and EEG data are obtained using separate systems (e.g., NIRScout and BrainAMP) and synchronized during analysis via host computer software [32]. While simpler to implement, this method may lack the precision needed for microsecond-level EEG analysis.

• Unified Processor Synchronization: A single processor simultaneously acquires and processes both EEG and fNIRS signals, achieving highly precise synchronization [32]. This method requires more complex system design but provides superior temporal alignment accuracy.

For post-acquisition temporal alignment, techniques include:

• Timestamp Normalization: Standardizing timestamps across devices and using interpolation for asynchronous sampling or missing data [87]
• Dynamic Time Warping (DTW): Algorithm that adjusts sequences with variable speeds or delays to align similar patterns despite timing differences [87]
• Sliding Window Methods: Segmenting continuous data streams into manageable chunks for real-time, near-synchronous fusion [87]

How can researchers achieve optimal spatial alignment?

Spatial alignment requires precise co-registration of EEG electrodes and fNIRS optodes. Current helmet fusion approaches include:

Table 2: Spatial Alignment Hardware Configurations

Helmet Design Approach	Advantages	Limitations & Considerations
Integrated EEG/fNIRS Cap [32]	Simple implementation; maintains probe coupling	Elastic fabric may cause variable source-detector distances; inconsistent scalp contact pressure
3D-Printed Custom Helmet [32]	Perfect individual fit; flexible component positioning	Higher production costs; longer fabrication time
Cryogenic Thermoplastic Sheet [32]	Cost-effective; lightweight; customized shaping	Potential rigidity; may exert uncomfortable pressure on head
Flexible Cap with Modifications [32]	Utilizes existing equipment; relatively straightforward	Probe stability issues during movement; placement inconsistencies

Critical steps for spatial alignment:

• Digitization: Use 3D magnetic space digitizers (e.g., Fastrak, Polhemus) to record optode positions relative to anatomical landmarks (nasion, inion, preauricular points) [63]
• Coordinate Transformation: Map all recording locations to standard brain atlas space (e.g., MNI space) for group-level analysis
• Inter-Optode Spacing: Maintain consistent distance (typically 2.5-3.5 cm) between fNIRS sources and detectors [63]

Troubleshooting Common Alignment Issues

How can researchers resolve temporal drift between fNIRS and EEG systems?

Temporal drift occurs when recording systems operate on independent clocks, causing gradually increasing misalignment.

Solution Protocol:

• Implement Hardware Synchronization: Use a common trigger signal (TTL pulses) to initiate both systems simultaneously [32]
• Embed Temporal Markers: Introduce predefined events (e.g., visual/auditory stimuli) at regular intervals to create alignment reference points
• Post-Hoc Correction: Apply cross-correlation algorithms to identify optimal time-shift between modalities
• Validation Check: Ensure key neural events (e.g., event-related potentials) temporally correspond with appropriate hemodynamic responses

What approaches fix poor probe-to-scalp contact in combined helmets?

Poor scalp contact creates signal artifacts and reduces data quality in both modalities.

Troubleshooting Strategies:

• Conductive Gel Optimization: Use appropriate EEG electrolyte gels that don't interfere with optical signals
• Customized Mounting: Create subject-specific fixtures using 3D-printed or thermoplastic materials [32]
• Pressure Monitoring: Implement force sensors to ensure consistent, comfortable pressure application
• Hair Management: Develop protocols for gently parting hair under optodes and electrodes without compromising contact

How can researchers address electromagnetic interference between systems?

EEG systems are susceptible to interference from fNIRS electronic components.

Mitigation Approaches:

• Shielding Implementation: Apply appropriate electromagnetic shielding to fNIRS components
• Cable Management: Route power and data cables to minimize cross-talk with EEG electrodes
• Grounding Optimization: Ensure proper grounding schemes that don't create ground loops
• Component Placement: Position fNIRS electronics as far from EEG amplifiers as physically possible within helmet constraints

Experimental Protocol: Motor Execution Study

What is a validated experimental design demonstrating successful fNIRS-EEG alignment?

The following protocol is adapted from a published study investigating neural activity during motor execution, observation, and imagery [63].

Research Goal: To examine shared and distinct neural mechanisms of motor execution (ME), motor observation (MO), and motor imagery (MI) using aligned fNIRS-EEG.

Participants: 21 healthy adults (16 right-handed, 5 ambidextrous), aged 18-65 years

Equipment & Setup:

• 24-channel continuous-wave fNIRS system (Hitachi ETG-4100) measuring HbO/HbR at 10Hz sampling rate
• 128-electrode EEG system (Electrical Geodesics, Inc.) embedded with fNIRS optodes in same cap
• 3D magnetic space digitizer (Fastrak, Polhemus) for optode/electrode localization
• Inter-optode distance: 2.88 ± 0.13 cm (range: 2.16-3.26 cm)

Alignment Methodology:

• Spatial: fNIRS probes positioned over sensorimotor and parietal cortices bilaterally; digitization of all optodes/electrodes relative to nasion, inion, and preauricular landmarks
• Temporal: Unified acquisition system with synchronized data streaming

Experimental Conditions:

• Motor Execution (ME): Participants grasp and move a cup using right hand after audio cue "Your turn"
• Motor Observation (MO): Participants observe experimenter performing the same cup-moving task after audio cue "My turn"
• Motor Imagery (MI): Participants mentally rehearse the cup-moving task without physical movement

Data Fusion Analysis: Structured Sparse Multiset Canonical Correlation Analysis (ssmCCA) identified brain regions consistently detected by both modalities, revealing activation in left inferior parietal lobe, superior marginal gyrus, and post-central gyrus across all conditions [63].

Research Reagent Solutions

Table 3: Essential Materials for fNIRS-EEG Alignment Research

Item	Function	Technical Specifications
Simultaneous fNIRS-EEG Cap [32] [63]	Integrated mounting platform for both modalities	Elastic fabric with embedded electrode/optode fixtures; multiple sizes for head circumference variation
3D Magnetic Digitizer [63]	Precise spatial localization of sensors	Fastrak/Polhemus systems; accuracy ±0.2-0.8mm; records nasion, inion, preauricular landmarks
Conductive EEG Gel	Ensures electrical contact for EEG electrodes	Electrolyte composition; non-interfering with optical signals; appropriate viscosity for stability
Cryogenic Thermoplastic [32]	Customizable helmet substrate for improved fit	Softens at ~60°C; moldable to head shape; retains stability when cooled; lightweight
Optical Phantoms	fNIRS signal calibration and validation	Tissue-simulating materials with known optical properties; used for system performance verification

Frequently Asked Questions

Can I achieve adequate alignment using separate fNIRS and EEG systems?

Yes, but with important limitations. Separate systems synchronized via software (e.g., NIRScout and BrainAMP) can improve real-time EEG classification accuracy [32]. However, this approach may not achieve the precision required for microsecond-level EEG analysis, and unified processor systems are recommended for research requiring high temporal precision.

What is the minimum required contrast ratio for visualization elements in alignment diagrams?

For accessibility compliance, standard text should have a contrast ratio of at least 7:1 against its background, while large-scale text (18pt or 14pt bold) requires at least 4.5:1 [89] [90]. These standards ensure that researchers with visual impairments or color vision deficiencies can interpret alignment diagrams and data visualizations.

How does alignment address the fundamental technical limitations of each modality?

Alignment creates synergy by combining EEG's high temporal resolution (milliseconds) with fNIRS's superior spatial resolution (centimeters) [86] [32]. EEG provides direct measurement of neural electrical activity, while fNIRS measures indirect hemodynamic responses through neurovascular coupling [86]. Proper alignment enables researchers to leverage the complementary strengths of both modalities while mitigating their individual limitations.

What computational methods are available for fNIRS-EEG data fusion?

Several analytical approaches exist:

• EEG-informed fNIRS analysis: Using temporal features from EEG to inform hemodynamic response modeling
• fNIRS-informed EEG analysis: Using spatial information from fNIRS to constrain EEG source localization
• Parallel fNIRS-EEG analysis: Analyzing modalities separately then combining results
• Structured Sparse Multiset CCA (ssmCCA): Advanced fusion method that identifies relationships between multiple datasets while incorporating structural constraints [63]

Optimizing Fusion with Structured Sparse Methods and Data-Driven Approaches

Technical Support Center: fNIRS-EEG Fusion Research

Frequently Asked Questions (FAQs)

Q1: What is the primary motivation for fusing fNIRS and EEG signals in brain imaging research? fNIRS and EEG are complementary neuroimaging techniques. EEG provides millisecond-level temporal resolution to capture neural electrical activity, while fNIRS measures hemodynamic responses with better spatial localization and is more robust to motion artifacts. Their integration provides a more comprehensive picture of brain activity by combining fast neural dynamics with improved spatial information. [8] [24]

Q2: What are the most common types of noise and artifacts affecting fNIRS signals, and how can they be addressed? fNIRS signals are contaminated by various physiological noises, including cardiac (~1 Hz), respiratory (~0.3 Hz), and Mayer waves (~0.1 Hz). Motion artifacts are also prevalent. Common remediation strategies include using digital filters (particularly band-pass finite impulse response filters), motion artifact correction algorithms, and employing short-separation measurements to regress out superficial contaminants. [23] [8]

Q3: How do artifact removal approaches typically differ between EEG and fNIRS in multimodal studies? While many studies incorporate robust artifact handling for EEG (e.g., for ocular and muscle activity), confounder correction in fNIRS often remains limited primarily to filtering or motion artifact removal. Furthermore, short-separation measurements and other auxiliary signals for fNIRS remain underutilized in many fusion studies. [8]

Q4: What are the main categories of fusion strategies for integrating EEG and fNIRS data? Fusion methods can be categorized as:

• Data Concatenation: Combining raw or preprocessed data from both modalities early in the pipeline.
• Model-based: Using frameworks like the General Linear Model (GLM) to analyze both signals.
• Source-decomposition: Identifying latent components that represent shared neural processes.
• Decision-level: Combining the outputs of classifiers applied to each modality separately, often using advanced methods like Dempster-Shafer theory. [8] [57]

Q5: Why is data preprocessing standardization particularly important for fNIRS research? The fNIRS research landscape shows significant heterogeneity in analysis approaches and pre-processing procedures. Often, there is a lack of complete methodological description, making study replication and results comparison challenging. Standardization enhances the reliability, repeatability, and traceability of reported findings. [23] [91]

Troubleshooting Guides

Issue 1: Poor Signal-to-Noise Ratio in fNIRS Data

Problem: Hemodynamic responses are masked by strong physiological noise or motion artifacts.

Solution:

• Verify Filter Parameters: Ensure an appropriate band-pass filter is applied. Simulation studies suggest a 1000th order band-pass Finite Impulse Response (FIR) filter can optimally recover the hemodynamic response before applying the General Linear Model (GLM). [23]
• Inspect for Motion Artifacts: Implement a dedicated motion artifact correction algorithm (e.g., using wavelet-based methods or robust regression).
• Utilize Short-Separation Channels: If available, use signals from short-separation channels as regressors to remove systemic physiological noise originating from the scalp. [8]

Issue 2: Suboptimal Performance in Multimodal Classification

Problem: A hybrid fNIRS-EEG Brain-Computer Interface (BCI) or decoding model yields lower accuracy than expected.

Solution:

• Re-evaluate Fusion Strategy: Simple concatenation may not capture complex cross-modality relationships. Consider implementing a cross-modal attention mechanism that allows the model to dynamically weigh the importance of features from each modality. [24]
• Model Uncertainty: For decision-level fusion, advanced methods like Dirichlet distribution parameter estimation and Dempster-Shafer Theory (DST) can model uncertainty and improve the robustness of the final decision. [57]
• Check Temporal Alignment: Ensure the high-temporal-resolution EEG signals and the slower fNIRS hemodynamic responses are properly aligned, accounting for the inherent delay in the hemodynamic response.

Issue 3: Inconsistent Replication of Published Studies

Problem: Inability to reproduce results from a published fNIRS or fNIRS-EEG study.

Solution:

• Check Methodological Reporting: Refer to best practices guidelines for fNIRS publications. Ensure all critical parameters are reported, including participant demographics (especially hair color/thickness), detailed optode locations, filter cut-offs, and specific motion correction techniques used. [91]
• Standardize Preprocessing Pipeline: Implement a standardized preprocessing workflow. The table below summarizes key steps and parameters for fNIRS and EEG.

Table 1: Standardized Preprocessing Steps for fNIRS and EEG Data

Step	fNIRS	EEG
Raw Data Inspection	Visual inspection for motion artifacts, heart beat oscillations. [23]	Visual inspection for amplitude jumps, muscle artifacts. [92]
Filtering	Band-pass filter (e.g., 0.01-0.2 Hz) to remove drift and cardiac noise. [23]	High-pass (e.g., 0.5-1 Hz) & Low-pass (e.g., 40 Hz) to remove slow drifts and line noise. [92] [93]
Artifact Correction	Motion artifact correction (e.g., wavelet, tPCA). [23]	ICA for ocular and muscle artifact removal. [92]
Referencing	-	Re-referencing (e.g., to average reference). [92]
Epoching	Epoching relative to task onset, with a sufficiently long baseline.	Epoching relative to event markers.
Bad Channel/ Trial Rejection	Signal quality inspection, CV/SNR-based rejection.	Statistical methods (e.g., Autoreject). [92]

Experimental Protocols & Methodologies

Detailed Methodology: Motor Imagery Data Collection

This protocol is adapted from a multimodal dataset containing simultaneous EEG-fNIRS recordings during motor imagery tasks. [93]

1. Participants:

• Cohort: 18 right-handed, healthy subjects.
• Screening: No neurological/psychiatric conditions. Normal or corrected-to-normal vision.
• Pre-experiment Instructions: Avoid alcohol, tea, and coffee on the day of and the day before the experiment.

2. Experimental Paradigm:

• Task: Eight different motor imagery tasks involving the right upper limb (e.g., hand open/close, wrist flexion/extension, elbow and shoulder movements).
• Trial Structure: Each trial lasts 18-20 seconds.
  • Rest (2 s): A white fixation cross is displayed.
  • Cue (2 s): A text and video cue indicates the upcoming task.
  • Imagery (4 s): The subject performs kinesthetic motor imagery of the cued movement.
  • Rest (10-12 s): A random rest period allows the hemodynamic response to return to baseline.
• Session Design: One session with 8 blocks. Each block contains 40 randomized trials (5 per task), totaling 320 trials per subject.

3. Data Acquisition Specifications: Table 2: Data Acquisition Parameters for a Multimodal Experiment

Parameter	EEG	fNIRS
System	Neuroscan SynAmps2 amplifier	NIRScout system (NIRx)
Channels	64 electrodes	8 sources, 8 detectors (forming 24 channels)
Sampling Rate	1000 Hz	7.8125 Hz
Reference	Left mastoid (M1)	-
Filter during Acquisition	0.5-100 Hz Band-pass, 50 Hz Notch	-
Optode/Electrode Placement	International 10-20 system	Left hemisphere, International 10-5 system
Impedance/Quality Check	< 10 kΩ	Visual inspection for good optical coupling

The Scientist's Toolkit

Table 3: Essential Research Reagent Solutions for fNIRS-EEG Fusion

Item	Function/Application
NIRScout System (fNIRS)	A continuous-wave fNIRS device used for measuring changes in oxy- and deoxy-hemoglobin concentrations in the cortex. [93]
Neuroscan SynAmps2 (EEG)	An amplifier system for high-quality, multi-channel EEG data acquisition. [93]
BrainVision Analyzer	Software for comprehensive analysis of EEG and fNIRS data, including preprocessing, visualization, and statistical analysis. [94]
MNE-Python	An open-source Python package for exploring, visualizing, and analyzing human neurophysiological data (EEG, MEG, fNIRS). It supports preprocessing, source decomposition, and machine learning. [92]
Short-Separation fNIRS Channels	Special fNIRS source-detector pairs placed close together (~8 mm) to selectively measure systemic physiological noise from the scalp, enabling its regression from standard channels. [8]
Dirichlet Distribution & DST	Mathematical frameworks for modeling uncertainty in classifier outputs and fusing decisions in a robust manner, improving final classification accuracy. [57]
Cross-Modal Attention Mechanism	A deep learning component that allows a model to dynamically focus on the most relevant features from EEG and fNIRS modalities for a given task. [24]

Workflow Visualization

fNIRS-EEG Preprocessing and Fusion Workflow

Multimodal Fusion Strategy Decision Diagram

Frequently Asked Questions

FAQ 1: What are the most critical steps to ensure our fNIRS-EEG pipeline is reproducible? Reproducibility hinges on robust data management practices from the start. Key steps include:

• Prioritize and Plan: Create a shortlist of 1-3 reproducible research practices (e.g., standardized data structures, version control) to implement per project to avoid being overwhelmed [95].
• Adopt Open Science Practices: Make your research process and outputs accessible. This includes publishing manuscripts, methods, data, and code openly to allow others to recreate your results [95].
• Use Version Control and Standardized Scripts: Employ systems like Git for code and data, and use automated scripts for data preprocessing and analysis instead of manual, point-and-click methods to ensure every step is documented and repeatable [95] [96].

FAQ 2: We experience significant artifacts in our fNIRS data during participant movement. How can we correct for this? Motion artifacts are a common challenge. A multi-pronged approach is recommended:

• Technical Solutions: Utilize robust artifact removal techniques. For fNIRS, this can include filtering and motion correction algorithms. Short-separation measurements are an advanced technique to better isolate brain-based signals but remain underutilized [8].
• Hardware Integration: Ensure your acquisition helmet provides a secure and consistent fit. Custom 3D-printed or thermoplastic helmets can offer better probe-to-scalp contact pressure than standard elastic caps, reducing motion-induced signal fluctuations [32].
• Data-Driven Methods: Implement machine learning approaches that can help distinguish evoked neuronal activity from complex systemic physiological noise and motion artifacts [8].

FAQ 3: Our EEG and fNIRS signals are not properly synchronized. What is the best method for temporal alignment? Precise synchronization is fundamental for fusion. There are two primary methods:

• Software Synchronization: Using a host computer to send simultaneous event markers to separate EEG and fNIRS systems. While simpler to set up, this method may lack the precision required for microsecond-level EEG analysis [32].
• Hardware Synchronization: Using a unified processor to acquire and process both EEG and fNIRS signals simultaneously. This method is more complex but achieves highly precise synchronization, streamlining the analytical process. This is the most widely used approach for concurrent recording [32] [97].

FAQ 4: At which stage of analysis should we fuse EEG and fNIRS data for the best classification results? Research indicates that the fusion stage impacts performance. A study on motor imagery classification found that early-stage fusion of EEG and fNIRS, where raw or pre-processed data from both modalities is combined before feature extraction, yielded significantly higher accuracy compared to middle-stage (feature-level) or late-stage (decision-level) fusion [59]. A Y-shaped neural network architecture can be an effective design for implementing early-stage fusion [59].

---

Troubleshooting Guides

Problem: Inconsistent Data Quality Across Recording Sessions

• Symptoms: High variance in signal-to-noise ratio, inconsistent activation patterns across sessions with the same participant and paradigm.
• Possible Causes & Solutions:
  • Cause 1: Inconsistent optode/electrode placement or pressure [32].
    • Solution: Use a customized acquisition helmet (e.g., 3D-printed) for consistent placement. For standard caps, digitize optode positions for each subject to account for placement variances [32] [63].
  • Cause 2: Inadequate participant instruction or engagement, especially in Motor Imagery tasks [97].
    • Solution: Implement a participant preparation protocol. For MI, use a grip strength calibration procedure with a dynamometer or stress ball to reinforce the kinesthetic sensation before data acquisition [97].

Problem: Poor Performance of Multimodal Classification Algorithm

• Symptoms: Classification accuracy is not better than using a single modality; model fails to generalize to new data.
• Possible Causes & Solutions:
  • Cause 1: Suboptimal fusion strategy [59].
    • Solution: Experiment with fusion stages. Consider implementing an early-stage fusion model. Test simpler feature-level concatenation if using a complex model [8] [59].
  • Cause 2: Lack of a public, standardized dataset for benchmarking [8] [97].
    • Solution: Utilize emerging public datasets like HEFMI-ICH for algorithm validation and optimization. This provides a benchmark against your data and methods [97].

Problem: Pipeline is Not Reproducible by Other Team Members

• Symptoms: Colleagues cannot replicate your results using your code and data; manual steps are required.
• Possible Causes & Solutions:
  • Cause 1: Manual, non-scripted data preprocessing steps [95].
    • Solution: Replace all manual steps (e.g., filtering, artifact rejection) with automated scripts. Use workflow orchestration tools to chain these steps into a single, automated pipeline [98] [99].
  • Cause 2: Lack of version control for data, code, and model parameters [95].
    • Solution: Implement version control systems (e.g., Git) for all code and analysis scripts. Use data versioning tools or clear naming conventions for different dataset versions. Maintain a detailed lab journal or README files documenting all changes and parameters [95].

---

Experimental Protocols & Data Specifications

The table below summarizes key specifications from a seminal study that created a hybrid EEG-fNIRS dataset for motor imagery, serving as a reference for designing reproducible experiments [97].

Table 1: Experimental Protocol and Data Specifications from the HEFMI-ICH Dataset

Aspect	Specification
Participants	17 normal subjects (12M/5F, 23.6 ± 1.8 yrs) & 20 ICH patients (17M/3F, 50.8 ± 10.3 yrs) [97]
Primary Paradigm	Left-hand vs. Right-hand Motor Imagery (MI) [97]
Trial Structure	Visual cue (2s) → Execution/MI (10s) → Rest (15s) [97]
EEG System	g.HIamp amplifier [97]
fNIRS System	NirScan, continuous-wave system [97]
EEG Sampling Rate	256 Hz [97]
fNIRS Sampling Rate	11 Hz [97]
EEG Channels	32 electrodes [97]
fNIRS Channels	90 measurement channels from 32 sources & 30 detectors [97]
Synchronization	Event markers from E-Prime 3.0 simultaneously triggered both systems [97]

Table 2: Performance of Different Fusion Strategies in Motor Imagery Classification

Fusion Stage	Description	Average Accuracy	Key Advantage
Early-Stage Fusion	Combining raw/pre-processed data before feature extraction [59].	76.21% [59]	Allows the model to learn complementary features directly from the data.
Middle-Stage (Feature-Level) Fusion	Extracting features from each modality first, then concatenating them [8].	~65-70% (EEG-only ~65%) [59]	Leverages domain knowledge for feature engineering.
Late-Stage (Decision-Level) Fusion	Each modality has a separate classifier, and their outputs are combined [8].	~57-65% (fNIRS-only ~57%) [59]	Robust to failures in one modality.

---

The Scientist's Toolkit

Table 3: Essential Research Reagents and Materials for fNIRS-EEG Fusion

Item	Function in the Experiment
Hybrid EEG-fNIRS Cap	A custom cap that integrates EEG electrodes and fNIRS optodes in a predefined geometry, ensuring co-registration of both modalities. Can be based on standard elastic caps or custom 3D-printed/thermoplastic designs [32] [97].
Continuous-Wave fNIRS System	Measures changes in oxygenated (HbO) and deoxygenated hemoglobin (HbR) concentrations in the cortex using near-infrared light, providing hemodynamic data [8] [97].
Bioamplifier for EEG	Records electrical potentials from the scalp with high temporal resolution, capturing neural electrophysiological activity [97].
3D Magnetic Digitizer	Records the precise 3D locations of fNIRS optodes and EEG electrodes on a participant's head. This is critical for accurate spatial localization and co-registration with brain anatomy [63].
Structured Sparse Multiset CCA (ssmCCA)	An advanced data fusion algorithm used to identify brain regions where both electrical (EEG) and hemodynamic (fNIRS) activities are consistently detected, strengthening findings [63].
Public Dataset (e.g., HEFMI-ICH)	Provides a benchmark for validating and optimizing new algorithms. Ensures that methods are tested against standardized, clinically relevant data [97].

---

Signaling Pathways and Experimental Workflows

Diagram 1: Data Preprocessing and Fusion Pipeline

Diagram 2: fNIRS-EEG Integration for Action Observation Network (AON)

Validating, Benchmarking, and Applying Fusion Pipelines in Research

Frequently Asked Questions (FAQs)

Q1: What are the most critical signal quality issues when simultaneously acquiring EEG and fNIRS data?

Both EEG and fNIRS signals are contaminated by various artifacts that can compromise data quality and subsequent analysis. For EEG, the primary contamination sources include ocular activity (EOG) and head/neck muscle activity (EMG), which introduce noise across much of the relevant EEG spectrum [8]. For fNIRS, contamination originates from scalp circulation, brain motion in cerebro-spinal fluid, and systemic physiology affecting brain vasculature (e.g., blood pressure changes, breathing, heart rate) [8]. Critically, while the same physiological source (e.g., cardiac activity) can affect both modalities, it manifests with distinct temporal, spatial, and amplitude characteristics in each [8].

Q2: How can I determine if my classification accuracy for a hybrid BCI system is acceptable?

Classification accuracy must be evaluated against baseline performance and state-of-the-art benchmarks. The table below summarizes representative accuracy values from recent studies:

Table 1: Classification Accuracy Benchmarks for Hybrid EEG-fNIRS BCI Systems

Study / Model	Task	Modality	Reported Accuracy	Benchmark Comparison
Deep Learning & Evidence Theory [57]	Motor Imagery	EEG-fNIRS Fusion	83.26%	3.78% improvement over state-of-the-art
EFRM (Few-shot learning) [56]	Various	EEG-fNIRS	Competitive with supervised models	High performance with minimal labeled data
Typical fNIRS-only [56]	Various	fNIRS-only	Lower than multimodal	Improved via shared domain learning with EEG
ShallowConvNet (EEG-only) [93]	Motor Imagery (Hand vs. Shoulder)	EEG-only	65.49%	Baseline for complex joint MI tasks

Q3: What metrics should I use beyond classification accuracy to evaluate my model properly?

While accuracy is a fundamental metric, a comprehensive evaluation should include additional dimensions:

• Signal Quality Metrics: For fNIRS, analyze the signal-to-noise ratio (SNR) of hemodynamic responses (HbO/HbR). For EEG, consider noise floor levels in key frequency bands (e.g., alpha, beta) [93].
• Model Robustness: Especially important for clinical applications, evaluate performance consistency across subjects and sessions. Note that EEG exhibits considerable inter-subject variability, hindering cross-subject generalizability [24].
• Temporal Dynamics: Assess the model's performance in real-time or single-trial classification scenarios, as this is crucial for practical BCI applications [56].

Q4: My fusion model isn't performing better than my single-modality model. What could be wrong?

This common issue often stems from inadequate fusion strategy or misalignment of multimodal data. Consider these points:

• Fusion Strategy: Simple feature concatenation may not fully exploit cross-modality correlations. Advanced strategies like cross-modal attention mechanisms [24] or evidence theory-based decision fusion [57] can more effectively integrate complementary information.
• Temporal Alignment: EEG captures millisecond-scale electrical activity, while fNIRS tracks slow hemodynamic changes (peaking 5-8 seconds post-stimulus) [8] [24]. Models must account for this neurovascular coupling delay [56]. Techniques like EEG-Guided Temporal Alignment (EGTA) have been proposed to address this [56].
• Data Quality: The benefits of fusion are only realized if both modalities provide quality signals. If one modality is excessively noisy, it can degrade the overall model performance.

Troubleshooting Guides

Issue 1: Poor EEG Signal Quality with Excessive Noise

Table 2: EEG Artifact Troubleshooting Guide

Symptom	Potential Cause	Solution	Preventive Measures
High-frequency noise	Muscle artifact (EMG) from head, neck, or jaw tension	Apply notch filter (e.g., 50/60 Hz) and band-pass filter (e.g., 0.5–100 Hz) [93].	Instruct subjects to relax jaw and minimize swallowing during task periods [93].
Low-frequency drifts	Poor electrode contact or skin potentials	Check and improve electrode impedance (maintain below 10 kΩ) [93].	Proper skin abrasion and use of high-conductivity electrolyte gel.
Large, slow deflections	Ocular artifact (EOG) from eye blinks or movements	Implement artifact removal algorithms (e.g., ICA, regression) in preprocessing.	Instruct subjects to fixate on a point and suppress blinks during critical task intervals [93].

Issue 2: Weak or Unreliable fNIRS Hemodynamic Response

Table 3: fNIRS Signal Troubleshooting Guide

Symptom	Potential Cause	Solution	Preventive Measures
No detectable HbO/HbR concentration change	Improper optode contact or scalp coupling	Verify signal quality at each channel before experiment; re-adjust cap.	Use a cap size appropriate for the subject's head circumference [97]. Ensure hair does not block optodes.
Signal appears saturated or abnormally high	Optode pressure on scalp, causing poor blood flow	Readjust cap to ensure firm but comfortable contact without excessive pressure.	Regularly monitor raw light intensity levels during data acquisition.
High-amplitude, sharp spikes in signal	Motion artifacts from head movement	Apply motion artifact correction algorithms (e.g., PCA, wavelet-based methods) [8].	Stabilize subject's head with a chin rest or headrest. Use short-separation channels to regress out superficial noise [8].
Systemic physiological contamination	Cardiac, respiratory cycles, blood pressure waves	Employ filtering (e.g., 0.01–0.2 Hz bandpass for HbO/HbR) and use short-separation regression [8].	Maintain a comfortable lab environment to minimize subject anxiety and physiological fluctuations.

Issue 3: Suboptimal Performance of Multimodal Fusion Model

Step 1: Verify Individual Modality Performance

• Ensure that each modality (EEG and fNIRS) can achieve reasonable classification accuracy on its own before attempting fusion. If one modality performs poorly, address its specific signal quality issues first.

Step 2: Re-examine the Fusion Strategy and Model Architecture

• Early vs. Late Fusion: Experiment with different fusion levels. Early fusion combines raw or low-level features, while late fusion (or decision fusion) combines high-level features or classifier outputs [24]. For instance, one innovative late-fusion approach used Dirichlet distribution parameter estimation to model uncertainty before fusing evidence with Dempster-Shafer Theory [57].
• Advanced Fusion Mechanisms: If simple concatenation fails, implement more sophisticated architectures designed to capture cross-modality relationships, such as:
  • Cross-modal attention mechanisms that dynamically weigh the importance of features from each modality [24].
  • Representation-learning models (like EFRM) that learn both modality-specific and shared representations, which is particularly beneficial when labeled data is scarce [56].

Step 3: Ensure Proper Temporal Synchronization and Alignment

• Confirm that the data acquisition systems for EEG and fNIRS are synchronized using a common trigger [97].
• Account for the inherent hemodynamic delay in fNIRS relative to EEG. This can be done by aligning the fNIRS signal with the EEG signal using a temporal offset or using alignment layers within the model architecture [56].

Experimental Protocols & Workflows

Standard Operating Procedure for Joint EEG-fNIRS Measurement

This protocol outlines the key steps for a simultaneous EEG-fNIRS experiment, from setup to data preprocessing [100].

Decision Workflow for Selecting a Fusion Strategy

Choosing the right fusion approach depends on data characteristics and the research goal. The following diagram outlines a logical decision process.

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 4: Essential Materials and Equipment for Hybrid EEG-fNIRS Research

Item	Specification / Example	Primary Function	Key Considerations
EEG Amplifier	g.HIamp (g.tec) [97], Neuroscan SynAmps2 [93]	Records electrical brain activity from scalp electrodes.	Check sampling rate (≥256 Hz), number of channels, input impedance, and synchronization capability.
fNIRS System	NIRScout (NIRx) [93], NirScan (Danyang Huichuang) [97]	Measures hemodynamic changes by detecting near-infrared light attenuation.	Check number of sources/detectors, wavelengths, sampling rate (≥10 Hz), and portability if needed.
Integrated Cap	Custom-designed hybrid cap with EEG electrodes and fNIRS optodes [97]	Holds EEG electrodes and fNIRS optodes in a stable, geometrically defined layout.	Ensure compatibility of montage with your target brain areas (e.g., motor cortex). Verify head size options.
Electrodes & Gel	Ag/AgCl sintered electrodes, high-conductivity electrolyte gel [93]	Ensure high-fidelity electrical contact between scalp and amplifier.	Maintain impedance below 10 kΩ throughout the experiment [93].
fNIRS Optodes	Sources (lasers/LEDs), Detectors (photodiodes/APDs)	Emit and detect near-infrared light through the scalp and brain tissue.	Source-detector separation is typically 3 cm for adult cerebral measurements [97].
Stimulus Presentation Software	E-Prime, PsychoPy, Presentation	Prescribes the experimental paradigm and delivers synchronized triggers.	Must be able to send simultaneous, low-latency triggers to both EEG and fNIRS recording systems.
Data Processing & Analysis Platform	MATLAB, Python (MNE, NiLearn, PyTorch)	Provides environment for implementing preprocessing, fusion, and classification algorithms.	Choose platforms with active community support and specialized toolboxes for EEG/fNIRS analysis.

Within the data preprocessing pipelines for hybrid functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) research, a critical decision point is the level at which data from these two modalities are integrated. This integration, known as fusion, typically occurs at different stages, with early fusion and late fusion representing two predominant strategies. The choice between them significantly impacts the performance and interpretability of brain-computer interface (BCI) systems. This guide provides a technical breakdown of these methods, their experimental protocols, and performance outcomes to assist researchers in selecting and troubleshooting the optimal fusion approach for their specific applications.

FAQs: Understanding Fusion Strategies

1. What is the fundamental difference between early and late fusion?

Early fusion, also known as feature-level fusion, involves combining the raw or pre-processed features from EEG and fNIRS before feeding them into a single, unified classification model. In contrast, late fusion (decision-level fusion) processes each modality through separate models and combines the final decisions or classifier outputs [101].

2. Why does the choice of fusion level matter for fNIRS-EEG research?

EEG and fNIRS capture complementary aspects of brain activity; EEG provides millisecond-level temporal resolution of electrical activity, while fNIRS offers better spatial resolution for slower hemodynamic changes [32] [27]. The fusion strategy determines how effectively this complementary information is exploited. An suboptimal strategy may fail to capture critical cross-modal interactions, leading to reduced classification accuracy and robustness [9].

3. What does the evidence say about which fusion method performs better?

Recent comparative studies and deep learning models have consistently demonstrated the superiority of early-stage fusion. For instance, one study using a Y-shaped neural network found that early-stage fusion significantly outperformed middle and late-stage fusion in classifying left-or-right hand motor imagery tasks [54] [102]. Another study proposing the "DeepSyncNet" framework reported that its early and deep fusion strategy outperformed traditional multimodal fusion techniques [9].

4. What are the practical hardware requirements for implementing early fusion?

Successful early fusion relies on high-quality, synchronized data. This typically requires:

• A unified or synchronized data acquisition system for EEG and fNIRS [32] [103].
• A custom-designed hybrid cap that integrates EEG electrodes and fNIRS optodes in a fixed, co-registered configuration to ensure spatial correspondence [32] [97].
• Systems that can handle the higher computational load and data dimensionality inherent in early fusion approaches.

Troubleshooting Guides

Problem: Low classification accuracy after feature concatenation (Early Fusion).

• Potential Cause: The concatenated feature vector may contain highly redundant or irrelevant features from the two modalities, which can confuse the classifier.
• Solution: Implement a feature selection algorithm before classification. Using a mutual information-based feature selection approach can optimize the feature set by maximizing complementarity and relevance while minimizing redundancy between EEG and fNIRS features [101].

Problem: Model fails to learn meaningful cross-modal relationships.

• Potential Cause: Simple concatenation of features may not be sufficient for the model to learn complex, non-linear interactions between the electrophysiological (EEG) and hemodynamic (fNIRS) data streams.
• Solution: Employ advanced deep learning architectures designed for early fusion. For example, use a model with an Attentional Fusion (AF) mechanism that adaptively weights and integrates features from both modalities at the feature extraction stage, allowing for richer cross-modal interaction [9].

Problem: System is unable to achieve precise temporal synchronization.

• Potential Cause: Using separate, unsynchronized systems for EEG and fNIRS acquisition can lead to misaligned data, making fusion unreliable.
• Solution: Utilize a single, unified processor to acquire and process both EEG and fNIRS signals simultaneously [32]. Ensure the data streams are synchronized at the hardware level or via precise software markers [97] [103].

Experimental Protocols & Performance Data

The following table summarizes key experimental findings from seminal studies comparing fusion methods, providing a quantitative basis for decision-making.

Table 1: Comparative Performance of EEG-fNIRS Fusion Methods

Study / Model	Fusion Type	Task (Dataset)	Key Methodology	Performance (Accuracy)
Y-shaped Network [54] [102]	Early-Stage	Motor Imagery (Dataset A)	Y-shaped ANN integrating bimodal data in initial layers.	76.21% (average, N=29)
Y-shaped Network [54] [102]	Middle-Stage	Motor Imagery (Dataset A)	Y-shaped ANN fusing data in intermediate network layers.	Lower than early-stage (P < 0.05)
Y-shaped Network [54] [102]	Late-Stage	Motor Imagery (Dataset A)	Y-shaped ANN combining data in final decision layers.	Lower than early-stage (P < 0.05)
Mutual Information & Feature Selection [101]	Feature-Level	Visuo-Mental Task (ALS patients/controls)	Mutual information-based feature selection to optimize complementarity.	Considerably improved hybrid performance vs. single modality and conventional fusion.
DeepSyncNet [9]	Early and Deep Fusion	Motor Imagery & Mental Arithmetic	3D tensor input, Attentional Fusion (AF), and Spatiotemporal Attention (STA).	Outperformed traditional multimodal fusion techniques.

Detailed Experimental Protocol: Early Fusion with a Y-shaped Network

This protocol is based on the study that directly compared fusion stages [54] [102].

1. Dataset:

• Dataset A from Shin et al. (2017): 29 participants performing left-hand vs. right-hand motor imagery.
• Task Structure: 2s visual cue, 10s motor imagery execution, 10-12s rest. 30 trials per task.

2. Data Preprocessing:

• EEG:
  • Downsampled from 200 Hz to 128 Hz.
  • Band-pass filtered (8-25 Hz) to retain μ and low-β bands.
  • Re-referenced to common average.
  • Selected 8 channels over the sensorimotor cortex.
  • Amplitude normalized to [-1, 1].
• fNIRS:
  • The study utilized the second and third modules of the EEGNet architecture, adapted for the lower temporal resolution of fNIRS signals [54].

3. Fusion & Classification Architecture:

• A Y-shaped artificial neural network was implemented with:
  • Two separate input branches (encoders) for EEG and fNIRS.
  • Early Fusion: The features from both modalities were merged at the initial stages of the network.
  • The fused features were then processed through subsequent layers for final classification.
• The model was evaluated using leave-one-out cross-validation.

4. Key Finding: The early-stage fusion configuration achieved statistically significantly higher performance than both middle-stage and late-stage fusion, establishing its efficacy for motor imagery classification [54] [102].

Signaling Pathways & Experimental Workflows

The following diagram illustrates the core architectural difference between early and late fusion strategies in a hybrid fNIRS-EEG pipeline.

The Scientist's Toolkit: Research Reagent Solutions

The table below lists essential "research reagents"—in this context, key datasets, algorithms, and hardware solutions—that are fundamental for conducting research in fNIRS-EEG fusion.

Item	Function / Description	Relevance to Fusion Research
Public Datasets	Provide standardized, annotated data for developing and benchmarking fusion algorithms.	Shin's Dataset [54]: A foundational public dataset for motor imagery and mental arithmetic. HEFMI-ICH [97]: The first hybrid dataset including Intracerebral Hemorrhage (ICH) patients, crucial for clinical translation.
Mutual Information-based Feature Selection [101]	A filter-based feature selection algorithm that maximizes feature complementarity and relevance while minimizing redundancy.	Critical for optimizing feature-level (early) fusion, preventing classifier confusion from high-dimensional, concatenated feature vectors.
Y-shaped Neural Network [54] [102]	A deep learning architecture with separate input branches for each modality that merge at a specific fusion stage.	The standard model for empirically comparing early, middle, and late fusion performance within a consistent framework.
g.tec g.HIamp & g.Nautilus [103]	Biosignal amplifiers that can be integrated with fNIRS add-ons (g.SENSOR) for simultaneous data acquisition.	Provides hardware-level synchronization, a prerequisite for reliable early fusion.
Custom Hybrid Caps [32] [97]	EEG caps with integrated holders for fNIRS optodes, ensuring fixed and co-registered sensor placement.	Maintains consistent spatial correspondence between EEG electrodes and fNIRS channels, which is vital for meaningful feature fusion.
Attentional Fusion (AF) Mechanism [9]	A deep learning module that uses a gating mechanism to adaptively weight and integrate features from EEG and fNIRS.	Represents an advanced early fusion technique that dynamically enhances cross-modal information interaction.

This technical support center is designed to assist researchers and scientists working at the intersection of motor imagery (MI) based Brain-Computer Interfaces (BCIs) and multi-modal neuroimaging. The guidance provided here is framed within a broader research context focusing on data preprocessing pipelines for functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) fusion. The following sections address frequently encountered technical challenges, provide detailed experimental protocols, and offer standardized solutions to ensure reproducible research outcomes in both academic and drug development settings.

Frequently Asked Questions (FAQs) & Troubleshooting Guides

FAQ 1: What are the most common causes of poor classification accuracy in MI-BCI systems, and how can they be mitigated?

Poor classification accuracy often stems from multiple sources, including insufficient training data, inappropriate feature extraction, and suboptimal channel selection.

• Insufficient or Poor-Quality Calibration Data: The system requires numerous examples of each MI class to learn effectively. A common mistake is collecting data in blocked trials (e.g., all left-hand trials followed by all right-hand trials), which makes it difficult for the model to generalize. Furthermore, single trials are insufficient; the system typically needs 20–50 repeats of each class when using a full-sized EEG headset, presented in a randomized order [104].
• Suboptimal Feature Extraction: Relying on basic bandpower features may not capture the complex spatial patterns of MI. Consider implementing advanced spatial filtering techniques like Common Spatial Patterns (CSP) to improve the separation between MI classes [105] [106].
• Non-Ideal Channel Selection: Using too many or too few channels can harm performance. The international 10-20 system is the standard for placement, with key areas for MI being the sensorimotor cortex (e.g., C3, Cz, C4). Review your channel configuration to ensure it covers these relevant regions [104] [106].

FAQ 2: How can I resolve persistent noise and data quality issues during fNIRS-EEG同步采集？

Simultaneous fNIRS-EEG data acquisition is prone to specific noise types that require targeted pre-processing strategies.

• Physiological Noise: Signals from cardiac pulsation, respiration, and blood pressure changes (Mayer waves) can obscure the functional hemodynamic response. Apply a bandpass filter (e.g., 0.01–0.1 Hz for fNIRS) to isolate the neurovascular coupling band [12] [21].
• Motion Artifacts (MA): These are a major source of noise in both modalities. For fNIRS, use algorithms like Savitzky–Golay filtering or wavelet-based methods to detect and correct for MAs. For EEG, tools like Artifact Subspace Reconstruction (ASR) can effectively remove transient artifacts [12] [21].
• Synchronization Errors: Ensure the synchronization error between your multi-modal signals is less than 100 ms. Using a dedicated synchronization protocol from the acquisition system is critical for valid joint analysis [35].

FAQ 3: My OpenBCI/Cyton board is showing a 'RAILED' error in the GUI. What does this mean and how do I fix it?

A 'RAILED' error indicates that the input signal is exceeding the dynamic range of the analog-to-digital converter, causing the signal to "rail" or clip at the maximum or minimum voltage.

• Solution: This is typically resolved by selecting a smaller gain on the affected Cyton channels.
  • In the OpenBCI GUI, start a session and open the 'Hardware Settings'.
  • Lower the gain setting from the default of 24x to a lower value, such as 8x, 12x, or 16x.
  • Click 'Send' to apply the new settings. The optimal gain varies depending on the individual user's skin impedance [107].

FAQ 4: What advanced machine learning models are recommended for improving MI classification performance beyond traditional methods?

While Linear Discriminant Analysis (LDA) and Support Vector Machines (SVM) are common, recent advances in deep learning offer significant improvements.

• Transformer-based Networks: These models, with their self-attention mechanisms, can capture long-range dependencies in EEG time-series data, leading to superior classification performance for MI tasks [105].
• Specialized Architectures: Models like EEGNet and ConvLSTM are explicitly designed for EEG signal processing, offering a good balance between performance and computational efficiency [105].
• Riemannian Geometry-Based Approaches: These methods can extract more robust spatial features from the covariance matrices of EEG signals, which are often very effective for MI-BCI [105].

Experimental Protocols & Methodologies

Protocol 1: Calibration Data Collection for Motor Imagery BCI

This protocol is essential for training a user-specific MI classifier [104] [105].

• Equipment Setup: Use an EEG system (e.g., OpenBCI Cyton, Explore Pro) with electrodes placed on the 10-20 locations C3, Cz, C4, and surrounding sites. Ensure impedances are below 5 kΩ for high-quality data [104] [106].
• Software Configuration: Stream EEG data using the Lab Streaming Layer (LSL) protocol. Use a presentation script (e.g., motorimg_calibrate.py) to display visual cues ('L' for left hand, 'R' for right hand) and record corresponding marker streams.
• Paradigm:
  • Each trial begins with a fixation cross.
  • A visual cue is displayed instructing the user which movement to imagine.
  • The user engages in kinesthetic motor imagery for a defined period (e.g., 3 seconds).
  • A rest period follows before the next trial.
• Data Specifications: Collect a minimum of 20-50 trials per class, presented in a randomized (non-blocked) order to prevent the classifier from learning the sequence rather than the brain signals [104].

Protocol 2: Synchronous EEG-fNIRS Data Preprocessing Pipeline

This protocol outlines a standardized workflow for fusing EEG and fNIRS data, critical for neurovascular coupling analysis [35] [21].

Table 1: Synchronous EEG-fNIRS Preprocessing Steps

Step	Modality	Toolbox	Parameters & Actions
Data Import & Synchronization	EEG & fNIRS	Custom Script	Align data streams using recorded timing markers.
Preprocessing	EEG	EEGLAB	1. Downsample to 250 Hz.2. High-pass filter at 1 Hz.3. Remove line noise (cleanline).4. Reject bad channels and interpolate.5. Apply Artifact Subspace Reconstruction (ASR).6. Re-reference to global average [21].
	fNIRS	HOMER3	1. Convert light intensity to optical density.2. Detect and correct motion artifacts (e.g., Savitzky-Golay filter).3. Bandpass filter (0.01 - 0.1 Hz).4. Convert to chromophore concentration (HbO, HbR) using the Modified Beer-Lambert Law [12] [21].
Feature Extraction	EEG	EEGLAB	Filter into classic frequency bands: Delta (1-4Hz), Theta (4-8Hz), Alpha (8-13Hz), Beta (13-30Hz), Gamma (30-40Hz) [106].
	Joint	Custom Script	Perform Multi-Band Local Neurovascular Coupling (MBLNVC) analysis by reconstructing EEG source activity onto fNIRS channel locations [35].

The following workflow diagram illustrates the integrated preprocessing pipeline for synchronous EEG-fNIRS data:

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 2: Key Hardware and Software for MI-BCI and fNIRS-EEG Research

Item Name	Category	Specification / Version	Primary Function in Research
OpenBCI Cyton Board	Hardware	Board with Daisy Module	A versatile, bio-sensing board for high-quality EEG data acquisition [104].
Explore Pro	Hardware	64-channel EEG device	High-density EEG recording system compatible with LSL for real-time data streaming [105].
fNIRS System	Hardware	e.g., NirSmart-6000A	Measures cortical hemodynamic changes (HbO/HbR) via near-infrared light [35].
Lab Streaming Layer (LSL)	Software	Protocol	A unified system for the simultaneous recording of brain signals and event markers across multiple devices [104] [105].
NeuroPype	Software	Suite	A comprehensive software for designing and executing BCI pipelines, including CSP and GLM [104].
EEGLAB	Software	Toolbox	An open-source MATLAB toolbox for processing EEG signals, including ICA for artifact removal [35] [21].
HOMER3	Software	Toolbox	An open-source package for converting, processing, and visualizing fNIRS data [21].

Table 3: Standardized Filtering Parameters for Preprocessing

Noise Type	Modality	Filter Type	Recommended Cut-off Frequencies	Purpose
Low-Frequency Drift	EEG	High-Pass	1 Hz	Remove slow signal drifts [21].
Powerline Noise	EEG	Notch	50 Hz / 60 Hz	Remove mains electricity interference [105].
High-Frequency Noise	EEG	Low-Pass	40 Hz	Remove muscle and other high-freq. artifacts [35].
Neurovascular Coupling Signal	fNIRS	Bandpass	0.01 - 0.1 Hz	Isolate the functional hemodynamic response [12] [21].
Systematic Physiological Noise	fNIRS	Bandpass / Wavelet	Custom (e.g., 0.5-2 Hz for cardiac)	Remove cardiac, respiratory, and Mayer wave influences [12].

Table 4: Comparison of Classifiers for Motor Imagery BCI

Classifier	Key Advantages	Key Disadvantages	Reported Accuracy Range
Linear Discriminant Analysis (LDA)	Simple, fast, works well on linearly separable data.	Assumes normal distribution and equal covariance; struggles with complex patterns.	Foundational algorithm, widely used [105].
Support Vector Machine (SVM)	Effective in high-dimensional spaces; versatile with kernel functions.	Can be computationally intensive; performance depends on kernel choice.	Can achieve high accuracy with good features [105].
Random Forest (RF)	Robust to overfitting; handles non-linear relationships well.	Less interpretable than linear models; can be computationally heavy.	~80.5% (as reported in [35])
EEG-Specific Transformers	Captures complex temporal dependencies; state-of-the-art performance.	Requires large datasets; computationally intensive to train.	~82.1% (SVM in a fusion study [35])

Cognitive State Decoding in Working Memory Tasks

Frequently Asked Questions (FAQs)

FAQ 1: What are the primary advantages of using a combined fNIRS-EEG system over either modality alone for cognitive state decoding?

The primary advantage is the synergistic combination of their complementary strengths. EEG provides direct measurement of neural electrical activity with high temporal resolution (millisecond precision), making it ideal for capturing rapid cognitive processes. fNIRS measures the hemodynamic response linked to neural activity with better spatial resolution (around 1 cm), helping to localize the brain regions involved. Using them together provides a more comprehensive picture of brain function, capturing both the fast dynamics and the localized areas of activation [108] [109] [110].

FAQ 2: My decoding performance is poor. Could my EEG preprocessing choices be the cause?

Yes, preprocessing choices significantly influence decoding performance. A 2025 study systematically varying preprocessing steps found that choices like filtering and artifact correction considerably impact classification results [111].

• Filters: Using a higher high-pass filter cutoff consistently increased decoding performance.
• Artifact Correction: Methods like ICA and Autoreject generally decreased decoding performance. This can occur because these artifacts are sometimes systematically related to the task, and removing them also removes this predictive signal. However, not correcting for artifacts sacrifices interpretability, as the model might be relying on structured noise rather than neural signals [111].
• Other Steps: Baseline correction and linear detrending also showed effects, which could vary based on the specific experiment or classifier used [111].

FAQ 3: How reproducible are fNIRS analysis results, and what factors affect them?

Reproducibility in fNIRS can be variable and is influenced by several factors. A large-scale initiative (FRESH) found that nearly 80% of research teams agreed on group-level results when hypotheses were strongly supported by literature. However, agreement was lower at the individual level. Key sources of variability include [53]:

• Data Quality: Reproducibility improves with better data quality.
• Analysis Pipelines: How poor-quality data is handled, how hemodynamic responses are modeled, and how statistical analyses are conducted are major drivers of differing results.
• Researcher Experience: Teams with higher self-reported confidence, which correlated with more years of fNIRS experience, showed greater agreement [53].

FAQ 4: How can I effectively synchronize EEG and fNIRS systems during data acquisition?

Precise synchronization is crucial for multimodal analysis. Two common methods are:

• Shared Hardware Triggers: Using a shared electrical signal (e.g., a TTL pulse via a parallel port) to mark events in both systems simultaneously [110].
• Software-Based Synchronization: Using a communication protocol like the Lab Streaming Layer (LSL), which allows unified collection of data streams from different devices and software-based event marking [110]. Some systems use a unified processor to acquire both signals, which achieves high-precision synchronization but requires a more complex system design [109].

FAQ 5: What is the best way to handle motion artifacts in fNIRS signals?

Motion artifacts are a significant challenge and can be addressed through hardware and algorithmic solutions.

• Hardware-Based Solutions: Using accelerometers attached to the head is a common approach. The accelerometer data serves as a reference for the motion, which can then be used with techniques like Adaptive Filtering or Active Noise Cancelation to remove the artifact from the fNIRS signal [37].
• Algorithmic Solutions: Many signal processing techniques exist, including:
  • Moving Average / Savitzky-Golay Filtering: Effective for smoothing and removing high-frequency motion spikes [21] [37].
  • Wavelet-Based Methods: These can identify and remove artifacts in the time-frequency domain [37].
  • PCA/ICA: These methods attempt to separate the signal into statistically independent components, allowing you to remove those correlated with motion [37].

Troubleshooting Guides

Issue 1: Poor Classification Accuracy in Neural Decoding

Problem: Your machine learning model fails to reliably decode cognitive states (e.g., different levels of working memory load) from your fNIRS-EEG data.

Potential Causes and Solutions:

• Cause: Inadequate Preprocessing.

  • Solution: Systematically optimize your preprocessing pipeline. For EEG, test the impact of different high-pass filters (e.g., 1 Hz vs. 0.5 Hz) and evaluate whether artifact correction methods like ICA are necessary or detrimental for your specific task and decoder [111]. For fNIRS, ensure motion artifacts are properly corrected using methods mentioned in FAQ 5 [37].

• Cause: Non-Informative Features.

  • Solution: Move beyond basic features. For EEG, consider extracting neural synchronization features such as:
    • Functional Connectivity: Measure the synchrony between different brain regions using metrics like phase-locking value or coherence [112].
    • Cross-Frequency Coupling: Examine the interaction between oscillations of different frequencies [112].
    • Neural-Behavioral Synchronization: Explore the mutual information between neural signals and behavioral outcomes [112]. Combining these with traditional power spectral features has been shown to improve classification accuracy for cognitive states like mind-wandering [112].

• Cause: Poor Experimental Design for Multimodality.

  • Solution: Design your task to accommodate the different temporal scales of EEG and fNIRS. Combine event-related designs (optimal for EEG/ERP analysis) with block designs (optimal for fNIRS hemodynamic response analysis). For example, present individual trials for EEG markers, but group these trials into longer blocks for robust fNIRS analysis [110].

Issue 2: Low Signal Quality in Simultaneous Recordings

Problem: The recorded EEG or fNIRS signals are excessively noisy when the systems are used together.

Potential Causes and Solutions:

• Cause: Physical Interference Between Sensors.

  • Solution: Use a specialized cap designed for multimodal recordings. The cap should have a large number of slits to flexibly host both EEG electrodes and fNIRS optodes. It is recommended to use a cap with a black fabric to reduce unwanted optical reflections for fNIRS [110]. Ensure holders are securely placed to maintain consistent optode-scalp contact pressure [109].

• Cause: Persistent Motion Artifacts.

  • Solution: For fNIRS, implement a robust motion artifact correction pipeline. A common and effective approach is to use an accelerometer-based method like ABAMAR or Active Noise Cancelation, as the hardware provides a direct measure of the motion causing the artifact [37]. For EEG, ensure that the cap is fitted tightly and comfortably to minimize movement.

• Cause: Improper Referencing or Filtering in EEG.

  • Solution: Experiment with different referencing schemes (e.g., common average reference, Cz reference) during preprocessing, as this choice can affect decoding performance [111]. Also, verify that your band-pass filter settings are appropriate for your analysis (e.g., a passband of 0.1-40 Hz for ERPs).

Experimental Protocols & Methodologies

The following table summarizes a representative experimental protocol for semantic decoding, which can be adapted for working memory tasks involving different types of mental imagery.

Table 1: Experimental Protocol for Semantic Category Decoding with Mental Imagery

Component	Description
Core Task	Differentiate between semantic categories (e.g., Animals vs. Tools) during various mental imagery tasks [113].
Participants	Native speakers (if task involves silent naming); right-handed individuals; normal/corrected-to-normal vision [113].
Stimuli	18 images from each category (e.g., Animals: cat, dog, elephant; Tools: hammer, saw, scissors). Images are gray-scaled, cropped, and presented on a white background [113].
Mental Tasks	1. Silent Naming: Silently name the object.2. Visual Imagery: Visualize the object in the mind.3. Auditory Imagery: Imagine the sounds associated with the object.4. Tactile Imagery: Imagine the feeling of touching the object [113].
Trial Structure	1. Stimulus presentation.2. Cued mental task execution for a fixed duration (e.g., 3-5 seconds).3. Inter-trial interval [113].
Data Acquisition	Record simultaneous EEG and fNIRS. Instruct participants to minimize physical movements during the mental task period [113].

Diagram: Simultaneous fNIRS-EEG Experimental Workflow

The diagram below illustrates the logical workflow for a typical simultaneous fNIRS-EEG experiment, from setup to data fusion.

The Scientist's Toolkit: Essential Research Reagents & Materials

Table 2: Key Materials for fNIRS-EEG Fusion Research

Item	Function in Research
Simultaneous EEG-fNIRS Cap	A specialized headcap (e.g., actiCAP with 128+ slits) that physically accommodates both EEG electrodes and fNIRS optodes, often using the international 10-20 system for co-registration [109] [110].
fNIRS System	Hardware that emits near-infrared light and detects its attenuation after passing through brain tissue to measure changes in oxygenated and deoxygenated hemoglobin concentrations, providing an indirect measure of neural activity [108] [109].
EEG System	Hardware that amplifies and records electrical potentials from the scalp, providing a direct, high-temporal-resolution measure of neural population activity [108] [109].
Accelerometer / IMU	Auxiliary hardware often attached to the acquisition cap to provide a reference signal for motion, which is crucial for effective motion artifact removal in fNIRS data [37].
Synchronization Interface	Hardware (e.g., parallel port for TTL triggers) or software (e.g., Lab Streaming Layer - LSL) that ensures temporal alignment of events and data streams between the EEG and fNIRS systems [109] [110].
Preprocessing Software Toolboxes	Open-source software packages like EEGLab (for EEG) and HOMER3 (for fNIRS) that provide standardized functions for filtering, artifact removal, and other preprocessing steps [21].

Data Presentation and Preprocessing Pipelines

The tables below summarize key quantitative data and preprocessing steps for both modalities.

Table 3: Characteristic Comparison of EEG and fNIRS

Feature	EEG	fNIRS
What It Measures	Electrical activity from post-synaptic potentials	Hemodynamic response (HbO & HbR concentration)
Temporal Resolution	High (milliseconds) [108] [110]	Low (seconds) [108] [110]
Spatial Resolution	Low (centimeter-level) [108]	Moderate (better than EEG, ~1 cm) [108] [110]
Sensitivity to Motion	High [108]	Low to Moderate [108]
Best Use Cases	Fast cognitive tasks, ERPs, rapid neural dynamics [108]	Sustained cognitive states, localization, naturalistic studies [108]

Diagram: Complementary Nature of EEG and fNIRS Signals

This diagram conceptualizes the complementary relationship between EEG and fNIRS signals in measuring brain activity.

Table 4: Example Preprocessing Steps for EEG and fNIRS

Step	EEG Protocol [21]	fNIRS Protocol [21]
Downsampling	To 250 Hz	-
Filtering	High-pass at 1 Hz; Remove line noise	Bandpass filter (e.g., 0.01 - 0.1 Hz) for neurovascular coupling
Artifact Removal	Clean rawdata; ASR; ICA	Motion artifact correction (e.g., Savitzky-Golay filtering, PCA/ICA)
Other Steps	Re-referencing (e.g., to global average); Spherical spline interpolation for bad channels	Convert optical density to chromophore concentration (e.g., HbO)

Troubleshooting Guides & FAQs

Q1: Our hybrid EEG-fNIRS classification accuracy for stroke patients is poor. What are the main challenges and solutions?

A: Poor classification accuracy in patient populations often stems from neurophysiological heterogeneity and signal artifacts. Key challenges and solutions include:

• Challenge: Pathological Neurovascular Uncoupling. In ICH patients, the relationship between neural electrical activity and hemodynamic response can be disrupted [97] [114].
  • Solution: Do not assume a fixed neurovascular coupling model. Instead of simple feature concatenation, use advanced fusion methods like mutual information-based feature selection to identify complementary features from each modality, optimizing for complementarity, redundancy, and relevance to the clinical label (e.g., healthy vs. patient) [101].
• Challenge: High Cross-Subject Variability. Models trained on healthy subjects often fail to generalize to patients due to divergent neural signatures [97] [115].
  • Solution: Implement transfer learning. One effective approach uses a Wasserstein metric to quantify neural distribution divergence and select optimal source domains from healthy templates for transfer to patient data, significantly improving accuracy [115].
• Challenge: Low Signal-to-Noise Ratio (SNR) in Patient Data.
  • Solution: For fNIRS, ensure proper preprocessing: convert raw intensity to optical density, then to hemoglobin concentrations (HbO/HbR) using the modified Beer-Lambert law, followed by band-pass filtering (e.g., 0.05–0.7 Hz) to remove cardiac noise and slow drifts [22] [23]. For EEG, apply high-pass (≥1 Hz) and low-pass (≤35 Hz) filters and use Independent Component Analysis (ICA) to remove ocular and muscle artifacts [116] [117].

Q2: How do we validate that our fused EEG-fNIRS features are truly capturing clinically relevant biomarkers?

A: Clinical validation requires linking multimodal features to established clinical scales and outcomes.

• Correlate with Gold-Standard Assessments. Extract known neurophysiological features and test their correlation with clinical scores. For motor recovery, Event-Related Desynchronization (ERD) in the EEG alpha/beta bands and oxygenated hemoglobin (HbO) concentration from fNIRS have been successfully used as predictors in linear regression models to predict balance function scores like the Berg Balance Scale (BBS) [116].
• Leverage Quantitative EEG (qEEG) Parameters. Parameters like the Brain Symmetry Index (BSI) and Power Ratio Index (PRI) are validated biomarkers. Higher BSI (indicating inter-hemispheric asymmetry) and higher PRI (more slow-wave activity) are correlated with worse motor outcomes on scales like the Fugl-Meyer Assessment (FMA) and Modified Rankin Scale (mRS) [114].
• Benchmark Performance. Compare your fusion model's classification accuracy against unimodal baselines. Clinically validated hybrid systems have demonstrated improvements of 5-10% in classification accuracy compared to using EEG or fNIRS alone [101] [97] [115].

Table 1: Clinically Validated Multimodal Biomarkers for Stroke/ICH Recovery

Biomarker	Modality	Clinical Correlation	Associated Clinical Scale
Power Ratio Index (PRI)	EEG	Higher PRI predicts poorer functional motor outcome [114].	FMA, mRS
Brain Symmetry Index (BSI)	EEG	Higher asymmetry correlates with worse neurological status and motor function [114].	NIHSS, FMA-UE
Event-Related Desynchronization (ERD)	EEG	Magnitude correlates with residual motor function in the paretic arm [116].	BBS, FMA
HbO Concentration	fNIRS	Increased activation in motor areas correlates with better balance function [116].	BBS
Multimodal Fusion Accuracy	EEG-fNIRS	~5-10% improvement in classification accuracy over unimodal approaches [101] [97].	Diagnostic/Prediction Accuracy

Table 2: Key Datasets for Clinical Validation of EEG-fNIRS Fusion

Dataset Name	Population	Task	Key Clinical Relevance
HEFMI-ICH [97]	17 Normal subjects, 20 ICH patients	Left/Right hand Motor Imagery (MI)	First hybrid dataset specifically for ICH rehabilitation research.
Private Data [115]	17 Normal controls, 13 ICH patients	Motor Imagery (MI)	Enables transfer learning from healthy templates to patient data.

Experimental Protocols

Protocol 1: Motor Imagery for ICH Rehabilitation [97] [115]

• Objective: To collect synchronized EEG-fNIRS data for developing and validating MI-based BCI classifiers in ICH patients.
• Participants: Healthy controls and patients with intracerebral hemorrhage.
• Equipment: A 32-channel EEG amplifier (e.g., g.HIamp) and a continuous-wave fNIRS system (e.g., NirScan) integrated into a single cap. fNIRS source-detector pairs should be placed for coverage of sensorimotor cortices.
• Paradigm:
  • Preparation: Conduct a grip strength calibration with a dynamometer or stress ball to enhance the kinesthetic sensation and vividness of MI.
  • Trial Structure:
    • Visual Cue (2s): A directional arrow (left/right) is displayed.
    • Execution (10s): Participants perform kinesthetic MI of grasping with the indicated hand at ~1 grasp/second.
    • Rest (15s): Blank screen for baseline recovery.
  • Session: At least 2 sessions, each with 15 trials per hand (60 trials total minimum).
• Data Acquisition: EEG sampled at 256 Hz, fNIRS at 11 Hz. Temporal synchronization is achieved using event markers from stimulus presentation software (e.g., E-Prime 3.0).

Protocol 2: Ankle Dorsiflexion for Balance Prediction [116]

• Objective: To predict balance function in stroke survivors using EEG and fNIRS features during a ankle movement task.
• Participants: Stroke survivors with lower limb motor deficits.
• Clinical Assessment: Prior to the experiment, balance is assessed using the Berg Balance Scale (BBS).
• Task:
  • Participants are seated with feet flat on the ground.
  • A single trial lasts 11.5 seconds:
    • Preparation (1s)
    • Dynamic Ankle Dorsiflexion (2.5s)
    • Static Hold (3s)
    • Rest (5s)
  • Multiple sessions (e.g., 5) with 10 trials each are conducted, with rest between sessions.
• Key Measured Features: ERD from EEG (at Cz electrode) and HbO from fNIRS in the motor cortex.

Signaling Pathways & Workflows

EEG-fNIRS Clinical Validation Pipeline

Troubleshooting Low Classification Accuracy

The Scientist's Toolkit

Table 3: Essential Research Reagents & Solutions for EEG-fNIRS Clinical Studies

Item	Function/Description	Example/Specification
Hybrid EEG-fNIRS Cap	Integrated headgear with co-located electrodes and optodes for simultaneous data acquisition.	Custom-designed cap with 32 EEG electrodes, 32 fNIRS sources, and 30 detectors [97].
Synchronization Interface	Ensures temporal alignment of EEG and fNIRS data streams with experimental events.	E-Prime 3.0 software sending event markers to both acquisition systems [97] [116].
Clinical Assessment Scales	Standardized tools to quantify patient impairment and recovery for biomarker validation.	Fugl-Meyer Assessment (FMA), Berg Balance Scale (BBS), Modified Rankin Scale (mRS) [97] [114] [116].
Mutual Information Toolbox	Computational tool for feature selection to maximize complementarity and minimize redundancy.	Used to optimize fused feature subsets for improved classifier performance [101].
Structured Sparse Multiset CCA (ssmCCA)	Advanced data fusion algorithm to identify brain regions consistently active in both EEG and fNIRS.	Used to fuse electrical and hemodynamic responses and pinpoint shared neural regions [63].

Benchmarking on Public Datasets and the Role of Synthetic Data

Troubleshooting Guides & FAQs

Frequently Asked Questions

Q1: What are the most common pitfalls when starting with a public fNIRS-EEG dataset, and how can I avoid them? A1: The most common pitfalls involve ignoring temporal misalignment and modality-specific artifacts. fNIRS has an inherent physiological delay compared to EEG; fNIRS measures slow hemodynamic responses (peaking 5-8 seconds post-stimulus), while EEG captures instantaneous electrical activity [8] [118]. To avoid this:

• Always verify synchronization markers in your data acquisition software.
• Apply appropriate temporal alignment techniques during preprocessing, considering the hemodynamic response function for fNIRS.
• Be aware that EEG is highly susceptible to electrical noise and muscle artifacts, whereas fNIRS is more robust to these but suffers from systemic physiological noise like cardiac and respiratory cycles [8].

Q2: My hybrid model isn't performing better than my unimodal model. What could be wrong? A2: This often indicates an issue with feature-level fusion or model architecture. Simply concatenating features from both modalities does not guarantee better performance.

• Check feature compatibility: Ensure the features extracted from EEG and fNIRS are on comparable scales. Z-score standardization is often recommended.
• Consider the information overlap: The power of hybrid systems comes from complementary information. If your task is best solved by fast temporal dynamics (EEG's strength) and does not benefit from improved spatial localization (fNIRS's strength), the gains may be minimal [118].
• Explore advanced fusion architectures: Instead of simple feature concatenation, try models that can learn joint representations, such as attention-based mechanisms that dynamically weight the contribution of each modality [119].

Q3: I have limited data for my specific task. How can synthetic data help? A3: Synthetic data can significantly enhance training sets, mitigating overfitting and improving model generalization.

• For data augmentation, generative models like Denoising Diffusion Probabilistic Models (DDPM) can create realistic EEG-fNIRS samples that expand your training diversity [119].
• To address the scarcity of clinical data, synthetic datasets can be generated to simulate pathological conditions, allowing for preliminary algorithm development where real patient data is scarce [8].
• A framework combining DDPM with added Gaussian noise has been shown to provide both diversity and robustness, enhancing classifier performance on tasks like motor imagery and mental arithmetic [119].

Troubleshooting Common Experimental Issues

Issue 1: Poor Classification Accuracy in Motor Imagery Tasks

• Potential Cause: Ineffective feature extraction or high intra-subject variability.
• Solution:
  • Validate data quality: Confirm the presence of expected physiological patterns. For EEG, look for event-related desynchronization (ERD) in the mu (8-13 Hz) and beta (13-30 Hz) rhythms over the sensorimotor cortex. For fNIRS, confirm a clear increase in oxygenated hemoglobin (HbO) in the same area during motor imagery [93].
  • Leverage multimodal information: Use fNIRS's spatial specificity to guide or confirm the EEG analysis. If fNIRS shows strong lateralized activation, it can reinforce the EEG features from corresponding electrodes.
  • Apply data augmentation: Use techniques like adding Gaussian noise or employing generative models (e.g., DDPM) to artificially increase your trial count and improve model robustness [119].

Issue 2: Handling Motion Artifacts and Physiological Noise

• Potential Cause: Subject movement and inherent biological signals corrupting the neural data of interest.
• Solution:
  • For EEG: Implement robust artifact removal pipelines like Independent Component Analysis (ICA) to identify and remove components related to eye blinks (EOG) and muscle activity (EMG) [8].
  • For fNIRS:
    • Use short-separation channels (e.g., 8 mm) to regress out the systemic physiological noise originating from the scalp [8] [17].
    • Apply motion artifact correction algorithms (e.g., wavelet-based, correlation-based signal improvement).
  • Multimodal Advantage: The modalities can be used complementarily; for instance, fNIRS can help interpret EEG signals contaminated by hemodynamic-based artifacts, and vice versa [8].

Issue 3: Synchronizing EEG and fNIRS Data Streams

• Potential Cause: Hardware latencies or unsynchronized data recording systems.
• Solution:
  • Design Phase: Use a single trigger source (e.g., from your stimulus presentation software like E-Prime) to send simultaneous event markers to both the EEG and fNIRS amplifiers [97].
  • Post-processing Phase:
    • Manually align the marker channels from both systems based on a shared, well-defined event (e.g., trial onset).
    • For fine-tuning, cross-correlate inherent physiological signals present in both modalities, such as the cardiac cycle (EEG's ECG channel vs. fNIRS's pulse signal) [8].

Benchmarking on Public Datasets

The table below summarizes key publicly available datasets for fNIRS-EEG fusion research, highlighting their scope and application.

Dataset Name	Modality	Subjects & Cohorts	Key Tasks	Key Specifications	Primary Research Use
Multi-modal EEG-fNIRS [93]	EEG & fNIRS	18 Healthy	8 MI tasks of hand, wrist, elbow, shoulder	64-channel EEG, 24-channel fNIRS; 5760 trials	Developing decoding algorithms for multi-joint MI
HEFMI-ICH [97]	EEG & fNIRS	17 Healthy, 20 ICH Patients	Left/Right hand MI	32-channel EEG, 90-channel fNIRS; synchronized acquisition	ICH rehabilitation, clinical BCI translation
WBCIC-MI [120]	EEG	62 Healthy	2-class & 3-class MI (hand, foot)	64-channel EEG; multi-session (3 days)	Cross-session and cross-subject MI-BCI research
Synthetic fNIRS-EEG [8]	Synthetic fNIRS & EEG	Simulated	Finger-tapping motor task	Simulates shared neuronal source with ground truth	Method development and validation for fusion algorithms

Experimental Protocols for Key Studies

1. Protocol: Multi-joint Motor Imagery [93]

• Purpose: To collect a benchmark dataset for decoding eight different motor imagery tasks from the same upper limb.
• Subjects: 18 right-handed, healthy subjects.
• Paradigm:
  • Each trial (18-20s) began with a 2s fixation cross.
  • A text and video cue was shown for 2s, indicating the MI task (e.g., hand open/close, wrist flexion).
  • Subjects performed kinesthetic motor imagery for 4s.
  • A random 10-12s rest period was included to allow fNIRS signals to return to baseline.
• Data Collection: 64-channel EEG (1000 Hz) and 24-channel fNIRS (7.8125 Hz) were recorded simultaneously from the left hemisphere.
• Outcome: The dataset provides 5760 trials. A baseline classification accuracy of 65.49% was achieved between hand and shoulder MI tasks using a deep-learning model.

2. Protocol: HEFMI-ICH Clinical Dataset [97]

• Purpose: To collect hybrid data for intracerebral hemorrhage (ICH) rehabilitation research.
• Subjects: 17 healthy controls and 20 ICH patients.
• Paradigm:
  • To enhance MI vividness, a grip strength calibration with a dynamometer and stress ball was performed before the experiment.
  • Each trial (27s) included a 2s visual cue, a 10s motor imagery execution period (imagining grasping at 1 Hz), and a 15s rest.
• Data Collection: A custom hybrid cap with 32 EEG electrodes and 62 fNIRS optodes (resulting in 90 channels) was used. Systems were synchronized via triggers from E-Prime.
• Outcome: Provides a rare resource of synchronized EEG-fNIRS from a clinical stroke population, enabling the development of patient-specific algorithms.

The Role of Synthetic Data

Synthetic data generation addresses the critical challenge of data scarcity in training deep learning models, which often require large amounts of data to generalize effectively [119].

• Advanced Generation Techniques: The EFDA-CDG framework uses a Denoising Diffusion Probabilistic Model (DDPM) combined with traditional Gaussian noise addition to augment EEG-fNIRS data. This approach first creates a unified spatiotemporal representation of the multimodal data before generating new, realistic samples [119].
• Proven Efficacy: This method has been validated on public databases, achieving high classification accuracies (e.g., 82.02% for motor imagery) in a participant-dependent training setup. The generated samples show high similarity to original data and enhance classifier robustness [119].
• Bridging the Clinical Data Gap: Realistic synthetic datasets simulating a finger-tapping task with a known ground truth have been created to help overcome the scarcity of multimodal public data, particularly for method development and validation [8] [17].

The following diagram illustrates a hybrid data processing and augmentation pipeline that integrates both real and synthetic data to improve model performance.

The Scientist's Toolkit: Research Reagent Solutions

This table lists essential tools and software used in the featured experiments for fNIRS-EEG research.

Tool / Solution	Function / Description	Example Use in Research
Neuroscan SynAmps2	EEG signal amplifier for high-quality data acquisition.	Used with a 64-channel cap for EEG recording at 1000 Hz [93].
NIRScout System (NIRx)	fNIRS device for measuring hemodynamic responses.	Configured with 8 sources and 8 detectors to create 24 fNIRS channels [93].
g.HIamp (g.tec) & NirScan	Synchronized hybrid EEG-fNIRS acquisition systems.	Employed for simultaneous recording in clinical ICH studies [97].
E-Prime	Software for designing and running experimental paradigms.	Used to present visual/auditory cues and send trigger markers to both EEG and fNIRS systems [97].
Denoising Diffusion Probabilistic Model (DDPM)	A deep generative model for creating synthetic data.	Applied for EEG-fNIRS data augmentation to improve classifier generalization [119].
ShallowConvNet / EEGNet	Deep learning models specifically designed for EEG and time-series classification.	Served as baseline models for benchmarking classification performance on MI tasks [93] [120].

Conclusion

The effective fusion of fNIRS and EEG hinges on robust, standardized preprocessing pipelines that handle the distinct artifacts and characteristics of each modality while leveraging their complementary information through neurovascular coupling. Methodological advancements in data-driven fusion, cross-modal attention, and deep learning are significantly enhancing the spatiotemporal resolution and decoding accuracy of multimodal brain signals. Looking forward, the development of low-code, reproducible software frameworks and the creation of comprehensive, clinically relevant datasets will be crucial for translating these techniques from research labs into practical tools for drug development, personalized neurorehabilitation, and a deeper understanding of brain function in both healthy and pathological states.

References

• 1. EEG brain waves and alpha rhythms: Past, current and ... [https://www.sciencedirect.com/science/article/pii/S0149763425002891]
• 2. EEG brain waves and alpha rhythms: Past, current ... [https://pubmed.ncbi.nlm.nih.gov/40669525/]
• 3. Editorial: EEG rhythms: decoding the evolutionary enigma ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1560294/full]
• 4. EEG Trouble Shooting | Center for ... [https://www.centerforsleepandcognition.com/eeg-trouble-shooting]
• 5. Two common issues in synchronized multimodal ... [https://www.sciencedirect.com/science/article/pii/S0168010223002201]
• 6. Two common issues in synchronized multimodal ... [https://pubmed.ncbi.nlm.nih.gov/38141782]
• 7. Selecting methods for a modular EEG pre-processing ... [https://www.sciencedirect.com/science/article/pii/S1746809423012636]
• 8. Multimodal fNIRS–EEG sensor fusion: Review of data-driven ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12592382/]
• 9. DeepSyncNet:Deep synchronized fusion network for EEG- ... [https://www.sciencedirect.com/science/article/abs/pii/S1746809425016118]
• 10. Preprocessing of EEG data [https://www.fieldtriptoolbox.org/workshop/nigeria2025/preprocessing/]
• 11. How fNIRS technology enables hemodynamic response [https://www.thinkiesystem.com/blog/how-fnirs-technology-enables-hemodynamic-response?srsltid=AfmBOoqoycd-vesqTP4hKg55KVnxrNQPbNODchPRFtl4R9yH3_TcqF1Q]
• 12. Data Processing in Functional Near-Infrared Spectroscopy ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC8151801/]
• 13. Investigation of the Pattern of the Hemodynamic Response ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2018.00371/full]
• 14. Optimal hemodynamic response model for functional near- ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC4468613/]
• 15. Modified Beer-Lambert Law [https://openfnirs.org/2024/01/01/modified-beer-lambert-law/]
• 16. Modified Beer-Lambert law for blood flow - PMC [https://pmc.ncbi.nlm.nih.gov/articles/PMC4242038/]
• 17. Multimodal fNIRS-EEG sensor fusion: Review of data- ... [https://pubmed.ncbi.nlm.nih.gov/41211102/]
• 18. Preprocessing functional near-infrared spectroscopy ... [https://mne.tools/1.7/auto_tutorials/preprocessing/70_fnirs_processing.html]
• 19. Enhanced spatiotemporal resolution imaging of neuronal ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7778454/]
• 20. STA-Net: Spatial–temporal alignment network for hybrid ... [https://www.sciencedirect.com/science/article/abs/pii/S156625352500096X]
• 21. fNIRS–EEG data preprocessing [https://bio-protocol.org/exchange/minidetail?id=18156217&type=30]
• 22. Preprocessing functional near-infrared spectroscopy ... [https://mne.tools/stable/auto_tutorials/preprocessing/70_fnirs_processing.html]
• 23. An Investigation of Diverse Signal Filtering Methods Within ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC6336925/]
• 24. Multimodal MBC-ATT: cross-modality attentional fusion of ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2025.1660532/full]
• 25. Neurovascular Coupling by Functional Near Infra-Red ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2020.00042/full]
• 26. fNIRS vs EEG: Comparing Non-Invasive Brain Imaging ... [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/?srsltid=AfmBOop_hyvDOIWJwc-MzUL-q-hXm_ePiMat7heGIvmZv_01Gc4ILqfV]
• 27. Fusion of fNIRS and EEG: a step further in brain research [https://www.bitbrain.com/blog/fusion-fnirs-eeg-brain-activity-research]
• 28. A Practical Guide for Combined fNIRS & EEG Recordings [https://www.gtec.at/2021/04/12/combined-fnirs-eeg-recordings/?srsltid=AfmBOoohVJLszs8nVGZqdKwGabgzH3OwPlWmPMrwxOXmmIk90BX2WEwC]
• 29. What should I know about fNIRS imaging? [https://www.biopac.com/knowledge-base/fnir-faq/]
• 30. The Advantages of Combining fNIRS and EEG for Brain ... [https://www.artinis.com/blogpost-all/advantages-of-combining-fnirs-and-eeg]
• 31. An EEG-fNIRS neurovascular coupling analysis method to ... [https://pubmed.ncbi.nlm.nih.gov/37196454/]
• 32. A Cross-Disciplinary Review of the fNIRS-EEG Dual-Modality ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11506513/]
• 33. Improved classification performance of EEG-fNIRS ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2022.973959/full]
• 34. Real time detection of cognitive load using fNIRS: A deep ... [https://www.sciencedirect.com/science/article/abs/pii/S1746809422006814]
• 35. Feature fusion analysis approach based on synchronous EEG ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11828439/]
• 36. Motion Artifacts Correction from Single-Channel EEG and ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9102309/]
• 37. Motion artifacts removal and evaluation techniques for ... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.878750/full]
• 38. Feature Extraction and Classification Methods for Hybrid ... [https://www.frontiersin.org/journals/human-neuroscience/articles/10.3389/fnhum.2018.00246/full]
• 39. fNIRS Analysis | NIRS Analysis Techniques, Homer2, NIRS ... [https://nirx.net/fnirs-analysis]
• 40. Simultaneous EEG-fNIRS Data on Learning Capability via ... [https://www.mdpi.com/2306-5729/10/8/131]
• 41. Preprocessing functional Near Infrared Spectroscopy ... [https://nimrobotics.com/blog/2025/fnirs-preprocessing/]
• 42. Cortical Signal Analysis and Advances in Functional Near ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC4899446/]
• 43. Lock-in functional near-infrared spectroscopy for ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9045899/]
• 44. Software [https://fnirs.org/resources/data-analysis/software/]
• 45. A statistically robust EEG re-referencing procedure to ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC4160811/]
• 46. a. Filtering [https://eeglab.org/tutorials/05_Preprocess/Filtering.html]
• 47. Baseline correction and re-referencing for ERPs [https://neuroimage.usc.edu/forums/t/baseline-correction-and-re-referencing-for-erps/21162]
• 48. Issues with OpenBCI EEG preprocessing pipeline or ... [https://openbci.com/forum/index.php?p=/discussion/3377/issues-with-openbci-eeg-preprocessing-pipeline-or-eeg-recording-process]
• 49. A Survey on Event-Related Potential Visualization Practices [https://apertureneuro.org/article/116386-the-art-of-brainwaves-a-survey-on-event-related-potential-visualization-practices]
• 50. The Perils of EEG Preprocessing [https://sapienlabs.org/perils-of-eeg-preprocessing/]
• 51. Investigating the interaction between EEG and fNIRS [https://www.sciencedirect.com/science/article/pii/S1877750324002096]
• 52. A Systematic Review of Techniques for Artifact Detection ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12473706/]
• 53. fNIRS reproducibility varies with data quality, analysis ... [https://www.nature.com/articles/s42003-025-08412-1]
• 54. Early-stage fusion of EEG and fNIRS improves ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9869134/]
• 55. Research on drug addiction detection based on AR-TSNET ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12542443/]
• 56. EFRM: A Multimodal EEG–fNIRS Representation-learning ... [https://www.sciencedirect.com/science/article/abs/pii/S0010482525016464]
• 57. EEG–fNIRS signal integration for motor imagery ... [https://www.sciencedirect.com/science/article/pii/S2772528625000299]
• 58. Error-related brain state analysis using ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9733771/]
• 59. Early-stage fusion of EEG and fNIRS improves ... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.1062889/full]
• 60. A bidirectional cross-modal transformer representation ... [https://www.sciencedirect.com/science/article/abs/pii/S0957417424029488]
• 61. Transformer-based deep learning model for predicting ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12618017/]
• 62. Best practices for fNIRS publications - PMC - PubMed Central [https://pmc.ncbi.nlm.nih.gov/articles/PMC7793571/]
• 63. Simultaneous multimodal fNIRS-EEG recordings reveal ... [https://www.nature.com/articles/s41598-023-31609-5]
• 64. MNE-Python Installer Error - Support & Discussions [https://mne.discourse.group/t/mne-python-installer-error/11324]
• 65. Error handling - MNE-BIDS-Pipeline [https://mne.tools/mne-bids-pipeline/dev/settings/error_handling.html]
• 66. fNIRS analysis toolbox series – MNE/Python [https://www.artinis.com/blogpost-all/2021/fnirs-analysis-toolbox-series-mne-python]
• 67. BrainFusion: a Low‐Code, Reproducible, and Deployable ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12407257/]
• 68. Decoding Analysis — MNE-NIRS 0.8.0.dev4+gddcedfe85 ... [https://mne.tools/mne-nirs/dev/auto_examples/general/plot_50_decoding.html]
• 69. Tutorials — MNE 1.11.0 documentation [https://mne.tools/stable/auto_tutorials/index.html]
• 70. Programme [https://fnirsuk.gitlab.io/programme.html]
• 71. Feature Selection Methods for Zero-Shot Learning of ... [https://www.frontiersin.org/journals/neuroinformatics/articles/10.3389/fninf.2017.00041/full]
• 72. Multimodal fusion of EEG-fNIRS: a mutual information- ... [https://opg.optica.org/boe/abstract.cfm?uri=boe-12-3-1635]
• 73. Neural Decoding and Feature Selection Techniques for ... - PMC [https://pmc.ncbi.nlm.nih.gov/articles/PMC11185693/]
• 74. Improved classification performance of EEG-fNIRS multimodal ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9388144/]
• 75. Preprocessing and averaging of single-channel NIRS data [https://www.fieldtriptoolbox.org/tutorial/nirs/nirs_singlechannel/]
• 76. Systemic physiology augmented functional near-infrared ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9272976/]
• 77. Effects of Systemic Physiology on Mapping Resting-State ... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.803297/full]
• 78. Preprocessing Troubleshooting - Forum [https://openfnirs.org/community/homer3-forum/preprocessing-troubleshooting/]
• 79. Removal of Artifacts from EEG Signals: A Review - PMC [https://pmc.ncbi.nlm.nih.gov/articles/PMC6427454/]
• 80. A Comparison of Approaches for Motion Artifact Removal ... [https://www.mdpi.com/1424-8220/25/15/4810]
• 81. EEG artifact removal with Blind Source Separation [https://sccn.ucsd.edu/~jung/Site/EEG_artifact_removal.html]
• 82. fNIRS vs EEG: Comparing Non-Invasive Brain Imaging ... [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/?srsltid=AfmBOooXzYUOYYyxj0_7RfyreMcIIk--c66vu1QfG86ft1HlUTwRBGY2]
• 83. NEAR: An artifact removal pipeline for human newborn ... [https://www.sciencedirect.com/science/article/pii/S1878929322000123]
• 84. AnEEG: leveraging deep learning for effective artifact ... [https://www.nature.com/articles/s41598-024-75091-z]
• 85. EEG is better when cleaning effectively targets artifacts [https://pubmed.ncbi.nlm.nih.gov/41092567/]
• 86. Concurrent fNIRS and EEG for Brain Function Investigation... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9371171/]
• 87. Top 5 Techniques to Achieve Multimodal Data Alignment [https://www.sapien.io/blog/5-smart-strategies-to-align-time-space-semantics]
• 88. Alignment Helps Make the Most of Multimodal Data [https://arxiv.org/html/2405.08454v3]
• 89. Text has enhanced contrast | ACT Rule | WAI [https://www.w3.org/WAI/standards-guidelines/act/rules/09o5cg/]
• 90. Text has enhanced contrast - Rules - Siteimprove [https://alfa.siteimprove.com/rules/sia-r66]
• 91. Best practices for fNIRS publications [https://www.spiedigitallibrary.org/journals/neurophotonics/volume-8/issue-01/012101/Best-practices-for-fNIRS-publications/10.1117/1.NPh.8.1.012101.full]
• 92. Preprocessing pipeline tips and tricks for EEG data - MNE Forum [https://mne.discourse.group/t/preprocessing-pipeline-tips-and-tricks-for-eeg-data/3552]
• 93. A multi-modal dataset of electroencephalography and ... [https://www.nature.com/articles/s41597-025-05286-0]
• 94. Support & Resources [https://www.brainproducts.com/support-resources/]
• 95. Ten simple rules for implementing open and reproducible ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC9815586/]
• 96. Best Practices for Improving the Computational ... [https://www.asist.org/meetings-events/webinars/best-practices-for-improving-the-computational-reproducibility-of-research/]
• 97. HEFMI-ICH: a hybrid EEG-fNIRS motor imagery dataset for ... [https://www.nature.com/articles/s41597-025-06100-7]
• 98. What Is An Automated Data Pipeline — Examples & Use Cases [https://estuary.dev/blog/what-is-an-automated-data-pipeline/]
• 99. Automated Machine Learning Pipelines: Accelerating ... [https://www.pecan.ai/blog/automated-machine-learning-pipelines/]
• 100. Protocol for Simultaneous EEG and fNIRS Measurement in ... [https://pubmed.ncbi.nlm.nih.gov/40606568/]
• 101. Multimodal fusion of EEG-fNIRS: a mutual information-based ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC7984774/]
• 102. Early-stage fusion of EEG and fNIRS improves ... [https://pubmed.ncbi.nlm.nih.gov/36699533/]
• 103. A Practical Guide for Combined fNIRS & EEG Recordings [https://www.gtec.at/2021/04/12/combined-fnirs-eeg-recordings/?srsltid=AfmBOooJMBjdZ1QdoMT0QcbXucG4151LNWm0n5iI12z8YYprmFweN26O]
• 104. Motor Imagery [https://docs.openbci.com/Deprecated/MotorImagery/]
• 105. Motor Imagery BCI [https://wiki.mentalab.com/applications/motor-imagery-bci/]
• 106. A Comprehensive Review on Critical Issues and Possible ... [https://www.mdpi.com/1424-8220/21/6/2173]
• 107. GUI Troubleshooting [https://docs.openbci.com/Troubleshooting/GUI_Troubleshooting/]
• 108. fNIRS vs EEG: Comparing Non-Invasive Brain Imaging ... [https://imotions.com/blog/learning/research-fundamentals/fnirs-vs-eeg/?srsltid=AfmBOorybaSlUKhrwXAWq8IMcu7VqIzROtdVF1l4GpI1PbMhIMPZqYzB]
• 109. A Cross-Disciplinary Review of the fNIRS-EEG Dual ... [https://www.mdpi.com/2076-3425/14/10/1022]
• 110. EEG-fNIRS Cookbook [https://www.brainproducts.com/support-resources/eeg-fnirs-cookbook/]
• 111. How EEG preprocessing shapes decoding performance [https://www.nature.com/articles/s42003-025-08464-3]
• 112. EEG synchronization signatures for decoding attentional ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12540398/]
• 113. Simultaneous EEG and fNIRS recordings for semantic ... [https://www.nature.com/articles/s41597-025-04967-0]
• 114. Current implications of EEG and fNIRS as functional neuroimaging... [https://pmc.ncbi.nlm.nih.gov/articles/PMC11629311/]
• 115. A Novel Transfer Learning‐Based Hybrid EEG‐fNIRS Brain ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12631896/]
• 116. Frontiers | Prediction of balance function for stroke based on EEG and... [https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2022.968928/full]
• 117. EEG Signal Processing for Dummies [https://www.neuroelectrics.com/blog/eeg-signal-processing-for-dummies]
• 118. Exploring Imagined Movement for Brain–Computer Interface ... [https://pmc.ncbi.nlm.nih.gov/articles/PMC12468679/]
• 119. Neurophysiological data augmentation for EEG-fNIRS ... [https://www.sciencedirect.com/science/article/abs/pii/S0169260725000112]
• 120. A multi-day and high-quality EEG dataset for motor imagery ... [https://www.nature.com/articles/s41597-025-04826-y]