Behavioral data
d-prime
The LME model analysis of d-prime values revealed a main effect of sensory Modality (F(1,156) = 46.59, p < 0.0001, β = 0.29), with higher d-prime values for bisensory visuotactile stimuli (adj-M = 4.74 [4.50, 5.00]) compared to unisensory stimuli (adj-M = 4.12, [3.88, 4.36]; t(156) = 6.83, p < 0.0001, BF10 > 100). There was no main effect of Group (F(1,52) = 3.53, p = 0.066, β = 0.20), with HC (adj-M = 4.64, [4.32, 4.95]) not being statistically different from SZ (adj-M = 4.22, [3.90, 4.54]; t(52) = 1.88, p = 0.066, BF10 = 1.16). However, there was a main effect of Attention (F(1,156) = 13.52, p < 0.0003, β = 0.15), with tactile stimuli (adj-M = 4.59, [4.35, 4.83]) showing higher d-prime values than visual stimuli (adj-M = 4.26, [4.02, 4.50]; t(156) = 3.68, p = 0.0003, BF10 = 50.99). There was also a significant interaction between Group and sensory Modality (F(1,156) = 4.48, p = 0.036, β = −0.09), with a significant difference between HC (adj-M = 4.43, [4.09, 4.76]) and SZ (adj-M = 3.82, [3.48, 4.15]; t(69.8) = 2.54, p = 0.0135, BF10 = 3.66) for unisensory stimuli, but no evidence of a difference between HC (adj-M = 4.85, [4.51, 5.19]) and SZ (adj-M = 4.62, [4.28, 4.96]; t(69.8) = 0.95, p = 0.348, BF10 = 0.40) for bisensory stimuli. The difference of differences for these two contrasts was also significant (unisensory minus bisensory = −0.38, t(156) = −2.12, p = 0.0358, BF10 = 1.70). This overall indicates that people with SZ have impaired unisensory processing relative to HC and suggests IA deficits in patients (Fig. 3a). The criterion of responses did not show evidence of differences between groups on any of the four response categories measured with non-parametric Mann-Whitney-U tests. Bisensory visual [Medsz = 2.71 [IQR = 0.76], Medhc = 2.71 [IQR = 0.95]; W = 354, p = 0.86, BF10 = 0.30]; bisensory tactile [Medsz = 2.71 [IQR = 0.61], Medhc = 2.71 [IQR = 0.81]; W = 434.5, p = 0.74, BF10 = 0.74]; unisensory visual [Medsz = 2.48 [IQR = 1.26], Medhc = 2.71 [IQR = 1.25]; W = 374.5, p = 0.87, BF10 = 0.27]; unisensory tactile [Medsz = 2.72 [IQR = 0.81], Medhc = 2.72 [IQR = 0.69]; W = 365, p = 1, BF10 = 0.30].
a Behavioral data showing deficits in IA for unisensory visual and unisensory tactile stimuli but not bisensory VT stimuli. The figure contrasts individuals with SZ (SZ, turquoise dots) and HC (red dots). Unisensory deficits were present in both visual and tactile attention conditions. There was an overall better performance for bisensory stimuli than for unisensory stimuli across groups and sensory modalities. People with SZ performed worse than HC specifically on the unisensory stimuli, whereas there were no significant performance differences on bisensory stimuli. b Behavioral data showing RT differences for bisensory vs. unisensory stimuli. In the attend-tactile blocks, differences between bisensory and unisensory stimuli were found, with bisensory stimuli showing faster RTs than unisensory tactile stimuli. This was not apparent for the attend-visual blocks. The large dots represent mean values with the error bars representing ± 1SE. The smaller dots represent individual datapoints. Significant main effects and interactions are represented with asterisks (n.s, p > 0.05, *p < 0.05, **p < 0.01).
Reaction Time (RT)
The LME model analysis of RTs revealed a main effect of Attention (F(1,156) = 393.42, p < 0.0001, β = 0.17), with longer RTs for tactile targets (m = 0.43, [0.40, 0.47]) compared to visual targets (m = 0.39, [0.36, 0.42], t(156) = 6.72, p < 0.0001). There was no main effect of Group (F(1,52) = 0.52, p = 0.475, β = 0.09), but a main effect of sensory Modality (F(1,156) = 198.43, p < 0.0001, β = −0.12). Follow-up contrasts showed faster responses for bisensory (adj-M = 0.40, [0.36-0.43]) over unisensory stimuli (adj-M = 0.43, [0.39, 0.46], t(156) = −4.46, p < 0.0001, BF10 > 100). In addition, there was a significant interaction between Attention and Modality (F(1,156) = 46.44, p = 0.033, β = −0.06). Follow-up Bonferroni adjusted contrasts showed that in the attend-tactile condition, RTs to bisensory stimuli were significantly shorter (adj-M = .41, [0.38, 0.44]) than RTs to unisensory tactile targets (adj-M = 0.46, [0.42, 0.49], t(156) = −4.67, p < 0.0001, BF10 > 100). This was not the case for the attend-visual condition in which there was no difference in RTs between unisensory (adj-M = 0.40, [0.36, 0.43]) and bisensory (adj-M = 0.38, [0.35, 0.42]; t(156) = −1.63, p = 0.106, BF10 = 0.81) stimuli. The difference of differences for these two contrasts was also significant (attend-visual minus attend-tactile = −0.03; t(156) = −2.15, p = 0.033, BF10 = 1.79). This overall suggests an elevated difficulty in the attend-tactile compared to the attend-visual condition, which especially affected RTs to unisensory stimuli (Fig. 3b).
Oscillatory responses
Figure 4 depicts topographic plots of the surface electrode activity for ongoing alpha (8–12 Hz) and beta (13–20 Hz) activity. This overview of activity shows the strongest ongoing alpha oscillations over occipital and sensorimotor regions. For the beta oscillations, the SZ group shows the strongest variation over sensorimotor regions. These are tested in more detail in the virtual electrodes analyses of occipital and sensorimotor regions that follow. LME models were applied for relative change in alpha as well as beta band activity, testing the effects of Group as well as VOI, with individual as random factor. It is possible that our approach of averaging over the predefined TOI obscured potential dynamic changes in this response. Such changes have been found previously in studies using attentional cues43,44, and patients with SZ also have deficits in temporal expectation processing45, which may have played a role the current study. To test this, we examined power fluctuations in the prestimulus interval from -800 ms to -200 ms, but did not find systematic patterns of power changes or fluctuations differences in IA effects between groups for either alpha or beta oscillations in the three VOIs (Supplement A).
Plots show alpha (left) and beta (right) power for HZ and individuals with SZ for each attention condition as well as the relative percent difference between visual and tactile attention conditions.
Alpha power
The analysis of IA effects on ongoing alpha oscillations in the visual cortex, which we first performed for the control group only, revealed an overall effect of VOI (F(2,52) = 6.93, p = 0.002), showing differences between the attention effects across the three different target regions. Follow-up contrasts showed significant IA effects on alpha oscillations for the occipital VOI (adj-M = −9.41 [-14.54, 4.27], t(45) = −3.69, p = 0.002, BF10 = 54.34) but not for the left sensorimotor VOI (adj-M = −5.87 [-11.00, -0.73], t(45) = −2.30, p = 0.078, BF10 = 2.33) or the right sensorimotor VOI (adj-M = −1.01 [-6.14, 4.13], t(45) = −0.39, p = 1, BF10 = 0.39). The negative values here indicate that alpha oscillations were stronger in the attend-tactile condition compared to the attend-visual condition. Inspection of the graph (Fig. 5) suggests that this result could have been driven by an extreme value. However, the effects were still present when this value was removed from the analysis [main effect of VOI: (F(2,50) = 5.85, p = 0.005) occipital (adj-M = −7.39 [-11.85, 2.89], t(47.4) = −3.31, p = 0.005, BF10 = 54.31)]. Our finding is in line with previous reports of cue related IA effect on occipital alpha oscillations in healthy individuals15,33.
Left panel: The control group showed the predicted stronger alpha power in the visual cortex when attention was directed to tactile compared to when it was directed to visual stimuli VOI (left panel). This was the case for when one outlier value was removed. Right panel: There were no significant attention effects on beta power (after removing one outlier value). The small arrows with asterisks represent Bonferroni-corrected follow-up contrasts between the attention conditions, *p < 0.05, **p < 0.01,***p < 0.001 n.s. = non-significant.
Examining both study groups in the LME model analysis which included both groups, we found a main effect of VOI for alpha activity (F(2, 104) = 16.43, p < 0.0001), but not for Group (F(1, 52) = 3.23, p = 0.078) (Fig. 6). Furthermore, there was no significant interaction between VOI and Group (F(2, 104) = 0.52, p = 0.60). Follow-up Bonferroni adjusted contrasts showed that there was a greater relative change in the right sensorimotor VOI, compared to left (diff-M = −6.46, t(104) = −3.69, p = 0.001, BF10 = 55.61) and occipital (diff-M = −9.88, t(104) = −5.65, p < 0.0001, BF10 > 100) VOIs, with no evidence of difference between left and occipital VOIs (diff-M = 3.42, t(104) = 1.96, p = 0.16, BF10 = 0.68). It is possible that there are systematic differences in alpha frequency between SZ and HC participants45. Therefore, we calculated an additional analysis of the individual alpha frequency. First, we observed that the average alpha frequency was slightly lower in the SZ group (adj-M = 9.91, 9.62-10.20) than HC (adj-M = 10.35, [10.06–10.60]; t(52) = 2.17, p = 0.035, BF10 = 1.85). Thus, for the LME model analysis with individual alpha frequency, we observed the same effects as in the analysis of the predefined alpha frequency range (Supplement B).
a Time-frequency representations for the occipital VOI (left), left sensorimotor VOI (middle), the right sensorimotor VOI (right), showing visual-attention (V) and tactile-attention (T) conditions and the contrast between conditions. b Alpha (left) and beta (right) power for both study groups (HC, red, and SZ turquoise). The y-axis scale represents the relative percentage change between conditions, i.e., (V – T)/T * 100. Values below zero represents a relatively stronger power in attend-tactile condition, whereas a difference above zero represents a stronger power in attend-visual condition. Individual points represent participants, large dots show mean with ± 1 SE CI. The purple box highlights the significant Group*VOI interaction for beta power in the right sensorimotor VOI, **p < 0.01.
Beta activity
The analysis of IA effects on ongoing beta oscillations in the sensorimotor cortex in the HC group only did not reveal the predicted larger beta activity for attend-visual compared to attend-tactile conditions (m = −2.38, [-5.65, 0.90], t(54.8) = −1.45, p = 0.455, BF10 = 0.52). Furthermore, the left sensorimotor VOI showed no significant effects of attention (adj-M = −0.93, [-4.20, 2.34], t(54.8) = 0.57, p = 1, BF10 = 0.24). Hence, previous findings of cue-related IA effects on sensorimotor beta oscillations[15] could not be found for block-wise IA effects in the present study (Fig. 5). This is detailed as follows. The model’s main effect of VOI was non significant (F(2,52) = 1.77, p = 0.180). Examining relative beta activity in occipital regions for HC showed larger beta oscillations in attend-tactile condition compared to the attend-vis condition (adj-M = −4.81, [-8.43, -1.184], t(58.5) = −2.66, p = 0.031, BF10 = 3.69). However, this was outlier driven: when the extreme value was removed, this was no longer significant (adj-M = −3.29, [-6.57, -0.22], t(54.8) = −2.02, p = 0.146). In addition, in the control group, there was no significant difference in beta activity between the attend-visual and attend-tactile conditions in the right sensorimotor VOI (m = −2.38, [-5.65, 0.90], t(54.8) = −1.45, p = 0.455, BF10 = 0.52).
Examining both study groups (Fig. 6) in the LME model analysis, there was a main effect of VOI for beta activity (F(2, 104) = 9.53, p = 0.0002), but not for Group (F(1, 52) = 1.55, p = 0.219). The Group*VOI interaction was significant (F(2, 104) = 5.23, p = 0.007). Follow-up Bonferroni adjusted contrasts showed that the group difference was specific to the right sensorimotor region, with SZ (adj-M = 4.23, [1.11, 7.36]) showing higher relative beta activity than HC (adj-M = −2.71, [-5.83, 0.42], t(131) = −3.11, p = 0.0023, BF10 = 9.36), whereas left (diff-M = 0.60, t(131) = 0.27, p = 0.79, BF10 = 0.10) and occipital (diff-M = 0.22, t(131) = 0.10, p = 0.92, BF10 = 0.10) regions did not show group differences. The interaction contrasts showed that the sensorimotor group differences were greater than occipital VOIs (diff-M = 7.17, t(104) = 2.73, p = 0.008, BF10 = 3.62), and that there was no evidence of difference between occipital or left sensorimotor VOIs (diff-M = 0.37, t(104) = 0.14, p = 0.89, BF10 = 0.11).
Right sensorimotor beta activity, d-prime and RT
We tested whether the IA effect in the beta band of the right sensorimotor region could be associated with differences in d-prime and RTs reported above. The relation between the relative change in beta power for visual vs. tactile attention conditions was also analyzed in a linear mixed effects model. The RT was collapsed to a single relative variable, the relative change in RT for (v – t)/t. This was then regressed against beta band relative change ((v – t)/t), Mode (multisensory vs unisensory) and Group. Similarly, d-prime was collapsed into a single variable (v – t/t), for both unisensory and bisensory conditions. This was then regressed against Group, Mode, and IA Effect in the beta band. For both d-prime and RT, the IA effect did not predict d-prime scores, nor did it interact with the other factors (Table 2).
Right sensorimotor beta activity and BACS
Regressing BACS overall scores for both groups against relative beta power in the right sensorimotor cortex showed an overall significant model (r2 = 23.3%, F(3,50) = 6.37, p = 0.0010) as well as main effects of the IA effect as a predictor of BACS scores (β = −0.06, t = −2.28, p = 0.027, BF10 = 2.24) and of Group (β = −0.95, t = −2.99, p = 0.0043, BF10 = 9.46), with an interaction between the two variables (β = 0.27, t = 2.07, p = 0.044, BF10 = 1.56). However, this effect was driven by an outlier value (Supplement C). When the outlier was removed from the analysis, the interaction was no longer found (β = −0.26, t = 1.88, p = 0.066, BF10 = 1.56). Therefore, we did not calculate follow-up tests for this interaction.
Right sensorimotor beta activity and PANSS dimensions
For the five dimensions of the PANSS, we carried out a multivariate multiple linear model to ascertain the overall relationship between symptomatology in patients and the attentionally modulated changes in right sensorimotor beta power. The overall model was significant (F(5,21) = 4.31, p = 0.0074). Follow-up univariate tests showed that positive symptoms (adj-r2 = 17.7%, β = −0.24, F(1,25) = 6.6, p = 0.0166) and cognitive symptoms (adj-r2 = 15.7%, β = −0.14, F(1,25) = 5.85, p = 0.023) were negatively related to the IA effects (Fig. 7, yellow and blue lines). Thus, a stronger IA effect, i.e., larger differences in beta oscillations between attention conditions, predict lower pathological symptomatology in positive and cognitive dimensions of patients. The relationships between the attention effects on beta activity and the other three dimensions were not significant: (negative (adj-r2 = 8.4%, β = −0.26, F(1,25) = 3.39, p = 0.077); excitation (adj-r2 = 5.32%, β = −0.15, F(1,25) = 2.46, p = 0.129); depression (adj-r2 = 0.4%, β = 0.09, F(1,25) = 1.12, p = 0.303)).
The analysis revealed that a stronger IA effect (i.e. higher beta activity in the non-attended condition) was associated with lower SZ symptomatology in the positive and cognitive dimensions of the PANSS. Other dimensions (in gray) were not significantly predicted by relative beta power change.
