This research investigates how augmenting small fMRI datasets with synthetic data from a large, pre-trained model (TRIBE v2) can significantly improve the accuracy of image decoding, showing up to a 68% gain over real-data-only training.
Brain decoding is often limited by the scarcity of labeled neural data, especially in low-data regimes. To overcome this, the study explores using synthetic data generated by TRIBE v2, a large encoding model trained on over 1000 hours of fMRI responses to video, audio, and language, to augment small fMRI datasets.
The researchers evaluated systematic grids showing performance improvements based on the amount of synthetic data used. Using two datasets (7T fMRI Natural Scenes and 3T fMRI BOLD5000), the approach yielded up to 68% improvement in Top-10 image-retrieval accuracy compared to decoders trained solely on real data.
Crucially, the study suggests that the required proportion of augmented data depends on the data source. Furthermore, image decoders trained exclusively on synthetic fMRI data demonstrated the potential to perform above chance in some settings, indicating that large models like TRIBE v2 may support zero-shot brain-to-image decoding. These results establish a foundation for improving data efficiency in neuroimaging tasks through the use of large-scale multimodal models.