Diffusion Tensor Imaging (DTI) is an MRI-based neuroimaging technique used to measure the diffusion process of water molecules in the brain. Data from DTI scan are often summarized by a 3×3 positive definite matrix for each voxel. To make full use of the matrix-valued data, we propose a spatial Wishart process which captures spatial dependence between nearby matrices while assuming diffusion tensors marginally follow a Wishart distribution. We propose a spatial Wishart process with varying coefficients to model spatial dependence and test for covariate effects. Because the spatial Wishart process has a complicated density function, we develop approximations based on the Cholesky decomposition. Due to the computational problem caused by massive MRI data, we further adopt Nearest Neighbor Gaussian Processes (NNGP) approximation for fast computation. In simulations, we demonstrate the improved performance compared to standard methods for detecting regions of the brain affected by covariates. We also apply our method on cocaine users data and controls to detect regions of difference.