For deeply understanding the arsenic mobilization mechanisms of groundwater in Hetao Basin of Inner Mongolia, sediments and groundwater samples from two multi-level wells (up to 80 m) were analyzed. The results showed that the sediment As content from well with high As concentration (K1) is higher than the well with low As concentration groundwater (K2), with the average As concentrations 14.7 and 12.8 mg kg−1 respectively. Interestingly, we found a gray–black peat layer (around 28 m)from the K2, with arsenic concentration 322 mg kg–1, while the TOC content and TS contents are up to 9.6% and 1.8%, respectively. The pyrite also was found in this layer. We conclude that the organic matter triggers the formation of arsenic–sequestering sulphides under strongly reducing conditions, therefore immobilization of As into the sediments. By bonding arsenic in this way, the peat layer occurred in the aquifer plays an active role for arsenic immobilization into the sediments.