How to more effectively and comprehensively utilize recycled aggregate resources to prepare a richer new low-carbon concrete material system is the key path to achieve low-carbon development in China’s construction field. Based on the material properties of graphite tailings (GT), this article explores the organic combination of GT to replace sand and recycled aggregates (RA) to replace natural stones, in order to propose an efficient recycling path for multiple solid wastes and a new and cost-effective recycled aggregate concrete (RAC). This article focuses on investigating the influences of GT (0–40%) and RA (0–40%) on the workability, water absorption, and mechanical properties of GT reinforced recycled aggregate concrete (GTRAC). Simultaneously, the hydration products, microstructure morphology, key chemical bonds, and pore structure characteristics were analyzed by a combination of microscopic tests such as X-ray diffraction, scanning electron microscope, Fourier transform infrared and mercury intrusion porosimetry to propose the key factors affecting the macroscopic performance of GTRAC. The results show that incorporating 10–30% GT has significant positive effects on water absorption, surface water content, compressive strength, and elastic modulus of RAC. RAC’s micro-crack density, hydration, and pore structure distribution can be maximally improved by appropriate GT (20%). However, the high content of GT will lead to the degradation of its mechanical properties. Therefore, 20% GT and 30% recycled aggregate are the optimum combinations of the concrete material system. However, it might be changed when the ratio of water to binder varies. Finally, this article also comprehensively analyzes and evaluates the performance and price of different types of RACs, which provides theoretical support for evaluating and predicting the practical engineering application value of GTRAC. In summary, the research results in this study provide theoretical guidance for understanding the mechanism of GT on the enhancement of mechanical properties of RAC and exploring new ways of practical engineering application of it.