This paper uses the knowledge discovery technique to build an effective and transparent mathematic thermal error model for tool machinery. Our proposed thermal error modeling methodology (called KRL) integrates the schemes of K-means theory (KM), rough-set theory (RS), and linear regression model (LR). Firstly, to explore the machine tool’s thermal behavior, an integrated measurement system is designed to measure the temperature ascents at selected characteristic points and the thermal deformations at spindle nose at the same time under suitable real machining conditions. Secondly, the obtained data are classified by the KM method and further reduced by the RS scheme, and eventually a linear thermal error model is then established by the LR technique. To evaluate the performance of our proposed model, an adaptive neural fuzzy inference system (ANFIS) thermal error model is introduced for comparison. At last, a verification experiment is carried out and results reveal that the proposed KRL model has good predictive ability of thermal behavior in tool machinery. Our proposed KRL model takes the advantages of transparency and easy-understanding to users, and can be easily programmed as well as modified to adapt to any different machining conditions.