Geographically Weighted Regression (GWR) is a local technique that models spatially varying relationships, where Euclidean distance is traditionally used as default in its calibration. However, empirical work has shown that the use of non-Euclidean distance metrics in GWR can improve model performance, at least in terms of predictive fit. Furthermore, the relationships between the dependent and each independent variable may have their own distinctive response to the weighting computation, which is reflected by the choice of distance metric. Thus, we propose a back-fitting approach to calibrate a GWR model with parameter-specific distance metrics. To objectively evaluate this new approach, a simple simulation experiment is carried out that not only enables an assessment of prediction accuracy, but also parameter accuracy. The results show that the approach can provide both more accurate predictions and parameter estimates, than that found with standard GWR. Accurate localised parameter estimation is crucial to GWR’s main use as a method to detect and assess relationship non-stationarity.