In many physical geography settings, principal component analysis (PCA) is applied without consideration for important spatial effects, and in doing so, tends to provide an incomplete understanding of a given process. In such circumstances, a spatial adaptation of PCA can be adopted, and to this end, this study focuses on the use of geographically weighted principal component analysis (GWPCA). GWPCA is a localized version of PCA that is an appropriate exploratory tool when a need exists to investigate for a certain spatial heterogeneity in the structure of a multivariate data set. This study provides enhancements to GWPCA with respect to: (i) finding the scale at which each localized PCA should operate; and (ii) visualizing the copious amounts of output that result from its application. An extension of GWPCA is also proposed, where it is used to detect multivariate spatial outliers. These advancements in GWPCA are demonstrated using an environmental freshwater chemistry data set, where a commentary on the use of preprocessed (transformed and standardized) data is also presented. The study is structured as follows: (1) the GWPCA methodology; (2) a description of the case study data; (3) the GWPCA application, demonstrating the value of the proposed advancements; and (4) conclusions. Most GWPCA functions have been incorporated within the GWmodel R package.