Benchmarking performance improvements with power data

When using power data to benchmark performance improvements, how do we account for the fact that a riders power output can be influenced by factors such as bike fit, aerodynamics, and even the type of roads they ride on, which can greatly impact the accuracy of the data. Should we be looking at average power output over a set period of time, or is there a better metric to use when comparing performance improvements over time, such as a riders ability to maintain a high power output over a longer period of time. And how do we differentiate between a genuine improvement in fitness and a rider simply becoming more efficient at generating power.