Physics Practical Skills Part 1 – Validity, Reliability and Accuracy of Experiments
Posted on June 29, 2017 by DJ Kim
Practical assessments are designed to test your practical skills: how well you can design and carry out an experiment and analyse results, but also your understanding of the purpose of the experiment and its limitations.
One aspect of this are the reliability, validity, and accuracy of the experiment. So what do these mean and how do they affect each other?
What is an experiment?
An experiment is a set of measurements that are analysed to test a link or relationship between different things. The measurements are usually analysed in an experimental report, which is set out in a clear format to help the reader understand different aspects of the experiment, like the aim, the equipment and method used, the results obtained, how they were analysed, and what conclusions can be drawn.
Definition of reliability
Reliability is about how close repeated measurements are to each other. You can consider the reliability of a measurement, or of the entire experiment. A measurement is reliable if you repeat it and get the same or a similar answer over and over again, and an experiment is reliable if it gives the same result when you repeat the entire experiment. This is summarised in the table below:
|Reliability of single measurements||Reliability of entire experiment’s final result|
|Definition||Repeating single measurements gives the same values.||Repeating the entire experiment gives the same final result.|
|How to improve||Through experimental method, e.g. fix control variables, choice of equipment.||Improve the reliability of single measurements and/or increase the number of repetitions of each measurement and use averaging e.g. line of best fit.|
|How to test||Repeat single measurements and look at difference in values.||Repeat entire experiment and look at difference in final results.|
You can test reliability through repetition. The more similar repeated measurements are, the more reliable the results. However, repetition alone doesn’t make your measurements reliable, it just allows you to check whether or not they are reliable.
Improving reliability is a different matter to testing it. The reliability of single measurements is not improved through repetition, but through the design of the experiment. Implementing a method that reduces random errors will improve reliability. However, the entire result of the experiment can be improved through repetition and analysis, as this may reduce the effect of random errors.
Definition of validity
Validity relates to the experimental method and how appropriate it is in addressing the aim of the experiment. “Is my experiment suitable?” or “Does it test what it’s meant to test?” or “Am I actually measuring what I’m trying to measure?”. Several aspects of the experiment can contribute to validity: the equipment, the experimental method, and the analysis of the results.
Although it may seem obvious, the appropriate equipment needs to be used. The equipment must be suitable for carrying out the experiment and taking the necessary measurements.
The experiment is ultimately testing a relationship between cause and effect: how changing X affects Y. To address this, you must only change X, and see what happens to Y. If you allow other changes at the same time, then you cannot make a valid conclusion about how X affected Y, since Y may have been affected by the other changes as well.
The correct way to describe this is in terms of the independent, dependent, and control variables. The independent variable in an experiment is the one you set (X). The dependent variable is the one you measure (Y, because it depends on X). All other variables are called control variables, and they must be kept constant to prevent them from affecting the dependent variable. This forms part of the experimental method.
The method (including the analysis) may contain some assumptions that need to be satisfied, e.g. maybe something has been simplified, or an equation being used is an approximation. The experimental method must ensure that all the assumptions are satisfied, otherwise you will end up using a method or analysis that is inappropriate, and the result will be invalid. You may be able to identify invalid measurements and discard them from the analysis.
If your experiment is invalid, then the result is meaningless because either the equipment, method or analysis were not appropriate for addressing the aim.
As examples, consider the pendulum experiment from the Year 12 Space topic. To analyse the pendulum experiment, the equation T = 2π(L/g)½ is used. This is an approximation that is valid if the angle of the swing is less than 10°. If this requirement is not adhered to in the method, the experiment will be invalid as the equation used will no longer describe the experiment. Another example, from the Motors & Generators topic, is an experiment with transformers, using the transformer equation: Np/Ns = Vp/Vs. This equation assumes there is no flux leakage, so suitable equipment such as a ferromagnetic core must be used. If this assumption is not satisfied, then the experiment will be invalid.
Validity and reliability
Reliability can be affected by the validity of the experiment.
If an experiment is invalid because of an inappropriate method being used, the result may still be reliable, it just won’t address the aim of the experiment.
However, if an experiment is invalid because the control variables are not constant, then they may be affecting measurements in an unpredictable way, making the result unreliable.
Definition of accuracy
Accuracy is much easier to define: the accuracy of an experiment is how close the final result is to the correct or accepted value. The closer it is, the more accurate the experiment. The accuracy can be improved through the experimental method if each single measurement is made more accurate, e.g. through the choice of equipment. Implementing a method that reduces systematic errors will improve accuracy.
|Accuracy of single measurement||Accuracy of entire experiment’s result|
|Definition||How close the measurement is to the value expected from theory.||How close the final experimental result is to the accepted value.|
|How to improve||Reduce systematic error by calibrating equipment.||Improve accuracy of individual measurements.|
|How to test||Compare measurement to value expected form theory.||Compare final experimental result to accepted value.|
Note that precision is a separate aspect which is not directly related to accuracy. Precision refers to the maximum resolution or the number of significant figures in a measurement. For example, a clock has a precision of 1 s, whereas a stopwatch has a precision of 0.01 s. Whether or not a measurement is accurate does not depend on the precision.
Reliability and accuracy
Reliability and accuracy are separate aspects of an experiment and the relationship between them is sometimes misunderstood. Consider the following table:
|Accurate||The correct answer all the time.||The correct answer on average, but answers vary between repetitions.|
|Inaccurate||The same incorrect answer all the time.||An incorrect answer overall, and answers vary between repetitions.|
A result can be reliable and inaccurate if you get the same incorrect answer all the time (e.g. your friend is always 10 minutes late), and it can also be accurate and unreliable (e.g. your friend is more or less on time, but sometimes early, sometimes late). An example usually given is one of shooting at a target, as shown below.
Some steps can be taken to improve both accuracy and reliability. For example, if you use better quality equipment, your measurements can be more reliable and more accurate. Considering the pendulum experiment again, you need to time the period of the pendulum which is a short time (around 1-2 s). If you time it by hand, your reaction time will introduce an error in the measurement. You can make the measurement more reliable by using light gates and a computer. You can also measure the time for 10 periods, and divide by 10. The latter measures a longer time (around 20 s) so any error is a smaller fraction of the measurement, and you are more likely to get the correct measurement (i.e. the same result) each time. If the measurement is easier to do, then you’re more likely to get the same result in each repetition.
Ultimately though, you cannot make general conclusions about the reliability from the accuracy and vice versa. The reason is that they are affected by different types of experimental errors as mentioned above: accuracy is affected by systematic error, and reliability is affected by random errors.
See Part 2 of this blog Physics Practical Skills Part 2 – Systematic VS Random Errors
You may also be interested in:
© Matrix Education and www.matrix.edu.au, 2017. Unauthorised use and/or duplication of this material without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Matrix Education and www.matrix.edu.au with appropriate and specific direction to the original content.
Found this article interesting or useful? Share the knowledge!