# ASTM E1488-12e1

Designation: E1488 − 12´1An American National StandardStandard Guide forStatistical Procedures to Use in Developing and ApplyingTest Methods1This standard is issued under the fixed designation E1488; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon (´) indicates an editorial change since the last revision or reapproval.ε1NOTE—Editorial corrections were made to 3.1.25 in April 2013.1. Scope1.1 This guide identifies statistical procedures for use indeveloping new test methods or revising or evaluating existingtest methods, or both.1.2 This guide also cites statistical procedures especiallyuseful in the application of test methods.2. Referenced Documents2.1 ASTM Standards:2E177 Practice for Use of the Terms Precision and Bias inASTM Test MethodsE178 Practice for Dealing With Outlying ObservationsE456 Terminology Relating to Quality and StatisticsE691 Practice for Conducting an Interlaboratory Study toDetermine the Precision of a Test MethodE1169 Practice for Conducting Ruggedness TestsE1402 Guide for Sampling DesignE2282 Guide for Defining the Test Result of a Test MethodE2489 Practice for Statistical Analysis of One-Sample andTwo-Sample Interlaboratory Proficiency Testing ProgramsE2554 Practice for Estimating and Monitoring the Uncer-tainty of Test Results of a Test Method Using ControlChart TechniquesE2586 Practice for Calculating and Using Basic StatisticsE2587 Practice for Use of Control Charts in StatisticalProcess ControlE2655 Guide for Reporting Uncertainty of Test Results andUse of the Term Measurement Uncertainty in ASTM TestMethods3. Terminology3.1 Definitions—For a more extensive list of terms in E11standards, see Terminology E456.3.1.1 bias, n—the difference between the expectation of thetest results and an accepted reference value. E1773.1.1.1 Discussion—Statistical procedures include the sam-pling considerations or the experiment design for the collectionof data, or both, and the numerical and graphical approaches tosummarize and analyze the collected data.3.1.2 coeffıcient of variation, CV, n—for a nonnegativecharacteristic, the ratio of the standard deviation to the meanfor a population or sample. E25863.1.3 component of variance, n—a part of a total varianceidentified with a specified source of variability.3.1.4 control chart, n—chart on which are plotted a statis-tical measure of a subgroup versus time of sampling along withlimits based on the statistical distribution of that measure so asto indicate how much common, or chance, cause variation isinherent in the process or product. E25873.1.5 observation, n—the process of obtaining informationregarding the presence or absence of an attribute of a testspecimen, or of making a reading on a characteristic ordimension of a test specimen. E22823.1.6 observed value, n—the value obtained by making anobservation. E22823.1.7 precision, n—the closeness of agreement betweenindependent test results obtained under stipulated conditions.E1773.1.8 proficiency testing, n—determination of laboratorytesting performance by means of interlaboratory comparisons.E24893.1.9 repeatability, n—precision under repeatability condi-tions. E1773.1.10 repeatability conditions, n—conditions where inde-pendent test results are obtained with the same method onidentical test items in the same laboratory by the same operatorusing the same equipment within short intervals of time. E1771This guide is under the jurisdiction of ASTM Committee E11 on Quality andStatistics and is the direct responsibility of Subcommittee E11.20 on Test MethodEvaluation and Quality Control.Current edition approved Aug. 1, 2012. Published September 2012. Originallyapproved in 1992. Last previous edition approved in 2009 as E1488 – 09. DOI:10.1520/E1488-12E01.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at service@astm.org. For Annual Book of ASTMStandards volume information, refer to the standard’s Document Summary page onthe ASTM website.Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States13.1.11 repeatability limit r, n—the value below which theabsolute difference between two individual test results obtainedunder repeatability conditions may be expected to occur with aprobability of approximately 0.95 (95 %). E1773.1.12 repeatability standard deviation, sr,n—the standarddeviation of test results obtained under repeatability condi-tions. E1773.1.13 reproducibility, n—precision under reproducibilityconditions. E1773.1.14 reproducibility conditions, n—conditions where testresults are obtained with the same method on identical testitems in different laboratories with different operators usingdifferent equipment. E1773.1.15 reproducibility limit, R, n—the value below whichthe absolute difference between two test results obtained underreproducibility conditions may be expected to occur with aprobability of approximately 0.95 (95 %). E1773.1.16 reproducibility standard deviation, sR,n—the stan-dard deviation of test results obtained under reproducibilityconditions. E1773.1.17 ruggedness, n—insensitivity of a test method todepartures from specified test or environmental conditions.E11693.1.18 ruggedness test, n—a planned experiment in whichenvironmental factors or test conditions are deliberately variedin order to evaluate the effects of such variation. E11693.1.19 standard deviation, n—of a population, σ, the squareroot of the average or expected value of the squared deviationof a variable from its mean – of a sample x¯, the square root ofthe sum of the squared deviations of the observed values in thesample divided by the sample size minus 1. E25863.1.20 state of statistical control, n—process conditionwhen only common causes are operating on the process. E25873.1.21 statistical procedures, n—the organized techniquesand methods used to collect, analyze, and interpret data.3.1.21.1 Discussion—Statistical procedures include thesampling considerations or the experiment design for thecollection of data, or both, and the numerical and graphicalapproaches to summarize and analyze the collected data.3.1.22 test determination, n—the value of a characteristic ordimension of a single test specimen derived from one or moreobserved values. E22823.1.23 test method, n—a definitive procedure that producesa test result. E22823.1.24 test observation, n—see observation. E22823.1.25 test result, n—the value of a characteristic obtainedby carrying out a specified test method. E22824. Significance and Use4.1 The creation of a standardized test method generallyfollows a series of steps from inception to approval andongoing use. In all such stages there are questions of how wellthe test method performs.4.1.1 Assessments of a new or existing test method gener-ally involve statistical planning and analysis. This standardrecommends what approaches may be taken and indicateswhich standards may be used to perform such assessments.4.2 This standard introduces a series of phases which arerecommended to be considered during the life cycle of a testmethod as depicted in Fig. 1. These begin with a design phasewhere the standard is initially prepared. A development phaseinvolves a variety of experiments that allow further refinementand understanding of how the test method performs within alaboratory. In an evaluation phase the test method is thenexamined by way of interlaboratory studies resulting in preci-sion and bias statistics which are published in the standard.Finally, the test method is subject to a monitoring phase.4.3 All ASTM test methods are required to include state-ments on precision and bias.34.4 Since ASTM began to require all test methods to haveprecision and bias statements that are based on interlaboratorytest methods, there has been increased concern regarding whatstatistical experiments and procedures to use during the devel-opment of the test methods. Although there exists a wide rangeof statistical procedures, there is a small group of generallyaccepted techniques that are beneficial to follow. This guide isdesigned to provide a brief overview of these procedures and tosuggest an appropriate sequence of carrying out these proce-dures.4.5 Statistical procedures often result in interpretations thatare not absolutes. Sometimes the information obtained may beinadequate or incomplete, which may lead to additional ques-tions and the need for further experimentation. Informationoutside the data is also important in establishing standards andin the interpretation of numerical results.5. Summary of Guide5.1 Outlined below is a suggested sequence of four phasesuseful in the development of a test method. A flowchart isprovided in Fig. 1. Such a sequence of analyses may need to bemodified in specific situations. The assistance of a qualifiedstatistician is recommended at each review phase.5.2 Design Phase:5.2.1 This phase includes the formalization of the scope andthe significance and use sections. It may include determiningthe purpose and describing a general approach to the testmethod but usually does not involve statistical studies.5.2.2 Studies may be conducted to evaluate the basicperformance of the method. The draft test method is preparedand sampling requirements and the test result (see GuideE2282) are clearly defined.5.2.3 A flow chart is extremely valuable to identify thesequence of operations involved in a test method, for example,the sampling steps required to obtain the test specimens,definition of the test determination, how a test result is to becomputed, and running the tests on the specimens.5.3 Development Phase:3See the Form and Style Manual for ASTM Standards that specifies, whenpossible, precision statements shall be estimated based on the results of aninterlaboratory test program.E1488 − 12´125.3.1 The test method is examined for such concerns as itsstability, ruggedness, statistical control and the contributions tovariability. The completion of this phase should result inpreliminary estimates of precision and the identification andsuggested ways to estimate potential contributors to uncer-tainty.5.3.2 Evaluation of Short Term Control of Test Method—Atest method must exhibit an ability to provide consistent resultsat least over short time periods. Preliminary studies or a pilottest should be conducted to evaluate the short term stability ofthe test method. A small series of repeated tests should beconducted.5.3.3 Analysis of Variability—Statistically designed experi-ments conducted in one or two laboratories can be used toassess the relative magnitudes of different sources or potentialcontributors to variability of the test results. Such studies canprovide estimates of intermediate measures of precision.5.3.4 Ruggedness Test—A ruggedness test (see PracticeE1169) is a statistically designed experiment that helps identifyproblems in running the test method, clarifies errors, and pointsout possible environmental conditions, which may adverselyaffect the test method or point out need for tightening require-ments. The ruggedness test can assist in locating ways ofreducing variability in the test method.5.3.5 Preliminary Estimates of Precision—From the variousstudies conducted in accordance with 5.3.2 – 5.3.4, preliminaryestimates of repeatability standard deviations should be devel-oped and published in this test method. Until an interlaboratorystudy is performed, these estimates generally are considered tobe provisional. Information on how a lab should developuncertainty estimates should also be provided.5.3.6 Statistical Control—A test method must show capa-bility of performing in a consistent way over time. The use ofcontrol charts (see Guide E2655)3to monitor a proposed, orexisting, test method over time is one recommended way toexamine the controllability or stability of a test method. Thisstatistical control should be demonstrated in one or twolaboratories using homogeneous material (test specimen).5.4 Evaluation Phase:5.4.1 The test method is subjected to interlaboratory studiesto provide estimates of within-laboratory repeatability andbetween-laboratory reproducibility. Additional information issupplied from proficiency studies when conducted.5.4.2 Interlaboratory Study (ILS)—In accordance withASTM Form and Style Manual, whenever feasible, an inter-laboratory study must be conducted. This procedure willprovide specific estimates of variation anticipated when usingthe test method.5.4.3 Protocol for the ILS, Practice E691, provides a guidefor developing the ILS for the test method. A first step is thewriting of an ILS Protocol, which will set out what needs to bedone before the test specimens (or test materials) are distrib-uted to the participating laboratories.FIG. 1 Sequence of StepsE1488 − 12´135.4.4 Precision Statements—Using the estimates of varia-tion obtained in the interlaboratory test, one may prepareprecision statements using Practices E691 and E177 or equiva-lent procedures.5.5 Monitoring Phase:5.5.1 After a test method is approved and in use it isimportant to ensure that the published precision and biasstatistics for the test method remain achievable and consistentover time or amongst different groups conducting the tests.5.5.2 Monitoring Within a Single Location—It is importantfor any laboratory or organization that will use a particular testmethod over time that a means of monitoring to ensure themethod results using quality control samples are stable and incontrol. Regular evaluation of the uncertainty (Practice E2554)or use of a control charting method (Practice E2587) are twoways to monitor the test method.5.5.3 Between Laboratory Comparisons—Proficiency test-ing programs measure the typical variation amongst ordinarylaboratories. The specific laboratories involved also obtaininformation about how well they perform compared to otherlaboratories.6. Development of Test Method6.1 Proposed standards that are under development shouldbe treated in a formal manner following as many of thesuggested procedures as possible. Standards that are already inexistence as approved test methods or in general practicerequire periodic review that would include selected procedures.6.2 Under Development—The development stage involvestest methods that are in the preliminary stages during whichequipment may not have been fully tested, practices are notagreed upon, and operators have yet to be adequately trained.Often this stage also applies to standards that have not yet beenapproved.6.2.1 It is essential that tests for statistical control,ruggedness, and variability analyses be conducted prior to anyinterlaboratory test programs.6.2.2 After all major environmental contributors have beenidentified, controlled, and incorporated into the test method,and after adequate standardized equipment is available, aninterlaboratory test