Psychological traits are an objective existence

Thorndike: Everything that exists objectively has its quantity

McColl: Everything that has quantity can be measured

Psychometric scales are unstable (low reliability)

Not really measuring what we want to measure (low validity)

The test subject’s true level has not been properly demonstrated

Physical environment: temperature, light, sound, etc. at the measurement site

Testing time

unexpected interference

Main test factors

①ε(X)=T or ε(E)=0 The expected value of the observed score is the true score, and the expected value of the random error score is 0. Operational definition of a true fraction: the average of the results obtained from numerous measurements.

②ρ(T,E)=0 True fractions and random errors are independent of each other

③ρ(E1,E2)=0 Random errors on each parallel test are independent of each other

Test-retest reliability, also called test-retest reliability, refers to the degree of consistency of the results obtained by using the same measurement tool to test the same group of subjects twice under the same conditions, and reflects the results of the measurement tool. Affected by time interval factors. The most appropriate time interval varies depending on the purpose and nature of the test and the characteristics of the test subject. Generally, two weeks to four weeks are suitable, and preferably no more than six months.

The size of the test-retest reliability can be marked by calculating the test-retest coefficient or stability coefficient of the measurement tool. Specifically, it is to obtain the Pearson product-difference correlation coefficient between the scores of the same group of subjects on the two tests.

The individual psychological traits measured by the measurement tool should be relatively stable over time. Such as: personality test There should be no obvious practice effect and forgetting effect on the individual psychological traits measured by the measurement tool, and the effects of practice and forgetting basically cancel each other out. Such as: intelligence test (6 months) No special training or training should be conducted between the two administrations to ensure that the test-retest reliability reflects the influence of random factors.

Duplicate reliability refers to the degree of consistency of the results obtained by two duplicate tests (parallel tests) measuring the same group of subjects. The degree of replica reliability is calculated by calculating the Pearson product-difference correlation coefficient of the scores obtained by the same group of subjects on the two replica tests. Replicate reliability reflects measurement errors caused by differences in questions and time intervals.

Equivalence coefficient: the two duplicate tests are administered simultaneously and continuously;

Stability and equivalence coefficient/test-retest reliability: The two replicate tests are administered twice, separated by a period of time.

The effect of the presentation order of independent variables on the dependent variable. That is, when the same subjects receive different experimental treatments, the possible impact of the first experimental treatment on the second experimental treatment. The impact may be huge or slight, short-lived or long-lasting.

An experimental design technique that controls the order of experimental treatments to offset sequence errors caused by the order of experimental treatments.

1) Construct two or more truly parallel tests (i.e. Papers A and B); Duplicate or parallel tests: Two tests that use different items to measure the same content and whose test results have the same mean and standard deviation. 2) Subjects must be qualified to accept two tests. (time, money, etc.) 3) You should try your best to explain in detail the time interval between the two tests, the test sequence arrangement, the test experience of the subjects during the test, etc. in the test result report.

Internal consistency reliability, also called homogeneity reliability, mainly evaluates whether the same psychological traits are measured among the random components of the test, and reflects the degree of sampling consistency of the question content.

split-half reliability

Library-theoretical reliability

Cronbach's coefficient

Hoyt reliability

Interrater reliability refers to the degree of consistency with which multiple raters rate responses from the same group of people. Generally, the average consistency between pairs of trained raters is required to be above 0.90 before the rating is considered objective.

Two raters: Calculate the correlation coefficient (Pearson product-difference correlation or Spearman rank correlation) between the scores given by the two raters to the same batch of subjects' answer sheets.

More than two raters: estimated using Kendall's concord coefficient.

single subject

Heterogeneity of the subject group, average ability level of the subject group

tester

grader

Test length

Test difficulty