It refers to the degree of agreement between the content actually measured by a test and the content to be measured, that is, whether the test is a representative sample of the behavioral field to be measured. The content or behavioral area to be measured is determined based on the purpose of measurement, which usually includes the scope of knowledge to be measured and the degree of mastery required for each knowledge point within the scope.

Logical analysis/expert judgment

statistical analysis

Empirical speculation

Content Validity Advantages and Disadvantages

Also known as construct validity and construct validity, it refers to the extent to which a test actually measures the theoretical structures and characteristics to be measured, or whether the test results can confirm or explain the assumptions, terms or constructs of a certain theory.

①The size of the construct validity depends on the pre-assumed psychological structure theory; The construct validity of tests with different theoretical constructs cannot be compared. ②Construct validity is sometimes difficult to obtain. When it cannot be confirmed, there may be problems with the theory or the experimental design cannot properly test the hypothesis; ③ There is no single quantitative index for construct validity, and it is evaluated based on the accumulation of evidence from all aspects. ④The definition or explanation of the measured content or behavioral range of some tests is similar to the explanation of theoretical constructs, so high content validity essentially means high structural validity.

1. Search for evidence within the test 2. Search for evidence between tests 3. Empirical validity method of examination test 4. Multiple characteristics-multiple methods matrix method 5. Experimental operation method

1. Content validity 2. Characteristics of subjects’ responses to the questions 3. Homogeneity reliability

Compatible validity: Examine the correlation between the newly compiled test and an old test that is known to have high validity and can effectively measure the same trait. The higher the better.

Discriminant validity: Also called heterogeneous validity, if two tests measure different traits, the correlation between them should be very low even if they are measured using the same method.

Convergent validity: Also called identical validity and convergent validity, if two tests measure the same trait, the correlation between them should be high even if different measurement methods are used.

Factor analysis method: By conducting factor analysis on a group of tests, we find out the common factors that affect the test. The loading of each test on the common factors (that is, the correlation between the test and each factor) is the factor validity of the test. These factors may It’s the psychological trait/construct we want to measure. The proportion of the total variation in test scores that comes from relevant factors is an indicator of the construct validity of the test.

The nature and type of the criterion predicted by the test are used as structural validity indicators for analyzing the test

Measure two or more psychological traits simultaneously, and each psychological trait is measured using two or more methods. Suppose that multiple traits (such as A, B, C) are measured by multiple methods (such as 1, 2, 3, 4).

Suitable for tests that are susceptible to certain conditions, such as anxiety. Based on the theoretical conception of the trait to be measured, we can predict that under certain circumstances or after certain treatments, the subject's score will change.

Advantages: It enables researchers to focus on the test's goal of proposing and verifying hypotheses, and pays attention to its role as a theoretical research tool, not just as an auxiliary decision-making function, thus providing a broader development prospect for the test.

Disadvantages: The concepts of some constructs are vague, and there is no consistent definition. It is impossible to clarify its operational steps and unique and objective indicators to determine validity. (Operational difficulty)

Also known as criterion-related validity (referred to as criterion validity), it refers to the validity of a test in estimating the behavior of individuals in a specific situation, that is, using the practical effect as the test standard. Validity criterion: The behavior being estimated is the standard for testing the validity of the test.

meaning

feature

Good efficacy standard

Prevent calibration standard contamination

Related laws: Calculate the correlation coefficient between the test score and the validity criterion measurement. This correlation coefficient is called the validity coefficient. The most commonly used is the product-difference correlation method because test scores and criterion measures are usually continuous variables.

distinction method; Discrimination is a method of testing whether test scores can effectively differentiate between groups defined by a criterion. The specific method is to divide the subjects into high and low groups and analyze whether the difference in test scores between the two groups is significant.

Hit rate method: The hit rate method is a method that uses the proportion of correct decisions as a validity indicator when a test is used as a basis for making choices. ①Positive hit rate It refers to the ratio of the number of people selected by the test who are actually selected correctly. ②Negative hit rate Refers to the ratio of those who were eliminated by the test to the number of people who really should have been eliminated. ③Total hit rate It refers to the ratio of the sum of the number of people who were tested correctly and the number of people who were eliminated correctly to the total number of people.

pass rate

extreme grouping

Calculation of discrimination index: D=PH-PL D: Project identification index PH: high group pass rate PL: Low group pass rate

Division of extreme groups

1. Point-to-two column correlation: scope of application: items are scored as 0 or 1 (or dichotomous variables), while criterion scores or total test scores are continuous variables.

2. Bi-column correlation: Scope of application: Both are continuous variables, but one of the variables is artificially divided into two categories. Item scores are continuous, while criterion scores or total test scores are divided into two categories: high, low, or passing or failing; The criterion score or total test score is continuous, while the item scores are divided into two categories: true, false, or pass or fail.

3.φ correlation: Scope of application: Both variables are dichotomous variables. Four-square table: high and low groups of standard scores or total test scores, passing and failing of project scores.

4. Product-difference correlation: Scope of application: Both variables are continuous variables. If the total test score is used, the greater the item score, the greater the discrimination.

Test whether there is a significant difference in the scores of each item between the subjects with high total test scores and the subjects with low total scores. If the difference is significant, the discrimination of the item is good.