Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Aug 6:9:1286.
doi: 10.3389/fpsyg.2018.01286. eCollection 2018.

Automatic Generation of Figural Analogies With the IMak Package

Diego Blum  1 , Heinz Holling  1
Affiliations

Automatic Generation of Figural Analogies With the IMak Package

Diego Blum et al. Front Psychol. .

Abstract

Automatic Item Generation (AIG) techniques are offering innovative ways to produce test items as they overcome many disadvantages involving standard item writing, such as time-consuming work and resource-intensive demands. Although this field is relatively new, it is progressing at a high speed, and several contributions have been accomplished. Nevertheless, a scarce amount of AIG software evidencing favorable psychometric properties of the generated items has been made accessible to the broad scientific community. This research had two goals: first, to present an empirical study of items produced with the aid of the Item Maker (IMak) package available online and, second, to present IMak itself for the automatic generation of figural analogies. We were particularly interested in assessing whether automatically created figural analogy rules could predict item psychometric difficulty. A total of 23 items were generated and administered to 307 participants, 49.51% from Germany. The mean age was 28.61 (SD = 10.19) and 57.65% of the participants were female. Results reveal adequate psychometric properties including convergent validity, that most of the manipulated rules contribute to item difficulty, and that rule-based difficulty prediction is possible to some extent. In other words, psychometric quality of the generated items is supported, which reveals the utility of the IMak package in assessment settings. Finally, the package is presented and its functions for figural analogy item generation are further described.

Keywords: Automatic Item Generation; Item Maker; LLTM; figural analogies; rules.

PubMed Disclaimer

Figures

FIGURE 1
FIGURE 1
Basic shape used for item construction.
FIGURE 2
FIGURE 2
Plot mode “A” of a four-rule-based item.
FIGURE 3
FIGURE 3
Rasch model-LLTM plots of mean-centered difficulty parameters when groups of five (left graphic) and nine (right graphic) rules are considered, based on traditional models.
FIGURE 4
FIGURE 4
Rasch model-LLTM plots of mean-centered difficulty parameters when groups of five (left graphic) and nine (right graphic) rules are considered, based on random-effects models.
See this image and copyright information in PMC

References

    1. Andersen E. B. (1973). A goodness of fit test for the Rasch model. Psychometrika 38 123–140. 10.1007/BF02291180 - DOI
    1. Arendasy M. (2002). GeomGen-Ein Itemgenerator Für Matrizentestaufgaben. Vienna: Eigenverlag.
    1. Arendasy M. (2005). Automatic generation of rasch-calibrated items: figural matrices test GEOM and endless-loops test EC. Int. J. Test. 5 197–224. 10.1207/s15327574ijt0503_2 - DOI
    1. Arendasy M. E., Sommer M. (2005). The effect of different types of perceptual manipulations on the dimensionality of automatically generated figural matrices. Intelligence 33 307–324. 10.1016/j.intell.2005.02.002 - DOI
    1. Arendasy M. E., Sommer M. (2010). Evaluating the contribution of different item features to the effect size of the gender difference in three-dimensional mental rotation using automatic item generation. Intelligence 38 574–581. 10.1016/j.intell.2010.06.004 - DOI