Skip to content
BY 4.0 license Open Access Published by De Gruyter Open Access June 29, 2019

Exploring Age-Related Metamemory Differences using Modified Brier Scores and Hierarchical Clustering

  • Chelsea Parlett-Pelleriti EMAIL logo , Grace C. Lin , Masha R. Jones , Erik Linstead and Susanne M. Jaeggi
From the journal Open Psychology


Older adults (OAs) typically experience memory failures as they age. However, with some exceptions, studies of OAs’ ability to assess their own memory functions—Metamemory (MM)— find little evidence that this function is susceptible to age-related decline. Our study examines OAs’ and young adults’ (YAs) MM performance and strategy use. Groups of YAs (N = 138) and OAs (N = 79) performed a MM task that required participants to place bets on how likely they were to remember words in a list. Our analytical approach includes hierarchical clustering, and we introduce a new measure of MM—the modified Brier—in order to adjust for differences in scale usage between participants. Our data indicate that OAs and YAs differ in the strategies they use to assess their memory and in how well their MM matches with memory performance. However, there was no evidence that the chosen strategies were associated with differences in MM match, indicating that there are multiple strategies that might be effective (i.e. lead to similar match) in this MM task.


Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445-459.10.3758/BF03193014Search in Google Scholar

Batchelor, J. H., & Miao, C. (2016). Extreme response style: A meta-analysis. Journal of Organizational Psychology, 16(2), 51-62.Search in Google Scholar

Belmont, J. M., & Borkowski, J. G. (1988). A group-administered test of children’s metamemory. Bulletin of the Psychonomic Society, 26(3), 206-208.10.3758/BF03337288Search in Google Scholar

Bird, H., Franklin, S., & Howard, D. (2001). Age of acquisition and imageability ratings for a large set of words, including verbs and function words. Behavior Research Methods, Instruments & Computers, 33(1), 73-79. doi: 10.3758/BF0319534910.3758/BF03195349Search in Google Scholar

Bunnell, J. K., Baken, D. M., & Richards-Ward, L. A. (1999). The effect of age on metamemory for working memory. New Zealand Journal of Psychology, 28(1), 23-29.Search in Google Scholar

Connor, L. T., Dunlosky, J., & Hertzog, C. (1997). Age-related differences in absolute but not relative metamemory accuracy. Psychology and Aging, 12(1), 50.10.1037/0882-7974.12.1.50Search in Google Scholar

Castel, A. D., Balota, D. A., & McCabe, D. P. (2009). Memory efficiency and the strategic control of attention at encoding: Impairments of value-directed remembering in Alzheimer’s disease. Neuropsychology, 23(3), 297.10.1037/a0014888Search in Google Scholar

Castel, A. D., Benjamin, A. S., Craik, F. I., & Watkins, M. J. (2002). The effects of aging on selectivity and control in short-term recall. Memory & Cognition, 30(7), 1078-1085.10.3758/BF03194325Search in Google Scholar

Chen, C., Lee, S.-Y., & Stevenson, H. W. (1995). Response style and cross-cultural comparisons of rating scales among east Asian and North American students. Psychological Science, 6(3), 170-175. in Google Scholar

DeMarie, D., & Ferron, J. (2003). Capacity, strategies, and metamemory: Tests of a three-factor model of memory development. Journal of Experimental Child Psychology, 84(3), 167-193.10.1016/S0022-0965(03)00004-3Search in Google Scholar PubMed

Dixon, R. A., Hultsch, D. F., & Hertzog, C. (1988). The metamemory in adulthood (MIA) questionnaire. Psychopharmacology Bulletin, 24(4), 671-688.Search in Google Scholar PubMed

Dunlosky, J., & Connor, L. T. (1997). Age differences in the allocation of study time account for age differences in memory performance. Memory & Cognition, 25(5), 691-700.10.3758/BF03211311Search in Google Scholar PubMed

Dunlosky, J., & Hertzog, C. (1997). Older and younger adults use a functionally identical algorithm to select items for restudy during multitrial learning. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 52(4), P178-P186.10.1093/geronb/52B.4.P178Search in Google Scholar

Dunlosky, J., & Metcalfe, J. (2009). Metacognition: A textbook of cognition, educational, life span, and applied psychology. Thousand Oaks, CA: Sage. Duncan, RB (1972). Characteristics of organizational environments and perceived environmental uncertainty. Administrative Science Quarterly, 17(3), 313-327.Search in Google Scholar

Dunlosky, J., & Thiede, K. W. (2013). Metamemory. In D. Reisberg (Ed.), The Oxford Handbook of Cognitive Psychology (pp. 283-298). New York: Oxford University Press.Search in Google Scholar

Emanuel Robinson, A., Hertzog, C., & Dunlosky, J. (2006). Aging, encoding fluency, and metacognitive monitoring. Aging, Neuropsychology, and Cognition, 13(3-4), 458-478.10.1080/13825580600572983Search in Google Scholar

Flavell, J., Wellman, H. M., Kail, R. V., & Hagen, J. W. (1977). Metamemory; Perspective on the development of memory and cognition.Search in Google Scholar

Gilewski, M. J., Zelinski, E. M., & Schaie, K. W. (1990). The Memory Functioning Questionnaire for assessment of memory complaints in adulthood and old age. Psychology and Aging, 5(4), 482.10.1037/0882-7974.5.4.482Search in Google Scholar

Hertzog, C., & Dunlosky, J. (2011). Metacognition in later adulthood: Spared monitoring can benefit older adults’ self-regulation. Current Directions in Psychological Science, 20(3), 167-173. doi: 10.1177/096372141140902610.1177/0963721411409026Search in Google Scholar

Hertzog, C., Dunlosky, J., & Sinclair, S. M. (2010). Episodic feeling-of-knowing resolution derives from the quality of original encoding. Memory & Cognition, 38(6), 771-784.10.3758/MC.38.6.771Search in Google Scholar

Hertzog, C., Kidder, D. P., Powell-Moman, A., & Dunlosky, J. (2002). Aging and monitoring associative learning: Is monitoring accuracy spared or impaired? Psychology and Aging, 17(2), 209.10.1037/0882-7974.17.2.209Search in Google Scholar

Kelemen, W. L, Frost, P. J., & Weaver III, C. A. (2000). Individual differences in metacognition: Evidence against a general metacognitive ability. Memory & Cognition, 28(1), 92-107.10.3758/BF03211579Search in Google Scholar

Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of experimental psychology: General, 126(4), 349.10.1037/0096-3445.126.4.349Search in Google Scholar

Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103(3), 490-517.10.1037/0033-295X.103.3.490Search in Google Scholar

Koriat, A., Ma’ayan, H., Sheffer, L., & Bjork, R. (2006). Exploring a mnemonic debiasing account of the underconfidence-with-practice effect. Journal of Experimental Psychology: Learning, Memory, & Cognition, 32(3), 595-608. doi: 10.1037/0278-7393.32.3.59510.1037/0278-7393.32.3.595Search in Google Scholar

Koriat, A., Sheffer, L., & Ma’ayan, H. (2002). Comparing objective and subjective learning curves: judgments of learning exhibit increased underconfidence with practice. Journal of Experimental Psychology: General, 131(2), 147.10.1037/0096-3445.131.2.147Search in Google Scholar

Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203-208.10.3758/BF03204766Search in Google Scholar

Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97(4), 723-731.10.1037/0022-0663.97.4.723Search in Google Scholar

McGillivray, S., & Castel, A. D. (2011). Betting on memory leads to metacognitive improvement by younger and older adults. Psychology and Aging, 26(1), 137.10.1037/a0022681Search in Google Scholar

Metcalfe, J. (2002). Is study time allocated selectively to a region of proximal learning? Journal of Experimental Psychology: General, 131(3), 349.10.1037/0096-3445.131.3.349Search in Google Scholar

Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Quantitative Methods in Psychology, 95(1), 109-133.10.1037/0033-2909.95.1.109Search in Google Scholar

Price, J., Hertzog, C., & Dunlosky, J. (2010). Self-regulated learning in younger and older adults: Does aging affect metacognitive control? Aging, Neuropsychology, and Cognition, 17(3), 329-359.10.1080/13825580903287941Search in Google Scholar

Price, J., & Murray, R. G. (2012). The region of proximal learning heuristic and adult age differences in self-regulated learning. Psychology and Aging, 27(4), 1120.10.1037/a0029860Search in Google Scholar

Rast, P., & Zimprich, D. (2009). Age differences in the underconfidence-with-practice effect. Experimental Aging Research, 35(4), 400-431.10.1080/03610730903175782Search in Google Scholar PubMed

Rhodes, M. G., & Tauber, S. K. (2011). The influence of delaying judgments of learning on metacognitive accuracy: A meta-analytic review. Psychological Bulletin, 137(1), 131.10.1037/a0021705Search in Google Scholar

Rogers, W. A., Hertzog, C., & Fisk, A. D. (2000). An individual differences analysis of ability and strategy influences: Age-related differences in associative learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(2), 359.10.1037/0278-7393.26.2.359Search in Google Scholar

Rousseeuw, P. J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20, 53-65.10.1016/0377-0427(87)90125-7Search in Google Scholar

Rufibach, K. (2010). Use of Brier score to assess binary predictions. Journal of clinical epidemiology, 63(8), 938-939.10.1016/j.jclinepi.2009.11.009Search in Google Scholar

Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition Learning, 4, 33-45. doi: 10.1007/s11409-008-9031-310.1007/s11409-008-9031-3Search in Google Scholar

Schwartz, B. L., & Efklides, A. (2012). Metamemory and memory efficiency: Implications for student learning. Journal of Applied Research in Memory and Cognition, 1(3), 145-151.10.1016/j.jarmac.2012.06.002Search in Google Scholar

Souchay, C., & Isingrini, M. (2004). Age related differences in metacognitive control: Role of executive functioning. Brain and Cognition, 56(1), 89-99.10.1016/j.bandc.2004.06.002Search in Google Scholar

Stine-Morrow, E. A., Shake, M. C., Miles, J. R., & Noh, S. R. (2006). Adult age differences in the effects of goals on self-regulated sentence processing. Psychology and Aging, 21(4), 790.10.1037/0882-7974.21.4.790Search in Google Scholar

Tauber, S. K., & Rhodes, M. G. (2012). Multiple bases for young and older adults’ judgments of learning in multitrial learning. Psychology and Aging, 27(2), 474.10.1037/a0025246Search in Google Scholar

Tauber, S. K., & Witherby, A. E. (2015). Metacognition in older adulthood. Encyclopedia of Geropsychology, 1-15.Search in Google Scholar

Thompson, W. B., & Mason, S. E. (1996). Instability of individual differences in the association between confidence judgments and memory performance. Memory & Cognition, 24(2), 226-234.10.3758/BF03200883Search in Google Scholar PubMed

Touron, D. R., Hoyer, W. J., & Cerella, J. (2004). Cognitive skill learning: age-related differences in strategy shifts and speed of component operations. Psychology and Aging, 19(4), 565.10.1037/0882-7974.19.4.565Search in Google Scholar

Troyer, A. K., & Rich, J. B. (2002). Psychometric properties of a new metamemory questionnaire for older adults. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 57(1), P19-P27.Search in Google Scholar

Ward Jr, J. H. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58(301), 236-244.10.1080/01621459.1963.10500845Search in Google Scholar

Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45(4), 1191-1207.10.3758/s13428-012-0314-xSearch in Google Scholar

Received: 2018-08-10
Accepted: 2019-03-08
Published Online: 2019-06-29
Published in Print: 2019-01-01

© 2019 Chelsea Parlett-Pelleriti et al., published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 Public License.

Downloaded on 2.10.2023 from
Scroll to top button