Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Multicultural Learning and Teaching

Editor-in-Chief: Obiakor, Festus / Algozzine, Robert

Managing Editor: Banks, Tachelle

Online
ISSN
2161-2412
See all formats and pricing
Online
Institutional Subscription
€ [D] 220.00 / US$ 296.00 / GBP 181.00*
Individual Subscription
€ [D] 49.00 / US$ 74.00 / GBP 40.00*
*Prices in US$ apply to orders placed in the Americas only. Prices in GBP apply to orders placed in Great Britain only. Prices in € represent the retail prices valid in Germany (unless otherwise indicated). Prices are subject to change without notice. Prices do not include postage and handling if applicable. RRP: Recommended Retail Price.
More options …

Validation of Assessment Vignettes and Scoring Rubric of Multicultural and International Competency in Faculty Teaching

Sheila J. Henderson
  • Corresponding author
  • I-MERIT (International – Multicultural Initiatives), Alliant International University, 1 Beach Street, San Francisco, CA 94301, USA
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
 / Ruth A. Horton
  • I-MERIT (International – Multicultural Initiatives), Alliant International University, 1 Beach Street, San Francisco, CA 94301, USA
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
 / Paul K. Saito / Kumea Shorter-Gooden
Published Online: 2014-07-29 | DOI: https://doi.org/10.1515/mlt-2014-0002

30,00 € / $42.00 / £23.00

Get Access to Full Text

Abstract

The purpose of this research was to develop a new tool for assessing multicultural and international competency in faculty teaching through vignette scenarios of university classroom critical incidents – across disciplines of clinical and forensics psychology, business, and education. Construct and content validity of the initial draft vignettes and the associated scoring rubric was established by a panel of experts who approved the vignettes as consistent with current teaching models and multicultural and international competency models. Each vignette presented a critical incident needing resolution – a rupture in the relationship between the instructor and students vis-à-vis a diversity issue. These hypothetical ruptures took place in university classrooms in four different disciplines: clinical and forensic psychology, business, and education. The authors recruited ten faculty participants for the online study, who were personally known to the authors as interested in competency-related issues and who were teaching in graduate programs. Each participant responded to three questions for each vignette, which prompted their written feedback on (a) process and content issues surrounding the critical incident as well as (b) coaching for the hypothetical instructor on how to resolve the incident. Five scorers rated the participants’ written responses according to a five-point scoring rubric. In the data analysis, once statistical conclusion validity and interscorer reliability was established (Krippendorff’s α = 0.887), content validity was again explored by assessing whether the written responses were indeed captured by the guiding models. The results, which indicated preliminary validity and reliability of the vignettes and the rubric, suggest that the vignettes are worthy of further testing. These vignettes can also be used as discussion tools to stimulate dialogue in competency trainings for faculty.

Keywords: multicultural and international competency; assessment; faculty teaching; classroom vignettes

References

  • Angelides, P., & Gibbs, P. (2004). Supporting the continued professional development of teachers through the use of vignettes. Teacher Education Quarterly, Fall, 111–121. Retrieved from http://www.teqjournal.org/backvols/2006/33_4/18angelides&gibbs.pdf

  • Banks, J. A., McGee Banks, C. A., Cortez, C. E., Hahn, C. L., Merryfield, M. W., Moodley, K. A., … Parker, W. C. (2005). Democracy and diversity: Principles and concepts for educating citizens in a global age. Center for Multicultural Education, College of Education University of Washington, Seattle. Retrieved from http://education.washington.edu/cme/DemDiv.pdf

  • Barter, C., & Renold, E. (2000). “I wanna tell you a story”: Exploring the application of vignettes in qualitative research with children and young people. International Journal of Social Research Methodology, 3(4), 307–323. CrossrefGoogle Scholar

  • Bennett, J. (1993). Toward ethnorelativism: A developmental model of intercultural sensitivity. In R. M. Paige (Ed.), Education for the intercultural experience (pp. 21–71). Yarmouth, ME: Intercultural Press.Google Scholar

  • Bhawuk, D., & Brislin, R. (1992). The measurement of intercultural sensitivity using the concepts of individualism and collectivism. International Journal of Intercultural Relations, 16, 413–436. .CrossrefGoogle Scholar

  • Borrayo, E. (2008). Challenging dialogues: Working through resistance toward multicultural understanding. In J. White & S. Henderson (Eds.), Building multicultural competency: Development, training, and practice (pp.155–166). Lanham, MD: Rowman & Littlefield Publishers.Google Scholar

  • Burgoyne, S., Placier, P., Thomas, M., Welch, S., Ruffin C., Flores, L. Y., … Miller, M. (2007). Interactive theater and self-efficacy. New Directions for Teaching and Learning, 111, 21–26. .CrossrefGoogle Scholar

  • Cabrera, A. F., Colbeck, C. L., & Terenzini, P. T. (2001). Developing performance indicators for assessing classroom teaching practices and student learning. Research in Higher Education, 42(3), 327–352. CrossrefGoogle Scholar

  • Chapman, V. G., & Inman, M. (2009). A Conundrum: Rubrics or creativity/metacognitive development? Education Digest: Essential Readings Condensed For Quick Review, 75(2), 53–56.Google Scholar

  • Constantine, M. G., Gloria, A. M., & Ladany, N. (2002). The factor structure underlying three self-report multicultural counseling competence. Cultural Diversity and Ethnic Minority Psychology, 8(4), 334–345. CrossrefGoogle Scholar

  • Constantine, M., & Ladany, N. (2000). Self-report multicultural counseling competency scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology, 47, 155–164.CrossrefGoogle Scholar

  • Cooper, B., & Gargan, A. (2009). Rubrics in education: Old term, new meanings. Phi Delta Kappan, 91(1), 54–55.CrossrefGoogle Scholar

  • Cronbach, L. J., & Meehl, R. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.CrossrefGoogle Scholar

  • D’Andrea, M., Daniels, J., & Heck, R. (1991). Evaluating the impact of multicultural counseling training. Journal of Counseling & Development, 70, 143–150. CrossrefGoogle Scholar

  • Dovidio, J., & Gaertner, S. (1986). Prejudice, discrimination, and racism. Orlando, FL: Academic Press.Google Scholar

  • Fantini, A. (2007). Exploring and assessing intercultural competence. Retrieved from http://csd.wustl.edu/Publications/Documents/RP07-01.pdf

  • Finch, J. (1987). The vignette technique in survey research. Sociology, 21(1), 105–114. CrossrefGoogle Scholar

  • Hammer, M., Bennett, M., & Wiseman, R. (2003). Measuring intercultural sensitivity: The intercultural development inventory. International Journal of Intercultural Relations, 27, 421–443. CrossrefGoogle Scholar

  • Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77–89. CrossrefGoogle Scholar

  • Hays, D. (2008). Assessing Multicultural competence in counselor trainees: A review of instrumentation and future directions. Journal of Counseling & Development, 86, 95–101. CrossrefGoogle Scholar

  • Isaeva, T. E. (2005). Classification of the college instructor’s professional and personalcompetencies. Russian Education and Society, 49(9), 31–41.CrossrefGoogle Scholar

  • Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.CrossrefGoogle Scholar

  • Kelley, C., & Meyers, J. (1995). Cross-cultural adaptability index. Minneapolis, MN: National Computer Systems.Google Scholar

  • Kitano, M. (1996). What a course will look like after multicultural change. In A. Morey & M. Kitano (Eds.), Multicultural course transformation in higher education: A broader truth. Boston, MA: Allyn & Bacon.Google Scholar

  • Krathwohl, D. (2002). A revision of Bloom’s taxonomy. Theory into Practice, 41(4), 212–218. CrossrefGoogle Scholar

  • LaFromboise, T., Coleman, H., & Hernandez, A. (1991). Development and factor structure of the cross-cultural counseling inventory-revised. Professional Psychology: Research and Practice, 22, 380–388. CrossrefGoogle Scholar

  • Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2004). Practical resources for assessing and reporting intercoder reliability in content analysis research projects. Retrieved from: http://ils.indiana.edu/faculty/hrosenba/www/Research/methods/lombard_reliability.pdf

  • Miles, M. (1990). New methods for qualitative data collection and analysis: Vignettes and prestructured cases. International Journal of Qualitative Studies in Education, 3(1), 37–51. CrossrefGoogle Scholar

  • Mobley, M., Worthington, R. L., & Soth, A. M. (2006). The Missouri multicultural counseling self-efficacy scale: A preliminary investigation. Paper presentation at the Twenty-third Annual Teachers College Winter Roundtable on Cross-Cultural Psychology and Education, New York, NY.Google Scholar

  • Mueller, J. (2010). Authentic assessment toolbox. Retrieved from http://jonathan.mueller.faculty.noctrl.edu/toolbox/index.htm

  • Odgers, T., & Giroux, I. (2006). Internationalizing faculty: A phased approach to transforming curriculum design and instruction. Paper presented at Internationalizing Canada’s Universities: Practices, Challenges, and Opportunities symposium. Retrieved from http://www.viu.ca/internationalization/staff/internationalizingcurriculum.asp

  • Olebe, M., & Koester, J. (1989). Exploring the cross-cultural equivalence of the behavioral assessment scale for intercultural communication. International Journal of Intercultural Relations, 13(3), 333–347. CrossrefGoogle Scholar

  • Olson, C., Green, M., & Hill, B. (2005). Building a strategic framework for comprehensive internalization. Washington, DC: American Council on Education.Google Scholar

  • Olson, C., Green, M., & Hill, B. (2006). Advancing comprehensive internationalization: What institutions can do and what students should learn. Washington, DC: American Council on Education.Google Scholar

  • Olson, C., & Kroeger, K. (2001). Global competency and intercultural sensitivity. Journal of Studies in International Education, 5(2), 116–137. CrossrefGoogle Scholar

  • Paschall, M., Fishbein, D., Hubal, R., & Eldreth, D. (2004). Psychometric properties of virtual reality vignette performance: A novel approach for assessing adolescents’ social competency skills. Health Education Research, 20(1), 61–70. CrossrefGoogle Scholar

  • Platt, J. J. & Laszloffy, T. A. (2013). Critical patriotism: Incorporating nationality into MFT education and training. Journal of Marital and Family Therapy, 39(4), 441–456.CrossrefGoogle Scholar

  • Ponterotto, J., Gretchen, D., Utsey, S., Rieger, B., & Austin, R. (2002). A revision of the multicultural counseling awareness scale. Journal of Multicultural Counseling and Development, 30, 11–20. CrossrefGoogle Scholar

  • Ponterotto, J. G., Rieger, B. P., Barrett, A., & Sparks, R. (1994). Assessing multicultural counseling competence: A review of instrumentation. Journal of Counseling and Development, 72(3), 316–322. CrossrefGoogle Scholar

  • Pope-Davis, D. B., Coleman, H. L. K., Liu, W. M. & Toporek, R. L. (2003). Handbook of multicultural competencies in counseling and psychology. Thousand Oaks, CA: Sage.Google Scholar

  • Queener, J., & Smith, S. (2008). Managing resistance: The difference between success and failure of multicultural competency initiatives. In J. White & S. J. Henderson (Eds.), Building multicultural competency: Development, training, and practice (pp.143–154)). Lanham, MD: Rowman & Littlefield Publishers.Google Scholar

  • Rankin, S., & Reason, R. (2008). Transformational tapestry model: A comprehensive approach to transforming campus climate. Journal of Diversity in Higher Education, 1(4), 262–274. CrossrefGoogle Scholar

  • Roberson, L., Kulik, C. T., & Pepper, M. B. (2002). Assessing instructor cultural competence in the classroom: An instrument and a development process. Journal of Management Education, 26, 40–55. CrossrefGoogle Scholar

  • Ruben, B. (1976). Assessing communication competency for intercultural adaptation. Group and Organization Studies, 1, 334–354. CrossrefGoogle Scholar

  • Ruben, B., & Kealey D. (1979). Behavioral assessment of communication competency and the prediction of cross-cultural adaptation. International Journal of Intercultural Relations, 3, 15–48. CrossrefGoogle Scholar

  • Schoenberg, N., & Ravdal, H. (2000). Using vignettes in awareness and attitudinal research. International Journal of Social Research Methodology, 3(1), 63–74. CrossrefGoogle Scholar

  • Sheu H. B., Rigali-Oiler M., & Lent, R. W. (2012). Multicultural counseling self-efficacy scale – racial diversity form: Factor structure and test of a social cognitive model. Psychotherapy Research, 22(5), 527–542. CrossrefGoogle Scholar

  • Sinicrope, C., Norris, J., & Watanabe, Y. (2008). Understanding and assessing competence: A summary of research, theory, and practice (Technical report for the Foreign Language Evaluation Project). Unpublished manuscript, University of Hawaii at Manoa, Hawaii. Retrieved from http://www.hawaii.edu/sls/uhwpesl/26(1)/Norris.pdf

  • Smith, L., Constantine, M., Graham, S., & Dize, C. (2008). The territory ahead for multicultural competence: The “spinning” of racism. Professional Psychology: Research and Practice, 39(3), 337–335. CrossrefGoogle Scholar

  • Sodowsky, G., Taffe, R., Gutkin, T., & Wise, S. (1994). Development of the multicultural counseling inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41(2), 137–148. CrossrefGoogle Scholar

  • Spanierman, L. B., Oh, E., Heppner, P. P., Neville, H. A., Mobley, M., Wright, C. V., & Navarro, R. (2011). The multicultural teaching competency scale: Development and initial validation. Urban Education, 46(3), 440–464. CrossrefGoogle Scholar

  • Sue, D. W., Arredondo, P., & McDavis, R. (1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Counseling and Development, 70, 477–486. CrossrefGoogle Scholar

  • Sue, D. W., Bernier, Y., Durran, A., Feinberg, L., Pedersen, P., Smith, E., & Vasquesz-Nuttal, E. (1982). Position paper: Cross cultural counseling competencies. The Counseling Psychologist, 10, 45–52. CrossrefGoogle Scholar

  • Sue, D. W., Capodilupo, C., Torino, G., Bucceri, J., Holder, A., Nadal, K., & Esquilin, M. (2007). Racial microaggressions in everyday life: Implications for clinical practice. American Psychologist, 62(4), 271–286. CrossrefGoogle Scholar

  • Suskie, L. (2004). Assessing student learning: A common sense guide. Boston, MA: Anker Publishing Company.Google Scholar

  • West, P. (1982). Reproducing naturally occurring stories: Vignettes in survey research. Working Paper. Aberdeen: MRC Medical Sociology Unit. Retrieved from http://www.sphsu.mrc.ac.uk/library/other%20reports/WP-May-1982.pdf

  • White, J. & Henderson, S. J. (Eds.). (2008). Building multicultural competency: Development, training and practice. Lanham, MD: Rowman & Littlefield Publishers.Google Scholar

  • Wilks, T. (2004). The use of vignettes in qualitative research into social work values. Qualitative Social Work, 3(1), 78–87. CrossrefGoogle Scholar

  • Wise, L. (1993). Scoring rubrics for performance tasks: Lessons learned from job performance assessment in the military. Paper presented at the Annual Meeting of the National Council of Measurement in Education, Atlanta, GA. Retrieved from http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED358114&ERICExtSearch_SearchType_0=no&accno=ED358114

  • Worthington, R. L., Mobley, M., Franks, R. P., & Tan, J. A. (2000). Multicultural counseling competencies: Verbal content, counselor attributions, and social desirability. Journal of Counseling Psychology, 47, 460–468. CrossrefGoogle Scholar

About the article

Published Online: 2014-07-29

Published in Print: 2016-03-01

Citation Information: Multicultural Learning and Teaching, Volume 11, Issue 1, Pages 53–81, ISSN (Online) 2161-2412, ISSN (Print) 2194-654X, DOI: https://doi.org/10.1515/mlt-2014-0002.

Export Citation

©2016 by De Gruyter.Get Permission

Comments (0)

Please log in or register to comment.
Log in