Developing and Validating the Competency Profile for Teaching and Learning Research Integrity

  • Jurij Selan Faculty of Education, University of Ljubljana, Slovenia
  • Mira Metljak National Institute of Biology, Ljubljana, Slovenia
Keywords: competency profile, research integrity, responsible conduct of research, factor analysis, aritificial intelligence

Abstract

Since research integrity is not external to research but an integral part of it, it should be integrated into research training. However, several hindrances regarding contemporary research integrity education exist. To address them, we have developed a competency profile for teaching and learning research integrity based on four assumptions: 1) to include all levels of study (BA, MA, and PhD); 2) to integrate research integrity into research education itself; 3) to address research integrity issues in context-specific practices; and 4) to pay particular attention to the ‘grey zone’ or questionable research practices. To assess the validity of the content of the competency profile and to determine if some adjustments to the profile are needed, we translated the competencies of the profile into items of a measurement instrument (a questionnaire) and conducted a survey amongst University of Ljubljana students that allowed us to 1) obtain information about students’ attitudes toward issues of integrity in research; 2) analyse differences in these attitudes among BA, MA, and PhD students; and 3) statistically validate the competency profile and suggest possible improvements. The results showed that 1) students are highly aware of research integrity issues, as scores were high on all items assessed. However, there were some deviations to lower scores, especially in relation to questionable research practises, confirming our assumption that the ‘grey zone’ issues are those that should be particularly addressed and given special attention in contemporary research integrity education. 2) The differences in the attitudes of BA, MA, and PhD students showed that higher-level students showed significantly more awareness of integrity issues than lower-level students did, suggesting that research integrity issues should be given special attention at the BA study level. 3) The measurement characteristics showed that the reliability of the questionnaire was very high, suggesting a good overall structure of the competency profile. The principal component analysis also confirmed the four-field structure of the Competency profile (Values and Principles, Research Practise, Publication and Dissemination, and Violations). However, the analysis also showed that the substructure of the four main areas of the profile did not fully match the results of the factor analysis, suggesting that the distribution of competencies in the competency profile could be reconsidered, especially in the area of Research Practice. The most recent developments in the field of research integrity also suggest that the competency profile should be updated with issues regarding the impact of artificial intelligence on research integrity.

Downloads

Download data is not yet available.

References

Alemu, S. K. (2020). Transnational mobility of academics: Some academic impacts. Center for Educational Policy Studies Journal, 10(2), 77–99. https://doi.org/10.26529/cepsj.464

Antes A., & DuBois J. M. (2014). Aligning objectives and assessment in responsible conduct of research instruction. Journal of Microbiology & Biology Education, 15(2), 108–116. https://doi.org/10.1128/jmbe.v15i2.852

Bebeau, M. J. (2002a). Influencing the moral dimensions of professional practice: Implications for teaching and assessing for research integrity. In N. A., Steneck, & M. H. Sheetz (Eds.), Proceedings of the first ORI research conference on research integrity (pp. 179–187). Office of Research Integrity, Department of Health and Human Services. https://ori.hhs.gov/documents/proceedings_rri.pdf

Bebeau, M. J. (2002b). The defining issues test and the four component model: Contributions to professional education. Journal of Moral Education, 31(3), 271–295. https://doi.org/10.1080/0305724022000008115

Bebeau, M. J. (2002c). Outcome measures for assessing integrity in the research environment (Appendix B). In US National Research Council. Integrity in scientific research: Creating an environment that promotes responsible conduct (pp. 143-166). The National Academies Press. https://www.nap.edu/read/10430/chapter/11

Bebeau, M. J., & Thoma, S. J. (1999). Intermediate concepts and the connection to moral education. Educational Psychology Review, 11(4), 343–360. https://doi.org/10.1023/A:1022057316180

Bison, R. (2023, July 19). Rise of AI stokes fears research misconduct could accelerate. Research Professional News. https://www.researchprofessionalnews.com/rr-news-world-2023-7-rise-of-aistokes-fears-research-misconduct-could-accelerate/

Böttcher, F., & Thiel, F. (2018), Evaluating research-oriented teaching: A new instrument to assess university students’ research competences. Higher Education, 75(1), 91–110. https://doi.org/10.1007/s10734-017-0128-y

Brent, T. (2023, June 27). European research integrity code updated to reflect advances in artificial intelligence. Science Business. https://sciencebusiness.net/news/AI/european-research-integritycode-updated-reflect-advances-artificial-intelligence

Butler, N., Delaney. H., & Spoelstra S. (2017). The gray zone: questionable research practices in the business school. Academy of Management Learning & Education, 16(1), 94–109. https://doi.org/10.5465/amle.2015.0201

Council of Europe (2023, April 26). EduTalks@Council of Europe – Artificial Intelligence and Academic Integrity. Council of Europe. https://www.coe.int/en/web/education/-/edutalks-council-of-europe-artificial-intelligence-andacademic-integrity

Currie, G. M. (2023). Academic integrity and artificial intelligence: is ChatGPT hype, hero or heresy? Seminars in Nuclear Medicine, 53(5), 719–730. https://doi.org/10.1053/j.semnuclmed.2023.04.008

Davis, M., & Feinerman A. (2010). Assessing graduate student progress in engineering ethics. Science and Engineering Ethics, 18(2), 351–367. https://doi.org/10.1007/s11948-010-9250-2

Davis, M., & Riley K. (2008). Ethics across the graduate engineering curriculum: An experiment in teaching and assessment. Teaching Ethics, 9(1), 25–42. https://doi.org/10.5840/tej20089115

Dawson, A. G. (2023). Artificial intelligence and academic integrity. Aspen Publishing.

Diaz-Martinez, L. A., Fisher, G. R., Esparza, D., Bhatt, J. M., D’Arcy, C. E., Apodaca, J., Brownell, S., Corwin, L., Davis, W. B., Floyd, K. W., Killion, P. J., Madden, J., Marsteller, P., Mayfield-Meyer, T., McDonald, K. K., Rosenberg, M., Yarborough, M. A., & Olimpo, J. T. (2019). Recommendations for effective integration of ethics and responsible conduct of research (E/RCR) education into coursebased undergraduate research experiences: A meeting report. CBE—Life Sciences Education, 18(2), 1–10. https://doi.org/10.1187/cbe.18-10-0203

Eaton, S. E. (2023, March 4). Artificial intelligence and academic integrity, post-plagiarism. University World News. https://www.universityworldnews.com/post.php?story=20230228133041549

Eke, D. O. (2023). ChatGPT and the rise of generative AI: Threat to academic integrity? Journal of Responsible Technology, 13. https://doi.org/10.1016/j.jrt.2023.100060

Fanelli, D. (2013). Redefine misconduct as distorted reporting. Nature, 494(7436), 149. https://doi.org/10.1038/494149a

Gonzáles J., & Wagenaar R. (Eds.) (2008). Universities’ contribution to the Bologna Process: An introduction (2nd Edition). Universidad de Deusto. https://www.unideusto.org/tuningeu/publications/278-universitiesacontribution-to-the-bolognaprocess-an-introduction-english-version.html

Hall, J., & Martin, B. R. (2019). Towards a taxonomy of research misconduct: The case of business school research. Research Policy, 48(2), 414–427. https://doi.org/10.1016/j.respol.2018.03.006

Hauser, F., Reuter, R., Gruber, H., & Mottok, J. (2018). Research competence: Modification of a questionnaire to measure research competence at universities of applied sciences. In C. S. González González, M. Castro & M. Llamas Nistal (Eds.), Proceedings of 2018 IEEE Global Engineering Education Conference (EDUCON 2018) (pp. 109–117). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/EDUCON.2018.8363216

Hussain, M. M. (2023). The policy efforts to address racism and discrimination in higher education institutions: the case of Canada. Center for Educational Policy Studies Journal, 13(2), 9–29. https://doi.org/10.26529/cepsj.965

Kohlberg, L. (1969). Stage and sequence: The cognitive development approach to socialization. In D. A. Goslin (Ed.), Handbook of socialization theory (pp. 347–480). Rand McNally.

Kohlberg, L. (1976). Moral stages and moralization: The cognitive-developmental. In T. Lickona (Ed.), Moral development and behavior: Theory, research and social issues (pp. 31–53). Holt, Rinehart and Winston.

LinkedIn (2023, August 3). What are some best practices for validating and updating competency models? https://www.linkedin.com/advice/0/what-some-best-practices-validating-updating-1e

Nanda, N. (2021, December 7). Is artificial intelligence a threat to academic integrity? Original by Turnitin. https://www.ouriginal.com/is-ai-a-threat-to-academic-integrity/

NPA Core Competencies Committee (2007-2009). The NPA postdoctoral core competencies. https://www.nationalpostdoc.org/page/CoreCompetencies

OECD Future of Education and Skills 2030 (2019). OECD learning compass 2030: A series of concept notes. http://www.oecd.org/education/2030-project/teaching-and-learning/learning/learningcompass-2030/OECD_Learning_Compass_2030_Concept_Note_Series.pdf

Olatunde Oduoye, M., Javed, B., Gupta, N., & Valentina Sih, C. M. (2023). Algorithmic Bias and Research integrity; the role of non-human authors in shaping scientific knowledge with respect to Artificial Intelligence (AI); a perspective. International Journal of Surgery. Advance online publication. https://doi.org/10.1097/JS9.0000000000000552

Rest, J. (1983). Morality. In P. H. Mussen, J. Flavell, and E. Markman (Eds.), Handbook of Child Psychology, Vol. 3, Cognitive Development (4th ed.) (pp. 556–629). Wiley.

Schaper, N. (2017). Why is it necessary to validate models of pedagogical competency? GMS Journal for Medical Education, 34(4), 1–8. https://doi.org/10.3205/zma00112

Selan, J., Metljak, M., BerÄnik, S., & Dagarin Fojkar, M. (2021). Competency profile for teaching and learning research integrity. Faculty of Education University of Ljubljana. https://zalozba.pef.uni-lj.si/index.php/zalozba/catalog/book/176

Steneck, N. (2006). Fostering integrity in research: Definition, current knowledge, and future directions. Science and Engineering Ethics, 12(1), 53–74. https://doi.org/10.1007/PL00022268

The European Centre for the Development of Vocational Training (Cedefop) (2011). Glossary: Quality in education and training. Publications Office of the European Union. https://www.cedefop.europa.eu/files/4106_en.pdf

The Ohio State University (2023, April 6). Artificial intelligence and academic integrity. The Ohio State University. https://oaa.osu.edu/artificial-intelligence-and-academic-integrity

The US National Academies of Sciences, Engineering, and Medicine (2017). Fostering integrity in research. The National Academies Press. https://doi.org/10.17226/21896

The US National Research Council (2002). Integrity in scientific research: Creating an environment that promotes responsible conduct. The National Academies Press. https://doi.org/10.17226/10430

Trachtenberg, B. (2023). ChatGPT, artificial intelligence, and academic integrity. Office of Academic Integrity. University of Missouri. https://oai.missouri.edu/chatgpt-artificial-intelligence-and-academic-integrity/

Turnitin. (2023). Academic integrity in the age of AI. Turnitin. https://www.turnitin.com/resources/academic-integrity-in-the-age-of-AI

University of Cambridge (2023). Artificial Intelligence. University of Cambridge. https://www.plagiarism.admin.cam.ac.uk/what-academic-misconduct/artificial-intelligence

York University (2023, August 16). AI technology and academic integrity for instructors. York University. https://www.yorku.ca/unit/vpacad/academic-integrity/ai-technology-and-academic-integrity/

Zobel, J. (2023, May 19). We need to retain research integrity in the AI era. Pursuit and Research, University of Melbourne. https://pursuit.unimelb.edu.au/articles/we-need-to-retain-research-integrity-in-the-ai-era

Published
2023-09-28
How to Cite
Selan, J., & Metljak, M. (2023). Developing and Validating the Competency Profile for Teaching and Learning Research Integrity. Center for Educational Policy Studies Journal, 13(3), 33-74. https://doi.org/10.26529/cepsj.1618