This is an outdated version published on 2024-01-16. Read the most recent version.

Effect of Automated Writing Evaluation in Higher Education Academic Writing Performance

Authors

Mark Anthony Gavina a:1:{s:5:"en_US";s:22:"Saint Louis Univeristy";}
Leilani M. Ibay-Pamo School of Advanced Studies, Saint Louis University, Philippines

DOI:

https://doi.org/10.20414/edulangue.v6i2.8300

Keywords:

Grammarly, Academic Writing, Subject-Verb Agreement, Verb tense, AWE

Abstract

This study aimed to identify the effects of Automated Writing Evaluation (AWE) in the academic writing of thirty-seven (n=37) higher education students as it is believed to address common errors encountered by higher education students in academic writing, specifically in grammar (S-V-A and verb tense). The study used a one-group pretest-posttest design that determined the specific effect of the formulated AWE treatment on the participants. The statistical evidence showed that the AWE tool (Grammarly) feedback provision, when paired with the teacher’s role as facilitator, significantly improved students’ academic writing in terms of S-V-A and verb tense. The findings also imply that the AWE, if integrated within the instruction using its feedback provision, can significantly improve students’ grammar usage and, consequently, their overall academic writing. These inarguable positive results of the AWE intervention used in this study create new assumptions to redefine not only the teachers’ feedback provision in students’ academic writing but also teaching writing in general.

Downloads

Download data is not yet available.

References

Alinsunod, J. (2014). A study of common writing errors of engineering students: A asis for curriculum development. European Journal of English Language and Literature Studies, 2(3), 7-15.

Alzuhairy, U. (2016). The Frequency of The Twelve Verb Tenses in Academic Papers Written by Native Speakers. Electronic Theses and Dissertations. https://stars.library.ucf.edu/etd/5282/

Bacha, N. N. (2002). Developing Learners’ Academic Writing Skills in Higher Education: A Study for Educational Reform. Language and Education, 16(3), 161–177. https://doi.org/10.1080/09500780208666826

Bjo?rkL. A., Netlibrary, I., & Al, E. (2003). Teaching academic writing in European higher education. Kluwer Academic Publishers.

Diane Dewhurst Belcher, & Braine, G. (1995). Academic Writing in a Second Language. Praeger.

Bujang M.A, Khee H.Y, & Yee L.K. (2022). A step-by-step guide to questionnaire validation research. Institute for Clinical Research, NIH, Malaysia.

Campbell, M. (2019). Teaching academic writing in higher education. Education Quarterly Reviews, 2(3). https://doi.org/10.31014/aior.1993.02.03.92

Campbell, D. T., & Stanley, J. C. (2015). Experimental and Quasi-Experimental Designs for Research. Ravenio Books.

CHED (2017). Commission on Higher Education (CHED) Memorandum Order No. 20 series of 2013. https://ched.gov.ph

Chew, P. G. L. (2006). Asian realities in language learning: The case of Singapore. https://hdl.handle.net/10497/16151

Choueiry, G. (2021, September 14). One-Group Pretest-Posttest Design: An Introduction. https://quantifyinghealth.com/one-group-pretest-posttest-design/Conference Papers - The Writing Center. (2014). The writing center.

https://writingcenter.unc.edu/tips-and-tools/conference-papers/

Cotos, E. (2014). Automated writing evaluation. 40–64. https://doi.org/10.1057/9781137333377_3

Dodigovic, M., & Tovmasyan, A. (2021). Automated writing evaluation: the accuracy of Grammarly’s feedback on form. International Journal of TESOL Studies, 3(2), 71-87.

Erni, Y. R. (2018) Utilizing Grammarly in teaching writing recount text through genre based approach. International Journal of Science, Technology and Society. Vol. 6, DOI: 10.11648/j.ijsts.20180601.11

Fahmi, M. A., & Cahyono, B. Y. (2021). EFL students’ perception on the use of Grammarly and teacher feedback. JEES (Journal of English Educators Society), 6(1), 18–25. https://doi.org/10.21070/jees.v6i1.849

Fayol, M., Hupet, M. & Largy, P. The acquisition of subject-verb agreement in written French: From novices to experts' errors. Reading and Writing 11, 153–174 (1999). https://doi.org/10.1023/A:1008038127807

Feng, H.–H., & Chukharev-Hudilainen, E. (2022). Genre-based AWE system for engineering graduate writing: Development and evaluation. Language Learning & Technology, 26(2), 58–77. https://doi.org/10125/73479

Fitria, T. N. (2021). Grammarly as AI-powered English Writing Assistant: Students’ Alternative for Writing English. Metathesis: Journal of English Language, Literature, and Teaching, 5(1), 65. https://doi.org/10.31002/metathesis.v5i1.3519

Folse, K. S., & Betty Schrampfer Azar. (2016). Keys to teaching grammar to English language learners: a practical handbook. University of Michigan Press.

Garcia, J. N. C. (2014). Writing and research in the Philippines: Comparative notes. Kasarinlan: Philippine Journal of Third World Studies.

Grimes, D. (2010, March 2). Utility in a Fallible Tool: A Multi-Site case study of Automated writing evaluation. https://ejournals.bc.edu/index.php/jtla/article/view/1625

Grimes, D., & Warschauer, M. (2010). Utility in a fallible tool: A multi-site case study of automated writing evaluation. The Journal of Technology, Learning and Assessment, 8(6). https://ejournals.bc.edu/index.php/jtla/article/view/1625

Halliday, M. A. K., & Matthiessen, C. M. (2013). Halliday's introduction to functional grammar. Routledge.

Hanauer, D. I., Sheridan, C. L., & Englander, K. (2018). Linguistic injustice in the writing of research articles in English as a second language: Data from Taiwanese and Mexican researchers. Written Communication, 36(1), 136–154. ttps://doi.org/10.1177/0741088318804821

Harris, A. D., McGregor, J. C., Perencevich, E. N., Furuno, J. P., Zhu, J., Peterson, D. E., & Finkelstein, J. (2006). The use and interpretation of quasi-experimental studies in medical informatics. Journal of the American Medical Informatics Association, 13(1),

–23. https://doi.org/10.1197/jamia.m1749

Hayes, J. R., & Flower, L. (1987). On the structure of the writing process. Topics in Language Disorders, 7(4), 19–30. https://doi.org/10.1097/00011363-198709000-00004

Hinkel, E. (2003). Teaching academic ESL writing: Practical techniques in vocabulary and grammar. Routledge.

Hinkel, E. (2013). Research Findings on Teaching Grammar for Academic Writing. English Teaching, 68(4), 3–21. https://doi.org/10.15858/engtea.68.4.201312.3

Hockly, N. (2018). Automated writing evaluation. ELT Journal, 73(1), 82–88. https://doi.org/10.1093/elt/ccy044

Huisman, B., Saab, N., Van Den Broek, P., & Van Driel, J. (2018). The impact of formative peer feedback on higher education students’ academic writing: a Meta-Analysis. Assessment & Evaluation in Higher Education, 44(6), 863–880. https://doi.org/10.1080/02602938.2018.1545896

Jiang, L., Yu, S., & Wang, C. (2020). Second language writing instructors’ feedback practice in response to automated writing evaluation: A sociocultural perspective. System, 93,

https://doi.org/10.1016/j.system.2020.102302

Kaharuddin, K. (2021). Assessing the effect of using artificial intelligence on the writing skill

of Indonesian learners of English. Linguistics and Culture Review, 5(1), 288–304. https://doi.org/10.21744/lingcure.v5n1.1555

Khoshnevisan, B. (2019). The affordances and constraints of automatic writing evaluation (AWE) tools: A case for Grammarly. ARTESOL EFL Journal, 2(2), 12-25.

Lim, F. V., & Phua, J. Y. (2019). Teaching writing with language feedback technology. Computers and Composition, 54, 102518.

https://doi.org/10.1016/j.compcom.2019.102518

Li, J., Link, S., & Hegelheimer, V. (2015). Rethinking the role of automated writing evaluation (AWE) feedback in ESL writing instruction. Journal of Second Language Writing, 27, 1–18. https://doi.org/10.1016/j.jslw.2014.10.004

Masangya, E. M., & Lozada, L. (2009). An investigation on the relationship between the language exposures and errors in English essays of high school students. Philippine ESL Journal, 2, 31.

Nova, M. (2018). Utilizing Grammarly in evaluating academic writing: Anarrative research on EFL students’ experience. Premise: Journal of English Education and Applied Linguistics, 7(1). 2089-3345. https://doi.org/doi.org/10.24127/pj.v7i1

Özkayran, A., & Yilmaz, E. (2020). Analysis of Higher Education Students’ Errors in English Writing Tasks. Advances in Language and Literary Studies, 11(2), 48–58. https://eric.ed.gov/?id=EJ1267155

Pablo, J. C. I., & Lasaten, R. C. S. (2018). Writing difficulties and quality of academic essays of senior high school students. Asia Pacific Journal of Multidisciplinary Research, 6(4), 46-57.

Pangket, W. F., Pangesfan, S. K. K., Cayabas, J. P., & Madjaco, G. L. (2023). Research writing readiness of graduate students in a Philippine state college. International Journal of Learning, Teaching and Educational Research, 22(4), 141-159.

Parra G., L., & Calero S. , X. (2019). Automated Writing Evaluation Tools in the Improvement of the Writing Skill. International Journal of Instruction, 12(2), 209–226. https://eric.ed.gov/?id=EJ1211027

Portillo-San Miguel, E. J. (2021). Writing difficulties encountered by humanities and social sciences students in Philippine politics and governance. International Journal of Language and Literary Studies, 3(3), 156-167

Rahimi, M. & Zhang, L. J. (2018). Writing task complexity, students’ motivational beliefs, anxiety and their writing production in English as a second language. Reid, J. M. (1988). The process of composition. Prentice-Hall.

Sagor, R. (2011). The action research guidebook: A four-stage process for educators and school teams. Corwin Press.

Singh, C. K. S., Jageer Singh, A. K., Abd Razak, N. Q., & Ravinthar, T. (2017). Grammar errors made by ESL tertiary students in writing. English Language Teaching, 10(5), 16. https://doi.org/10.5539/elt.v10n5p16

Portilla, T., & Giovanny, M. (2019). Virtual writing tutor software in the development of academic writing skills. Repositorio.uta.edu.ec. http://repositorio.uta.edu.ec/jspui/handle/123456789/29599

Ware, P. (2011). Computer-generated feedback on student writing. Tesol Quarterly.

Warschauer, M., & Grimes, D. (2008). Automated writing assessment in the classroom. Pedagogies: An International Journal, 3(1), 22–36. https://doi.org/10.1080/15544800701771580.

Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10(2), 157-180.

White, M. G., MA. (2017). Definition of academic writing with examples. In YourDictionary. https://grammar.yourdictionary.com/word-definitions/definition-of- academic-writing.html

Wilson, J., & Roscoe, R. D. (2019). Automated writing evaluation and feedback: Multiple metrics of efficacy. Journal of Educational Computing Research, 58(1), 87–125. https://doi.org/10.1177/0735633119830764

Yamakami, T. (2003). Impact from mobile SPAM mail on mobile internet services. Lecture Notes in Computer Science (pp. 179–184). https://doi.org/10.1007/3-540- 37619-4_19

Yan, Y. (2010). Towards an Eclectic Framework for Teaching EFL Writing in a Chinese Context. (Vol. 7). ERIC. https://eric.ed.gov/?id=ED511226

Yulianti, E. (2018). Utilizing grammarly in teaching writing recount text through genre based approach. International Journal of Science, Technology and Society, 6(1), 1. https://doi.org/10.11648/j.ijsts.20180601.11

Zhang, Z. V. (2020). Engaging with automated writing evaluation (AWE) feedback on L2 writing: Student perceptions and revisions. Assessing Writing, 43, 100439. https://doi.org/10.1016/j.asw.2019.100439