Automated Essay Feedback Generation and Its Impact on Revision

Published on Oct 1, 2017in IEEE Transactions on Learning Technologies3.72
· DOI :10.1109/TLT.2016.2612659
Ming Liu15
Estimated H-index: 15
(SWU: Southwest University),
Yi Li1
Estimated H-index: 1
+ 1 AuthorsLi Liu19
Estimated H-index: 19
(Chongqing University)
Sources
Abstract
Writing an essay is a very important skill for students to master, but a difficult task for them to overcome. It is particularly true for English as Second Language (ESL) students in China. It would be very useful if students could receive timely and effective feedback about their writing. Automatic essay feedback generation is a challenging task, which requires understanding the relationship between the text features of the essay and feedback. In this study, we first analyzed 1,290 teacher comments on their 327 English-major students and annotated the feedback on seven aspects of writing, including the grammar, spelling, sentence diversity, structure, organization, supporting ideas, coherence, and conclusion , for each paper. Then, an automatic feedback classification experiment was conducted with the machine learning approach. Finally, we investigated the impact of the system generated-indirect corrective feedback (ICF) and human teachers’ direct corrective feedback (DCF) in two English writing classes ( {\mathrm{N\,= \,56}} in ICF class; {\mathrm{N\,= \,54}} in DCF class) at a key Chinese university through a web-based assignment management system. The study results indicated the feasibility of this approach that system generated ICF can be as useful as direct comments made by the teachers in terms of improving the quality of the content regarding to the structure, organization, supporting ideas, coherence, and conclusion, and encouraging students to spend more time on self-correction.
📖 Papers frequently viewed together
2018
2014ICDM: International Conference on Data Mining
References47
Newest
#1Ming Liu (USYD: University of Sydney)H-Index: 15
#2Rafael A. Calvo (USYD: University of Sydney)H-Index: 52
Last. Vasile Rus (U of M: University of Memphis)H-Index: 27
view all 3 authors...
Critical review skill is one important aspect of academic writing. Generic trigger questions have been widely used to support this activity. When students have a concrete topic in mind, trigger questions are less effective if they are too general. This article presents a learning-to-rank based system which automatically generates specific trigger questions from citations for critical review support. The performance of the proposed question ranking models was evaluated and the quality of generate...
#1Danielle S. McNamara (U of M: University of Memphis)H-Index: 87
#2Arthur C. Graesser (U of M: University of Memphis)H-Index: 109
Last. Zhiqiang Cai (U of M: University of Memphis)H-Index: 35
view all 4 authors...
Coh-Metrix is among the broadest and most sophisticated automated textual assessment tools available today. Automated Evaluation of Text and Discourse with Coh-Metrix describes this computational tool, as well as the wide range of language and discourse measures it provides. Section I of the book focuses on the theoretical perspectives that led to the development of Coh-Metrix, its measures, and empirical work that has been conducted using this approach. Section II shifts to the practical arena,...
Source
#1Dana R. Ferris (UC Davis: University of California, Davis)H-Index: 24
#2Hsiang Liu (CSUS: California State University, Sacramento)H-Index: 1
Last. Manuel Senna (UC Davis: University of California, Davis)H-Index: 1
view all 4 authors...
Abstract In this study, the controversial issue of written corrective feedback (WCF) is examined through a longitudinal (16-week semester) multiple-case study approach. Ten L2 writers (from “Generation 1.5” backgrounds) in a developmental ESL writing class at a U.S. university wrote four in-class texts, revised them after receiving WCF, and participated in retrospective interviews after each of the first three writing and revision sessions. Data collected included student background questionnair...
Source
#1Danielle S. McNamara (ASU: Arizona State University)H-Index: 87
#2Scott A. Crossley (GSU: Georgia State University)H-Index: 48
Last. Rod D. Roscoe (ASU: Arizona State University)H-Index: 22
view all 3 authors...
The Writing Pal is an intelligent tutoring system that provides writing strategy training. A large part of its artificial intelligence resides in the natural language processing algorithms to assess essay quality and guide feedback to students. Because writing is often highly nuanced and subjective, the development of these algorithms must consider a broad array of linguistic, rhetorical, and contextual features. This study assesses the potential for computational indices to predict human rating...
Source
#1Scott A. Crossley (GSU: Georgia State University)H-Index: 48
#2Danielle S. McNamara (U of M: University of Memphis)H-Index: 87
This study addresses research gaps in predicting second language (L2) writing proficiency using linguistic features. Key to this analysis is the inclusion of linguistic measures at the surface, textbase and situation model level that assess text cohesion and linguistic sophistication. The results of this study demonstrate that five variables (lexical diversity, word frequency, word meaningfulness, aspect repetition and word familiarity) can be used to significantly predict L2 writing proficiency...
Source
#1Tuomo Kakkonen (University of Eastern Finland)H-Index: 12
#2Erkki Sutinen (University of Eastern Finland)H-Index: 32
This article introduces a model for the assessment of free text that combines both computerised and human models of assessment. This semi-automatic assessment model represents a new generation of assessment models, which offer the benefits of fully automatic grading and feedback together with the advantages of a human assessor. A model such as this must take into account the various linguistic, technical and pedagogical issues that affect the assessment of texts. In this work, the model is appro...
Source
#1Scott A. Crossley (GSU: Georgia State University)H-Index: 48
#2Danielle S. McNamara (U of M: University of Memphis)H-Index: 87
This article reviews recent studies in which human judgements of essay quality are assessed using Coh-Metrix, an automated text analysis tool. The goal of these studies is to better understand the relationship between linguistic features of essays and human judgements of writing quality. Coh-Metrix reports on a wide range of linguistic features, affording analyses of writing at various levels of text structure, including surface, text-base, and situation model levels. Recent studies have examine...
Source
#1Ute Knoch (University of Melbourne)H-Index: 21
Abstract Rating scales act as the de facto test construct in a writing assessment, although inevitably as a simplification of the construct ( North, 2003 ). However, it is often not reported how rating scales are constructed. Unless the underlying framework of a rating scale takes some account of linguistic theory and research in the definition of proficiency, the validity of the scale will be limited ( Lantolf & Frawley, 1985 ). In this paper, the decision-making process when designing a rating...
Source
#1Rachel M. Rufenacht (U of M: University of Memphis)H-Index: 1
#2Philip M. McCarthy (U of M: University of Memphis)H-Index: 18
Last. Travis Alan Lamkin (U of M: University of Memphis)H-Index: 4
view all 3 authors...
Using the Gramulator, we analyzed the linguistic features of ESL texts and fairy tales. Our goal was to determine if fairy tales had the potential to be used as reading material for English language learners. The results of our analyses suggest that there are significant similarities between fairy tales and ESL texts, but that differences lie in the content of the text types with fairy tales appearing significantly more narrative in style and ESL texts appearing more expository.
Source
#1Scott A. Crossley (GSU: Georgia State University)H-Index: 48
This study investigates the importance of human evaluations of coherence in predicting human judgments of holistic essay quality. Of secondary interest is the potential for computational indices of cohesion and coherence to model human judgments of coherence. The results indicate that human judgments of coherence are the most predictive features of holistic essay scores and that computational indices related to text structure, semantic coherence, lexical sophistication, and grammatical complexit...
Cited By9
Newest
Reading comprehension, which has been defined as gaining an understanding of written text through a process of translating grapheme into meaning, is an important academic skill. Other language learning skills - writing, speaking and listening, all are connected to reading comprehension. There have been several measures proposed by researchers to automate the assessment of comprehension skills for second language (L2) learners, especially English as Second Language (ESL) and English as Foreign La...
#1Michael Loncar (NCKU: National Cheng Kung University)H-Index: 1
#2Wayne Schams (National Pingtung University)H-Index: 1
Last. Jong-Shing Liang (National Pingtung University)H-Index: 1
view all 3 authors...
The following review incorporates a systematic selection, coding, and analysis methodology in order to compile a corpus of empirical research studies that investigate the use of technology-mediated...
Source
#1Andrew Potter (UD: University of Delaware)H-Index: 1
#2Joshua Wilson (UD: University of Delaware)H-Index: 13
Automated Writing Evaluation (AWE) provides automatic writing feedback and scoring to support student writing and revising. The purpose of the present study was to analyze a statewide implementation of an AWE software (n = 114,582) in grades 4-11. The goals of the study were to evaluate (a) to what extent AWE features were used, (b) if equity and access issues influenced AWE usage, and (c) if AWE usage was associated with writing performance on a large-scale state writing assessment. Descriptive...
Source
#1Galina Deeva (Katholieke Universiteit Leuven)H-Index: 3
#2Daria Bogdanova (Katholieke Universiteit Leuven)H-Index: 3
Last. Jochen De Weerdt (Katholieke Universiteit Leuven)H-Index: 19
view all 5 authors...
Abstract Teacher feedback provided to learners in real-time is a crucial factor for their knowledge and skills acquisition. However, providing real-time feedback at an individual level is often infeasible, considering limited teaching resources. Fortunately, recent technological advancements have allowed for developing of various computer tutoring systems, which can support learners at any place and time by generating personalized feedback automatically. Such systems have emerged in various doma...
Source
#1Patricia Marybelle Davies (Prince Mohammad bin Fahd University)H-Index: 1
Last. Yanjun Gao (PSU: Pennsylvania State University)H-Index: 3
view all 4 authors...
Source
Source
#1Ming Liu (UTS: University of Technology, Sydney)H-Index: 15
#2Simon Buckingham Shum (UTS: University of Technology, Sydney)H-Index: 47
Last. Cherie Lucas (UTS: University of Technology, Sydney)H-Index: 10
view all 4 authors...
Reflective writing is widely acknowledged to be one of the most effective learning activities for promoting students’ self-reflection and critical thinking. However, manually assessing and giving feedback on reflective writing is time consuming, and known to be challenging for educators. There is little work investigating the potential of automated analysis of reflective writing, and even less on machine learning approaches which offer potential advantages over rule-based approaches. This study ...
Source
#1Prapa Rattadilok (UNNC: The University of Nottingham Ningbo China)H-Index: 6
#2Chris Roadknight (UNNC: The University of Nottingham Ningbo China)H-Index: 1
Assessment is central in effective teaching. This research sets out to discover the impact and effectiveness of timely assessment and feedback on student performance and engagement. Qualitative and quantitative data is collected from two cohorts of students with different levels of engagement. We have shown that more regular feedback and engagement resulted in a significantly improved pass rate and average mark. In conclusion, enabling timely assessment and feedback can improve student performan...
Source
With the rapid development of computer technology and the need of college Eng-lish teaching, the automatic grading of college students' English writing level is an inevitable trend. This is the inevitable result of the combination of computer tech-nology and English teaching. The theoretical basis of this paper is the feedback of English writing and related theories introduced as follows. This paper analyzes the influence of automatic scoring and feedback system on English writing level via test...
Source
#1Kohei Yamamoto (Rits: Ritsumeikan University)
#2Fumiya Kan (Rits: Ritsumeikan University)
Last. Nobuhiko Nishio (Rits: Ritsumeikan University)H-Index: 11
view all 5 authors...
Gesture-based recognition is one of the most intuitive methods for inputting information and is not subject to cumbersome operations. Recognition is performed on human’s consecutive motion without reference to retrial or alternation by user. We propose a gesture recognition model with a mechanism for correcting recognition errors that operates interactively and is practical. We applied the model to a setting involving a manual grading task in order to verify its effectiveness. Our system, named ...
Source
This website uses cookies.
We use cookies to improve your online experience. By continuing to use our website we assume you agree to the placement of these cookies.
To learn more, you can find in our Privacy Policy.