![\"\"](\"https:\/\/pressbooks.bccampus.ca\/tonybates\/wp-content\/uploads\/sites\/1475\/2019\/09\/Robot-vs-women-2.jpg\")
9.4.1<\/span> Focusing on AI's affordances for<\/span> teaching and learning<\/span><\/h2>\r\nArtificial intelligence (AI) is a daunting topic as there are so many issues with respect to its use in education. AI is also currently going through yet another period of extreme hype as a panacea for education, currently being at the top of the peak of inflated expectations, but this hype is driven mainly by successful applications outside the field of education, such as in finance, marketing and medical research<\/span>. Furthermore the term 'AI' is increasingly being used (incorrectly) as a general term for any complex<\/span> computational activity. <\/span>\r\n\r\nEven in education, there are very different possible areas of application of AI. Zeide (2019<\/a>) makes a very useful distinction between institutional, student support and instructional applications (Figure 9.4.2<\/span> below).<\/span>\r\n\r\n[caption id=\"attachment_298\" align=\"aligncenter\" width=\"755\"]![\"\"](\"https:\/\/pressbooks.bccampus.ca\/tonybates\/wp-content\/uploads\/sites\/1475\/2021\/08\/AI-applications-Zeide.png\")
Figure 9.4.2<\/span> AI applications in education Image: \u00a9 Zeide, 2019<\/span>[\/caption]\r\n\r\nAlthough AI applications for institutional or student support purposes are very important, this chapter is focused on the pedagogical affordances of different media and technologies (what Zeide calls 'instructional' applications). In particular, the focus in this section will be on the role of AI as a form of media or technology for teaching and learning, its pedagogical affordances, and its strengths and weaknesses in this area.<\/span>\r\n\r\nMoreover, AI is really a sub-set of computing. Thus all the general affordances of computing in education set out in Chapter 8, Section 5\u00a0<\/a>will apply to AI. This section aims to tease out the extra potential that AI can offer in teaching and learning.\u00a0This will mean particularly focusing on its role as a medium rather than a general technology in teaching, which means looking at a wider context than just the computational aspects of AI, in particular its pedagogical role.<\/span>\r\n9.4.2<\/span> What is artificial intelligence?<\/span><\/h2>\r\nThe original definition of artificial intelligence by McCarthy (1956, cited in Russell & Norvig, 2010<\/a>) is:<\/span><\/p>\r\nevery aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.<\/em><\/span><\/p>\r\nZawacki-Richter et al. (2<\/span><\/a>019<\/span><\/a>), in a review of the literature on AI in higher education, report that those authors that defined artificial intelligence tended to describe it as:<\/span>\r\nintelligent computer systems or intelligent agents with human features, such as the ability to memorise knowledge, to perceive and manipulate their environment in a similar way as humans, and to understand human natural language.<\/em><\/span><\/p>\r\nKlutka et al. (2018) also defined<\/span> AI in terms of what it can do in higher education (Figure 9.4.3<\/span> below):<\/span>\r\n\r\n[caption id=\"attachment_298\" align=\"aligncenter\" width=\"755\"]![\"\"](\"https:\/\/pressbooks.bccampus.ca\/tonybates\/wp-content\/uploads\/sites\/1475\/2021\/08\/AI-what-it-can-do.png\")
Figure 9.4.3<\/span> What AI can do in education Image: Klutka et al. (2018)[\/caption]\r\n\r\nThere are three basic computing requirements that set 'modern' AI apart from other computing applications:<\/span>\r\n\r\n \t- access to massive amounts of data;<\/span><\/li>\r\n \t
- computational power on a large scale to manage and analyze the data;<\/span><\/li>\r\n \t
- powerful and relevant algorithms for the data analysis.<\/span><\/li>\r\n<\/ul>\r\n
9.4.3<\/span> Why use artificial intelligence for teaching and learning?<\/span><\/h2>\r\nThere are two somewhat different goals for the general use of artificial intelligence. The first is to increase the efficiency of a system or organization, primarily by reducing the high costs of labour, namely by replacing relatively expensive human workers with relatively less costly machines (automation). Politicians, entrepreneurs and policy makers increasingly see a move to automation as a way of reducing the costs of education. However, in education in particular, teachers and instructors are the main cost. <\/span>\r\n\r\nThe second is to increase the effectiveness of teaching and learning, in economic terms to increase outputs: better learning outcomes and greater benefits for the same or more cost. With this goal, AI would be used alongside or supporting teachers and instructors.<\/span>\r\n\r\nKlutka et al. (2018<\/a>) provided a general statement of the potential of AI in higher education 'instruction' through Figure 9.4.4:<\/span>\r\n\r\n[caption id=\"attachment_298\" align=\"aligncenter\" width=\"650\"]![\"\"](\"https:\/\/pressbooks.bccampus.ca\/tonybates\/wp-content\/uploads\/sites\/1475\/2021\/08\/AI-goals-2-300x293.jpg\")
Figure 9.4.4<\/span> Goals for AI in higher education instruction Image: Klutka et al. (2018)[\/caption]\r\n\r\nThese are understandable goals, but we shall see later in this section that such goals to date are mainly aspirational rather than real.<\/span>\r\n\r\nIn terms of this book, a key focus is on developing the knowledge and skills required by learners in a digital age. The key test then for artificial intelligence is to what extent it can assist in the development of these higher level skills.<\/span>\r\n9.4.4<\/span> Affordances and examples of AI use in teaching and learning<\/span><\/h2>\r\nZawacki-Richter et al. (2019<\/span><\/a>) in a review of the literature on AI in education initially identified 2,656 research papers in English or Spanish, then narrowed the list down by eliminating duplicates, limiting publication to articles in peer-reviewed journals published between 2007 and 2018, and eliminating articles that turned out in the end not to be about the use of AI in education. This resulted in a final 145 articles which were then analysed. Zawacki-Richter et al. then classified these 145 papers into different uses of AI in education. This section draws heavily on this classification. (It should be noted that within the 145 articles, only 92 were focused on instruction\/student support. The rest were on institutional uses such as identifying at risk students before admission).<\/span>\r\n\r\nThe Zawacki-Richter study offers one insight into the main ways that AI has been used in education for teaching and learning over the ten years between 2007 and 2018, the closest we can come to 'affordances'. First, three main general 'instructional' categories (with considerable overlap) from the study are provided below, followed by some specific examples. (I have omitted Zawacki-Richter et al.'s category of profiling and prediction concerned with administrative issues such as admissions, course scheduling, and early warning systems for students at risk.)<\/span>\r\n9.4.4.1<\/span> Intelligent tutoring systems (29 out of 92 articles reviewed by Zawacki-Richter et al.)<\/span><\/h3>\r\nIntelligent tutoring systems:<\/span>\r\n\r\n \t- provide teaching content to students and, at the same time, support them by giving\u00a0<\/span>adaptive feedback and hints to solve questions related to the content, as well as detecting students'\u00a0<\/span>difficulties\/errors when working with the content or the exercises;<\/span><\/span><\/li>\r\n \t
- curate learning materials based on student needs, such as by providing specific recommendations regarding the type of reading material and exercises done, as well as personalised courses of action;<\/span><\/li>\r\n \t
- facilitate collaboration between learners, for instance, by providing automated feedback, generating automatic questions for discussion, and the analysis of the process.<\/span><\/li>\r\n<\/ul>\r\n
9.4.4.2<\/span> Assessment and evaluation (36 out of 92)<\/span><\/h3>\r\nAI supports assessment and evaluation through:<\/span>\r\n\r\n \t- automated grading;<\/span><\/li>\r\n \t
- \r\n
feedback, including a range of student-facing tools, such as intelligent agents that provide students with prompts or guidance when they are confused or stalled in their work;<\/span><\/p>\r\n<\/li>\r\n \t- \r\n
evaluation of student understanding, engagement and academic integrity.<\/span><\/p>\r\n<\/li>\r\n<\/ul>\r\n9.4.4.3<\/span> Adaptive systems and personalization (27 out of 92)<\/span><\/h3>\r\nAI enables adaptive systems and the personalization of learning by:<\/span>\r\n\r\n \t- teaching course content then diagnosing strengths or gaps in student knowledge, and providing automated feedback;<\/span><\/li>\r\n \t
- recommending personalized content;<\/span><\/li>\r\n \t
- supporting teachers in learning design by recommending appropriate teaching strategies based on student performance;<\/span><\/li>\r\n \t
- supporting representation of knowledge in concept maps.<\/span><\/li>\r\n<\/ul>\r\nKlutka et al. (2018<\/a>) identified several uses of AI for teaching and learning in universities in the USA.\u00a0<\/span>\r\n
\r\n \t- ECoach<\/a>,<\/a> developed at the University of Michigan, provides formative feedback for a variety of mainly large classes in the STEM field. It tracks students progress through a course and directs them to appropriate actions and activities on a personalized basis<\/span><\/li>\r\n \t
- sentiment analysis<\/a>\u00a0<\/span>(using students\u2019 facial expressions to measure their level of engagement in studying),\u00a0<\/span><\/li>\r\n \t
- an application to monitor student engagement in discussion forums<\/a>, and\u00a0<\/span><\/li>\r\n \t
- organizing commonly shared mistakes in exams into groups<\/a><\/span>\u00a0for the instructor to respond once to the group rather than individually.<\/span><\/li>\r\n<\/ul>\r\n
9.4.4.4<\/span> Chatbots<\/span><\/h3>\r\nA chatbot is programming that simulates the conversation or 'chatter' of a human being through text or voice interactions\u00a0(Rouse, 2018<\/a>). Chatbots in particular are a tool used to automate communications with students. Bayne (2014<\/a>) describes one such application in a MOOC with 90,000 subscribers. Much of the student activity took place outside the Coursera platform within social media.\u00a0The five academics teaching the MOOC were all active on Twitter, each with large networks, and Twitter activity around the MOOC hashtag (#edcmooc) was high across all instances of the course (for example, a total of around 180,000 tweets were exchanged on the first offering of the MOOC). A 'Teacherbot' was designed to roam the tweets using the course Twitter hashtag, using keywords to identify 'issues' then choosing pre-designed responses to these issues, which often entailed directing students to more specific research on a topic.\u00a0For a review of research on chatbots in education, see Winkler and S\u00f6llner (2018<\/a>).<\/a><\/span>\r\n9.4.4.5<\/span> Automated essay grading<\/span><\/h3>\r\nThompson (2022)<\/a> provides a simple explanation, aimed mainly at teachers, of how automated essay scoring (AES) works.<\/span>\r\n\u00a0 \u00a0 \u00a0The first and most critical thing to know is that there is not an algorithm that \u201creads\u201d the student essays.\u00a0 Instead, you need to\u00a0 train<\/strong> an algorithm....You have to\u00a0actually grade the essays\u00a0(or at least a large sample of them) and then use that data to fit a machine learning algorithm.<\/span><\/em><\/p>\r\nThis means identifying the rubrics you use in grading essays. You then mark a large number of essays to determine the weight (say on a five point scale) that you assign to each rubric to grade each assignment. You try several AI automated essay grading models and run your assignments through those models a number of times to see which correlates best with your own grading. The models will in fact 'learn' to get better the more essays you run through them.<\/span>\r\n\r\nAs Thompson puts it: <\/span>\r\n\u00a0 \u00a0 \u00a0There is a trade-off between simplicity and accuracy.\u00a0 Complex models might be accurate but take days to run.\u00a0 A simpler model might take 2 hours but with a 5% drop in accuracy....The general consensus in research is that AES algorithms work as well as a second human, and therefore serve very well in that role.\u00a0 But you shouldn\u2019t use them as the only score.<\/em><\/span><\/p>\r\nNatural language processing (NLP) artificial intelligence systems - often called automated essay scoring engines - are now either the primary or secondary grader on standardized tests in at least 21 states in the USA (Feathers, 2019<\/a>).\u00a0According to Feathers:<\/span>\r\nEssay-scoring engines don\u2019t actually analyze the quality of writing. They\u2019re trained on sets of hundreds of example essays to recognize patterns that correlate with higher or lower human-assigned grades. They then predict what score a human would assign an essay, based on those patterns.<\/em><\/span><\/p>\r\nFeathers though claims that research from psychometricians and AI experts show that these tools are susceptible to a common flaw in AI: bias against certain demographic groups (see Ongweso, 2019<\/a>).<\/span>\r\n\r\nLazendic et al. (2018<\/span>)<\/a>\u00a0offer a detailed\u00a0account of the plan for machine grading in Australian high schools. They\u00a0<\/span>state:<\/span>\r\nIt is ...crucially important to acknowledge that the human scoring models, which are developed for each NAPLAN writing prompt, and their consistent application, ensure and maintain the validity of NAPLAN writing assessments. Consequently, the statistical reliability of human scoring outcomes is fundamentally related to and is the key evidence for the validity of NAPLAN writing marking.<\/em><\/span><\/p>\r\nIn other words, the marking must be based on consistent human criteria. However, it was announced later (Hendry, 2018<\/a>) that\u00a0Australian education ministers agreed not to introduce automated essay marking for NAPLAN writing tests, heeding calls from teachers' groups to reject the proposal.<\/span>\r\n\r\nPerelman (2013<\/span>)<\/a> developed a computer program called the BABEL generator\u00a0that patched together strings of sophisticated words and sentences into meaningless gibberish essays. The nonsense essays consistently received high, sometimes perfect, scores when run through several different scoring engines.\u00a0See also Mayfield, 2013<\/span>,<\/a> for a thoughtful analysis of the issues in the automated marking of writing. For a good description of where automated essay scoring is headed, see Kumar and Boulanger (2020<\/a>).<\/span><\/span>\r\n\r\nAES may eventually have potential for marking massive numbers of assignments for nation-wide standard examinations such as the NAPLAN in Australia or the General Certificate of Secondary Education in the U.K., but such methods are still impractical for most individual teachers or instructors. <\/span><\/span>At the time of writing, despite considerable pressure to use automated essay grading for standardized exams, the technology still has many questions lingering over it.<\/span>\r\n9.4.4.6 Online proctoring<\/span><\/h3>\r\nEspecially as a result of the Covid-19\u00a0 pandemic, there has been a rapid increase in the use of AI-based proctoring services to verify whether students taking exams at home are cheating. There is a surprisingly large number of online proctoring companies, such as Examity, Mercer\/Mettle, Proctortrack, OnVUE (from Pearson Publishing), Meazure Learning (formerly ProctorU), and Proctorio. These use cameras installed either in the students' computer or provided by the proctoring company to be used in th students' home or wherever the exam is taken. Most online proctoring comes in two forms: live, with a remote person watching (usually contracted by the proctoring company); or automated. Sometimes there is a mix of both. <\/span>\r\n\r\nIncreasingly, these services use AI to identify possible proxies for cheating behaviour, such as <\/span>\r\n\r\n \t- the students' face not matching a photo ID uploaded prior to the exam,<\/span><\/li>\r\n \t
- 'distraction': movement of the student during the exam beyond the limit of the camera<\/span><\/li>\r\n \t
- other people in the room<\/span><\/li>\r\n \t
- extraneous human sound<\/span><\/li>\r\n \t
- books or other documents on the desk<\/span><\/li>\r\n \t
- a 360 degree view of the room where the student is taking the exam.<\/span><\/li>\r\n<\/ul>\r\nSome companies create, through the use of AI, a 'credibility index' as a result. Students usually have to provide personal data, such as name, address, student number and sometimes credit card information.\u00a0 Students - or even the institution or school requiring the use of the proctoring service - have no control over the use of this personal data, which can be and is often shared with third parties.<\/span>\r\n\r\nThe sensitive information collected by online proctoring companies has raised many concerns among students - and parents, who are automatically excluded from the exam process.<\/span>\r\n\r\nNigam et al. (2021) systematically reviewed 43 papers on AI- and non-AI based proctoring systems published between 2015 and 2021. were listed out from the year 2015 to 2021. They report:<\/span>\r\n
\u00a0 \u00a0 'Our analysis ...reveals that security issues associated with AIPS are multiplying and are a cause of legitimate concern. Major issues include security and privacy concerns, ethical concerns, trust in AI-based technology, lack of training among usage of technology, cost and many more. It is difficult to know whether the benefits of these online proctoring technologies outweigh their risks. The most reasonable conclusion we can reach in the present is that the ethical justification of these technologies and their various capabilities requires us to rigorously ensure that a balance is struck between these concerns and the possible benefits.'\u00a0<\/em><\/span><\/p>\r\nOnline proctoring is a good example of trying to adapt 19th century methods to 21st century technology. Online assessment is discussed more fully in Chapter 6.8.4<\/a>, which indicated that assessment can be done differently with online learning, using for instance continuous assessment, as student learning is automatically tracked through an LMS, or ePortfolios, which allow students to create an authentic digital portfolio of work. What should be avoided is the intrusiveness, lack of privacy, and lack of transparency that come with AI-based proctoring services.<\/span>\r\n9.4.5\u00a0<\/span> Strengths and weaknesses<\/span><\/h2>\r\nThere are several ways to assess the value of the teaching and learning affordances of particular applications of AI in teaching and learning:<\/span>\r\n\r\n \t- is the application based on the three core features of 'modern' AI: massive data sets, massive computing power; powerful and relevant algorithms? <\/span><\/li>\r\n \t
- does the application have clear benefits in terms of affordances over other media, and particularly general computing applications?<\/span><\/li>\r\n \t
- does the application facilitate the development of the skills and knowledge needed in a digital age?<\/span><\/li>\r\n \t
- is there unintended bias built into the algorithms? Does it appear to discriminate against certain categories of people?<\/span><\/li>\r\n \t
- is the application ethical in terms of student and teacher\/instructor privacy and their rights in an open and democratic society?<\/span><\/li>\r\n \t
- are the results of the application 'explainable'? For example, can a teacher or instructor or those responsible for the application understand and explain to students how the results or decisions made by the AI application were reached?<\/span><\/li>\r\n<\/ul>\r\nThese issues are addressed below.<\/span>\r\n
9.4.5.1<\/span> Is it really a 'modern' AI application in teaching and learning?<\/span><\/h3>\r\nLooking at the Zawacki-Richter et al. study and many other research papers published in peer-reviewed journals, very few so-called AI applications in teaching and learning meet the criteria of massive data, massive computing power and powerful and relevant algorithms. Much of the intelligent tutoring within conventional education is what might be termed 'old' AI: there is not a lot of processing going on, and the data points are relatively small. Many so-called AI papers focused on intelligent tutoring and adaptive learning are really just general computing applications.<\/span>\r\n\r\nIndeed, so-called intelligent tutoring systems, automated multiple-choice test marking, and automated feedback on such tests have been around since the early 1980s. The closest to modern AI applications appear to be automated essay grading of standardised tests administered across an entire education system, and use of AI in online proctoring<\/span>. However there are major problems with both such applications<\/span>. More development is clearly needed to make automated essay grading and AI-based online proctoring<\/span> more reliable and secure.<\/span><\/span>\r\n\r\nThe main advantage that Klutka et al. (2018<\/a>) identify for AI is that\u00a0it opens up the possibility for higher education services to become scalable at an unprecedented rate, both inside and outside the classroom.\u00a0However, it is difficult to see how 'modern' AI could be used within the current education system, where class sizes or even whole academic departments, and hence data points, are relatively small, in terms of the numbers needed for 'modern' AI.\u00a0It cannot be said to date that modern AI has been tried, and failed, in teaching and learning; it's not really even been tried.<\/span>\r\n\r\nApplications outside the current formal education <\/span>systems<\/span> are more realistic, for MOOCs, for instance, or for corporate training on an international scale, or for distance teaching universities with very large numbers of students. The requirement for massive data does suggest that the whole education system could be massively disrupted if the necessary scale could be reached by offering modern AI-based education outside<\/em> the existing education systems, for instance by large Internet corporations that could tap into and use<\/span> personal data from<\/span> their massive markets of consumers.<\/span>\r\n\r\nHowever, there is still a long way to go before AI makes that feasible. This is not to say that there could not be such applications of modern AI in the future, but at the moment, in the words of the old English bobby, 'Move along, now, there's nothing to see here.'<\/span>\r\n\r\nHowever, for the sake of argument, let's assume that the definition of AI offered here is too strict and that most of the applications discussed in this section are examples of AI. How do these applications of AI meet the other criteria above?<\/span>\r\n9.4.5.2<\/span> Do the applications facilitate the development of the skills and knowledge needed in a digital age?<\/span><\/h3>\r\nThis does not seem to be the case in most so-called AI applications for teaching and learning today. They are heavily focused on content presentation and testing for understanding and comprehension. In particular, Zawacki-Richter et al. make the point that most AI developments for teaching and learning - or at least the research papers - are by computer scientists, not educators. Since AI tends to be developed by computer scientists, they tend to use models of learning based on how computers or computer networks work (since of course it will be a computer that has to operate the AI). As a result, such AI applications tend to adopt a very behaviourist model of learning: present\/test\/feedback. Lynch (2017<\/a>) argues that:<\/span>\r\nIf AI is going to benefit education, it will require strengthening the connection between AI developers and experts in the learning sciences. Otherwise, AI will simply \u2018discover\u2019 new ways to teach poorly and perpetuate erroneous ideas about teaching and learning.<\/em><\/span><\/p>\r\nComprehension and understanding are indeed important foundational skills, but AI so far is not helping with the development of higher order skills in learners of critical thinking, problem-solving, creativity and knowledge-management.\u00a0Indeed, Klutka et al. (2018<\/a>) claim that that AI can handle many of the routine functions currently done by instructors and administrators, freeing them up to\u00a0solve more complex problems and connect with students on deeper levels. <\/i>This reinforces the view that the role of the instructor or teacher needs to move away from content presentation, content management and testing of content comprehension - all of which can be done by computing - towards skills development. The good news is that AI used in this way supports teachers and instructors, but does not replace them. The bad news is that many teachers and instructors will need to change the way they teach or they will become redundant.<\/span>\r\n9.4.5.3<\/span> Is there unintended bias in the algorithms?<\/span><\/h3>\r\nIt could be argued that all AI does is to encapsulate the existing biases in the system. The problem though is that this bias is often hard to detect in any specific algorithm, and that AI tends to scale up or magnify such biases. These are issues more for institutional uses of AI, but machine-based bias can discriminate against students in a teaching and learning context as well, and especially in automated assessment.<\/span><\/p>\r\n\r\n9.4.5.4<\/span> Is the application ethical?<\/span><\/h3>\r\nThere are many potential ethical issues arising from the use of AI in teaching and learning, mainly due to the lack of transparency in the AI software, and particularly the assumptions embedded in the algorithms. T<\/span>he literature review by Zawacki-Richter et al. (2019) concluded:\u00a0<\/span>\r\n...a stunning result of this review is the dramatic lack of critical reflection of the pedagogical and ethical implications as well as risks of implementing AI applications in higher education<\/em>.\u00a0<\/span><\/p>\r\nWhat data are being collected, who owns or controls it, how is it being interpreted, how will it be used? Policies will need to be put in place to protect students and teachers\/instructors (see for instance the U.S. Department of Education's student data policies<\/a> for schools or British Columbia's Ministry of Advanced Education and Skills Training's<\/span>
The original definition of artificial intelligence by McCarthy (1956, cited in Russell & Norvig, 2010<\/a>) is:<\/span><\/p>\r\n every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves.<\/em><\/span><\/p>\r\nZawacki-Richter et al. (2<\/span><\/a>019<\/span><\/a>), in a review of the literature on AI in higher education, report that those authors that defined artificial intelligence tended to describe it as:<\/span>\r\n intelligent computer systems or intelligent agents with human features, such as the ability to memorise knowledge, to perceive and manipulate their environment in a similar way as humans, and to understand human natural language.<\/em><\/span><\/p>\r\nKlutka et al. (2018) also defined<\/span> AI in terms of what it can do in higher education (Figure 9.4.3<\/span> below):<\/span>\r\n\r\n[caption id=\"attachment_298\" align=\"aligncenter\" width=\"755\"] feedback, including a range of student-facing tools, such as intelligent agents that provide students with prompts or guidance when they are confused or stalled in their work;<\/span><\/p>\r\n<\/li>\r\n \t evaluation of student understanding, engagement and academic integrity.<\/span><\/p>\r\n<\/li>\r\n<\/ul>\r\n \u00a0 \u00a0 \u00a0The first and most critical thing to know is that there is not an algorithm that \u201creads\u201d the student essays.\u00a0 Instead, you need to\u00a0 train<\/strong> an algorithm....You have to\u00a0actually grade the essays\u00a0(or at least a large sample of them) and then use that data to fit a machine learning algorithm.<\/span><\/em><\/p>\r\nThis means identifying the rubrics you use in grading essays. You then mark a large number of essays to determine the weight (say on a five point scale) that you assign to each rubric to grade each assignment. You try several AI automated essay grading models and run your assignments through those models a number of times to see which correlates best with your own grading. The models will in fact 'learn' to get better the more essays you run through them.<\/span>\r\n\r\nAs Thompson puts it: <\/span>\r\n \u00a0 \u00a0 \u00a0There is a trade-off between simplicity and accuracy.\u00a0 Complex models might be accurate but take days to run.\u00a0 A simpler model might take 2 hours but with a 5% drop in accuracy....The general consensus in research is that AES algorithms work as well as a second human, and therefore serve very well in that role.\u00a0 But you shouldn\u2019t use them as the only score.<\/em><\/span><\/p>\r\nNatural language processing (NLP) artificial intelligence systems - often called automated essay scoring engines - are now either the primary or secondary grader on standardized tests in at least 21 states in the USA (Feathers, 2019<\/a>).\u00a0According to Feathers:<\/span>\r\n Essay-scoring engines don\u2019t actually analyze the quality of writing. They\u2019re trained on sets of hundreds of example essays to recognize patterns that correlate with higher or lower human-assigned grades. They then predict what score a human would assign an essay, based on those patterns.<\/em><\/span><\/p>\r\nFeathers though claims that research from psychometricians and AI experts show that these tools are susceptible to a common flaw in AI: bias against certain demographic groups (see Ongweso, 2019<\/a>).<\/span>\r\n\r\nLazendic et al. (2018<\/span>)<\/a>\u00a0offer a detailed\u00a0account of the plan for machine grading in Australian high schools. They\u00a0<\/span>state:<\/span>\r\n It is ...crucially important to acknowledge that the human scoring models, which are developed for each NAPLAN writing prompt, and their consistent application, ensure and maintain the validity of NAPLAN writing assessments. Consequently, the statistical reliability of human scoring outcomes is fundamentally related to and is the key evidence for the validity of NAPLAN writing marking.<\/em><\/span><\/p>\r\nIn other words, the marking must be based on consistent human criteria. However, it was announced later (Hendry, 2018<\/a>) that\u00a0Australian education ministers agreed not to introduce automated essay marking for NAPLAN writing tests, heeding calls from teachers' groups to reject the proposal.<\/span>\r\n\r\nPerelman (2013<\/span>)<\/a> developed a computer program called the BABEL generator\u00a0that patched together strings of sophisticated words and sentences into meaningless gibberish essays. The nonsense essays consistently received high, sometimes perfect, scores when run through several different scoring engines.\u00a0See also Mayfield, 2013<\/span>,<\/a> for a thoughtful analysis of the issues in the automated marking of writing. For a good description of where automated essay scoring is headed, see Kumar and Boulanger (2020<\/a>).<\/span><\/span>\r\n\r\nAES may eventually have potential for marking massive numbers of assignments for nation-wide standard examinations such as the NAPLAN in Australia or the General Certificate of Secondary Education in the U.K., but such methods are still impractical for most individual teachers or instructors. <\/span><\/span>At the time of writing, despite considerable pressure to use automated essay grading for standardized exams, the technology still has many questions lingering over it.<\/span>\r\n \u00a0 \u00a0 'Our analysis ...reveals that security issues associated with AIPS are multiplying and are a cause of legitimate concern. Major issues include security and privacy concerns, ethical concerns, trust in AI-based technology, lack of training among usage of technology, cost and many more. It is difficult to know whether the benefits of these online proctoring technologies outweigh their risks. The most reasonable conclusion we can reach in the present is that the ethical justification of these technologies and their various capabilities requires us to rigorously ensure that a balance is struck between these concerns and the possible benefits.'\u00a0<\/em><\/span><\/p>\r\nOnline proctoring is a good example of trying to adapt 19th century methods to 21st century technology. Online assessment is discussed more fully in Chapter 6.8.4<\/a>, which indicated that assessment can be done differently with online learning, using for instance continuous assessment, as student learning is automatically tracked through an LMS, or ePortfolios, which allow students to create an authentic digital portfolio of work. What should be avoided is the intrusiveness, lack of privacy, and lack of transparency that come with AI-based proctoring services.<\/span>\r\n If AI is going to benefit education, it will require strengthening the connection between AI developers and experts in the learning sciences. Otherwise, AI will simply \u2018discover\u2019 new ways to teach poorly and perpetuate erroneous ideas about teaching and learning.<\/em><\/span><\/p>\r\nComprehension and understanding are indeed important foundational skills, but AI so far is not helping with the development of higher order skills in learners of critical thinking, problem-solving, creativity and knowledge-management.\u00a0Indeed, Klutka et al. (2018<\/a>) claim that that AI can handle many of the routine functions currently done by instructors and administrators, freeing them up to\u00a0solve more complex problems and connect with students on deeper levels. <\/i>This reinforces the view that the role of the instructor or teacher needs to move away from content presentation, content management and testing of content comprehension - all of which can be done by computing - towards skills development. The good news is that AI used in this way supports teachers and instructors, but does not replace them. The bad news is that many teachers and instructors will need to change the way they teach or they will become redundant.<\/span>\r\n It could be argued that all AI does is to encapsulate the existing biases in the system. The problem though is that this bias is often hard to detect in any specific algorithm, and that AI tends to scale up or magnify such biases. These are issues more for institutional uses of AI, but machine-based bias can discriminate against students in a teaching and learning context as well, and especially in automated assessment.<\/span><\/p>\r\n\r\n ...a stunning result of this review is the dramatic lack of critical reflection of the pedagogical and ethical implications as well as risks of implementing AI applications in higher education<\/em>.\u00a0<\/span><\/p>\r\nWhat data are being collected, who owns or controls it, how is it being interpreted, how will it be used? Policies will need to be put in place to protect students and teachers\/instructors (see for instance the U.S. Department of Education's student data policies<\/a> for schools or British Columbia's Ministry of Advanced Education and Skills Training's<\/span> Figure 9.4.3<\/span> What AI can do in education Image: Klutka et al. (2018)[\/caption]\r\n\r\nThere are three basic computing requirements that set 'modern' AI apart from other computing applications:<\/span>\r\n\r\n \t
9.4.3<\/span> Why use artificial intelligence for teaching and learning?<\/span><\/h2>\r\nThere are two somewhat different goals for the general use of artificial intelligence. The first is to increase the efficiency of a system or organization, primarily by reducing the high costs of labour, namely by replacing relatively expensive human workers with relatively less costly machines (automation). Politicians, entrepreneurs and policy makers increasingly see a move to automation as a way of reducing the costs of education. However, in education in particular, teachers and instructors are the main cost. <\/span>\r\n\r\nThe second is to increase the effectiveness of teaching and learning, in economic terms to increase outputs: better learning outcomes and greater benefits for the same or more cost. With this goal, AI would be used alongside or supporting teachers and instructors.<\/span>\r\n\r\nKlutka et al. (2018<\/a>) provided a general statement of the potential of AI in higher education 'instruction' through Figure 9.4.4:<\/span>\r\n\r\n[caption id=\"attachment_298\" align=\"aligncenter\" width=\"650\"]
Figure 9.4.4<\/span> Goals for AI in higher education instruction Image: Klutka et al. (2018)[\/caption]\r\n\r\nThese are understandable goals, but we shall see later in this section that such goals to date are mainly aspirational rather than real.<\/span>\r\n\r\nIn terms of this book, a key focus is on developing the knowledge and skills required by learners in a digital age. The key test then for artificial intelligence is to what extent it can assist in the development of these higher level skills.<\/span>\r\n9.4.4<\/span> Affordances and examples of AI use in teaching and learning<\/span><\/h2>\r\nZawacki-Richter et al. (2019<\/span><\/a>) in a review of the literature on AI in education initially identified 2,656 research papers in English or Spanish, then narrowed the list down by eliminating duplicates, limiting publication to articles in peer-reviewed journals published between 2007 and 2018, and eliminating articles that turned out in the end not to be about the use of AI in education. This resulted in a final 145 articles which were then analysed. Zawacki-Richter et al. then classified these 145 papers into different uses of AI in education. This section draws heavily on this classification. (It should be noted that within the 145 articles, only 92 were focused on instruction\/student support. The rest were on institutional uses such as identifying at risk students before admission).<\/span>\r\n\r\nThe Zawacki-Richter study offers one insight into the main ways that AI has been used in education for teaching and learning over the ten years between 2007 and 2018, the closest we can come to 'affordances'. First, three main general 'instructional' categories (with considerable overlap) from the study are provided below, followed by some specific examples. (I have omitted Zawacki-Richter et al.'s category of profiling and prediction concerned with administrative issues such as admissions, course scheduling, and early warning systems for students at risk.)<\/span>\r\n
9.4.4.1<\/span> Intelligent tutoring systems (29 out of 92 articles reviewed by Zawacki-Richter et al.)<\/span><\/h3>\r\nIntelligent tutoring systems:<\/span>\r\n
\r\n \t
9.4.4.2<\/span> Assessment and evaluation (36 out of 92)<\/span><\/h3>\r\nAI supports assessment and evaluation through:<\/span>\r\n
\r\n \t
9.4.4.3<\/span> Adaptive systems and personalization (27 out of 92)<\/span><\/h3>\r\nAI enables adaptive systems and the personalization of learning by:<\/span>\r\n
\r\n \t
\r\n \t
9.4.4.4<\/span> Chatbots<\/span><\/h3>\r\nA chatbot is programming that simulates the conversation or 'chatter' of a human being through text or voice interactions\u00a0(Rouse, 2018<\/a>). Chatbots in particular are a tool used to automate communications with students. Bayne (2014<\/a>) describes one such application in a MOOC with 90,000 subscribers. Much of the student activity took place outside the Coursera platform within social media.\u00a0The five academics teaching the MOOC were all active on Twitter, each with large networks, and Twitter activity around the MOOC hashtag (#edcmooc) was high across all instances of the course (for example, a total of around 180,000 tweets were exchanged on the first offering of the MOOC). A 'Teacherbot' was designed to roam the tweets using the course Twitter hashtag, using keywords to identify 'issues' then choosing pre-designed responses to these issues, which often entailed directing students to more specific research on a topic.\u00a0For a review of research on chatbots in education, see Winkler and S\u00f6llner (2018<\/a>).<\/a><\/span>\r\n
9.4.4.5<\/span> Automated essay grading<\/span><\/h3>\r\nThompson (2022)<\/a> provides a simple explanation, aimed mainly at teachers, of how automated essay scoring (AES) works.<\/span>\r\n
9.4.4.6 Online proctoring<\/span><\/h3>\r\nEspecially as a result of the Covid-19\u00a0 pandemic, there has been a rapid increase in the use of AI-based proctoring services to verify whether students taking exams at home are cheating. There is a surprisingly large number of online proctoring companies, such as Examity, Mercer\/Mettle, Proctortrack, OnVUE (from Pearson Publishing), Meazure Learning (formerly ProctorU), and Proctorio. These use cameras installed either in the students' computer or provided by the proctoring company to be used in th students' home or wherever the exam is taken. Most online proctoring comes in two forms: live, with a remote person watching (usually contracted by the proctoring company); or automated. Sometimes there is a mix of both. <\/span>\r\n\r\nIncreasingly, these services use AI to identify possible proxies for cheating behaviour, such as <\/span>\r\n
\r\n \t
9.4.5\u00a0<\/span> Strengths and weaknesses<\/span><\/h2>\r\nThere are several ways to assess the value of the teaching and learning affordances of particular applications of AI in teaching and learning:<\/span>\r\n
\r\n \t
9.4.5.1<\/span> Is it really a 'modern' AI application in teaching and learning?<\/span><\/h3>\r\nLooking at the Zawacki-Richter et al. study and many other research papers published in peer-reviewed journals, very few so-called AI applications in teaching and learning meet the criteria of massive data, massive computing power and powerful and relevant algorithms. Much of the intelligent tutoring within conventional education is what might be termed 'old' AI: there is not a lot of processing going on, and the data points are relatively small. Many so-called AI papers focused on intelligent tutoring and adaptive learning are really just general computing applications.<\/span>\r\n\r\nIndeed, so-called intelligent tutoring systems, automated multiple-choice test marking, and automated feedback on such tests have been around since the early 1980s. The closest to modern AI applications appear to be automated essay grading of standardised tests administered across an entire education system, and use of AI in online proctoring<\/span>. However there are major problems with both such applications<\/span>. More development is clearly needed to make automated essay grading and AI-based online proctoring<\/span> more reliable and secure.<\/span><\/span>\r\n\r\nThe main advantage that Klutka et al. (2018<\/a>) identify for AI is that\u00a0it opens up the possibility for higher education services to become scalable at an unprecedented rate, both inside and outside the classroom.\u00a0However, it is difficult to see how 'modern' AI could be used within the current education system, where class sizes or even whole academic departments, and hence data points, are relatively small, in terms of the numbers needed for 'modern' AI.\u00a0It cannot be said to date that modern AI has been tried, and failed, in teaching and learning; it's not really even been tried.<\/span>\r\n\r\nApplications outside the current formal education <\/span>systems<\/span> are more realistic, for MOOCs, for instance, or for corporate training on an international scale, or for distance teaching universities with very large numbers of students. The requirement for massive data does suggest that the whole education system could be massively disrupted if the necessary scale could be reached by offering modern AI-based education outside<\/em> the existing education systems, for instance by large Internet corporations that could tap into and use<\/span> personal data from<\/span> their massive markets of consumers.<\/span>\r\n\r\nHowever, there is still a long way to go before AI makes that feasible. This is not to say that there could not be such applications of modern AI in the future, but at the moment, in the words of the old English bobby, 'Move along, now, there's nothing to see here.'<\/span>\r\n\r\nHowever, for the sake of argument, let's assume that the definition of AI offered here is too strict and that most of the applications discussed in this section are examples of AI. How do these applications of AI meet the other criteria above?<\/span>\r\n
9.4.5.2<\/span> Do the applications facilitate the development of the skills and knowledge needed in a digital age?<\/span><\/h3>\r\nThis does not seem to be the case in most so-called AI applications for teaching and learning today. They are heavily focused on content presentation and testing for understanding and comprehension. In particular, Zawacki-Richter et al. make the point that most AI developments for teaching and learning - or at least the research papers - are by computer scientists, not educators. Since AI tends to be developed by computer scientists, they tend to use models of learning based on how computers or computer networks work (since of course it will be a computer that has to operate the AI). As a result, such AI applications tend to adopt a very behaviourist model of learning: present\/test\/feedback. Lynch (2017<\/a>) argues that:<\/span>\r\n
9.4.5.3<\/span> Is there unintended bias in the algorithms?<\/span><\/h3>\r\n
9.4.5.4<\/span> Is the application ethical?<\/span><\/h3>\r\nThere are many potential ethical issues arising from the use of AI in teaching and learning, mainly due to the lack of transparency in the AI software, and particularly the assumptions embedded in the algorithms. T<\/span>he literature review by Zawacki-Richter et al. (2019) concluded:\u00a0<\/span>\r\n