Learning engineering: Difference between revisions

Revision as of 02:55, 13 December 2020

Revision as of 15:35, 15 December 2020

Line 4:

Supporting learners as they learn is complex, and design of learning experiences and support for learners usually requires interdisciplinary teams. Learning engineers themselves might specialize in designing learning experiences that unfold over time, engage the population of learners, and support their learning; automated data collection and analysis; design of learning technologies; design of learning platforms; or some combination. The products of learning engineering teams include on-line courses (e.g., a particular MOOC), software platforms for offering online courses, learning technologies (e.g., ranging from physical manipulatives to electronically-enhanced physical manipulatives to technologies for simulation or modeling to technologies for allowing immersion), after-school programs, community learning experiences, formal curricula, and more. Learning engineering teams require expertise associated with the content that learners will learn, the targeted learners themselves, the venues in which learning is expected to happen, educational practice, software engineering, and sometimes even more.

Learning engineering teams employ an iterative design process for supporting and improving learning. Initial designs are informed by findings from the [[learning sciences]]. Refinements are informed by analysis of data collected as designs are carried out in the world. Methods from [[learning analytics]], [[design-based research]], and rapid large-scale experimentation are used to evaluate designs, inform refinements, and keep track of iterations.<ref>{{Cite web|last1=Dede|first1=Chris|last2=Richards|first2=John|last3=Saxberg|first3=Bror|date=2018|title=Learning Engineering for Online Education: Theoretical Contexts and Design-Based Examples|url=https://www.routledge.com/Learning-Engineering-for-Online-Education-Theoretical-Contexts-and-Design-Based/Dede-Richards-Saxberg/p/book/9780815394426|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Routledge & CRC Press|language=en}}</ref><ref>{{Cite document|last=Saxberg|first=Bror|s2cid=12156278|date=April 2017|title=Learning Engineering {{!}} Proceedings of the Fourth (2017) ACM Conference on Learning @ Scale|language=EN|doi=10.1145/3051457.3054019}}</ref><ref>{{Cite document|last1=Koedinger|first1=Ken|s2cid=29186611|date=April 2016|title=Learning Engineering {{!}} Proceedings of the Third (2016) ACM Conference on Learning @ Scale|language=EN|doi=10.1145/2876034.2876054}}</ref> According to the [[IEEE Standards Association]]'s IC Industry Consortium on Learning Engineering, <em>Learning Engineering is a process and practice that applies the learning sciences using human-centered engineering design methodologies and data-informed decision making to support learners and their development.</em><ref>{{Cite web|date=|title=IEEE ICICLE {{!}} a volunteer professional organization committed to the development of Learning Engineering as a profession and as an academic discipline.|url=https://sagroups.ieee.org/icicle/}}</ref>

== History ==

[[Herbert A. Simon|Herbert Simon]], a [[Cognitive psychology|cognitive psychologist]] and [[economist]], first coined the term “learning engineering” in 1967.<ref>{{Cite web|last=Simon|first=Herbert A.|date=Winter 1967|title=The Job of a College President|url=http://digitalcollections.library.cmu.edu/awweb/awarchive?type=file&item=33692|url-status=live|archive-url=|archive-date=|access-date=|website=Carnegie Mellon University University Libraries - Digital Collections}}</ref> However, associations between the two terms “learning” and “engineering” began emerging earlier, in the 1940s<ref name=":0">{{Cite web|last=Watters|first=Audrey|date=2019-07-12|title=The History of the Future of the 'Learning Engineer'|url=http://hackeducation.com/2019/07/12/learning-engineers|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Hack Education|language=en-US}}</ref> and as early as the 1920s.<ref name=":0" /><ref>{{Cite web|last1=Wilcox|first1=Karen E.|last2=Sarma|first2=Sanjay|last3=Lippel|first3=Philip|date=April 2016|title=Online Education: A Catalyst for Higher Education Reforms|url=https://oepi.mit.edu/files/2016/09/MIT-Online-Education-Policy-Initiative-April-2016.pdf|url-status=live|archive-url=|archive-date=|access-date=|website=MIT Online Education Policy Initiative}}</ref> Simon argued that the social sciences, including the field of education, should be approached with the same kind of mathematical principles as other fields like physics and engineering.<ref>{{Cite web|title=The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 1978|url=https://www.nobelprize.org/prizes/economic-sciences/1978/simon/biographical/|access-date=2020-07-21|website=NobelPrize.org|language=en-US}}</ref>

Simon’s ideas about learning engineering continued to reverberate at Carnegie Mellon University, but the term did not catch on until Bror Saxberg began using it in 2014

.<ref>{{Cite book|last1=Hess|first1=Frederik|last2=Saxberg|first2=Bror|date=2014|title= Breakthrough Leadership in the Digital Age: Using Learning Science to Reboot Schooling |publisher= Corwin Press |isbn= 9781452255491}}</ref> A clear line can be drawn from Simon to Saxberg. In 1978, Herb Simon helped bring [[John Robert Anderson (psychologist)|John Anderson]] to Carnegie Mellon and Anderson soon began to test his theory of cognition within intelligent tutoring systems. In 1998, [[Carnegie Learning]] was spun off producing the first widespread use of intelligent tutoring systems in K12 schools. In 2004, [[Kenneth Koedinger]] and [[Kurt Vanlehn]] started the [[Pittsburgh Science of Learning Center]], or LearnLab for short. Bror Saxberg brought his team from Kaplan to visit CMU. The team went back to Kaplan, armed with LearnLab’s KLI framework,<ref>{{cite journal |last1=Koedinger|first1=Ken|last2=Corbett |first2=Albert|last3=Perfetti|first3=Charles|date=2012 |title=Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning |url=http://pact.cs.cmu.edu/pubs/Koedinger,%20Corbett,%20Perfetti%202012-KLI.pdf|journal=Cognitive Science |volume=36 |issue=5 |pages=757–798|doi=10.1037/a0031955}}</ref> a theoretical framework linking cognition and instruction. They began executing what we now call learning engineering to enhance, optimize, and test their educational products. Bror Saxberg would later co-write the 2014 book using the term “learning engineering”. It caught on this time.

Subsequently, the term “learning engineering” has come to emphasize a focus on applied research (rather than foundational or theoretical research), as well as incorporating research findings about how people learn in order to support learning and improve real-life learning outcomes.<ref>{{Cite web|last=Lieberman|first=Mark|date=|title=Learning Inch Toward the Spotlight|url=https://www.insidehighered.com/digital-learning/article/2018/09/26/learning-engineers-pose-challenges-and-opportunities-improving|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Inside Higher Education|language=en}}</ref>

Line 22:

Learning Engineering initiatives aim to improve educational outcomes by leveraging computing to dramatically increase the applications and effectiveness of learning science as a discipline. Digital learning platforms have generated large amounts of data which can reveal immediately actionable insights.<ref>{{Cite book|last1=Koedinger|first1=Kenneth|last2=Cunningham|first2=Kyle|last3=Skogsholm|first3=Alida|last4=Leber|first4=Brett|last5=Stamper|first5=John|title=Handbook of Educational Data Mining|date=2010-10-25|chapter=A Data Repository for the EDM Community|series=Chapman & Hall/CRC Data Mining and Knowledge Discovery Series|volume=20103384|pages=43–55|chapter-url=https://www.researchgate.net/publication/254199600|doi=10.1201/b10274-6|isbn=978-1-4398-0457-5}}</ref>

The Learning Engineering field has the further potential to communicate educational insights automatically available to educators. For example, learning engineering techniques have been applied to the issue of [[Dropping out|drop-out]] or high failure rates. Traditionally, educators and administrators have to wait until students actually withdraw from school or nearly fail their courses to accurately predict when the drop out will occur. Learning engineers are now able to use data on “off-task behavior”<ref>{{Cite web|last1=Cocea|first1=Mihaela|last2=Hershkovitz|first2=Arnon|last3=Baker|first3=Ryan S.J.d.|date=|title=The Impact of Off-task and Gaming Behaviors on Learning: Immediate or Aggregate?|url=https://www.upenn.edu/learninganalytics/ryanbaker/CoceaHershkovitzBakerFinal.pdf|url-status=live|archive-url=|archive-date=|access-date=|website=Penn Center for Learning Analytics}}</ref> or “wheel spinning”<ref>{{Cite journal|last1=Beck|first1=Joseph E.|last2=Gong|first2=Yue|date=2013|editor-last=Lane|editor-first=H. Chad|editor2-last=Yacef|editor2-first=Kalina|editor3-last=Mostow|editor3-first=Jack|editor4-last=Pavlik|editor4-first=Philip|title=Wheel-Spinning: Students Who Fail to Master a Skill|journal=Artificial Intelligence in Education|series=Lecture Notes in Computer Science|volume=7926|language=en|___location=Berlin, Heidelberg|publisher=Springer|pages=431–440|doi=10.1007/978-3-642-39112-5_44|isbn=978-3-642-39112-5}}</ref> to better understand student engagement and predict whether individual students are likely to fail.

This data enables educators to spot struggling students weeks or months prior to being in danger of dropping out. Proponents of Learning Engineering posit that data analytics will contribute to higher success rates and lower drop-out rates.<ref>{{Cite journal|last1=Milliron|first1=Mark David|last2=Malcolm|first2=Laura|last3=Kil|first3=David|date=Winter 2014|title=Insight and Action Analytics: Three Case Studies to Consider|url=https://eric.ed.gov/?id=EJ1062814|journal=Research & Practice in Assessment|language=en|volume=9|pages=70–89|issn=2161-4210|via=}}</ref>

Learning Engineering can also assist students by providing automatic and individualized feedback.

Line 31:

== Common approaches ==

=== [[A/B testing|A/B Testing]] ===

A/B testing compares two versions of a given program and allows researchers to determine which approach is most effective. In the context of Learning Engineering, platforms like TeacherASSIST<ref>{{Cite web|last=Heffernan|first=Neil|date=|title=TEACHER ASSIST|url=https://sites.google.com/view/neiltheffernan/projects/funded-projects/teacher-assist|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=sites.google.com|language=en-US}}</ref> and [[Coursera]] use A/B testing to determine which type of feedback is the most effective for learning outcomes.<ref>{{Cite web|last=Saber|first=Dan|date=2018-06-15|title=How A/B Testing Powers Pedagogy on Coursera|url=https://medium.com/coursera-engineering/how-a-b-testing-powers-pedagogy-on-coursera-2cd10ed8365e|access-date=2020-07-21|website=Medium|language=en}}</ref>

[[Neil Heffernan]]’s work with TeacherASSIST includes hint messages from teachers that guide students toward correct answers. Heffernan’s lab runs A/B tests between teachers to determine which type of hints result in the best learning for future questions.<ref>{{Cite web|last1=Thanaporn|first1=Patikorn|last2=Heffernan|first2=Neil|date=|title=Effectiveness of Crowd-Sourcing On-Demand Tutoring from Teachers in Online Learning Platforms|url=https://drive.google.com/file/d/1ZRjjie6mMAUBcR1c2JFWZIJ-mKKvTF4C/view?usp=embed_facebook|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Google Docs}}</ref><ref>{{Cite web|last=|first=|date=|title=Programme|url=https://learningatscale.acm.org/las2020/programme/|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Learning @ Scale 2020|language=en-US}}</ref>

[https://www.upgrade-platform.org/ UpGrade] is an open-source platform for A/B testing in education. It allows EdTech companies to run experiments within their own software. [https://www.etrialstestbed.org/ ETRIALS] leverages ASSISTments and give scientists freedom to run experiments in authentic learning environments. [https://terracotta.education/ Terracotta] is a research platform that supports teachers' and researchers' abilities to easily run experiments in live classes.

Line 40:

Educational Data Mining involves analyzing data from student use of educational software to understand how software can improve learning for all students. Researchers in the field, such as [[Ryan S. Baker|Ryan Baker]] at the University of Pennsylvania, have developed models of student learning, engagement, and affect to relate them to learning outcomes.<ref>{{Cite journal|last1=Fischer|first1=Christian|last2=Pardos|first2=Zachary A.|last3=Baker|first3=Ryan Shaun|last4=Williams|first4=Joseph Jay|last5=Smyth|first5=Padhraic|last6=Yu|first6=Renzhe|last7=Slater|first7=Stefan|last8=Baker|first8=Rachel|last9=Warschauer|first9=Mark|s2cid=219091098|date=2020-03-01|title=Mining Big Data in Education: Affordances and Challenges|journal=Review of Research in Education|language=en|volume=44|issue=1|pages=130–160|doi=10.3102/0091732X20903304|issn=0091-732X}}</ref>

=== Platform Instrumentation ===

Education tech platforms link educators and students with resources to improve learning outcomes. For example, Phil Poekert at the [[University of Florida College of Education]]’s Lastinger Center for Learning has created Flamingo,<ref>{{Cite web|last=|first=|date=|title=Flamingo Learning System|url=https://lastinger.center.ufl.edu/innovations/flamingo-learning-system/|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=University of Florida Lastinger Center}}</ref> a platform that integrates critical functionalities like resources and teaching management systems along with a community-based forum.<ref>{{Cite web|last=Poekert|first=Phil|date=2019-10-28|title=Poekert: At the University of Florida, We Have Lots of Data on Students and Math. Now, We Need Researchers to Help Us Mine It|url=https://www.the74million.org/article/poekert-at-the-university-of-florida-we-have-lots-of-data-on-students-and-math-now-we-need-researchers-to-help-us-mine-it/|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=The 74 Million|language=en-US}}</ref>

Other platforms like [https://www.carnegielearning.com/products/software-platform/mathia-learning-software/ MATHia], [https://www.algebranation.com/ms/what-is-algebra-nation/?ref=about Algebra Nation], [https://learnplatform.com/about-us LearnPlatform], [https://coursekata.org/ coursekata], and [[ALEKS]] offer interactive learning environments created to align with key learning outcomes.

=== Dataset Generation ===

Datasets provide the raw material that researchers use to formulate educational insights. For example, Carnegie Mellon University hosts a large volume of learning interaction data in LearnLab's DataShop.<ref>{{Cite web|last=|first=|date=|title=Datashop|url=https://pslcdatashop.web.cmu.edu/index.jsp|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Pittsburgh Science of Learning Center Datashop}}</ref> Their datasets range from sources like Intelligent Writing Tutors<ref>{{Cite web|last=|first=|date=|title=Intelligent Writing Tutor|url=https://pslcdatashop.web.cmu.edu/Project?id=18|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Pittsburgh Science of Learning Center Datashop}}</ref> to Chinese tone studies<ref>{{Cite web|last=|first=|date=|title=Chinese tone study|url=https://pslcdatashop.web.cmu.edu/Project?id=4|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Pittsburgh Science of Learning Center Datashop}}</ref> to data from [[Carnegie Learning]]’s MATHia platform.

[[Kaggle]], a hub for programmers and open source data, regularly hosts machine learning competitions. In 2019, PBS partnered with Kaggle to create the 2019 Data Science Bowl.<ref>{{Cite web|last=|first=|date=|title=2019 Data Science Bowl|url=https://kaggle.com/c/data-science-bowl-2019|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Kaggle|language=en}}</ref> The DataScience Bowl sought machine learning insights from researchers and developers, specifically into how digital media can better facilitate early-childhood STEM learning outcomes.

Datasets, like those hosted by Kaggle PBS and Carnegie Learning, allow researchers to gather information and derive conclusions about student outcomes. These insights help predict student performance in courses and exams.<ref>{{Cite journal|last=Baker|first=Ryan S.J.D.|date=2010|title=Data mining for education|url=http://www.cs.cmu.edu/~rsbaker/Encyclopedia%20Chapter%20Draft%20v10%20-fw.pdf|journal=International Encyclopedia of Education|volume=7|pages=112–118|doi=10.1016/B978-0-08-044894-7.01318-X|via=}}</ref>

=== Learning Engineering in Practice ===

Combining education theory with data analytics has contributed to the development of tools that differentiate between when a student is “wheel spinning” (i.e., not mastering a skill within a set timeframe) and when they are persisting productively.<ref>{{Cite journal|last1=Kai|first1=Shimin|last2=Almeda|first2=Ma Victoria|last3=Baker|first3=Ryan S.|last4=Heffernan|first4=Cristina|last5=Heffernan|first5=Neil|date=2018-06-30|title=Decision Tree Modeling of Wheel-Spinning and Productive Persistence in Skill Builders|url=https://jedm.educationaldatamining.org/index.php/JEDM/article/view/210|journal=JEDM {{!}} Journal of Educational Data Mining|language=en|volume=10|issue=1|pages=36–71|doi=10.5281/zenodo.3344810|issn=2157-2100}}</ref> Tools like ASSISTments<ref>{{Cite web|last=|first=|date=|title=ASSISTments {{!}} Free Education Tool for Teachers & Students|url=https://new.assistments.org/|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=ASSISTments}}</ref> alert teachers when students consistently fail to answer a given problem, which keeps students from tackling insurmountable obstacles,<ref name=":1">{{Cite web|last=Heffernan|first=Neil|date=2019-10-09|title=Persistence Is Not Always Productive: How to Stop Students From Spinning Their Wheels - EdSurge News|url=https://www.edsurge.com/news/2019-10-09-persistence-is-not-always-productive-how-to-stop-students-from-spinning-their-wheels|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=EdSurge|language=en}}</ref> promotes effective feedback<ref name=":1" /> and educator intervention, and increases student engagement.

Studies have found that Learning Engineering may help students and educators to plan their studies before courses begin. For example, UC Berkeley Professor Zach Pardos uses Learning Engineering to help reduce stress for community college students matriculating into four-year institutions.<ref>{{Cite web|last=|first=|date=2019-09-30|title=Zach Pardos is Using Machine Learning to Broaden Pathways from Community College|url=https://www.ischool.berkeley.edu/news/2019/zach-pardos-using-machine-learning-broaden-pathways-community-college|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=UC Berkeley School of Information|language=en}}</ref> Their predictive model analyzes course descriptions and offers recommendations regarding transfer credits and courses that would align with previous directions of study.<ref>{{Cite web|last=Hodges|first=Jill|date=2019-09-30|title=This is Data Science: Using Machine Learning to Broaden Pathways from Community College {{!}} Computing, Data Science, and Society|url=https://data.berkeley.edu/news/data-science-using-machine-learning-broaden-pathways-community-college|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=UC Berkeley - Computing, Data Science, and Society}}</ref>

Similarly, researchers Kelli Bird and Benjamin Castlemen’s work focuses on creating an algorithm to provide automatic, personalized guidance for transfer students.<ref>{{Cite web|last1=Castleman|first1=Benjamin|last2=Bird|first2=Kelli|date=|title=Personalized Pathways to Successful Community College Transfer: Leveraging machine learning strategies to customized transfer guidance and support|url=https://www.povertyactionlab.org/evaluation/personalized-pathways-successful-community-college-transfer-leveraging-machine-learning|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=The Abdul Latif Jameel Poverty Action Lab (J-PAL)|language=en}}</ref> The algorithm is a response to the finding that while 80 percent of community college students intend to transfer to a four-year institution, only roughly 30 percent actually do so.<ref>{{Cite web|last1=Ginder|first1=S.|last2=Kelly-Reid|first2=J.E.|last3=Mann|first3=F.B.|date=2017-12-28|title=Enrollment and Employees in Postsecondary Institutions, Fall 2016; and Financial Statistics and Academic Libraries, Fiscal Year 2016: First Look (Provisional Data)|url=https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2018002|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=National Center for Employment Statistics|language=EN}}</ref> Such research could lead to a higher pass/fail rate<ref>{{Cite journal|last1=Kakish|first1=Kamal|last2=Pollacia|first2=Lissa|date=2018-04-17|title=Adaptive Learning to Improve Student Success and Instructor Efficiency in Introductory Computing Course|url=https://www.researchgate.net/publication/324574230}}</ref> and help educators know when to intervene to prevent student failure or drop out.<ref>{{Cite web|last=Delaney|first=Melissa|date=2019-05-31|title=Universities Use AI to Boost Student Graduation Rates|url=https://edtechmagazine.com/higher/article/2019/05/universities-use-ai-boost-student-graduation-rates|url-status=live|archive-url=|archive-date=|access-date=2020-07-21|website=Technology Solutions That Drive Education|language=en}}</ref><ref>{{Cite journal|last1=Kakish|first1=Kamal|last2=Pollacia|first2=Lissa|date=2018-04-17|title=Adaptive Learning to Improve Student Success and Instructor Efficiency in Introductory Computing Course|url=https://www.researchgate.net/publication/324574230}}</ref>

== Challenges ==

The multidisciplinary nature of learning engineering creates challenges. The problems that learning engineering attempts to solve often require expertise in diverse fields such as [[software engineering]], [[instructional design]], [[___domain knowledge]], [[pedagogy]]/[[andragogy]], [[psychometrics]], [[learning sciences]], [[data science]], and [[systems engineering]]. In some cases, an individual “learning engineer” with expertise in multiple disciplines might be sufficient. However, learning engineering problems often exceed any one person’s ability to solve.