Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK - Part 1
•Descargar como PPTX, PDF•
1 recomendación•433 vistas
Rienties, B et al 2015 Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK. Journal of Interactive Media in Education, X(X): X, pp. 1–11, DOI: http://dx.doi.org/10.5334/jime.az
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (11)
Similar a Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK - Part 1
Similar a Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK - Part 1 (20)
Último
Último (20)
Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK - Part 1
- 1. Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK Avinash Boroowa, Senior Project Manager, Open University
- 2. avinash.boroowa@open.ac.uk | @nashman11178 Acknowledgements 2 ● Project Lead: Kevin Mayles, Head of Analytics ● Implementation: Dr. Bart Rienties, Avinash Boroowa, Tom Olney, Tricia Winters ● Evaluation and analysis: Dr. Bart Rienties, Dr. Christothea Herodotou, Galina Naydenova @DrBartRienties @kevinmayles @nashman11178
- 3. avinash.boroowa@open.ac.uk | @nashman11178 3 Mileage at the start and end of trip Gasoline/petrol
- 4. avinash.boroowa@open.ac.uk | @nashman11178 The Open University Mission 4
- 5. avinash.boroowa@open.ac.uk | @nashman11178 OU Context 2014/15 174k students The average age of our new undergraduate students is 29 40% new undergraduates have 1 A-Level or lower on entry Over 21,000 OU students have disabilities 868k assessments submitted, 395k phone calls and 176k emails received from students 5
- 6. avinash.boroowa@open.ac.uk | @nashman11178 Analytics enhancement strategy The OU is developing its capabilities in 10 key areas that build the underpinning strengths required for the effective deployment of analytics Adapted from Barton and Court (2012) 6
- 7. avinash.boroowa@open.ac.uk | @nashman11178 Underpinning organisational strengths The OU recognised three equally important strengths are required for the effective deployment of analytics We need to ensure we have the right architecture and processes for collecting the right data and making them accessible for analytics – we need a ‘big data’ mind-set 7
- 8. avinash.boroowa@open.ac.uk | @nashman11178 Underpinning organisational strengths The OU recognised three equally important strengths are required for the effective deployment of analytics The university needs world class capability in data science to continually mine the data and build rapid prototypes of simple tools, and a clear pipeline for the outputs to be mainstreamed into operations 8
- 9. avinash.boroowa@open.ac.uk | @nashman11178 Underpinning organisational strengths The OU recognises that three equally important strengths are required for the effective deployment of analytics Benefits will be realised through existing business processes impacting on students directly and through enhancement of the student learning experience – we will develop an ‘analytics mind-set’ in these areas For/in/on-action adapted from Schön (1987) 9
- 10. avinash.boroowa@open.ac.uk | @nashman11178 Analytics enhancement strategy Early alert indicators using predictive analytics Policy on the ethical use of student data for learning analytics Analytics for action evaluation framework Impact of learning design on outcomes 10
- 13. avinash.boroowa@open.ac.uk | @nashman11178 Information for students 13
- 14. avinash.boroowa@open.ac.uk | @nashman11178 Problem Statement 14
- 15. avinash.boroowa@open.ac.uk | @nashman11178 15 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
- 16. avinash.boroowa@open.ac.uk | @nashman11178 16 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
- 17. avinash.boroowa@open.ac.uk | @nashman11178 Student Progression Reports 17
- 18. avinash.boroowa@open.ac.uk | @nashman11178 18 Faculty Dashboard
- 20. avinash.boroowa@open.ac.uk | @nashman11178 20 Module Dashboard
- 21. avinash.boroowa@open.ac.uk | @nashman11178 21 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
- 22. avinash.boroowa@open.ac.uk | @nashman11178 22 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
- 23. avinash.boroowa@open.ac.uk | @nashman11178 Developing the Framework The Social Sciences Brief 23 …but what IS ‘active presentation? We must be engaging in active presentation!
- 27. avinash.boroowa@open.ac.uk | @nashman11178 Supporting Module Teams 27 Briefed over 80 staff Working with 46 modules, meeting each team at least 3 times in the year
- 29. avinash.boroowa@open.ac.uk | @nashman11178 The A4A Toolkit 29 1. What is the trend in the gap between TMA submissions and registration? How does this compare with previous presentations? 2. What is the relationship between VLE activity and TMA submission? How does this compare with previous presentations? 3. How does the module's VLE activity / TMA submissions / retention rate compare with a comparable module on the same presentation? How do TMA submission rates compare on the module? Are there any significant drops in submissions (pinchpoints) between TMAs?
- 30. avinash.boroowa@open.ac.uk | @nashman11178 Support for module teams 30
- 31. avinash.boroowa@open.ac.uk | @nashman11178 31 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
- 32. avinash.boroowa@open.ac.uk | @nashman11178 32 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
- 33. avinash.boroowa@open.ac.uk | @nashman11178 Taking Action 33 Technology Enhanced Learning Team enabled actions Module Team enabled actions Student Support Team enabled actions Associate Lecturer enabled actions Library Services enabled actions
- 34. avinash.boroowa@open.ac.uk | @nashman11178 Taking action 34 Community of inquiry framework for Online Learning (adjusted based upon Stenbom, Cleveland-Innes and Hrastinski)
- 36. avinash.boroowa@open.ac.uk | @nashman11178 Evaluation Methodologies / Protocols 36 A/B Testing Randomised Control Trials (RCTs) Quasi- experimental Tests Apply to all students Pilot Study
- 37. avinash.boroowa@open.ac.uk | @nashman11178 Evaluation – what assistance is available? 37
- 38. avinash.boroowa@open.ac.uk | @nashman11178 38 “We don’t need more and more data – we need information” “There appears to be information overload – I'd like to know how to get through this forest of data” “OK, I can see there is a problem – what do I do about it?”
Notas del editor
- Distance Learning Open Entry Research Explain Module study – 30-60 credits --- 360 over 6-7 years
- Open entry Distance learning
- The OU recognised three equally important strengths are required for the effective deployment of analytics Availability of Data Analysis and Creation of insight Processes that impact student success
- Kevin Right data – data gaps Access – data warehouse / integration Technology – visualisation tool
- Prototyping – predictive modelling – experimental Operation models – have to operate at scale – mature models Interpretation – cycles of activity that align with our business processes to incorporate change / enhancement
- reflect in action (while doing something) and on action (after you have done it) Read more: Reflection and Reflective Practice http://www.learningandteaching.info/learning/reflecti.htm#ixzz49e2X361u Under Creative Commons License: Attribution Non-Commercial No Derivatives
- Explain Module study Module Teams Tutor –Student Ratio Curriculum presentation cycles Investment & review? Speed up review cycle Dist learning – issue with retention.
- Past – MI – module focussed Change to student focus based on study goals
- What is the purpose of this report? The reports are being developed to show student progression and highlight issues that can affect student retention and progression. Students may be counted multiple times where they are active on more than one qualification. Questions answered: Which Qualifications - which qualifications are being studied by students within a faculty? Which Modules - which modules are being studied by students within a faculty? Multiple Module Study: All mod students - what is the study intensity of all students studying a module within a faculty? Multiple Modules: Faculty qual students - what is the study intensity of all qualifications students within a faculty? Assessment Measures: Module - which modules have the lowest TMA01 submission rates within a faculty? Assessment Measures: Qualification - which qualifications have the lowest TMA01 submission rates within a faculty?
- What is the purpose of the report? The reports are being developed to show students’ progression through a qualification and highlight issues that can affect student retention and progression at the qualification level. The tabs show different sections within the qualification dashboard. Currently they are: Qualification Study Plans – how many are studying, in which modules, future planned study, last study. Concurrent Study – students studying 2 or more modules within current presentation, or plans to overlap study. Study intensity - what other modules are currently being studied - i.e. combinations of 2, 3, 4 and more modules Assessment measures - TMA01* results by modules linked to the selected qualification
- What is the purpose of this report? The reports are being developed to show students progression through a module and highlight issues that can affect student retention and progression at the module level. The tabs show different sections within the qualification dashboard. Currently they are: Module Students - this shows a breakdown of what module students are studying Concurrent Study - shows a breakdown of what other modules students are currently studying Module Combinations - shows which other modules students are studying Withdrawers - A view of the number of students who have either withdrawn themselves or by the university 5 and 6. Assessment results by module and broken down by qualification.
- Split Add example of L192 – induction Session E102 K101
- Split Add example of L192 – induction Session E102 K101