SlideShare una empresa de Scribd logo
1 de 31
Descargar para leer sin conexión
Introduction to quantitative and
qualitative research
Dr Liz FitzGerald
Institute of Educational Technology
Research and research methods
• Research methods are split broadly into
quantitative and qualitative methods
• Which you choose will depend on
– your research questions
– your underlying philosophy of research
– your preferences and skills
Basic principles of research design
Four main features of research design, which are distinct, but closely related
• Ontology: How you, the researcher, view the world and the assumptions that you
make about the nature of the world and of reality
• Epistemology: The assumptions that you make about the best way of investigating
the world and about reality
• Methodology: The way that you group together your research techniques to make
a coherent picture
• Methods and techniques: What you actually do in order to collect your data and
carry out your investigations
• These principles will inform which methods you choose: you need to understand
how they fit with your ‘bigger picture’ of the world, and how you choose to
investigate it, to ensure that your work will be coherent and effective
Four main schools of ontology
(how we construct reality)
Ontology Realism Internal Realism Relativism Nominalism
Summary
The world is ‘real’, and
science proceeds by
examining and
observing it
The world is real, but
it is almost impossible
to examine it directly
Scientific laws are
basically created by
people to fit their
view of reality
Reality is entirely
created by people,
and there is no
external ‘truth’
Truth There is a single truth
Truth exists, but is
obscure
There are many
truths
There is no truth
Facts
Facts exist, and can be
revealed through
experiments
Facts are concrete,
but cannot always be
revealed
Facts depend on
the viewpoint of
the observer
Facts are all human
creations
However, none of these positions are absolutes.
They are on a continuum, with overlaps between them.
Epistemology
i.e. the way in which you choose to investigate the world
Two main schools are positivism and social constructionism:
• Positivists believe that the best way to investigate the world
is through objective methods, such as observations.
Positivism fits within a realist ontology.
• Social constructionists believe that reality does not exist by
itself. Instead, it is constructed and given meaning by
people. Their focus is therefore on feelings, beliefs and
thoughts, and how people communicate these. Social
constructionism fits better with a relativist ontology.
Methodology
• Epistemology and ontology will have implications for your
methodology
• Realists tend to have positivist approach
 tend to gather quantitative sources of data
• Relativists tend to have a social constructionist approach
 tend to gather qualitative sources of data
• Remember these are not absolutes! People tend to work
on a continuum  role for mixed methods and approaches
• Also consider the role of the researcher*: internal/external;
involved or detached?
* See also Adams, Anne; FitzGerald, Elizabeth and Priestnall, Gary (2013). Of catwalk
technologies and boundary creatures. ACM Transactions on Computer-Human Interaction
(TOCHI), 20(3), article no. 15. http://oro.open.ac.uk/35323/
A note about data
• Quantitative data is about quantities, and
therefore numbers
• Qualitative data is about the nature of the thing
investigated, and tends to be words rather than
numbers
• Difference between primary and secondary data
sources
• Be aware of research data management practices
and archives of data sets (both in terms of
downloading and uploading)
Choosing your approach
• Your approach may be influenced by your colleagues’ views, your organisation’s
approach, your supervisor’s beliefs, and your own experience
• There is no right or wrong answer to choosing your research methods
• Whatever approach you choose for your research, you need to consider five
questions:
– What is the unit of analysis? For example, country, company or individual.
– Are you relying on universal theory or local knowledge? i.e. will your results be generalisable,
and produce universally applicable results, or are there local factors that will affect your
results?
– Will theory or data come first? Should you read the literature first, and then develop your
theory, or will you gather your data and develop your theory from that? (N.B. this will likely be
an iterative process)
– Will your study be cross-sectional or longitudinal? Are you looking at one point in time, or
changes over time?
– Will you verify or falsify a theory? You cannot conclusively prove any theory; the best that you
can do is find nothing that disproves it. It is therefore easier to formulate a theory that you can
try to disprove, because you only need one ‘wrong’ answer to do so.
Quantitative approaches
• Attempts to explain phenomena by collecting and analysing
numerical data
• Tells you if there is a “difference” but not necessarily why
• Data collected are always numerical and analysed using
statistical methods
• Variables are controlled as much as possible (RCD as the gold
standard) so we can eliminate interference and measure the
effect of any change
• Randomisation to reduce subjective bias
• If there are no numbers involved, its not quantitative
• Some types of research lend themselves better to quant
approaches than others
Quantitative data
• Data sources include
– Surveys where there are a large number of
respondents (esp where you have used a Likert
scale)
– Observations (counts of numbers and/or coding
data into numbers)
– Secondary data (government data; SATs scores
etc)
• Analysis techniques include hypothesis
testing, correlations and cluster analysis
Black swans and falsifiability
• Hypothesis testing
• Start with null hypothesis
i.e. H0 – that there will be no difference
https://www.flickr.com/photos/lselibrary/
IMAGELIBRARY/5
• Falsifiability or refutability of a
statement, hypothesis, or theory is the
inherent possibility that it can be proven
false
• Karl Popper and the black swan;
deductive c.f. inductive reasoning
CC BY-SA 3.0,
https://commons.wikimedia.org/w/index.php?curid=1243220
Type I and Type II errors
Analysing quant data
• Always good to group and/or visualise the
data initially  outliers/cleaning data
• What average are you looking for?
Mean, median or mode?
• Spread of data:
– skewness/distribution
– range, variance and standard deviation
What are you looking for?
• Trying to find the signal from the noise
• Generally, either a difference (between/within
groups) or a correlation
• Choosing the right test to use:
parametric vs non-parametric (depends what
sort of data you have – interval/ratio vs
nominal/ordinal and how it is distributed)
• Correlation does not imply causation!
Example correlations
From ‘Spurious
correlations’ website
http://www.tylervigen.com
/spurious-correlations
Interpreting test statistics
• Significance level – a fixed probability of wrongly
rejecting the null hypothesis H0, if it is in fact true.
Usually set to 0.05 (5%).
• p value - probability of getting a value of the test
statistic as extreme as or more extreme than that
observed by chance alone, if the null hypothesis H0, is
true.
• Power – ability to detect a difference if there is one
• Effect size – numerical way of expressing the strength
or magnitude of a reported relationship, be it causal or
not
Example of quant data/analysis*
• Matched users were those who learning styles were matched with
the lesson plan e.g. sequential users with a sequential lesson plan.
Mismatched participants used a lesson plan that was not matched
to their learning style, e.g. sequential users with a global lesson
plan.
• H0 – there will be no statistically significant difference in knowledge
gained between users from different experimental groups
• H1 – students who learn in a matched environment will learn
significantly better than those who are in mismatched environment
• H2 – students who learn in a mismatched environment will learn
significantly worse than those who learn in a matched environment
* Case study taken from: Brown, Elizabeth (2007) The use of learning styles in adaptive
hypermedia. PhD thesis, University of Nottingham. http://eprints.nottingham.ac.uk/10577/
Interpreting test statistics
• Statistical testing was carried out using a univariate ANOVA in
SPSS, to determine if there was any significant difference in
knowledge gained.
• Initial conjecture suggests that the mismatched group actually
performed better than the matched group.
• However, the difference between the two groups was not
significant (F(1,80)=0.939, p=0.34, partial eta squared = 0.012)
and hence hypotheses 1 and 2 can be rejected.
What quant researchers worry about
• Is my sample size big enough?
• Have I used the correct statistical test?
• have I reduced the likelihood of making Type I
and/or Type II errors?
• Are my results generalisable?
• Are my results/methods/results reproducible?
• Am I measuring things the right way?
What’s wrong with quant research?
• Some things can’t be measured – or measured
accurately
• Doesn’t tell you why
• Can be impersonal – no engagement with human
behaviours or individuals
• Data can be static – snapshots of a point in time
• Can tell a version of the truth (or a lie?)
“Lies, damned lies and statistics” – persuasive
power of numbers
Qualitative approaches
• Any research that doesn’t involve numerical
data
• Instead uses words, pictures, photos, videos,
audio recordings. Field notes, generalities.
Peoples’ own words.
• Tends to start with a broad question rather
than a specific hypothesis
• Develop theory rather than start with one
 inductive rather than deductive
Gathering qual data
• Tends to yield rich data to explore how and why things
happened
• Don’t need large sample sizes (in comparison to
quantitative research)
• Some issues may arise, such as
– Respondents providing inaccurate or false information – or
saying what they think the researcher wants to hear
– Ethical issues may be more problematic as the researcher
is usually closer to participants
– Researcher objectivity may be more difficult to achieve
Sources of qual data
• Interviews (structured, semi-structured or
unstructured)
• Focus groups
• Questionnaires or surveys
• Secondary data, including diaries, self-reporting,
written accounts of past events/archive data and
company reports;
• Direct observations – may also be recorded
(video/audio)
• Ethnography
Analysing qual data
• Content analysis
• Grounded analysis
• Social network analysis (can also be quant)
• Discourse analysis
• Narrative analysis
• Conversation analysis
Example of qual data research*
• Describing and comparing two
types of audio guides: person-
led and technology-led
• Geolocated audio to enable
public, informal learning of
historical events
• Data sources: questionnaires,
researcher observations, and
small focus groups
* Taken from: FitzGerald, Elizabeth; Taylor, Claire and Craven, Michael (2013). To the
Castle! A comparison of two audio guides to enable public discovery of historical events.
Personal and Ubiquitous Computing, 17(4) pp. 749–760. http://oro.open.ac.uk/35077/
Data analysis and findings
• Comparison of the two different walks
– Differences/similarities of the walks
– Issues surrounding participant engagement
• Thematic analysis
– Mode of delivery
– Number of participants and social interactions
– Geographical affordances of places and locations
– User experience
– Opportunities for learning
– Other factors
• Findings, lessons learned, recommendations
What qual researchers worry about
• Have I coded my data correctly?
• Have I managed to capture the situation in a
realistic manner?
• Have I described the context in sufficient
detail?
• Have I managed to see the world through the
eyes of my participants?
• Is my approach flexible and able to change?
What’s wrong with qual research?
• It can be very subjective
• It can’t always be repeated
• It can’t always be generalisable
• It can’t always give you definite answers in the
way that quantitative research can
• It can be easier to carry out (or hide) ‘bad’
(poor quality) qual research than ‘bad’ quant
research
Other aspects of research design
• Validity
• Reliability
• Trustworthiness*
– Dependability: showing that the findings are consistent
and could be repeated
– Confirmability: a degree of neutrality or the extent to
which the findings of a study are shaped by the
respondents and not researcher bias, motivation, or
interest
– Credibility: confidence in the 'truth' of the findings
– Transferability: showing that the findings have applicability
in other contexts
* See Lincoln, YS. & Guba, EG. (1985). Naturalistic Inquiry.
Newbury Park, CA: Sage Publications.
Summary
• The type of approach you choose will be determined
by your research question, your epistemological and
ontological stances and your skills or ability to utilise a
certain appoach
• For most people in ed tech, a mixed methods approach
will be used
• So long as you make an informed choice and can justify
it, it should be fine 
• Just be aware of the limitations of your approach(es)
and try to compensate where necessary
Acknowledgments and further links
• Some content borrowed from SkillsYouNeed website
(http://www.skillsyouneed.com/learn/research-methods.html)
Other useful links:
• Introduction to Quantitative and Qualitative Research Models (William
Bardebes). PDF at http://tinyurl.com/qq-models
• Methods Map: http://www.methodsmap.org
• Ready To Research: http://readytoresearch.ac.uk
• Methods@Manchester:
http://www.methods.manchester.ac.uk/resources/categories
• Research Data Management training: http://datalib.edina.ac.uk/mantra/

Más contenido relacionado

La actualidad más candente

Quantitative Data Analysis
Quantitative Data AnalysisQuantitative Data Analysis
Quantitative Data AnalysisAsma Muhamad
 
Pragmatic approach to research
Pragmatic approach to researchPragmatic approach to research
Pragmatic approach to researchayoub babar
 
Mixed research-methods (1)
Mixed research-methods (1)Mixed research-methods (1)
Mixed research-methods (1)Rakibul islam
 
Quantitative reseach method
Quantitative reseach methodQuantitative reseach method
Quantitative reseach methodmetalkid132
 
Research Types & Characteristicsch types & characteristics
Research Types & Characteristicsch types & characteristicsResearch Types & Characteristicsch types & characteristics
Research Types & Characteristicsch types & characteristicsmamuntaj
 
research Qualitative vs. quantitative research
research Qualitative vs. quantitative researchresearch Qualitative vs. quantitative research
research Qualitative vs. quantitative researchgagan deep
 
Mixed Method Research.pptx
Mixed Method Research.pptxMixed Method Research.pptx
Mixed Method Research.pptxDevarajuBn
 
Longitudinal vs. Cross-sectional Studies
Longitudinal vs. Cross-sectional StudiesLongitudinal vs. Cross-sectional Studies
Longitudinal vs. Cross-sectional StudiesEda Nur Ozcan
 
Qualitative research - type of data, analysis of qualitative data, software f...
Qualitative research - type of data, analysis of qualitative data, software f...Qualitative research - type of data, analysis of qualitative data, software f...
Qualitative research - type of data, analysis of qualitative data, software f...Dr.Preeti Tiwari
 

La actualidad más candente (20)

Quantitative Data Analysis
Quantitative Data AnalysisQuantitative Data Analysis
Quantitative Data Analysis
 
Pragmatic approach to research
Pragmatic approach to researchPragmatic approach to research
Pragmatic approach to research
 
Qualitative research
Qualitative researchQualitative research
Qualitative research
 
Mixed research-methods (1)
Mixed research-methods (1)Mixed research-methods (1)
Mixed research-methods (1)
 
Case Study Research Methods
Case Study Research MethodsCase Study Research Methods
Case Study Research Methods
 
The research instruments
The research instrumentsThe research instruments
The research instruments
 
Quantitative reseach method
Quantitative reseach methodQuantitative reseach method
Quantitative reseach method
 
Research Types & Characteristicsch types & characteristics
Research Types & Characteristicsch types & characteristicsResearch Types & Characteristicsch types & characteristics
Research Types & Characteristicsch types & characteristics
 
Case study Research
Case study Research Case study Research
Case study Research
 
Mixed method
Mixed methodMixed method
Mixed method
 
research Qualitative vs. quantitative research
research Qualitative vs. quantitative researchresearch Qualitative vs. quantitative research
research Qualitative vs. quantitative research
 
Qualitative research designs
Qualitative research designsQualitative research designs
Qualitative research designs
 
Grounded Theory
Grounded TheoryGrounded Theory
Grounded Theory
 
PHENOMENOLOGICAL RESEARCH
PHENOMENOLOGICAL RESEARCHPHENOMENOLOGICAL RESEARCH
PHENOMENOLOGICAL RESEARCH
 
Research design
Research designResearch design
Research design
 
Questionnaire development
Questionnaire developmentQuestionnaire development
Questionnaire development
 
Mixed Method Research.pptx
Mixed Method Research.pptxMixed Method Research.pptx
Mixed Method Research.pptx
 
Types of research
Types of researchTypes of research
Types of research
 
Longitudinal vs. Cross-sectional Studies
Longitudinal vs. Cross-sectional StudiesLongitudinal vs. Cross-sectional Studies
Longitudinal vs. Cross-sectional Studies
 
Qualitative research - type of data, analysis of qualitative data, software f...
Qualitative research - type of data, analysis of qualitative data, software f...Qualitative research - type of data, analysis of qualitative data, software f...
Qualitative research - type of data, analysis of qualitative data, software f...
 

Destacado

Quantitative And Qualitative Research
Quantitative And Qualitative ResearchQuantitative And Qualitative Research
Quantitative And Qualitative Researchdoha07
 
Primary and secondary research
Primary and secondary researchPrimary and secondary research
Primary and secondary researchemilyselim
 
Primary and Secondary Research - Quantitative / Qualitative data
Primary and Secondary Research - Quantitative / Qualitative dataPrimary and Secondary Research - Quantitative / Qualitative data
Primary and Secondary Research - Quantitative / Qualitative datachrisnaufel
 
Difference Between Qualitative and Quantitative Research
Difference Between Qualitative and Quantitative ResearchDifference Between Qualitative and Quantitative Research
Difference Between Qualitative and Quantitative ResearchMuhammad Ahsan Nawaz
 
What is an experimental research (1)
What is an experimental research (1)What is an experimental research (1)
What is an experimental research (1)M Usama Sehgal
 
research paper writing guide
research paper writing guideresearch paper writing guide
research paper writing guideSriaarry Prawini
 
How to develop a research topic
How to develop a research topicHow to develop a research topic
How to develop a research topicphoebeleung
 
Selecting And Narrowing Research Topics
Selecting And Narrowing Research TopicsSelecting And Narrowing Research Topics
Selecting And Narrowing Research TopicsDeanna Blevins AUC
 
How to choose a Research topic
How to choose a Research topicHow to choose a Research topic
How to choose a Research topicAvinash Advani
 
2012 choosing a research topic
2012 choosing a research topic2012 choosing a research topic
2012 choosing a research topiccherylyap61
 
Qualitative and Quantitative Research
Qualitative and Quantitative ResearchQualitative and Quantitative Research
Qualitative and Quantitative Researchgirlie
 
Developing good research questions
Developing good research questionsDeveloping good research questions
Developing good research questionsbrannow
 
Choosing a Research Topic
Choosing a Research TopicChoosing a Research Topic
Choosing a Research TopicAndrew Walsh
 
difference between the qualitative and quantitative researcher, variables, co...
difference between the qualitative and quantitative researcher, variables, co...difference between the qualitative and quantitative researcher, variables, co...
difference between the qualitative and quantitative researcher, variables, co...laraib asif
 
Experimental research
Experimental researchExperimental research
Experimental researchdhinnar
 
Qualitative Vs Quantitative Research
Qualitative Vs Quantitative ResearchQualitative Vs Quantitative Research
Qualitative Vs Quantitative ResearchSam Ladner
 
Quantitative and Qualitative Research
Quantitative and Qualitative ResearchQuantitative and Qualitative Research
Quantitative and Qualitative ResearchMohammad Hassan
 
Identification of research topic
Identification of research topicIdentification of research topic
Identification of research topicBruno Mmassy
 

Destacado (20)

Quantitative And Qualitative Research
Quantitative And Qualitative ResearchQuantitative And Qualitative Research
Quantitative And Qualitative Research
 
Primary and secondary research
Primary and secondary researchPrimary and secondary research
Primary and secondary research
 
Primary and Secondary Research - Quantitative / Qualitative data
Primary and Secondary Research - Quantitative / Qualitative dataPrimary and Secondary Research - Quantitative / Qualitative data
Primary and Secondary Research - Quantitative / Qualitative data
 
Difference Between Qualitative and Quantitative Research
Difference Between Qualitative and Quantitative ResearchDifference Between Qualitative and Quantitative Research
Difference Between Qualitative and Quantitative Research
 
What is an experimental research (1)
What is an experimental research (1)What is an experimental research (1)
What is an experimental research (1)
 
research paper writing guide
research paper writing guideresearch paper writing guide
research paper writing guide
 
How to develop a research topic
How to develop a research topicHow to develop a research topic
How to develop a research topic
 
Choosing a research topic
Choosing a research topicChoosing a research topic
Choosing a research topic
 
Selecting And Narrowing Research Topics
Selecting And Narrowing Research TopicsSelecting And Narrowing Research Topics
Selecting And Narrowing Research Topics
 
How to choose a Research topic
How to choose a Research topicHow to choose a Research topic
How to choose a Research topic
 
2012 choosing a research topic
2012 choosing a research topic2012 choosing a research topic
2012 choosing a research topic
 
Qualitative and Quantitative Research
Qualitative and Quantitative ResearchQualitative and Quantitative Research
Qualitative and Quantitative Research
 
Developing good research questions
Developing good research questionsDeveloping good research questions
Developing good research questions
 
Choosing a Research Topic
Choosing a Research TopicChoosing a Research Topic
Choosing a Research Topic
 
difference between the qualitative and quantitative researcher, variables, co...
difference between the qualitative and quantitative researcher, variables, co...difference between the qualitative and quantitative researcher, variables, co...
difference between the qualitative and quantitative researcher, variables, co...
 
Experimental research
Experimental researchExperimental research
Experimental research
 
Qualitative Vs Quantitative Research
Qualitative Vs Quantitative ResearchQualitative Vs Quantitative Research
Qualitative Vs Quantitative Research
 
Quantitative and Qualitative Research
Quantitative and Qualitative ResearchQuantitative and Qualitative Research
Quantitative and Qualitative Research
 
Identification of research topic
Identification of research topicIdentification of research topic
Identification of research topic
 
Experimental research
Experimental researchExperimental research
Experimental research
 

Similar a Introduction to quantitative and qualitative research

PSYA4 - Research methods
PSYA4 - Research methodsPSYA4 - Research methods
PSYA4 - Research methodsNicky Burt
 
Research and its types
Research and its typesResearch and its types
Research and its typesRijitha R
 
Lecture 01 & 02 (Research Basics).ppt
Lecture 01 & 02 (Research Basics).pptLecture 01 & 02 (Research Basics).ppt
Lecture 01 & 02 (Research Basics).pptKamiBhutta
 
321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)Iin Angriyani
 
Difference between quantitative and qualitative research
Difference between quantitative and qualitative researchDifference between quantitative and qualitative research
Difference between quantitative and qualitative researchSafi Nawam
 
Presentation on research methodologies
Presentation on research methodologiesPresentation on research methodologies
Presentation on research methodologiesBilal Naqeeb
 
The scientific method
The scientific methodThe scientific method
The scientific methodGreg Kleponis
 
Qualitative and quantatitve research
Qualitative and quantatitve researchQualitative and quantatitve research
Qualitative and quantatitve researchHeather Lambert
 
Designing Qualitative Research
Designing Qualitative ResearchDesigning Qualitative Research
Designing Qualitative ResearchMike Crabb
 
Introduction to research
Introduction to researchIntroduction to research
Introduction to researchKumar
 
Research Paradigms.pptx
Research Paradigms.pptxResearch Paradigms.pptx
Research Paradigms.pptxDanaiDen1
 
You Want Me to Measure What?
You Want Me to Measure What?You Want Me to Measure What?
You Want Me to Measure What?Dave Hogue
 

Similar a Introduction to quantitative and qualitative research (20)

PSYA4 - Research methods
PSYA4 - Research methodsPSYA4 - Research methods
PSYA4 - Research methods
 
Methodology and IRB/URR
Methodology and IRB/URRMethodology and IRB/URR
Methodology and IRB/URR
 
Research and its types
Research and its typesResearch and its types
Research and its types
 
Lecture 01 & 02 (Research Basics).ppt
Lecture 01 & 02 (Research Basics).pptLecture 01 & 02 (Research Basics).ppt
Lecture 01 & 02 (Research Basics).ppt
 
321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)
 
Difference between quantitative and qualitative research
Difference between quantitative and qualitative researchDifference between quantitative and qualitative research
Difference between quantitative and qualitative research
 
Presentation on research methodologies
Presentation on research methodologiesPresentation on research methodologies
Presentation on research methodologies
 
Amsale Read.ppt
Amsale Read.pptAmsale Read.ppt
Amsale Read.ppt
 
RM-1 (1).pptx
RM-1 (1).pptxRM-1 (1).pptx
RM-1 (1).pptx
 
The scientific method
The scientific methodThe scientific method
The scientific method
 
What is research
What is researchWhat is research
What is research
 
Qualitative and quantatitve research
Qualitative and quantatitve researchQualitative and quantatitve research
Qualitative and quantatitve research
 
Lecture 7 research methodology in counselling
Lecture 7 research methodology in counsellingLecture 7 research methodology in counselling
Lecture 7 research methodology in counselling
 
Designing Qualitative Research
Designing Qualitative ResearchDesigning Qualitative Research
Designing Qualitative Research
 
ETHNOGRAPHY IV: Mixed Research Methods.pptx
ETHNOGRAPHY IV: Mixed Research Methods.pptxETHNOGRAPHY IV: Mixed Research Methods.pptx
ETHNOGRAPHY IV: Mixed Research Methods.pptx
 
Introduction to research
Introduction to researchIntroduction to research
Introduction to research
 
Qrm 210 unit 1
Qrm 210 unit 1Qrm 210 unit 1
Qrm 210 unit 1
 
Introduction of research
Introduction of researchIntroduction of research
Introduction of research
 
Research Paradigms.pptx
Research Paradigms.pptxResearch Paradigms.pptx
Research Paradigms.pptx
 
You Want Me to Measure What?
You Want Me to Measure What?You Want Me to Measure What?
You Want Me to Measure What?
 

Más de Liz FitzGerald

Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’
Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’
Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’Liz FitzGerald
 
Keynote speech: The promises and pitfalls of personalised eLearning
Keynote speech: The promises and pitfalls of personalised eLearningKeynote speech: The promises and pitfalls of personalised eLearning
Keynote speech: The promises and pitfalls of personalised eLearningLiz FitzGerald
 
Dimensions of personalisation in TEL: a framework and implications for design
Dimensions of personalisation in TEL: a framework and implications for designDimensions of personalisation in TEL: a framework and implications for design
Dimensions of personalisation in TEL: a framework and implications for designLiz FitzGerald
 
Video analysis techniques
Video analysis techniquesVideo analysis techniques
Video analysis techniquesLiz FitzGerald
 
Situ8: browsing and capturing geolocated user-created content
Situ8: browsing and capturing geolocated user-created contentSitu8: browsing and capturing geolocated user-created content
Situ8: browsing and capturing geolocated user-created contentLiz FitzGerald
 
Catwalk technologies and researching in the wild
Catwalk technologies and researching in the wildCatwalk technologies and researching in the wild
Catwalk technologies and researching in the wildLiz FitzGerald
 
Learning in the wild: designing for location-based experiences
Learning in the wild: designing for location-based experiencesLearning in the wild: designing for location-based experiences
Learning in the wild: designing for location-based experiencesLiz FitzGerald
 
Using augmented reality and mobile learning: opportunities and challenges
Using augmented reality and mobile learning: opportunities and challengesUsing augmented reality and mobile learning: opportunities and challenges
Using augmented reality and mobile learning: opportunities and challengesLiz FitzGerald
 
Augmented reality and mobile learning: the state of the art
Augmented reality and mobile learning: the state of the artAugmented reality and mobile learning: the state of the art
Augmented reality and mobile learning: the state of the artLiz FitzGerald
 
Assessing informal learning: a case study using historical audio guides
Assessing informal learning: a case study using historical audio guidesAssessing informal learning: a case study using historical audio guides
Assessing informal learning: a case study using historical audio guidesLiz FitzGerald
 
Lies, damned lies and statistics: an evaluation of learning styles in AEH
Lies, damned lies and statistics: an evaluation of learning styles in AEHLies, damned lies and statistics: an evaluation of learning styles in AEH
Lies, damned lies and statistics: an evaluation of learning styles in AEHLiz FitzGerald
 
The OU's Digital Humanities seminar series: the Pelagios project
The OU's Digital Humanities seminar series: the Pelagios projectThe OU's Digital Humanities seminar series: the Pelagios project
The OU's Digital Humanities seminar series: the Pelagios projectLiz FitzGerald
 
Geolocated audio tours
Geolocated audio toursGeolocated audio tours
Geolocated audio toursLiz FitzGerald
 
Augmented reality for mobile learning
Augmented reality for mobile learningAugmented reality for mobile learning
Augmented reality for mobile learningLiz FitzGerald
 
Educational concepts: learning styles
Educational concepts: learning stylesEducational concepts: learning styles
Educational concepts: learning stylesLiz FitzGerald
 
Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...
Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...
Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...Liz FitzGerald
 
IET Technology Coffee Morning - Location-based learning: education in the Wild
IET Technology Coffee Morning - Location-based learning: education in the WildIET Technology Coffee Morning - Location-based learning: education in the Wild
IET Technology Coffee Morning - Location-based learning: education in the WildLiz FitzGerald
 
Creating location-based mobile learning experiences
Creating location-based mobile learning experiencesCreating location-based mobile learning experiences
Creating location-based mobile learning experiencesLiz FitzGerald
 
Hidden Histories: a Towards Pervasive Media feasibility study
Hidden Histories: a Towards Pervasive Media feasibility studyHidden Histories: a Towards Pervasive Media feasibility study
Hidden Histories: a Towards Pervasive Media feasibility studyLiz FitzGerald
 

Más de Liz FitzGerald (20)

Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’
Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’
Impact of the OpenLearn Create course ‘Support Through Court: Domestic Abuse’
 
Keynote speech: The promises and pitfalls of personalised eLearning
Keynote speech: The promises and pitfalls of personalised eLearningKeynote speech: The promises and pitfalls of personalised eLearning
Keynote speech: The promises and pitfalls of personalised eLearning
 
Dimensions of personalisation in TEL: a framework and implications for design
Dimensions of personalisation in TEL: a framework and implications for designDimensions of personalisation in TEL: a framework and implications for design
Dimensions of personalisation in TEL: a framework and implications for design
 
Video analysis techniques
Video analysis techniquesVideo analysis techniques
Video analysis techniques
 
Situ8 overview
Situ8 overviewSitu8 overview
Situ8 overview
 
Situ8: browsing and capturing geolocated user-created content
Situ8: browsing and capturing geolocated user-created contentSitu8: browsing and capturing geolocated user-created content
Situ8: browsing and capturing geolocated user-created content
 
Catwalk technologies and researching in the wild
Catwalk technologies and researching in the wildCatwalk technologies and researching in the wild
Catwalk technologies and researching in the wild
 
Learning in the wild: designing for location-based experiences
Learning in the wild: designing for location-based experiencesLearning in the wild: designing for location-based experiences
Learning in the wild: designing for location-based experiences
 
Using augmented reality and mobile learning: opportunities and challenges
Using augmented reality and mobile learning: opportunities and challengesUsing augmented reality and mobile learning: opportunities and challenges
Using augmented reality and mobile learning: opportunities and challenges
 
Augmented reality and mobile learning: the state of the art
Augmented reality and mobile learning: the state of the artAugmented reality and mobile learning: the state of the art
Augmented reality and mobile learning: the state of the art
 
Assessing informal learning: a case study using historical audio guides
Assessing informal learning: a case study using historical audio guidesAssessing informal learning: a case study using historical audio guides
Assessing informal learning: a case study using historical audio guides
 
Lies, damned lies and statistics: an evaluation of learning styles in AEH
Lies, damned lies and statistics: an evaluation of learning styles in AEHLies, damned lies and statistics: an evaluation of learning styles in AEH
Lies, damned lies and statistics: an evaluation of learning styles in AEH
 
The OU's Digital Humanities seminar series: the Pelagios project
The OU's Digital Humanities seminar series: the Pelagios projectThe OU's Digital Humanities seminar series: the Pelagios project
The OU's Digital Humanities seminar series: the Pelagios project
 
Geolocated audio tours
Geolocated audio toursGeolocated audio tours
Geolocated audio tours
 
Augmented reality for mobile learning
Augmented reality for mobile learningAugmented reality for mobile learning
Augmented reality for mobile learning
 
Educational concepts: learning styles
Educational concepts: learning stylesEducational concepts: learning styles
Educational concepts: learning styles
 
Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...
Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...
Geological Society Higher Education Network (HEN 2012) meeting - 18 Jan 2012 ...
 
IET Technology Coffee Morning - Location-based learning: education in the Wild
IET Technology Coffee Morning - Location-based learning: education in the WildIET Technology Coffee Morning - Location-based learning: education in the Wild
IET Technology Coffee Morning - Location-based learning: education in the Wild
 
Creating location-based mobile learning experiences
Creating location-based mobile learning experiencesCreating location-based mobile learning experiences
Creating location-based mobile learning experiences
 
Hidden Histories: a Towards Pervasive Media feasibility study
Hidden Histories: a Towards Pervasive Media feasibility studyHidden Histories: a Towards Pervasive Media feasibility study
Hidden Histories: a Towards Pervasive Media feasibility study
 

Último

3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptxmary850239
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17Celine George
 
Vani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational TrustVani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational TrustSavipriya Raghavendra
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICESayali Powar
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...Nguyen Thanh Tu Collection
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapitolTechU
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17Celine George
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17Celine George
 
How to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using CodeHow to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using CodeCeline George
 
A gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceA gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceApostolos Syropoulos
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfYu Kanazawa / Osaka University
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?TechSoup
 
EBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting BlEBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting BlDr. Bruce A. Johnson
 
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...Dr. Asif Anas
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...raviapr7
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...CaraSkikne1
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesCeline George
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.EnglishCEIPdeSigeiro
 
Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....Riddhi Kevadiya
 

Último (20)

3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx3.26.24 Race, the Draft, and the Vietnam War.pptx
3.26.24 Race, the Draft, and the Vietnam War.pptx
 
How to Solve Singleton Error in the Odoo 17
How to Solve Singleton Error in the  Odoo 17How to Solve Singleton Error in the  Odoo 17
How to Solve Singleton Error in the Odoo 17
 
Vani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational TrustVani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
Vani Magazine - Quarterly Magazine of Seshadripuram Educational Trust
 
Quality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICEQuality Assurance_GOOD LABORATORY PRACTICE
Quality Assurance_GOOD LABORATORY PRACTICE
 
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
CHUYÊN ĐỀ DẠY THÊM TIẾNG ANH LỚP 11 - GLOBAL SUCCESS - NĂM HỌC 2023-2024 - HK...
 
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdfPersonal Resilience in Project Management 2 - TV Edit 1a.pdf
Personal Resilience in Project Management 2 - TV Edit 1a.pdf
 
CapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptxCapTechU Doctoral Presentation -March 2024 slides.pptx
CapTechU Doctoral Presentation -March 2024 slides.pptx
 
How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17How to Add a New Field in Existing Kanban View in Odoo 17
How to Add a New Field in Existing Kanban View in Odoo 17
 
How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17How to Create a Toggle Button in Odoo 17
How to Create a Toggle Button in Odoo 17
 
How to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using CodeHow to Send Emails From Odoo 17 Using Code
How to Send Emails From Odoo 17 Using Code
 
A gentle introduction to Artificial Intelligence
A gentle introduction to Artificial IntelligenceA gentle introduction to Artificial Intelligence
A gentle introduction to Artificial Intelligence
 
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdfP4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
P4C x ELT = P4ELT: Its Theoretical Background (Kanazawa, 2024 March).pdf
 
What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?What is the Future of QuickBooks DeskTop?
What is the Future of QuickBooks DeskTop?
 
EBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting BlEBUS5423 Data Analytics and Reporting Bl
EBUS5423 Data Analytics and Reporting Bl
 
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
Unveiling the Intricacies of Leishmania donovani: Structure, Life Cycle, Path...
 
Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...Patient Counselling. Definition of patient counseling; steps involved in pati...
Patient Counselling. Definition of patient counseling; steps involved in pati...
 
5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...5 charts on South Africa as a source country for international student recrui...
5 charts on South Africa as a source country for international student recrui...
 
How to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 SalesHow to Manage Cross-Selling in Odoo 17 Sales
How to Manage Cross-Selling in Odoo 17 Sales
 
Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.Easter in the USA presentation by Chloe.
Easter in the USA presentation by Chloe.
 
Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....Riddhi Kevadiya. WILLIAM SHAKESPEARE....
Riddhi Kevadiya. WILLIAM SHAKESPEARE....
 

Introduction to quantitative and qualitative research

  • 1. Introduction to quantitative and qualitative research Dr Liz FitzGerald Institute of Educational Technology
  • 2. Research and research methods • Research methods are split broadly into quantitative and qualitative methods • Which you choose will depend on – your research questions – your underlying philosophy of research – your preferences and skills
  • 3. Basic principles of research design Four main features of research design, which are distinct, but closely related • Ontology: How you, the researcher, view the world and the assumptions that you make about the nature of the world and of reality • Epistemology: The assumptions that you make about the best way of investigating the world and about reality • Methodology: The way that you group together your research techniques to make a coherent picture • Methods and techniques: What you actually do in order to collect your data and carry out your investigations • These principles will inform which methods you choose: you need to understand how they fit with your ‘bigger picture’ of the world, and how you choose to investigate it, to ensure that your work will be coherent and effective
  • 4. Four main schools of ontology (how we construct reality) Ontology Realism Internal Realism Relativism Nominalism Summary The world is ‘real’, and science proceeds by examining and observing it The world is real, but it is almost impossible to examine it directly Scientific laws are basically created by people to fit their view of reality Reality is entirely created by people, and there is no external ‘truth’ Truth There is a single truth Truth exists, but is obscure There are many truths There is no truth Facts Facts exist, and can be revealed through experiments Facts are concrete, but cannot always be revealed Facts depend on the viewpoint of the observer Facts are all human creations However, none of these positions are absolutes. They are on a continuum, with overlaps between them.
  • 5. Epistemology i.e. the way in which you choose to investigate the world Two main schools are positivism and social constructionism: • Positivists believe that the best way to investigate the world is through objective methods, such as observations. Positivism fits within a realist ontology. • Social constructionists believe that reality does not exist by itself. Instead, it is constructed and given meaning by people. Their focus is therefore on feelings, beliefs and thoughts, and how people communicate these. Social constructionism fits better with a relativist ontology.
  • 6. Methodology • Epistemology and ontology will have implications for your methodology • Realists tend to have positivist approach  tend to gather quantitative sources of data • Relativists tend to have a social constructionist approach  tend to gather qualitative sources of data • Remember these are not absolutes! People tend to work on a continuum  role for mixed methods and approaches • Also consider the role of the researcher*: internal/external; involved or detached? * See also Adams, Anne; FitzGerald, Elizabeth and Priestnall, Gary (2013). Of catwalk technologies and boundary creatures. ACM Transactions on Computer-Human Interaction (TOCHI), 20(3), article no. 15. http://oro.open.ac.uk/35323/
  • 7. A note about data • Quantitative data is about quantities, and therefore numbers • Qualitative data is about the nature of the thing investigated, and tends to be words rather than numbers • Difference between primary and secondary data sources • Be aware of research data management practices and archives of data sets (both in terms of downloading and uploading)
  • 8. Choosing your approach • Your approach may be influenced by your colleagues’ views, your organisation’s approach, your supervisor’s beliefs, and your own experience • There is no right or wrong answer to choosing your research methods • Whatever approach you choose for your research, you need to consider five questions: – What is the unit of analysis? For example, country, company or individual. – Are you relying on universal theory or local knowledge? i.e. will your results be generalisable, and produce universally applicable results, or are there local factors that will affect your results? – Will theory or data come first? Should you read the literature first, and then develop your theory, or will you gather your data and develop your theory from that? (N.B. this will likely be an iterative process) – Will your study be cross-sectional or longitudinal? Are you looking at one point in time, or changes over time? – Will you verify or falsify a theory? You cannot conclusively prove any theory; the best that you can do is find nothing that disproves it. It is therefore easier to formulate a theory that you can try to disprove, because you only need one ‘wrong’ answer to do so.
  • 9. Quantitative approaches • Attempts to explain phenomena by collecting and analysing numerical data • Tells you if there is a “difference” but not necessarily why • Data collected are always numerical and analysed using statistical methods • Variables are controlled as much as possible (RCD as the gold standard) so we can eliminate interference and measure the effect of any change • Randomisation to reduce subjective bias • If there are no numbers involved, its not quantitative • Some types of research lend themselves better to quant approaches than others
  • 10. Quantitative data • Data sources include – Surveys where there are a large number of respondents (esp where you have used a Likert scale) – Observations (counts of numbers and/or coding data into numbers) – Secondary data (government data; SATs scores etc) • Analysis techniques include hypothesis testing, correlations and cluster analysis
  • 11. Black swans and falsifiability • Hypothesis testing • Start with null hypothesis i.e. H0 – that there will be no difference https://www.flickr.com/photos/lselibrary/ IMAGELIBRARY/5 • Falsifiability or refutability of a statement, hypothesis, or theory is the inherent possibility that it can be proven false • Karl Popper and the black swan; deductive c.f. inductive reasoning CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=1243220
  • 12. Type I and Type II errors
  • 13. Analysing quant data • Always good to group and/or visualise the data initially  outliers/cleaning data • What average are you looking for? Mean, median or mode? • Spread of data: – skewness/distribution – range, variance and standard deviation
  • 14. What are you looking for? • Trying to find the signal from the noise • Generally, either a difference (between/within groups) or a correlation • Choosing the right test to use: parametric vs non-parametric (depends what sort of data you have – interval/ratio vs nominal/ordinal and how it is distributed) • Correlation does not imply causation!
  • 15. Example correlations From ‘Spurious correlations’ website http://www.tylervigen.com /spurious-correlations
  • 16. Interpreting test statistics • Significance level – a fixed probability of wrongly rejecting the null hypothesis H0, if it is in fact true. Usually set to 0.05 (5%). • p value - probability of getting a value of the test statistic as extreme as or more extreme than that observed by chance alone, if the null hypothesis H0, is true. • Power – ability to detect a difference if there is one • Effect size – numerical way of expressing the strength or magnitude of a reported relationship, be it causal or not
  • 17. Example of quant data/analysis* • Matched users were those who learning styles were matched with the lesson plan e.g. sequential users with a sequential lesson plan. Mismatched participants used a lesson plan that was not matched to their learning style, e.g. sequential users with a global lesson plan. • H0 – there will be no statistically significant difference in knowledge gained between users from different experimental groups • H1 – students who learn in a matched environment will learn significantly better than those who are in mismatched environment • H2 – students who learn in a mismatched environment will learn significantly worse than those who learn in a matched environment * Case study taken from: Brown, Elizabeth (2007) The use of learning styles in adaptive hypermedia. PhD thesis, University of Nottingham. http://eprints.nottingham.ac.uk/10577/
  • 18. Interpreting test statistics • Statistical testing was carried out using a univariate ANOVA in SPSS, to determine if there was any significant difference in knowledge gained. • Initial conjecture suggests that the mismatched group actually performed better than the matched group. • However, the difference between the two groups was not significant (F(1,80)=0.939, p=0.34, partial eta squared = 0.012) and hence hypotheses 1 and 2 can be rejected.
  • 19. What quant researchers worry about • Is my sample size big enough? • Have I used the correct statistical test? • have I reduced the likelihood of making Type I and/or Type II errors? • Are my results generalisable? • Are my results/methods/results reproducible? • Am I measuring things the right way?
  • 20. What’s wrong with quant research? • Some things can’t be measured – or measured accurately • Doesn’t tell you why • Can be impersonal – no engagement with human behaviours or individuals • Data can be static – snapshots of a point in time • Can tell a version of the truth (or a lie?) “Lies, damned lies and statistics” – persuasive power of numbers
  • 21. Qualitative approaches • Any research that doesn’t involve numerical data • Instead uses words, pictures, photos, videos, audio recordings. Field notes, generalities. Peoples’ own words. • Tends to start with a broad question rather than a specific hypothesis • Develop theory rather than start with one  inductive rather than deductive
  • 22. Gathering qual data • Tends to yield rich data to explore how and why things happened • Don’t need large sample sizes (in comparison to quantitative research) • Some issues may arise, such as – Respondents providing inaccurate or false information – or saying what they think the researcher wants to hear – Ethical issues may be more problematic as the researcher is usually closer to participants – Researcher objectivity may be more difficult to achieve
  • 23. Sources of qual data • Interviews (structured, semi-structured or unstructured) • Focus groups • Questionnaires or surveys • Secondary data, including diaries, self-reporting, written accounts of past events/archive data and company reports; • Direct observations – may also be recorded (video/audio) • Ethnography
  • 24. Analysing qual data • Content analysis • Grounded analysis • Social network analysis (can also be quant) • Discourse analysis • Narrative analysis • Conversation analysis
  • 25. Example of qual data research* • Describing and comparing two types of audio guides: person- led and technology-led • Geolocated audio to enable public, informal learning of historical events • Data sources: questionnaires, researcher observations, and small focus groups * Taken from: FitzGerald, Elizabeth; Taylor, Claire and Craven, Michael (2013). To the Castle! A comparison of two audio guides to enable public discovery of historical events. Personal and Ubiquitous Computing, 17(4) pp. 749–760. http://oro.open.ac.uk/35077/
  • 26. Data analysis and findings • Comparison of the two different walks – Differences/similarities of the walks – Issues surrounding participant engagement • Thematic analysis – Mode of delivery – Number of participants and social interactions – Geographical affordances of places and locations – User experience – Opportunities for learning – Other factors • Findings, lessons learned, recommendations
  • 27. What qual researchers worry about • Have I coded my data correctly? • Have I managed to capture the situation in a realistic manner? • Have I described the context in sufficient detail? • Have I managed to see the world through the eyes of my participants? • Is my approach flexible and able to change?
  • 28. What’s wrong with qual research? • It can be very subjective • It can’t always be repeated • It can’t always be generalisable • It can’t always give you definite answers in the way that quantitative research can • It can be easier to carry out (or hide) ‘bad’ (poor quality) qual research than ‘bad’ quant research
  • 29. Other aspects of research design • Validity • Reliability • Trustworthiness* – Dependability: showing that the findings are consistent and could be repeated – Confirmability: a degree of neutrality or the extent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest – Credibility: confidence in the 'truth' of the findings – Transferability: showing that the findings have applicability in other contexts * See Lincoln, YS. & Guba, EG. (1985). Naturalistic Inquiry. Newbury Park, CA: Sage Publications.
  • 30. Summary • The type of approach you choose will be determined by your research question, your epistemological and ontological stances and your skills or ability to utilise a certain appoach • For most people in ed tech, a mixed methods approach will be used • So long as you make an informed choice and can justify it, it should be fine  • Just be aware of the limitations of your approach(es) and try to compensate where necessary
  • 31. Acknowledgments and further links • Some content borrowed from SkillsYouNeed website (http://www.skillsyouneed.com/learn/research-methods.html) Other useful links: • Introduction to Quantitative and Qualitative Research Models (William Bardebes). PDF at http://tinyurl.com/qq-models • Methods Map: http://www.methodsmap.org • Ready To Research: http://readytoresearch.ac.uk • Methods@Manchester: http://www.methods.manchester.ac.uk/resources/categories • Research Data Management training: http://datalib.edina.ac.uk/mantra/

Notas del editor

  1. The Role of the Researcher The researcher can be either involved, or external, detached. These two positions, again, tend to link to the ontology and epistemology, with the positivist approach leading to a detached view, and the social constructionists tending towards the researcher being part of the world and therefore influencing, and being influenced by, events. Choices and Trade-Offs The choice of any particular research design, from ontology, through epistemology to methodology and then methods and techniques, involves trade-offs. All of the main research traditions have strengths and weaknesses. The most important aspect of designing your research is what you want to find out. Whatever methods you use, together with their underpinning philosophy, must answer your chosen research questions. Find more at: http://www.skillsyouneed.com/learn/research-methods.html#ixzz40KrH4t80
  2. Popper uses falsification as a criterion of demarcation to draw a sharp line between those theories that are scientific and those that are unscientific. The classical view of the philosophy of science is that it is the goal of science to prove hypotheses like "All swans are white" or to induce them from observational data. Popper argued that this would require the inference of a general rule from a number of individual cases, which is inadmissible in deductive logic.[2]:4 However, if one finds one single swan that is not white, deductive logic admits the conclusion that the statement that all swans are white is false. Falsificationism thus strives for questioning, for falsification, of hypotheses instead of proving them. For a statement to be questioned using observation, it needs to be at least theoretically possible that it can come in conflict with observation. A key observation of falsificiationism is thus that a criterion of demarcation is needed to distinguish those statements that can come in conflict with observation and those that cannot (Chorlton, 2012). Popper chose falsifiability as the name of this criterion.
  3. Have I made any Type I (false positive – i.e. rejecting null hypothesis incorrectly) or Type II errors (false negative – i.e. rejecting alternate hypothesis incorrectly)?
  4. When most people say average, they are talking about the mean. It has the advantage that it uses all the data values obtained and can be used for further statistical analysis. However, it can be skewed by ‘outliers’, values which are atypically large or small. As a result, researchers sometimes use the median instead. This is the mid-point of all the data. The median is not skewed by extreme values, but it is harder to use for further statistical analysis. The mode is the most common value in a data set. It cannot be used for further statistical analysis. The values of mean, median and mode are not the same, which is why it is really important to be clear which ‘average’ you are talking about. The range is the difference between the largest and smallest values. Researchers often quote the interquartile range, which is the range of the middle half of the data, from 25%, the lower quartile, up to 75%, the upper quartile, of the values (the median is the 50% value). To find the quartiles, use the same procedure as for the median, but take the quarter- and three-quarter-point instead of the mid-point. The standard deviation measures the average spread around the mean, and therefore gives a sense of the ‘typical’ distance from the mean. The variance is the square of the standard deviation. They are calculated by: calculating the difference of each value from the mean; squaring each one (to eliminate any difference between those above and below the mean); summing the squared differences; dividing by the number of items minus one. This gives the variance. To calculate the standard deviation, take the square root of the variance. Find more at: http://www.skillsyouneed.com/num/simple-statistical-analysis.html#ixzz40Puk0Bhw
  5. If you get it wrong you risk using an incorrect statistical procedure or you may use a less powerful procedure. Non-paramteric statistical procedures are less powerful because they use less information in their calulation. For example, a parametric correlation uses information about the mean and deviation from the mean while a non-parametric correlation will use only the ordinal position of pairs of scores. The basic distinction for paramteric versus non-parametric is: If your measurement scale is nominal or ordinal then you use non-parametric statistics If you are using interval or ratio scales you use parametric statistics. There are other considerations which have to be taken into account: You have to look at the distribution of your data. If your data is supposed to take parametric stats you should check that the distributions are approximately normal.The best way to do this is to check the skew and Kurtosis measures from the frequency output from SPSS. For a relatively normal distribution: skew ~= 1.0 kurtosis~=1.0 If a distribution deviates markedly from normality then you take the risk that the statistic will be inaccurate. The safest thing to do is to use an equivalent non-parametric statistic. www.csse.monash.edu.au/~smarkham/resources/scaling.htm - Nominal, ordinal etc.
  6. Significance level of a statistical hypothesis test is a fixed probability of wrongly rejecting the null hypothesis H0, if it is in fact true. The probability value (p-value) of a statistical hypothesis test is the probability of getting a value of the test statistic as extreme as or more extreme than that observed by chance alone, if the null hypothesis H0, is true. Small p-values suggest that the null hypothesis is unlikely to be true. The smaller it is, the more convincing is the rejection of the null hypothesis. It indicates the strength of evidence for say, rejecting the null hypothesis H0, rather than simply concluding "Reject H0' or "Do not reject H0". The power of a statistical hypothesis test measures the test's ability to reject the null hypothesis when it is actually false - that is, to make a correct decision. Ability to detect a difference.
  7. If the null hypothesis is true, you expect F to have a value close to 1.0 most of the time. A large F ratio means that the variation among group means is more than you'd expect to see by chance. You'll see a large F ratio both when the null hypothesis is wrong (the data are not sampled from populations with the same mean) and when random sampling happened to end up with large values in some groups and small values in others. The P value is determined from the F ratio and the two values for degrees of freedom shown in the ANOVA table. Degrees of freedom - http://ron.dotsch.org/degrees-of-freedom/ to describe the number of values in the final calculation of a statistic that are free to vary. partial eta squared = measure of effect size. .02 ~ small .13 ~ medium .26 ~ large
  8. owever, there are some pitfalls to qualitative research, such as: If respondents do not see a value for them in the research, they may provide inaccurate or false information. They may also say what they think the researcher wishes to hear. Qualitative researchers therefore need to take the time to build relationships with their research subjects and always be aware of this potential. Although ethics are an issue for any type of research, there may be particular difficulties with qualitative research because the researcher may be party to confidential information. It is important always to bear in mind that you must do no harm to your research subjects. It is generally harder for qualitative researchers to remain apart from their work. By the nature of their study, they are involved with people. It is therefore helpful to develop habits of reflecting on your part in the work and how this may affect the research. See our page on Reflective Practice for more. Find more at: http://www.skillsyouneed.com/learn/quantitative-and-qualitative.html#ixzz40SxYY76I
  9. 1. Content Analysis Here, you start with some ideas about hypotheses or themes that might emerge, and look for them in the data that you have collected. You might, for example, use a colour-coding or numbering system to identify text about the different themes, grouping together ideas and gathering evidence about views on each theme. 2. Grounded Analysis This is similar to content analysis, in that it uses similar techniques for coding. However, in grounded analysis, you do not start from a defined point. Instead, you allow the data to ‘speak for itself’, with themes emerging from the discussions and conversations. In practice, this may be much harder to achieve because it requires you to put aside what you have read and simply concentrate on the data. Some people, such as Myers-Briggs 'P' types, may find this form of analysis much easier to achieve than others. 3. Social Network Analysis This form of analysis examines the links between individuals as a way of understanding what motivates behaviour. It has been used, for example, as a way of understanding why some people are more successful at work than others, and why some children were more likely to run away from home. This type of analysis may be most useful in combination with other methods, for example after some kind of content or grounded analysis to identify common themes about relationships. It’s often helpful to use a visual approach to this kind of analysis to generate a network diagram showing the relationships between members of a network. 4. Discourse Analysis This approach not only analyses conversation, but also takes into account the social context in which the conversation occurs, including previous conversations, power relationships and the concept of individual identity. It may also include analysis of written sources, such as emails or letters, and body language to give a rich source of data surrounding the actual words used. 5. Narrative Analysis This looks at the way in which stories are told within an organisation or society to try to understand more about the way in which people think and are organised within groups. There are four main types of narrative: bureaucratic, which is highly structured and logical, and often about imposing control; quest, where the ambition is to have the most compelling story and lead others to success; chaos, where the story is lived, rather than told; and postmodern, which is rather like chaos narratives, in that it is lived, but where the ‘narrator’ is aware of the story and what they are trying to achieve. 6. Conversation Analysis This is largely used in ethnographic research. It assumes that conversations are all governed by rules and patterns which remain the same whoever is talking. It also assumes that what is said can only be understood by looking at what went before and after. Conversation analysis requires a detailed examination of the data, including exactly which words are used, in what order, whether speakers overlap their speech, and where the emphasis is placed. There are therefore detailed conventions used in transcribing for conversation analysis.
  10. http://www.crec.co.uk/docs/Trustworthypaper.pdf Lincoln and Guba's Evaluative Criteria Lincoln and Guba posit that trustworthiness of a research study is important to evaluating its worth.  Trustworthiness involves establishing: Credibility - confidence in the 'truth' of the findings Transferability - showing that the findings have applicablity in other contexts Dependability - showing that the findings are consistent and could be repeated Confirmability - a degree of neutraility or the extent to which the findings of a study are shaped by the respondents and not researcher bias, motivation, or interest. http://www.qualres.org/HomeLinc-3684.html