Blueprint of exam questions

Blueprint of Examination Questions
1st Professional Examination of
MD Program in FMHS at UPM
Prof. Dr. Shahid Hassan
MHPE
H/O Unit of Medical Education
Faculty of Medicine and Health Sciences

Universiti Putr Malaysia

1st Professional Examination of MD Program in FMHS at UPM

Definition:
A blueprint of examination also known as the test
specification provides examination strategy of an institution at
a glance.
•  Blueprint is the matrix or chart reporting the number and
type of test questions represented across the topics in
content area, consistent with learning objective and relative
weight on test given to each topic.
•  A blueprint also identifies the % weighting of cognitive
dimensions as the level of competence tested in each
konowledge domain.


Purpose of Blueprints:
The purpose of constructing examination using a blueprint is
to provide a conceptual map of examination format and the
content area represented in assessment. Blueprint of
examination is the widely used assessment tool that provides
list of information about:
1. Type of measurement tools and proportion of each question
format in assessment shown in respective weighting colunm.
2. Topics and the level of training for each topic and the
relevant learning objectives.


Purpose of Blueprints:
3. Topics and the relative weighting given to each topic in
content area.
4. Competencies as unidimesional cognitive process tested in
knowledge and skills domains of Bloom’s taxonomy of
educatioal objective and Miller’s pyramid of competence
learning respectively.
5. Knowledge as multidimensional process of learning
proposed as taxonomy table by Anderson including factual
knowledge, conceptual knowledge and procedural knowledge.


Bloom’s Unidimensional Taxonomy of Educational Objective

Learning Objective/Outcome
Cognitive
dimension

Knowledge
Comprehension
Analysis
Application
Synthesis
Evaluation

1

2

3

4

5

6


Anderson’s Multidimensional Taxonomy Table

Learning
Domains è
Knowledge
Dimension
ê

Factual
Knowledge
Conceptual
Knowledge
Procedural
Knowledge
Metacognitiv
e Knowledge

The Cognitive Process Dimension
Recall Understand

Apply

Analyze Evaluate Create


Who will Use the Blueprint:
The blueprint is a globally accepted tool used to construct
examination questions for ready reference of stakeholders
engaged in assessment process.
Following stakeholders can use the blueprints readily
available.
1. Candidates to use examination blueprints to ensure
prpeparation of assessment for its time management and
strategy to achieve the desired outcom of passing the
examination.


Eevery student however, must be informed that:

•  All topics in blueprint may not be listed and a curriculum
detail on content should always be sought by students.
•  Topics not listed in blueprints are still part of content,
which can be tested through assignments, student centered
learning (SCL) and continuous (formative) assessment.
•  Students must take the responsibility of acquiring a broadbased knowledge of all those topics not listed in blueprint.


Who will Use the Blueprints:
2. Teaching faculty inclusive of package and module
coordinators to ensure the validity of assessment in term of
assessment questions being representative of content area
documented in curriculum.
3. External examiners to know the assessment specification
with optimal information in given time to help facilitate the
assessment and review the learning process.


4. Members of vetting committee to ensure specification of
questions being consistent with konowledge and skills domain
versus cognitive dimension as per the learning objectives of
each module.
•  Blueprint also provides a readily available opportunity to
vetting members to ensure that the essential topics are not
overlooked.


Essentials of Blueprints:
The objective of examination at different level of training will
vary from topic to topic. A blueprint of examination should
not only be consistent with those objectives listed for each
topic but should also reflect to achieving them for reasons:
1.  Candidats’ are required to attain the competencies listed in
assessment in order to be be promoted to next level in
training.
• 

A blueprint therefore must reflect that a candiadte has
acquired the competencies to be able to advance to next
level.


2. Blueprint is seen as readily available document, which
contains the list of topics covered under each module with its
identified learning objectives as blueprint.
•  However, to develop the blueprint of examination
questions in MD program, we take overall objectives of
individual modules in each package as the prelude for
listing the outcome compentency of each objective.
.


3. Learning outcome is viewed as module competencies
essential for each module and a blueprint provide the
eveidence of as many modules as possible reperesented in
percentage weighting.
•  This will reflect on content validity of assessment in
blueprint.
•  A blueprint representating the percentage weighting of
content across assessment is one of the essentials of
assessment validity.

1st

Professional Examination of MD Program in FMHS at UPM

Guideline to Prepare Blueprints:
Blueprint of examination is produced as three separate tables:
Table 1:
A table reflecting the type of questions and their weighting
respectively.
• 

Ideally each measurement tool and items in a mesure
should also have percentage weighting depending on
cognitive demand of question.

• 

In fact time allocation should also vary between questions
of same construct such as MTF vs. SBAQ.


•  Currently all question regardless of cognitive demand
carried the same marks weighting. We however, follow the
practice of marks weighting between the tools but not
between the items (all MTF/SBAQ questions carry equal
marks).
•  We have allocated equal marks to each item in MCQ
irrespective of complexity of cognitive demand it is
intended to test.
•  The current practice somehow sends a wrong message to
students that questions with factual recall is as mportant as
applications and analysis of knowledge.


Table 2:
A series of tables showing the contents with its objectives and
percentage weighting and preferably the level of training for
each topic/module and this may need several tables for each
topic or module in a package


Table 3:
A table potraying the ultimate blueprint developed as a chart
representing each module. Total number of questions with its
percentage weighting and number of questions weighted
against their ability to test the levels of cognitive dimension
described by Bloom’s taxonomy of knowledge is the
characterstics of blueprint


Structural Components of Blueprint:
Table 1. Type of Measurement Tools in Assessment:
•  This table comprises of examination question guided by
type of assessment tools and their approximate weighting.
This table is the first step towards developing the actual
blueprint
(See Table 1)


Table 2: Topics with objectives and learning outcome:
•  The next table comprised of a series of matrix containing
the code and the title of course content (topics/module),
learning objectives, learning outcome (module
competencies), weighting and preferably the level of
training
(See Table 2)
•  This may help student and faculty to know about
competencies to be learned in the course, highlighted in
blueprint.


•  A summary of detail weighting of content area from each
module in a package currently practiced in 1st professional
eaxamination is developed with % weighting is calculated.
(See Table 3)

Table 3: Summary of weighting of each package and their respective modules calculated for number of weeks of study (content
area) per module versus weighting currently practiced in 1st Professional Exmination.


•  It can be ensured that the weighting of assessment in term
of content area calculated from number of weeks allocated
to deliver that module is accordingly represented in
assessment.
•  An obvious impression from eyeball rolling over this
blueprint clearly shows that the content is not evenly
distributed. Some of the modules are not even given a
place in weighting whilw other are overrepresented in
assessment of professional examoination.



Table 3: Number and Weighting of Questions in
Assessment:
•  After knowing the learning objectives of each contentent
areas as per their modules in preclinical phase of training
(see table 2) and the type of measurement tools in 1st
professional examination (see table 1) it is easy to finally
create the examination blueprint.

(See Table 4-6)

Table 4: Template of examination blueprint showing number of questions (%) determined by content area (weeks of study) as
defined for each packages/modules and their learning domain (cognitive dimension) determined by their respective lerning
objectives in 60 questions MCQ/SBAQ (30/30) of 5 marks each from Paper I/II in 1st professional examination of MD course.

*
Indicates recommended percent wise representation of questions with respect to their level of competence (cognitive
dimension) in different modules however, exact number may not follow this strickly and can be determined in vetting

Table 5: Template of examination blueprint showing number of questions (%) determined by content area (weeks of study) as
defined for each packages/modules and their learning domain (cognitive dimension) determined by their lerning objectives in
12 questions SEQ/MEQ (8/4) of 15 marks each from Paper I/II in 1st professional examination. Of MD course

*

Table 6: Template of examination blueprint representing the number of station demarcated into packages and modules with skills
domains (level of competence) determined by their lerning objectives in 15 stations OSPE and 5 stations OSCE of 5 marks each
in 1st professional examination of MD course.

*



Table 3: Number and Weighting of Questions in
Assessment:
•  Examination blueprint is tabulated as chart representing the
total number of questions within each module and their
distribution as per level of competence (cognitive
dimension) in term of knowledge and skills domains
respectively.
•  This will ensure the desired coverage of topics in term of
modules and their relative importance per their lerning
objectives and content area in the curriculum.


Rationale of Blueprints:
1. Assessment is a complex and multifaceted aspect of
learning and teaching, which may be viewed differently by
different users. However, a valid assessment determins the
acquired competencies identified as learning outcome in
curriculum.
2. The document that provides an overview to assessment and
can be readily used on stakeholders (particularly the external
examiners, MQA, MMC and Senate) demand of quality
assessment evident from its blueprint.


3. Another important aspect of assessment, the feedback on
students’ progress and teachers’ delivering of curriculum is
less often practiced simply for not well understood
assessment process by faculty.
•  A well design assessment documented as blueprint
provides an opportunity and confidence to faculty to give
feedback with realtive importance to measures and their
weighting.


4. From student’s point of view assessment always define
curriculum.
• 

What is to be learnt (content area in assessment) and how
to be learnt (superficial to deep learning approaches) will
depend how assessment is designed and how it is being
made available (blueprint) to students and assessors alike.


5. We all agree that most students demonstrate a preference
for learning approach.
• 

In this perspective teaching and learning context
determined as a blueprint of examination question in
assessment can influence the students’ actual approach in
practice.


6. We in FMHS would like to develop a blueprint of
examination question not because it is demanded by the
external examiners but to guide our students to adopt a
learning approach, moving from superficial to deeper level of
learning domain.
• 

This will enable students’ in handling the learning task
that our assessment approach not only tends to present but
now it is also provded as a blueprint of examination
questions.



7. Besides the external pressure to develop blueprint of exam
questions, assessment imperatives such as validity, reliability
and managebility takes the cetral position in idea behind
bluepriting.
•  Bloom’s taxonomy of knowledge as test of cognitive
dimension is primarily utilized to develop the entire
written and some aspect of practical questions.
•  Miller’s pyramid is used for judgment of practical and
clinical skills of competency to test students’ affective and
psychomotor abilities ensured in blueprint.


Systemic Reporting of Questions for Blueprint:
•  Further to blueprint a guideline for faculty member to
categorize each question into relevant columns of cognitive
dimension that the question intend to test will be requested.
•  It will be mandatory for each question setter to fill up
the proposed table while writing and submitting the
examination questions.
(See Table 7)

Table 7: Guidelines for questions writer to categorize each question to a relevant column og cognitive dimension that the
question is set to test.


Systemic Reporting of Questions for Blueprint:
•  All the questions categorized to one or more level of
competence refering to cognitive dimension will also be
vetted by the vetting committee in their routine meetings.
•  This practice will ensure that the examination questions
are consistent with target set for weghting of question in
blueprint.
•  This will also help the academic office to prepare the
blueprint of examination questions well on time for ready
referal of faculty and external examiners.


THANK YOU

Blueprint of exam questions

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Blueprint of exam questions

Similar a Blueprint of exam questions (20)

Último

Último (20)

Blueprint of exam questions