1. Introduction to critical appraisal

welcome to this series of training

videos developed by the Cochran common

mental disorders group at the University

of York in collaboration with

Northumberland Tyne and Wear NHS

Foundation Trust and teasers can wear

valleys NHS Foundation Trust we hope

that these videos will provide an

accessible resource for helping you

develop critical appraisal skills when

it comes to understanding and reviewing

research in your clinical practice we

have produced six training videos to

cover fundamental aspects of critical

appraisal for different experimental and

observational research designs because

different types of research questions

require different study designs each

video will provide a dedicated module

looking individually its systematic

reviews and meta-analysis randomized

control trials or RCTs and cohort

studies we will also take a look at case

control studies cross-sectional studies

and diagnostic studies each video module

will provide a summary of these

different designs and situations in

which they may be best employed to

answer a research question we're making

reference to examples we will walk

through how to critically appraise

different research designs using

recommended tools to help us do this by

using these video modules we hope to

equip you with the critical and

analytical tools needed to appraise

findings from research involving the

weighing up of evidence to assess its

validity the trustworthiness of the

results as well as its potential value

and relevance to a practice context by

way of learning outcomes at the end of

these videos you'll be able to firstly

understand and discuss critical

appraisal concepts and its role an

evidence-based practice secondly you

will be able to understand different

research designs and be able to answer

questions like have the researchers use

the most appropriate methodology to

answer their research question finally

you'll also be able to appraise results

of research its validity and its

relevance to your practice specifically

we all provide guidance in using the

critical appraisal skills program

checklist also known as a Casper


by learning these new concepts and

familiarizing yourself with a sample of

accessible critical appraisal tools

you'll be able to answer questions like

how would I rate the risk of bias and

the quality of this research and with

this effect how confident I am in

applying the findings into practice in

this video we will be more broadly

introducing the topic of critical

appraisal and the important terms you

will encounter so what do we actually

mean by critical appraisal fundamentally

it is the process of carefully and

systematically examining research to

judge its validity the trustworthiness

of the results and its value and

relevance if results are applied in a

particular context being able to

critically appraise research is an

essential skill for use in

evidence-based practice because it

allows clinicians to find and use

evidence reliably and efficiently these

are the three overarching key concepts

of the appraisal process and are

predominantly used by the review

checklist that we will take a look at

over the coming models as they are

applied differently depending on the

type of research you are appraising we

will briefly explore these concepts

individually in order to help you start

thinking about the ways you can apply

them in practice when making your own

critical appraisals when we talk about

validity this refers to the soundness or

the rigor of a particular study and the

extent to which the conclusions of the

research are true within the specific

context that the research was undertaken

this is also known as internal validity

a more valid research design method and

procedure produces less biased results

giving us more reliable generalizable

conclusions questions you may ask

relating to validity may include is

there a clear statement of aims in the

research you are appraising if authors

of the research have not provided a

clear research question that describes

the population of interest the

intervention given or the outcomes of

interest then there could be cause for

concern as to the validity of the

results of the research you also want to

be clear of the methods used in the

study to make sure they are suitable to

answer the question that's being posed

for example if you are critically

appraising a study the

is looking at the effect of an

intervention you'd be right in thinking

that a randomized control trial would be

the most reliable way to measure

treatment effect because these trials

aim to reduce bias by randomly

allocating participants to treatment

however it may be the case that the

authors of this type of studying may

well have stated that they randomized

participants but did not provide any

further details as to randomization

methods this may not necessarily mean

that authors did not use appropriate

selection and randomization methods in

the conduct of their research but if

their methods are not reported clearly

it makes it difficult to fairly assess

validity and may impact the conclusions

of your own critical appraisal as a

reviewer it would be your job to make a

judgement based on the published

research that's available to you as to

how reliable any conclusions of the

research might be and whether

participants relevant to the research

question was selected appropriately

you'll also want to appraise how Trust

whether you find the results being

presented in published research first of

all you'll want to know clearly what the

findings are but also you'll want to

know how precise they are depending on

the research question being asked data

may be qualitative or quantitative

observational or interventional and this

will influence the method of data

analysis for example if we were

appraising systematic review and

meta-analysis evidence for the effect of

an intervention we would want to know

the statistical measures used and make

an assessment as to whether they were

undertaken appropriately we would also

want to know whether the author

statement of findings are accurate and

precise assessments based on factors

such as the size of the p-value and the

confidence interval as well as the

overall quality of the evidence in the

field of evidence-based practice we

would also want to be critical about how

the results of research fit with the

existing body of evidence if results of

a study were to entirely contradict

established evidence-based practice you

would want to consider why this has

happened you may ask whether participant

populations and treatments were similar

enough in an interventional study or

whether in a case control study exposure

was accurately and equally measured

between groups

finally what can we determine of the

practical relevance of the research

findings who can the findings be applied

to beyond the confines of the article

you are appraising and can they be

applied locally in your own practice

context when you are critically

appraising research to consider whether

to make use of findings in local

practice you will want to investigate

whether the study setting and

participants covered are sufficiently

similar to your population consider the

outcomes how are they defined and

measured who are they important to and

are they similar to outcomes that you

your clinical team and your patient want

to achieve by being mindful of the

potential differences between the

research context and the context in

which you would hope to make use of

these findings you'll be able to provide

a thorough and considered critical

appraisal that could positively impact

on your evidence-based practice though

we have only touched on some very broad

issues relating to critical appraisal we

hope you've got you thinking more about

how to start approaching research in

future modules we will focus on specific

study designs and spend more time

exploring these different concepts and

how to interpret and apply them in

critical appraisal critical appraisal is

a valuable skill for health

professionals and commissioners as part

of their work in evidence-based practice

much money and many resources are

invested in undertaking healthcare

research internationally however not all

of it is of good quality it is important

to be able to a identify which study

design is most suitable to answer a

particular question be identify the

various sources of bias that may impact

on the reliability of research results

and see be able to judge the value and

practical relevance of research findings

only when we know how to locate

interpret and make sensible best use of

knowledge that is available to us can we

put it into practice and start making a

difference to the way that we provide

high quality evidence-based health care

the second module in this series we'll

look at systematic reviews and

meta-analysis and we will start putting

critical appraisal concepts into

practice we will also demonstrate how we

would critically appraise a systematic

review using a Kaspar checklist with the


to test your learning by appraising a

selected review using the methods we'll

discuss thank you for listening these

training videos have been developed by

the Cochran common mental disorders

group at the University of York with

support from T's Eskom where values NHS

foundation trust Northumberland Tyne and

Wear NHS Foundation Trust and the

Economic and Social Research Council if

you would like to test your knowledge on

the topics introduced in this first

module please follow the link below the

video which will take you to a short

online quiz