Guide to an Overall Critique of a Quantitative Research Report

Aspect of the Report

Critiquing Questions

Detailed

Critiquing

Guidelines

Title

Ø Is the title a good one, succinctly suggesting key variables and the study population?

Abstract

Ø Does the abstract clearly and concisely summarize the main features of the report (problem, methods, results, conclusions)?

Introduction

Statement of the problem

Ø Is the problem stated unambiguously, and is it easy to identify?

Ø Does the problem statement build a cogent, persuasive argument for the new study?

Ø Does the problem have significance for nursing?

Ø Is there a good match between the research problem and the paradigm and methods used? Is a quantitative approach appropriate?

Box 4.3, page 90

Hypotheses or research questions

Ø Are research questions and/or hypotheses explicitly stated? If not, is their absence justified?

Ø Are questions and hypotheses appropriately worded, with clear specification of key variables and the study population?

Ø Are the questions/hypotheses consistent with the literature review and the conceptual framework?

Box 4.3, page 90

Literature review

Ø Is the literature review up to date and based mainly on primary sources?

Ø Does the review provide a state-of-the-art synthesis of evidence on the problem?

Ø Does the literature review provide a sound basis for the new study?

Box 5.4, page 122

Conceptual/theoretical framework

Ø Are key concepts adequately defined conceptually?

Ø Is there a conceptual/theoretical framework, rationale, and/or map, and (if so) is it appropriate? If not, is the absence of one justified?

Box 6.3, page 145

Method

Protection of human rights

Ø Were appropriate procedures used to safeguard the rights of study participants? Was the study externally reviewed by an IRB/ethics review board?

Ø Was the study designed to minimize risks and maximize benefits to participants?

Box 7.3, page 170

Research design

Ø Was the most rigorous possible design used, given the study purpose?

Ø Were appropriate comparisons made to enhance interpretability of the findings?

Ø Was the number of data collection points appropriate?

Ø Did the design minimize biases and threats to the internal, construct, and external validity of the study (e.g., was blinding used, was attrition minimized)?

Box 9.1, page 230;

Box 10.1, page 254

Population and sample

Ø Is the population described? Is the sample described in sufficient detail?

Ø Was the best possible sampling design used to enhance the sample’s representativeness? Were sampling biases minimized?

Ø Was the sample size adequate? Was a power analysis used to estimate sample size needs?

Box 12.1, page 289

Data collection and measurement

Ø Are the operational and conceptual definitions congruent?

Ø Were key variables operationalized using the best possible method (e.g., interviews, observations, and so on) and with adequate justification?

Ø Are specific instruments adequately described and were they good choices, given the study purpose, variables being studied, and the study population?

Ø Does the report provide evidence that the data collection methods yielded data that were reliable and valid?

Box 13.1, page 309; Box 14.1, page 347

Procedures

Ø If there was an intervention, is it adequately described, and was it rigorously developed and implemented? Did most participants allocated to the intervention group actually receive it? Is there evidence of intervention fidelity?

Ø Were data collected in a manner that minimized bias? Were the staff who collected data appropriately trained?

Box 9.1, page 230; Box 10.1, page 254

Results

Data analysis

Ø Were analyses undertaken to address each research question or test each hypothesis?

Ø Were appropriate statistical methods used, given the level of measurement of the variables, number of groups being compared, and assumptions of the tests?

Ø Was the most powerful analytic method used (e.g., did the analysis help to control for confounding variables)?

Ø Were Type I and Type II errors avoided or minimized?

Ø In intervention studies, was an intention-to-treat analysis performed?

Ø Were problems of missing values evaluated and adequately addressed?

Box 16.1, page 400;

Box 17.1, page 429

Findings

ØIs information about statistical significance presented? Is information about effect size and precision of estimates (confidence intervals) presented?

Ø Are the findings adequately summarized, with good use of tables and figures?

ØAre findings reported in a manner that facilitates a meta-analysis, and with sufficient information needed for EBP?

Box 17.1, page 429;

Box 28.1, page 687

Discussion

Interpretation of the findings

Ø Are all major findings interpreted and discussed within the context of prior research and/or the study’s conceptual framework?

ØAre causal inferences, if any, justified?

Ø Are interpretations well-founded and consistent with the study’s limitations?

Ø Does the report address the issue of the generalizability of the findings?

Box 19.1, page 482

Implications/ recommendations

Ø Do the researchers discuss the implications of the study for clinical practice or further research—and are those implications reasonable and complete?

Box 19.1, page 482

Global Issues

Presentation

Ø Is the report well-written, organized, and sufficiently detailed for critical analysis?

Ø Is the report written in a manner that makes the findings accessible to practicing nurses?

Box 28.2, page 698

Researcher credibility

Ø Do the researchers’ clinical, substantive, or methodologic qualifications and experience enhance confidence in the findings and their interpretation?

Summary assessment

Ø Despite any limitations, do the study findings appear to be valid—do you have confidence in the truth value of the results?

Ø Does the study contribute any meaningful evidence that can be used in nursing practice or that is useful to the nursing discipline?