Is there a statement of the problem, Is background information

Article critique guidelines

An article critique or evaluation should include two main sections; a summary and an evaluation. The evaluation section should be 2-3 pages (750-1000 words) long. Your critique should be longer than your summary. You should evaluate all aspects of the research paper from the importance of the problem investigated to the review of the literature, hypotheses, participants, instruments, procedure and design, analysis and results and finally the conclusion section. Your task is to comment on both strong and weak elements of the studies. Do not try to fish for imperfections in the study. When you do find a weak element, make an informed comment with an appropriate recommendation to follow. Below are some questions that could help you structure your critique. You do not have to address all questions. However, you should address highlighted questions. The questions listed are there to help you learn what to look for in evaluating a research article. The format of your paper should NOT be like a Q & A list. Instead, you should integrate your answers into an essay format. Introduction

I. Problem

1. Is there a statement of the problem? 2. Is background information on the problem presented? 3. Is the educational/psychological significance of the problem discussed? 4. Does the problem statement indicate the variables of interest and the specific relationship between those variables which are investigated? When necessary, are variables directly or operationally defined?

II. Review of Related Literature

1. Is the review comprehensive? 2. Are all cited references relevant to the problem under investigation? 3. Have the references been critically analyzed and the results of various studies

compared and contrasted, i.e., is the review more than a series of abstracts or annotations?

4. Does the review conclude with a brief summary of the literature and its implications for the problem investigated?

5. Do the implications discussed form an empirical or theoretical rationale for the hypotheses which follow?

III. Hypotheses

1. Are specific questions to be answered listed or specific hypotheses to be tested stated? 2. Does each hypothesis state an expected relationship or difference? 3. If necessary, are variables directly or operationally defined? 4. Is each hypothesis testable?

Method

I. Participants

1. Are the size and major characteristics of the population studied described? 2. If a sample was selected, is the method of selecting the sample clearly described? 3. Is the method of sample selection described one that is likely to result in a representative,

unbiased sample? 4. Are the size and major characteristics of the sample described? 5. Does the sample size meet the suggested guideline for minimum sample size appropriate

for the method of research represented?

II. Instruments

1. Is the rationale given for the selection of the instruments (or measurements) used? 2. Is each instrument described in terms of purpose and content? 3. Are the instruments appropriate for measuring the intended variables? 4. Is evidence presented that indicates that each instrument is appropriate for the sample

under study? 5. Is instrument validity discussed and coefficients given if appropriate? 6. Is reliability discussed in terms of type and size of reliability coefficients? 7. If appropriate, are subtest reliabilities given? 8. If an instrument was developed specifically for the study, are the procedures involved in

its development and validation described? 9. If an instrument was developed specifically for the study, are administration, scoring or

tabulating, and interpretation procedures fully described?

III. Design and Procedure

1. Is the design appropriate for answering the questions or testing the hypotheses of the study?

2. Are the procedures described in sufficient detail to permit them to be replicated by another researcher?

3. If a pilot study was conducted, are its execution and results described as well as its impact on the subsequent study?

4. Are the control procedures described? 5. Did the researcher discuss or account for any potentially confounding variables that he or

she was unable to control for? Results

1. Are appropriate descriptive or inferential statistics presented? 2. Was the probability level, α, at which the results of the tests of significance were

evaluated, specified in advance of the data analyses?

3. If parametric tests were used, is there evidence that the researcher avoided violating the required assumptions for parametric tests?

4. Are the tests of significance described appropriate, given the hypotheses and design of the study?

5. Was every hypothesis tested? 6. Are the tests of significance interpreted using the appropriate degrees of freedom? 7. Are the results clearly presented? 8. Are the tables and figures (if any) well organized and easy to understand? 9. Are the data in each table and figure described in the text?

Discussion (Conclusions and Recommendation)

1. Is each result discussed in terms of the original hypothesis to which it relates? 2. Is each result discussed in terms of its agreement or disagreement with previous results obtained by other researchers in other studies? 3. Are generalizations consistent with the results? 4. Are the possible effects of uncontrolled variables on the results discussed? 5. Are theoretical and practical implications of the findings discussed? 6. Are recommendations for future action made? 7. Are the suggestions for future action based on practical significance or on statistical significance only, i.e., has the author avoided confusing practical and statistical significance?

8. Are recommendations for future research made?