Appendix 1: grading the recommendations

Grading Methodology

To accurately assess the strength of evidence available, the NHMRC methodology (FORM) was used in this clinical practice guideline to formulate and grade recommendations. The aim of this approach by NHMRC is to assist clinical practice guideline developers with a structured process for evaluating the evidence base corresponding to a particular key clinical question, in the context of the setting in which it is to be applied.⁹

The grading methodology allows for both the quality of the evidence and the strength of recommendations to be determined. Where insufficient evidence exists to formulate a grade, a practice point may be assigned instead. The NHMRC grading framework allows for these practice points to be included when developers consider it is important to provide non-evidence-based guidance.⁹

The NHMRC Evidence Statement Form sets out the basis for rating five key components of the ‘body of evidence’ for each recommendation. These components are:

1. The evidence base, in terms of the number of studies, level of evidence and quality of studies (risk of bias).
2. The consistency of the study results
3. The potential clinical impact of the proposed recommendation
4. The generalisability of the body of evidence to the target population for the guideline
5. The applicability of the body of evidence to the Australian healthcare context⁸.

The first two components describe the internal validity of the study data in support of efficacy (for an intervention), accuracy (for a diagnostic test), or strength of association (for a prognosis or aetiological question). As suggested, the third component gives the likely clinical impact of the proposed recommendation. The final two components assess external factors that may influence the effectiveness of the proposed recommendation in practice, in terms of generalisability of study results to the intended target population for the Guideline and setting of the proposed recommendation, and applicability to the Australian (or other local) health care system.⁸

These described components should be rated according to the body of evidence matrix (refer to Table 5). The matrix system is used to summarise the rating of the five key components which allows each recommendation to be assigned an overall NHMRC Grade of Recommendation (A-D).⁹

Table 5: NHMRC Body of evidence matrix⁸

Component	A	B	C	D
Description	Excellent	Good	Satisfactory	Poor
Evidence base#	Several level I or II studies with low risk of bias	One or two level II studies with low risk of bias or a SR/multiple level III studies with low risk of bias	Level III studies with low risk of bias, or level I or II studies with moderate risk of bias	Level IV studies, or level I to III studies with high risk of bias
Consistency*	All studies consistent	Most studies consistent and inconsistency may be explained	Some inconsistency reflecting genuine uncertainty around clinical question	Evidence is inconsistent
Clinical impact	Very large	Substantial	Moderate	Slight or restricted
Generalisability	Population/s studied in body of evidence are the same as the target population for the guideline	Population/s studied in body of evidence are similar to target population for the guideline	Populations/s studied in body of evidence differ to target population for guideline but it is clinically sensible to apply this evidence to target population^	Populations/s studied in body of evidence differ to target population and hard to judge whether it is sensible to generalise to target population
Applicability	Directly applicable to Australian healthcare context	Applicable to Australian healthcare context with few caveats	Probably applicable to Australian healthcare context with some caveats	Not applicable to Australian healthcare context

^#Level of evidence determined from the NHMRC evidence hierarchy

*If there is only one study, rank this component as ‘not applicable’

^{^}For example, results in adults that are clinically sensible to apply to children OR psychosocial outcomes for one cancer that may be applicable to patients with another cancer

There is also capacity to note any other relevant factors that were considered by the guideline developers and the respective Working Group when judging the body of evidence and developing the wording of the recommendation.

The NHMRC grades given (A-D) are intended to indicate the strength of the body of evidence underpinning the recommendation (refer to Table 6).  Grade A or B recommendations are generally based on a body of evidence that can be trusted to guide clinical practice, whereas Grades C or D recommendations must be applied cautiously to individual clinical and organisational circumstances and should be interpreted with care. A recommendation cannot be graded A or B unless evidence base and consistency of the evidence are both rated A and B respectively.⁹

Table 6: Definition of NHMRC grades of recommendations^8,9

Note: This table is replicated in Clinical practice recommendations and practice points

Grade of recommendation	Description
A	Body of evidence can be trusted to guide practice
B	Body of evidence can be trusted to guide practice in most situations
C	Body of evidence provides some support for recommendation(s) but care should be taken in its application
D	Body of evidence is weak and recommendation must be applied with caution

 

By referring to the statements of evidence in combination with the NHMRC body of evidence matrix, a grade for each recommendation was derived from the respective grades allocated to the five key components. Grading the components of consistency, clinical impact, generalisability and applicability, was undertaken by the Working Group members, who discussed each section, and based on consensus achieved across the Working Group, arrived at these ratings.

The use of the NHMRC evidence hierarchy Table, categorises the respective study level according to the study design (refer to Table 3). This is used to determine the respective grades for evidence base and consistency of the recommendation.

Implementing the NHMRC Evidence Hierarchy, each included study in a systematic review should be assessed according to the following three dimensions of evidence:

1. Strength of evidence (level of evidence, quality of evidence (risk of bias) and statistical precision.
2. Size of effect (assessing the clinical importance of the findings of each study and hence addressing the clinical impact component of the body of evidence matrix.
3. Relevance of evidence (translation of research evidence into clinical practice and is potentially the most subjective of the evidence assessments).

Table 7: NHMRC Evidence Hierarchy: designations of ‘levels of evidence’ according to type of research question⁸

Level	Intervention	Diagnostic accuracy	Prognosis	Aetiology	Screening Intervention
I	A systematic review of level II studies	A systematic review of level II studies	A systematic review of level II studies	A systematic review of level II studies	A systematic review of level II studies
II	A randomised controlled trial	A study of test accuracy with: an independent, blinded comparison with a valid reference standard, among consecutive persons with a defined clinical presentation	A prospective cohort study	A prospective cohort study	A randomised controlled trial
III-1	A pseudo randomised controlled trial(i.e. alternate allocation or some other method)	A study of test accuracy with: an independent, blinded comparison with a valid reference standard, among non-consecutive persons with a defined clinical presentation	All or none	All or none	A pseudo randomised controlled trial (i.e. alternate allocation or some other method)
III-2	A comparative study with concurrent controls: • Non-randomised, experimental trial • Cohort study • Case-control study • Interrupted time series with a control group	A comparison with reference standard that does not meet the criteria required for Level II and III-1 evidence	Analysis of prognostic factors amongst persons in a single arm of a randomised controlled trial	A retrospective cohort study	A comparative study with concurrent controls: • Non-randomised, experimental trial • Cohort study • Case-control study
III-3	A comparative study without concurrent controls: • Historical control study • Two or more single arm study • Interrupted time series without a parallel control group	Diagnostic case-control study	A retrospective cohort study	A case-control study	A comparative study without concurrent controls: • Historical control study • Two or more single arm study
IV	Case series with either post-test or pre-test/post-test outcomes	Study of diagnostic yield (no reference standard)	Case series, or cohort study of persons at different stages of disease	A cross-sectional study or case series	Case series