57 / 66
55
Concordant validity considerations
8.4.
Completion of all fields in the SCF by the treating surgeon requires some self-reflection. An example is where the
treating surgeon is asked to nominate any areas of consideration, concern or adverse events emanating from their
care of the patient. Such responses by the treating surgeon were compared to assessors’ responses to the same
question and the degree of concordance was estimated.
Analysis of concordance is a method of studying inter-relater reliability in reporting all clinical management issues.
Performing a full case note review on all reported deaths is not feasible for logistic reasons.
Gwet’s AC1 provided a more stable interrater reliability coefficient than Cohen’s Kappa and appears less affected by
prevalence and marginal probability and is represented in this report for better interpretation of interrater reliability
analysis.
(40)
The outcomes of concordance analysis shown below are reassuring, as they mirror the predicted outcomes.
•
Disagreement between first- and second-line assessors was most marked in the fluid balance, timing of the
operation, decision to operate, pre, intra and post-operative care and the clinical management section, with
second-line assessors perceiving more issues than the first-line assessors.
•
The tendency of second-line assessors to be more critical of clinical management events is foreseeable as they
have access to an independent description of the episode of care. However, evaluating the quality of the decisions
made by the treating surgeons allows preventative measures to be implemented during the peer review process
and recommendations for improved surgical care to be delivered to the treating clinical teams.
•
The decision to operate is one of the clinical management issues. The question of whether a patient should have
surgery is unclear and may have broader implications for surgical decision making.
(39)
For a detailed analysis of these qualitative interviews, please see the 2015 VASM technical report.