55 / 66
53
8. VASM evaluation
Treating surgeon’s appraisal of the VASM peer review process
8.1.
The VASM has uniquely implemented an extra step in the audit process, with the inclusion of a feedback form
alongside the assessor reports sent to the treating surgeon. This allows the treating surgeon to record their opinion of
the assessments provided. The form also contains a free-text field in which the treating surgeon, the person who is
most conversant with the clinical nuances of the patient’s course to death, can record their own perspective.
From its commencement of the surgeon’s appraisal survey on 1 January 2015 to the end of the current audit period 30
June 2015, VASM received 1,681 notifications of death and of those, 40.1% (674) had completed the full audit
process. Of the 674 peer assessments sent to the treating surgeon 31.9% (215) had provided the evaluation on the
peer review feedback. Of the 215, 76.3% (164) were for FLAs and 23.7% (51) for SLAs.
Of the 215 treating surgeons that completed the survey, 71.5% agreed with value of the peer review feedback
received, 21.2% were neutral about the feedback and 7.6% disagreed with the assessors’ opinions from the feedback
received. Additional comments were provided alongside the evaluation of the assessment reports on 32.6% of the
feedback forms (70).
Overall, 67.4% of treating surgeons agreed that the peer review feedback is a good source of information to improve
surgical care at their institution (145/215).
This evaluation survey pilot demonstrates that there is value in the audit process. The VASM audit continues to
identify, assess and review factors associated with surgical mortality and will continue to develop action plans,
educational programs and recommendations for further patient care improvements in Victoria.
For a detailed analysis of these qualitative surveys, please see the 2015 VASM technical report.
VASM Evaluation Survey
8.2.
With the release of each VASM Report an evaluation survey was sent to surgeons and hospitals. The survey sought
feedback on the perceived value of the reports, Case Note Review Booklets, the value of the personal feedback sent
to treating surgeons as part of the peer review process and the value of the new electronic interface. In addition there
were free-text sections soliciting suggestions for improvement and requesting topics that might be addressed with
future educational seminars. Surgeons were also asked if the outcomes from any part of the audit process had led to
any change in their practice.
The VASM surveyed 17.6% of Fellows (252/1,429) and 10.1% of hospital contacts (44/436) from the 126 health
services with surgical services.
The evaluation survey indicated that the VASM program was valuable and appropriate to the stakeholders. The
surgical Fellows provided an average score of 3.9 out of 5, and the Victorian hospital stakeholders provided an
average score of 4.3 out of 5.
For a detailed analysis of these surveys please see the 2015 VASM technical report.
The Perceived Quality of VASM Information
8.3.
The VASM has completed the first round of a small qualitative project seeking a range of feedback from its
stakeholders.
The VASM was externally audited in 2015 by Aspex Consulting. The external audit suggested the update of a new
KPI relating to: “The perceived value of information provided by VASM in order to promote ongoing improvements to
surgical safety, quality and confidence across the Victorian health system”.
(36)
This project conducted by VASM, the
Perceived Quality of VASM Information, is in response to the recommendations made by Aspex Consulting.
Data was collected in the form of quantitative and qualitative feedback. The mixed method approach was designed to
provide open ended explorations into stakeholder’s views, while also providing structured tools for annual trending
reports.
(37, 38)
Between November and December 2015, 38 hospital stakeholders were contacted and of those, 68.4% (26/38)
consented to the interview.
Participants were employed at private and public health services and represented different levels of management and
administration.