Building a mixed-methods evaluation plan

A mixed-methods evaluation plan for health intervention programs turns delivery goals into a coherent evidence strategy. It connects a program logic model to evaluation questions, indicators, data sources and decisions, so results can guide practice rather than remain in a final report.

The Lower New York State Public Health Training Center offers courses on logic models, qualitative research, mixed methods and focus groups. This collaborative approach suits Australian teams, where evidence must connect local practice with state and national obligations.

Australian teams work in a mixed market: a Commonwealth-funded program may sit within a state health service, a Primary Health Network or an Aboriginal Community Controlled Health Organisation. Staff call scope a “project brief,” use “co-design” for community participation and reserve “sign-off” for approval. These terms reveal where authority and accountability sit.

Good planning starts with decisions stakeholders need to make. It selects complementary quantitative and qualitative methods, protects participant dignity and builds a feasible path from fieldwork to action in Sydney, Melbourne, Darwin and regional or remote communities.

Frame the evaluation around program logic and use

Begin with a one-page logic model linking inputs, activities, outputs and outcomes, plus assumptions or external conditions affecting success. A theory of change can add the causal pathway, while an evaluation matrix links each question to an indicator, source, timing and owner.

Define which decisions the evaluation will inform: continue, adapt, scale, target or stop. Distinguish implementation questions about reach, adoption and fidelity from outcome questions about health, wellbeing or system change. Name intended users so each analysis has a clear job.

Specify measures, samples and statistical analysis

Operationalise every indicator before fieldwork. Specify its definition, denominator, source, baseline, target and collection point, including when change is expected. Include reach, dose, quality, cost and equity measures where they matter, rather than treating one clinical endpoint as the whole evaluation.

Select data that match the question: de-identified administrative records, surveys or clinical measures for patterns, and carefully sampled comparison groups where feasible. Pre-specify subgroup analyses by place, age, disability, language and Indigenous status, using suitable sample sizes. Report uncertainty and avoid causal claims the design cannot support.

Plan qualitative fieldwork and analysis

For qualitative work, use interviews, focus groups or observation to explore experience, mechanisms, barriers and unintended effects. Sample purposively across program intensity, demographic groups and implementation settings, recruiting people who can vary the explanation rather than simply adding the most accessible participants.

Prepare discussion guides around a few analytic themes, train facilitators, and test questions with interpreters or support needs. In northern Australia and Aboriginal communities, cultural protocols may require local leadership, community-controlled processes or formats such as yarning circles. Ask how participants wish to be identified and findings shared.

Record, transcribe and code data consistently, then develop themes across cases and sites. Keep an audit trail of coding decisions, write reflexive notes about the researcher’s position, and bring preliminary interpretations back to the relevant community or advisory group. This avoids treating lived experience as a decorative quotation.

Integrate the two evidence strands

Choose an integration design early. An explanatory sequence can use qualitative findings to explain an unexpected quantitative pattern; an exploratory sequence can use interviews to develop survey items or hypotheses; a convergent design compares parallel results. The sequence should follow the decisions, not a preferred methodology.

Build a joint display placing key statistics, participant accounts and case examples side by side. For an Aboriginal health promotion program in regional New South Wales, attendance and referral data might show strong uptake, while interviews reveal transport, language or trust barriers. Read convergence and contradiction together, then write a meta-inference explaining what each strand contributes.

Build governance, ethics and cultural safety

Create a small evaluation governance group with decision rights, not just a long invitation list. Include the program owner, commissioning body, frontline staff, data specialists, ethics expertise and people with lived experience. Assign responsibility for access, analysis, safeguarding, reporting and responses, and meet often enough to resolve barriers.

Check consent, privacy and data requirements before recruitment. The National Health and Medical Research Council’s National Statement, the Privacy Act 1988, state and territory health-records rules and local data agreements may apply. With Aboriginal and Torres Strait Islander data, use the relevant community’s governance process and make collective benefit, data sovereignty and secure stewardship explicit.

Evidence governance extends to sources used in training and reporting. Before citing external source material, the team should verify ownership, authorship, date and relevance, and record why it is credible for the claim being made.

Plan resources, timelines and quality assurance

Cost the evaluation as a core program activity. Include staff time, training, participant reimbursement, translation and accessibility, software, storage, reporting and community review. Australian grants and commissioning contracts often require specific budgets and performance reports, so align the timetable with those milestones and allow for ethics and data-access approvals.

Work backwards from the date results are needed. Pilot instruments, test the data dictionary, define extraction specifications, document analysis decisions and check quality before databases are locked. Assign task owners and keep a risk register for low recruitment, missing records, inconsistent collection and staff turnover.

International delivery models can prompt useful comparison for evaluation training. Dutch-language training resources may show how sessions and materials are framed for another health system, but teams should verify accreditation, terminology and cultural fit before adopting the approach locally.

Turn findings into funded action

Report in layers: a plain-language community summary, a concise funder dashboard, a technical report and a short team learning session. Present results by place and population where numbers permit, use respectful language, and separate findings from interpretation. Make limitations visible so decision makers do not mistake uncertainty for failure.

Link every conclusion to an advance decision rule: what would trigger continuation, redesign, additional investment or cessation? Assign an owner and response date to each recommendation, and schedule feedback with participants and operational teams. Evidence becomes useful when people can see what changed because it was heard.

Start the next evaluation cycle by drafting a one-page logic model, naming the decisions, assigning indicator owners and booking the first community review before data collection begins.