This is Part 4 of the article, where we are discussing the different phases of the validation processes an RTO should be following to ensure they meet regulatory requirements and industry expectations.
In the previous articles, we discussed the following regarding the validation of assessment resources:
- Explanation of assessment validation
- Typical benchmarks used during the validation processes
- Stages of validation (before, during and after the assessment judgements)
- Regulatory requirements for conducting validation
- Assessment systems
- Who conducts validation?
- How is validation different from moderation?
- How external consultants can help you with validation of assessment and learner resources?
- The regulatory requirements around validation of learner resources.
In this month’s article, we will explore the following:
- Why do you need to keep validators information
- Why validation of assessment and learner resources should be systematic and ongoing
- How you can schedule validation
- What is statistically valid sampling
- Validation outcomes
Why do you need to keep validators information?
You need to keep for validators information for the following reasons:
- To demonstrate that the RTO’s assessment system can consistently produce valid assessment judgements.
- Validation is undertaken by one person or by a team of people. The RTO must ensure the review process is completed by people who collectively hold:
- vocational competencies and current industry skills relevant to the assessment being validated
- current knowledge and skills in vocational teaching and learning, and
- the TAE40110 Certificate IV in Training and Assessment (or its successor) or the TAESS00001 Assessor Skills Set (or its successor).
- A regulatory body such as Australian Skills Quality Authority (ASQA) may request evidence of it during audit activity.
Reasons why validation of assessment and learner resources should be systematic and ongoing.
A unit of competency needs to be regularly reviewed to ensure that it meets current industry and regulatory requirements, therefore, you must have a systematic and ongoing mechanism to track any changes and to ensure all your resources are up-to-date and current.
How to schedule validation
The first step is to develop a “validation schedule” used to validate each training product (AQF qualification, skill set, unit of competency, accredited short course and module) on your scope of registration.
A validation schedule is a five year plan and each training product must be reviewed at least once in that five-year period, and at least 50% of your training products must be validated in the first three years of the schedule. Your RTO might choose to validate its training products more often, for example, if risk indicators demonstrate that more frequent validation is required. Indicators of risk might include:
- the use of new assessment tools
- delivery of training products where safety is a concern
- the level and experience of the assessor, or
- changes in technology, workplace processes, legislation, and licensing requirements.
Once you have your validation schedule you need to complete a validation plan with dates and timelines. The more detailed your plan is with regards to the who (needs to be included), when (what date and for how long), and what (which units is being validate, what information will be needed, from where are we getting it) the better your outcomes for your validation will be.
What is statistically valid sampling?
A statistically valid sample is one that is:
- large enough that the validation outcomes of the sample can be applied to the entire set of judgements, and
- taken randomly from the set of assessment judgements being considered.
Use ASQA’s validation sample size calculator for more information. It can be found here:
Validation outcomes
You must keep all records as a soft-copy/scanned format or in hard-copy. The records must include all the tools used to conduct validation such as assessment resources, validation forms and checklists, profiles of validators etc.
The validation outcomes should identify recommendations for improvement to the assessment tool, assessment process or assessment outcome.
Sometimes the validation outcome can identify critical issues in the collection of valid evidence.
When this occurs, you may:
- increase the validation sample size to assist in identifying patterns of issue
- validate completed assessments from other units of competency to see if the issue is spread across the whole of the qualification, and
- look for patterns of error (for example, consider if it is one assessor making invalid judgements—this could indicate the assessor requires further training in competency-based assessment).
Your validation plan must clarify how you will document and act on validation outcomes. For example, an assessment validation checklist addressing the principles of assessment and rules of evidence may be utilised to document the validation outcomes. Outcomes of validation may be acted upon through your RTO’s continuous improvement processes.
Your RTO must have a records management process to retain the evidence of the validation. You should retain evidence of:
- the person/people leading and participating in the validation activities (including their qualifications, skills and knowledge)
- the sample pool
- the validation tools used
- all assessment samples considered, and
- the validation outcomes.
If the validation outcomes recommend improvements to the assessment tool, you should implement these recommendations across all training products, not only those included in the sample. If you make changes to the assessment tool, complete quality checks and review the revised tool prior to implementation.