NCVER has heard user requirements from the sector
NCVER engaged Nous Group (Nous) for a discovery process to develop a response on behalf of the VET sector to questions about the VET information management system, including:
1. What are the necessary requirements for a renewed, modern and fit-for-purpose national VET information management system that improves quality, timeliness and access?
2. What is the set of possible opportunities and actions that can deliver on this for all stakeholders across the VET system?
The discovery process has concluded, and found that user requirements of the VET data system tend to support three over-arching needs:
• to better understand what is happening in the VET sector in order to play their role effectively,
• to achieve this in a way that does not unduly compromise ongoing operations, and
• to support students in their journey through training with timely information and accurate records of achievement.
The ways in which the VET data system can improve its offering to support these needs is something that different stakeholders had different perspectives on, depending on the nature of their interactions with data. For example, changing the system such that RTOs only need to submit data once would be of significant benefit to RTOs, whereas other stakeholders may not experience as strong a benefit from this type of system change.
For this reason, the set of user requirements described in this website represent a range of stakeholder perspectives.
The user requirements were developeD via multiple engagement methods
To undertake the discovery process, Nous conducted interviews, received written submissions, and conducted a Discovery Workshop. The word clouds below give an indication of the over-arching themes surfaced by these three communication methods .
Many users are ready for change
Nous distilled the requirements of the VET sector into a set of 8 user requirements. These are listed in the image below, and are described in the sections which follow. Each user requirement is accompanied by a table to outline the benefits it will deliver to four stakeholder groups:
- SMS Vendors
- Policy makers, contract managers, and regulators
Timely and frequent data releases
In the current VET data system, data provision and access is slow and delayed. Stakeholders commented that annual data reports, which are produced based on academic year figures, are generally not published until the upcoming financial year has already begun. This means that organisations must use data that is 18 months old when planning for upcoming financial years.
With regards to government-funded data, State Training Authorities currently have very timely data access, as data is first reported by RTOs to their relevant STA at a monthly or quarterly interval. The STA then holds the data itself before later passing it on to NCVER. Hence when referring to ‘timeliness’ as a user requirement, STAs mean that they want to continue to have at least the same degree of instant access to data that they currently enjoy.
Stakeholders not only want data to be more timely, but also want it to be released at more frequent intervals. Minimising the delay to receiving annual reports will not be enough – stakeholders want quarterly or even monthly access to data releases.
The table below provides detail on the advantages for different stakeholder groups of more timely and frequent VET data release.
High quality and complete data
NCVER has very effective systems for ensuring that the data it holds is of a very high statistical quality, having been subjected to a number of checks before being accepted into NCVER’s data collections.
Stakeholders have noted, however, that the strict validation process results in instances where some data submitters focus more on having data that is valid, rather than data that is complete. For example, an RTO which has had an upload rejected might simply delete the portions of data which are causing the rejection and re-submit the data without them, rather than investigating the invalid data and amending it.
When discussing enhancing data completeness, some stakeholders expressed positivity toward a system in which data is accepted on initial submission, and clear and understandable error messages are provided to the submitter to prompt them to repair the offending data. This of course requires the system to enable data to be edited after it has already been submitted.
Ensuring both high quality (ie. data that is valid and accurate) and completeness (data which represents a very high proportion of actual activity) will provide many benefits to stakeholders as outlined in the table below.
Shared data collection and management
Stakeholders spoke of adopting a shared approach to collection and management, envisaging a system whereby data is collected once, and used many times. To reduce the number of times that data must be purposefully collected requires that it be stored in a shared data collection at the earliest convenient moment after it is created. This shared data collection would be held in a partnership between NCVER, States and Territories, and the Commonwealth. This approach would enable each subsequent user of data to extract it from the shared storage location, rather than depending on receiving it directly from another participant in the sector.
Benefits of a shared collection and management approach are outlined in the table below.
Analysis and reporting tools
A major underlying need that stakeholders have of the VET data system is to have a strong understanding of activity in the VET sector. High quality analysis and reporting provides this understanding. Despite this, analysis and reporting was not among the most mentioned requirements, as some individual organisations already have very strong abilities in this area – all they need instead is quality, timely data to analyse themselves. Other organisations, however, are not as strong in this area, and indicated that outside support would help them to operate more effectively.
Such support could be provided through web-based analytics and reporting portals which give stakeholders information both at a national and local level. This may include providing a deeper understanding of student trajectories both inside and outside of particular operating jurisdictions. The table below describes the benefits for different stakeholders of strengthening the analytics and reporting offering of the system.
Linkage ready data
Linking data sets can build the sector’s understanding of student journeys before, during, and after their interaction with the VET system. Examples of data sets that may provide interesting insights include the ATO, Centrelink, and higher education data. This can help answer questions such as:
- What types of courses do job-seekers most enrol into?
- What types of courses most lead to changes in employment status?
- What types of courses most frequently lead to higher education enrolment?
Stakeholders indicated that while linking VET activity data to existing data sets will take some co-ordination and preparation, there was a large potential benefit to doing so. These benefits are outlined in the table below.
More flexible and agile data standards
There is a strong desire for a new data standard. Under present arrangements, AVETMISS standards are updated every two years. When the AVETMISS standards change, this triggers a series of actions within the sector: each State Training Authority then updates their own data standards, which are passed on to RTOs. Subsequently SMS Vendors must adapt their systems to comply with the AVETMISS standards, and also to comply with a combination of different State and Territory standards, depending on which states their client RTOs operate in.
Stakeholders indicated that they want to be able to engage in a collaborative process in updating data standards, and they would benefit from standards updating on a shortened timeframe.
One approach which had been raised by some stakeholders is a system in which standards are updated in a way which connects to the SMS systems directly. This would minimise the impact of standard changes on RTOs, as standards would be communicated to the software directly.
Advantages to more flexible and agile AVETMISS standards are outlined in the table below.
An integrated survey system
Stakeholders noted that students receive surveys at different times from different organisations. This repetition may impact on response rates and certainly would impact on students’ experiences. Stakeholders suggested integrating the sector’s approach to conducting surveys, and combining the questions from the multiple entities that generally have their own surveys (RTOs, State Training Authorities, NCVER). The benefits to stakeholders of this approach are outlined in the table below.
Alignment of government, regulators, and departments regarding privacy and confidentiality issues
Stakeholders noted that interpretations of privacy laws and policies are not clear between different entities in the VET sector. Further, some stakeholders noted that some practices appear to be in contradiction to current privacy legislation, while other policies appear more conservative than legislation requires. For example, after jurisdictions submit data, they later receive it back in an aggregated, de-identified form.
The benefits to stakeholders of aligning the relevant entities on privacy issues are outlined in the table below.
Considerations for the future
Along with satisfying user needs with regards to data quality, timeliness and ease of submission, NCVER will need to lead the sector toward a future state which expands its data offering in other ways. This includes supporting new datasets, enriching existing datasets, enabling different linkage techniques and supporting the use of advanced analytics. These are presented in the table below.
NCVER will consider these areas for expansion as it undergoes future planning.