Nous

 

NCVER has heard user requirements from the sector

NCVER engaged Nous Group (Nous) for a discovery process to develop a response on behalf of the VET sector to questions about the VET information management system, including:

1.      What are the necessary requirements for a renewed, modern and fit-for-purpose national VET information management system that improves quality, timeliness and access?

2.      What is the set of possible opportunities and actions that can deliver on this for all stakeholders across the VET system?

The discovery process has concluded, and found that user requirements of the VET data system tend to support three over-arching needs:
•    to better understand what is happening in the VET sector in order to play their role effectively, 
•    to achieve this in a way that does not unduly compromise ongoing operations, and
•    to support students in their journey through training with timely information and accurate records of achievement.  

The ways in which the VET data system can improve its offering to support these needs is something that different stakeholders had different perspectives on, depending on the nature of their interactions with data. For example, changing the system such that RTOs only need to submit data once would be of significant benefit to RTOs, whereas other stakeholders may not experience as strong a benefit from this type of system change. 

For this reason, the set of user requirements described in this website represent a range of stakeholder perspectives.  

 

The user requirements were developeD via multiple engagement methods

To undertake the discovery process, Nous conducted interviews, received written submissions, and conducted a Discovery Workshop.  The word clouds below give an indication of the over-arching themes surfaced by these three communication methods .

Image: Key words from written submissions

Image: Key words from written submissions

Image: Key words from interviews

Image: Key words from interviews

Image: Key words from Discovery Workshop

Image: Key words from Discovery Workshop

 
thin line.png

 

Many users are ready for change

Nous distilled the requirements of the VET sector into a set of 8 user requirements.  These are listed in the image below, and are described in the sections which follow.   Each user requirement is accompanied by a table to outline the benefits it will deliver to four stakeholder groups:

  • RTOs
  • SMS Vendors
  • Policy makers, contract managers, and regulators
  • Students
Image: List of user requirement categories, weighted by  their frequency of occurrence in a discovery process exercise

Image: List of user requirement categories, weighted by  their frequency of occurrence in a discovery process exercise

 

Timely and frequent data releases

In the current VET data system, data provision and access is slow and delayed. Stakeholders commented that annual data reports, which are produced based on academic year figures, are generally not published until the upcoming financial year has already begun. This means that organisations must use data that is 18 months old when planning for upcoming financial years. 
With regards to government-funded data, State Training Authorities currently have very timely data access, as data is first reported by RTOs to their relevant STA at a monthly or quarterly interval. The STA then holds the data itself before later passing it on to NCVER. Hence when referring to ‘timeliness’ as a user requirement, STAs mean that they want to continue to have at least the same degree of instant access to data that they currently enjoy.
Stakeholders not only want data to be more timely, but also want it to be released at more frequent intervals. Minimising the delay to receiving annual reports will not be enough – stakeholders want quarterly or even monthly access to data releases. 
The table below provides detail on the advantages for different stakeholder groups of more timely and frequent VET data release.

Table: Benefits of timely and frequent data releases

Table: Benefits of timely and frequent data releases

High quality and complete data

NCVER has very effective systems for ensuring that the data it holds is of a very high statistical quality, having been subjected to a number of checks before being accepted into NCVER’s data collections.
Stakeholders have noted, however, that the strict validation process results in instances where some data submitters focus more on having data that is valid, rather than data that is complete. For example, an RTO which has had an upload rejected might simply delete the portions of data which are causing the rejection and re-submit the data without them, rather than investigating the invalid data and amending it.
When discussing enhancing data completeness, some stakeholders expressed positivity toward a system in which data is accepted on initial submission, and clear and understandable error messages are provided to the submitter to prompt them to repair the offending data. This of course requires the system to enable data to be edited after it has already been submitted.
Ensuring both high quality (ie. data that is valid and accurate) and completeness (data which represents a very high proportion of actual activity) will provide many benefits to stakeholders as outlined in the table below.

Table: Benefits of high quality and complete data

Table: Benefits of high quality and complete data

Shared data collection and management

Stakeholders spoke of adopting a shared approach to collection and management, envisaging a system whereby data is collected once, and used many times. To reduce the number of times that data must be purposefully collected requires that it be stored in a shared data collection at the earliest convenient moment after it is created. This shared data collection would be held in a partnership between NCVER, States and Territories, and the Commonwealth. This approach would enable each subsequent user of data to extract it from the shared storage location, rather than depending on receiving it directly from another participant in the sector.
Benefits of a shared collection and management approach are outlined in the table below.

Table: Benefits of shared data collection and management

Table: Benefits of shared data collection and management

Analysis and reporting tools

A major underlying need that stakeholders have of the VET data system is to have a strong understanding of activity in the VET sector. High quality analysis and reporting provides this understanding. Despite this, analysis and reporting was not among the most mentioned requirements, as some individual organisations already have very strong abilities in this area – all they need instead is quality, timely data to analyse themselves. Other organisations, however, are not as strong in this area, and indicated that outside support would help them to operate more effectively.
Such support could be provided through web-based analytics and reporting portals which give stakeholders information both at a national and local level. This may include providing a deeper understanding of student trajectories both inside and outside of particular operating jurisdictions. The table below describes the benefits for different stakeholders of strengthening the analytics and reporting offering of the system.

Table: Benefits of high quality analysis and reporting tools

Table: Benefits of high quality analysis and reporting tools

Linkage ready data

Linking data sets can build the sector’s understanding of student journeys before, during, and after their interaction with the VET system. Examples of data sets that may provide interesting insights include the ATO, Centrelink, and higher education data. This can help answer questions such as:

  • What types of courses do job-seekers most enrol into?
  • What types of courses most lead to changes in employment status?
  •  What types of courses most frequently lead to higher education enrolment?

Stakeholders indicated that while linking VET activity data to existing data sets will take some co-ordination and preparation, there was a large potential benefit to doing so. These benefits are outlined in the table below.

Table: Benefits of linkage ready data

Table: Benefits of linkage ready data

More flexible and agile data standards

There is a strong desire for a new data standard. Under present arrangements, AVETMISS standards are updated every two years. When the AVETMISS standards change, this triggers a series of actions within the sector: each State Training Authority then updates their own data standards, which are passed on to RTOs. Subsequently SMS Vendors must adapt their systems to comply with the AVETMISS standards, and also to comply with a combination of different State and Territory standards, depending on which states their client RTOs operate in.
Stakeholders indicated that they want to be able to engage in a collaborative process in updating data standards, and they would benefit from standards updating on a shortened timeframe.
One approach which had been raised by some stakeholders is a system in which standards are updated in a way which connects to the SMS systems directly. This would minimise the impact of standard changes on RTOs, as standards would be communicated to the software directly.
Advantages to more flexible and agile AVETMISS standards are outlined in the table below.

Table: Benefits of flexible and agile data standards

Table: Benefits of flexible and agile data standards

An integrated survey system

Stakeholders noted that students receive surveys at different times from different organisations. This repetition may impact on response rates and certainly would impact on students’ experiences. Stakeholders suggested integrating the sector’s approach to conducting surveys, and combining the questions from the multiple entities that generally have their own surveys (RTOs, State Training Authorities, NCVER). The benefits to stakeholders of this approach are outlined in the table below.

Table: Benefits of an integrated survey system

Table: Benefits of an integrated survey system

Alignment of government, regulators, and departments regarding privacy and confidentiality issues

Stakeholders noted that interpretations of privacy laws and policies are not clear between different entities in the VET sector. Further, some stakeholders noted that some practices appear to be in contradiction to current privacy legislation, while other policies appear more conservative than legislation requires. For example, after jurisdictions submit data, they later receive it back in an aggregated, de-identified form.
The benefits to stakeholders of aligning the relevant entities on privacy issues are outlined in the table below.

Table: Benefits of alignment regarding privacy and confidentiality issues

Table: Benefits of alignment regarding privacy and confidentiality issues

thin line.png

Considerations for the future

Along with satisfying user needs with regards to data quality, timeliness and ease of submission, NCVER will need to lead the sector toward a future state which expands its data offering in other ways. This includes supporting new datasets, enriching existing datasets, enabling different linkage techniques and supporting the use of advanced analytics.   These are presented in the table below.

NCVER will consider these areas for expansion as it undergoes future planning.  

Table: Areas of expansion for NCVER’s service offering

Table: Areas of expansion for NCVER’s service offering

 
thin line.png

 

Next steps

We thank you for your involvement in the process so far, and will be in touch with future updates.  

Please direct your questions regarding the discovery process to VET_data_streamlining@nousgroup.com.au 

To view the original Issues Paper as a website, please see here.

 

Opportunities
thin line.png