REF2: Outputs

References on this page refer to the paragraph numbers within the 'Guidance on submissions' (GOS) and the 'Panel criteria and working methods' (PC).

REF 2021 introduces a new approach to output selection. No longer will each staff member be required to submit four outputs for review, but instead each submission will consist of a pool of outputs, subject to minimum and maximum limits per individual. This decoupling of staff and outputs is intended to provide increased flexibility to institutions in building the portfolio of outputs for submission and shift the focus from quantity to quality of outputs.

FAQs

For more FAQs relating to outputs, please visit the REF website for an up-to-date list.

• Can we submit the outputs of former staff? Yes. Outputs of former Category A Eligible staff can be submitted. This includes staff still employed by the University who have moved to ineligible contract types.
• Do we have to submit the outputs of former staff? No. There is no requirement for us to submit outputs of former staff.
• Do former staff contribute to the FTE submitted? No. They do not contribute to the FTE count for the number of outputs and impact case studies required.
• Can we submit the outputs of Category A Eligible staff that have moved onto other contract types, such as a teaching or admin contract? Yes. These staff are treated in the same way as former staff.
• Can we submit the outputs of Category A Submitted staff that were published before they were employed by Greenwich? Yes. All outputs published since January 2014 are eligible for submission, regardlessof wherea member of staff was employed when the output was published or what type of contract they held.
• Can we submit outputs published in 2013? Outputs 'pre-published' during the REF2014 period (2008-2013) are eligible provided that the final version of the output was published in this census period (2014-2020), and that it was not submitted to REF2014.
• Can we submit the same output to different UOAs? Yes, where a different co-author is submitted to each unit.
• How do double-weighted outputs contribute to the maximum of 5 per person? A double-weighted output counts as two outputs, therefore the maximum any individual can be attributed to is two double-weighted and one single-weighted output.

Number of outputs required

Each submission need to have a number of outputs equal to 2.5 times the combined FTE of Category A Submitted staff, rounded to the nearest whole number (GOS 205). So, for example, a submission

of 39 staff, totalling 37.2 FTE, will need 93 outputs. This number may be adjusted as appropriate to take account of approved requests for unit reductions due to equality-related circumstances.

A minimum of one outputis required for each Category A Submitted staff member, and then further outputs up to the total required for the unit need to be submitted, with no more than five attributed to any individual (including former staff) (GOS 207).

So, continuing the example above, each of the 39 staff will require one outputattributed to them (Pool A), while the remaining 54 outputs (93 - 39 = 54) can come from any mix of current and former staff, up to a total maximum of 5 outputs allocated per person (Pool A + Pool B).

Output eligibility

To be eligible, an output must be:

1. The product of research AND
2. First made first publicly available between 1 January 2014 and 31 December 2020 AND
3. Compliant with the open access policy (where applicable) AND
4. Attributable to either:
   1. Category A Submitted staff member who made a substantial research contribution to the output, regardless of where the member of staff was employed at the time they produced that output,
   2. OR a former Category A Eligible staff member who was employed by the University when the output was first made publicly available.

Outputs of former staff

We can submit the outputs of former staff that were first made publicly available while they were employed by us as Category A Eligible staff (GOS 211-216). This includes both leavers and current staff who have moved onto ineligible contract types (such as a Teaching contract, or a contract that is less than 0.2 FTE).

Outputs published when the individual was not Category A Eligible (such as a student or on teaching contract) are ineligible, as are outputs first published before or after their employment at Greenwich, regardless of when it was submitted or accepted. 

Timing of publication

Outputs must be first made first publicly available between 1 January 2014 and 31 December 2020 (GOS 256) to be eligible for submission.

Outputs due for publication between 27 November 2020 and 31 December 2020 can be submitted with a reserve output to mitigate the risk of an unclassified score in the event of delayed publication (GOS 266-267).

Co-authored outputs and author contribution

Outputs may normally be submitted only once by each unit. Where co-authors are in different units then the output may be submitted by each unit (PC 221, 226, 231).

In Main Panel D the same co-authored output may exceptionally be submitted twice within the same submission, up to 5% of the total outputs. This provision cannot be used in combination with double-weighting (PC 233-235).

Outputs may only be attributed to co-authors who made a substantial research contribution to the output (GOS 209).

UOAs 1-6 and 9 require additional information to determine the contribution of the attributed co-author where there are 16 or more authors and they are not the lead or corresponding author.

No additional information is required for the other UOAs, or for outputs in UOAs 1-6 and 9 with 15 or fewer authors. However, the substantial research contribution criteria still apply and are subject to audit.

If a contribution is cited on the paper then that will take precedence, regardless of the number of authors (PC 222-225, 228-231).

If the sub-panel is satisfied that the co-author's contribution is distinct and substantial then it will assess the quality of the output, taking no further regard of individual contributions. If they are not satisfied, then the output will be unclassified (PC 216-220).

Double-weighting

We can request that outputs of extended scale and scope be double-weighted (count as two outputs) in the assessment. The panels have described the criteria for double-weighting in the Panel Guidance (PC 242-247).

A 100-word statement is required explaining how the scale and scope of the output satisfies the panel criteria (PC 238).

Double-weighted outputs can be submitted with a reserve output to mitigate the risk of an unclassified score in the event that the request is not upheld (GOS 282-283).

Additional information for outputs

Main Panel A

Information about the research process and content is welcomed for all non-text or practice-based outputs where it is not evident within the output itself (up to 300 words). This includes, but is not limited to, patents, software or standards documents, and applies to all Units of Assessment in this Main Panel (PC 252). 

The above does not include information about significance. Any information about the significance of the output will be disregarded (PC 253). 

Main Panel B

All Units of Assessment within Main Panel B require a description of the research process and content where this is not evident within the output (up to 300 words). 

For reviews, they also welcome the identification of the original research content or new insights reported (up to 300 words) (PC 254-255).

Units of Assessment 11 and 12 only invite factual information (up to 100 words) to be provided to outline the significance of the output. This could include evidence of recognition, impact on the state of the art, where it led to further developments, or has been applied (PC 256). The information provided must be verifiable and externally referenced where possible. 

Main Panel C

For any submission where the research content and/or process is not evident in the output, such as non-text outputs or teaching materials, submissions should include a statement identifying the research questions, methodology and means of dissemination (up to 300 words) (PC 259).

For practice-based outputs, submissions should include an explanatory presentation in an easily-handled paper-based format to allow the panel to understand the output without visiting it and to make a judgement of its research contribution. These additional submissions can include images, figures and diagrams (PC 260).

Where the form of an output makes it necessary, this paper-based submission can be supplemented by additional visual material such as a video (PC 262).

For software and data sets, a full written description should be provided including details of how and where the data set or software can be accessed (PC 261).

Main Panel D

For any submission where the role of the researcher or the research process is not evident in the output, UoAs are strongly encouraged to include a statement identifying the research process, insights and means of dissemination (300 words) (PC 264-269).

For outputs where the role of the researcher is not evident, UoAs have the following submission options (PC 265):

• a single item with a 300-word supporting statement
• a multi-component output, with a 300-word supporting statement
• a single item supported by contextual information (formerly called a portfolio). In this case the 300-word statement should be used to indicate which elements are the output, and which the contextual information.
• a multi-component output supported by contextual information (formerly called a portfolio). In this case the 300-word statement should be used to indicate which elements are the output, and which the contextual information.

Where a research output is a contribution to an anthology, edited book, special issue of a journal, or curatorial project, the whole work should be submitted. Units of Assessment may provide a statement (up to 300 words) to clarify the nature of the individual's contribution (PC 270).

Groups of short items such as creative writing, encyclopaedia or dictionary entries may be submitted as a single output in a portfolio style where such work embodies research as defined in the Guidance for Submissions. A statement of up to 300 words must be included to explain the rationale behind the grouping (PC 271).

For research outputs in languages other than English, a short abstract in English (up to 100 words) should be provided to describe the content and nature of the work. This requirement is waived for UoA26 if the output is produced in any of the languages within the remit of the panel (PC 272).

Use of metrics

Citation data will be used to inform the assessment of outputs where panels consider this appropriate for their discipline (PC 274-282). These are:

• UOA 1 Clinical Medicine
• UOA 2 Public Health, Health Services and Primary Care
• UOA 3 Allied Health Professions, Dentistry, Nursing and Pharmacy
• UOA 4 Psychology, Psychiatry and Neuroscience
• UOA 5 Biological Sciences
• UOA 6 Agriculture, Food and Veterinary Sciences
• UOA 7 Earth Systems and Environmental Sciences
• UOA 8 Chemistry
• UOA 9 Physics
• UOA 11 Computer Science and Informatics
• UOA 16 Economics and Econometrics

The citation data will be provided by Clarivate Analytics (formerly known as Web of Science) and will be made available to panels in a consistent and transparent way.

We will be able to view and verify the citation data for outputs in the REF Submission System before submission.

Open Access

To be eligible, all journal articles and conference proceedings (with an ISSN) accepted for publication: 

• Between 1 April 2016 and 31 March 2018 must be deposited in a compliant open access repository within three months of first publication.
• After 1 April 2018 must be deposited in a compliant open access repository within three months of acceptance.

For the University of Greenwich this means that outputs should be uploaded to the Greenwich Academic Literature Archive (GALA).

For more information about GALA go to LibGuides.

Who to contact

Please contact Neil Cormack-Bissett on 020 8331 9274 or N.Cormack-Bissett@gre.ac.uk.

Queries relating to Open Access and GALA should directed to gala@gre.ac.uk.