Generative AI has rapidly become embedded across scientific research and the wider knowledge production pipeline. While its use has been increasingly normalised, we at the AI Accountability Lab have been reflecting on what it means to use this technology in the context of our own work, which is largely focused on evaluating the technology itself.
The environmental costs, labour exploitation, data extraction, concentration of power, and normalisation of surveillance underlying the technology make it difficult, if not impossible, to meaningfully operationalise and practice “ethical” and “responsible” use. Choosing not to make use of generative AI for these reasons is a valid position. Alongside that, we have developed this document to support in-depth, critical engagement on the topic of generative AI’s implications for scientific integrity, a process which we believe can sharpen our own work.
As a research lab primarily tasked with critically evaluating the technology and its downstream impacts, its use in research raises fundamental questions about scientific integrity and validity. Generative AI is now routinely used for text classification, data annotation, and similar tasks, making the technology not only an object of evaluation but also an instrument of evaluation. This raises important questions about methodological validity and reliability. What does it mean, in practice, to adhere to core scientific principles such as transparency, reproducibility, and accountability while using opaque, commercialised, and continually changing models? How confident can we be with knowledge claims (partly) produced using models that have since been deprecated as methodological tools?
In the following, we reflect on the values that we consider central to scientific integrity and examine how generative AI might undermine them. Many of these considerations apply to machine learning techniques broadly, and while we focus on generative AI in this policy, we encourage reflection regardless of model type. However, we believe it is particularly urgent in the context of generative AI given its uniquely widespread adoption, large scale exploitation, immense hype and commercialisation, and the centralisation of power in the hands of a few powerful actors.
This is not to say that every conceivable use produces harm, nor does it deny that particular uses may have legitimate research value. Yet, it places the burden of justification on the use of the technology rather than on its non-use. The purpose of this document is to make its costs and implications – specifically for credible scientific practice – harder to ignore and, ultimately, to help AIAL members, our collaborators, and the larger scientific community make informed decisions about the personal, epistemic, and societal cost of its use in the research process.
Even if the use of generative AI models might be justified, we request that researchers continually challenge their assumptions. Non-use is the default because we want use to be an intentional decision, reached through a reflexive process.
The following salient dimensions of generative AI usage should be considered when determining whether its use can be justified. We also strongly encourage exploring alternatives to determine whether non-AI tools can support your use case.
Motivation for your usage: Why are you using it - what is the underlying reason? E.g., because it is convenient; to save time doing an analysis you couldn’t otherwise; to understand a concept you are not very familiar with; to “improve” your writing? What is lost when doing so? Is it skill acquisition, learning, personal and intellectual growth or development? What dynamics, potentially at a systemic level, does it feed into? E.g., chronic “lack of time” in modern work, or the unintentional risk of plagiarism or scientific misconduct? What are the implications for your ability to be accountable for your own work?
- Guidance: Write down the main reasons why you are using it, if you choose to. This can clarify the trade-off between the risks and the anticipated benefits, and help you decide where you want to draw the line.
Alternatives: Is it possible to complete the same task using other means? Have you exhausted all possible alternatives within reason? What would using the AI model or tool enable you to do which you otherwise could not? Is there a person or a community of practice you can ask for help or guidance on it?
- Consider:
- Supporting and working with local experts as a first point of alternative, where possible.
- If you are a small research lab like us, curate an internal list of non-generative AI tools, resources, and libraries tailored to your needs. If you do have one, let us know via email to <[email protected]> :)
Type of task: We strongly question the use of generative AI in knowledge production (e.g., generating hypotheses, synthesising insights, ideation). However, when it comes to code generation, we acknowledge its potential for use, e.g., when the design is set by the researcher, not in itself a process of knowledge production, and in line with other considerations outlined in this policy.
Degree of usage: How you use AI requires as much care and deliberation as determining if you will use AI. How much is offloaded to AI poses a major concern. Ask yourself: what kind of work is being delegated? Is it rote work or would it entail intellectual work? Lower “levels” of usage may be acceptable in consideration with the rest of this policy. Higher “levels” are wholly unacceptable for any work done at the Lab, as it dramatically increases risks. For example, the more substantive the LLM use, the harder it becomes to establish the contribution of the researcher and the risks of inadvertent plagiarism (LLM use in scholarly writing poses a provenance problem). This is closely related to the below section, “Verifying the output.”
- For example, when using it to generate code:
- Lower level: Small snippets which can be verified manually against a specification; minor debugging.
- Higher level: hundreds of lines of generated code that are difficult to review and to test and often include many design decisions; agentic coding.
- For example, during research:
- High level: ideation and deliberation, literature review, data collection, data annotation, and data analysis.
- For example, in writing tasks:
- Lower level: spell checking, correcting typos.
- Low level: rephrasing in a grammatical sense, e.g. correct phrasing or grammatical tense.
- High level: changes to content or introduction of new words or phrases; e.g. rewrite to make the text clearer or more consistent, or auto-complete that rephrases text while introducing words or framing ideas in ways that give rise to risk beyond level 2. (See further reading).
- Higher level: generating entirely new content, e.g., creating an abstract, summarising a document, turning bullet points into prose.
Specificity: Was the model or tool developed for a specific use case that matches the one you are using it for? Discriminative models are typically more task-specific than generative models.
- Guidance: Use more specific tools when possible, weighing this consideration alongside the others in this policy. If the tool is overly broad and cannot be adapted relative to the task, reconsider its use.
Replicability: Can the tool be used in a deterministic way, such that your process and results can be replicated exactly? How crucial is this for your use case, as this can be task dependent?
- Factors for consideration:
- Can the LLM itself be used in a replicable way?
- Are the model weights downloadable such that you can run a specific model version yourself, now and in the future?
- Can someone else replicate this work without a highly specialised setup?
- Please see this and this for further elaboration of the notions of replicability and reproducibility in research involving LLM use.
- Guidance: Look into open-weight models, identify those which you are confident will remain available and can be replicated by others without highly specialized setups, and verify that you can reproduce the exact same output given an input.
- Model provenance: Please note that replicability and model provenance are related but distinct considerations. Open-weight models make model parameters available for download, allowing specific versions to be run and pinned. Open-source models, in theory, also make training data and training code available, allowing the model itself to be inspected and rebuilt. However, sometimes models described as “open-source,” such as Llama and Gemma, are open-weight rather than open-source. While open-weight models might be sufficient to reproduce a specific model version and, in principle, similar input-output behaviour, questions of model provenance require further details about how the model was built, including access to training data and code. Consider these separately when assessing whether a model can guarantee replicability and, ultimately, whether its use can be justified.
Verifying the output: Do you have the domain expertise to understand and check the output? Please note that understanding and verifying are not the same and that verifying can take substantial time. Furthermore, expertise is a spectrum, ranging from sufficient domain expertise for meaningfully assessing the output, broad familiarity with the area, to being entirely outside your knowledge. Your level of expertise, therefore, determines the degree to which you can understand and verify the output.
- Factors for consideration:
- Does the model/tool work for the specific task? Does it perform the task in accordance with well established standards?
- For information-seeking usage: Can you validate the information by checking independent sources and/or by validating any reference provided? Otherwise, the output cannot be trusted.
- Guidance:
- Where possible, compare the model’s output against those produced independently, using a sample, and report what you compared and how much of it. For example, for:
- Classification or annotation: compare against human coders on a held out sample (Cohen’s κ, or Krippendorff’s α for more than two coders), and/or against gold standard using precision, recall and F1, reporting the sample size and comparison method.
- Transcription: compare against a sample transcribed independently, using word error rate. Report the sample size and comparison method.
- Coding: verification may be more appropriate than a quality measure. For example, write tests or define expected outputs before reviewing the generated code, or compare the outputs against a trusted implementation or library on the same inputs.
- Other types of tasks: identify an established quality metric or independent reference to the task, and apply it to a sample of the output. Use this as a sanity check, not a final judgement.
- Where possible, compare the model’s output against those produced independently, using a sample, and report what you compared and how much of it. For example, for:
Transparency of your usage: Regardless of where you land on the considerations above, document and be transparent about any usage and your reasoning process. We aspire for transparency both in public-facing work as well as work and ideas shared within the lab. This covers any usage, from spellcheck and grammatical edits to anything more substantial, such as text summarization and ideation. We expect all collaborators to respect our policy and for AIAL members to communicate this prior to embarking on and throughout a collaborative project, similar to approaches taken by other labs.
- Guidance: Use the below format to declare the use of generative AI tools. Feel welcome to edit or expand it to fit your specific case.
- [Model/tool] was used for [purpose]. All generated [code/text] was thoroughly reviewed and verified [describe how e.g., line-by-line] by the authors, who remain responsible for the final output.
Multi-perspectival socio-political issues: LLMs are known to encode and, at times, exacerbate historical and existing societal norms, inequalities, and injustices. How might the values encoded in an LLM affect your work and learning, if you were to use it for tasks pertaining to dynamic, contextual, and multi-perspectival issues? Consider the nuanced and opaque ways the model might reproduce dominant assumptions, normative perspectives or biases while obscuring or excluding others. How might this shape what you learn or the conclusion you draw? Are there specific socially and politically contentious issues where the use of LLMs cannot be justified?
- Guidance:
- Aspire to adhere to the principle of “nothing about us without us”. Engage and familiarise yourself with relevant communities, and where possible, engage directly with people (potentially) affected by the issues. Where direct engagement is not possible, familiarise yourself with relevant literature and other sources to develop a nuanced and informed understanding.
- Consider if and how you can (re)shape the values a model encodes which are relevant to your use case, e.g., via fine-tuning. This may not always be possible and such methods (e.g., statistical fairness techniques) often have fundamental limitations themselves.
- Understand where data originated, how it was obtained, and its composition, and consider how this affects other points in this policy. Consider the principles of data feminism.
- Examine and challenge power: Analyse how power operates, challenge unequal power structures, and work toward justice.
- Elevate emotion and embodiment: Value multiple forms of knowledge, including the knowledge grounded in levied and embodied experience.
- Rethink binaries and hierarchies: Challenge systems of categorisation and classification (e.g., gender binary) that perpetuate oppression.
- Embrace pluralism: An inherently dynamic and fluid world means knowledge is never static and complete. However, comprehensive knowledge comes from synthesising multiple perspectives, with priority given to local, Indigenous, and experiential ways of knowing.
- Consider context: Recognise that data are not neutral or objective, but products of unequal social relations, and that this context is essential for conducting accurate and ethical analysis.
- Make labour visible: Recognise and make visible the labour involved in producing data and data-driven systems.
You might also consult broader policies within our University
- TCD GenAI Hub
- TCD GenAI Statement
- TCD General Regulations and Information which specifically outlines: “[…] Where the output of GenAI is used in a document or work output, this usage should be acknowledged and appropriately cited, as per Library guidelines on acknowledging and referencing GenAI. Where restrictions are applied around the use of GenAI in particular modules, those restrictions must be explicitly stated in module descriptions in Blackboard Ultra and/or in programme or module handbooks.”
Further reading:
- On DEAD (Decontextualized, Engineered, Anonymized, Disembodied) methodologies:
- On the reproduction of WEIRD biases in research:
- On the political economy of Big AI:
This living document reflects our position as of October 2026 and will be continually revised as the ecosystem develops. It has been developed through continuous deliberations and iterative discussions over an extended period of time amongst AIAL members and our broader community.
How to cite:
Artificial Intelligence Accountability Lab (AIAL). (October, 2026) AIAL Statement on Generative AI and Scientific Integrity. (Version 1.0) DOI: 10.5281/zenodo.23220545
BibTex:
@misc{AIAL2026,
Author = {{AIAL}},
Title = {{AIAL Statement on Generative AI and Scientific Integrity}},
Year = {2026},
Version = {1.0},
DOI = {10.5281/zenodo.23220545},
Publisher = {AIAL}
}