Information Security · Teachers College
Securing Research That Utilizes AI
Best practices, tools, and guidance to safeguard sensitive data and AI models — ensuring the integrity and security of research that utilizes artificial intelligence at TC.
Artificial intelligence tools are increasingly embedded in TC research — from data analysis and natural language processing to model training and automated decision-making. While AI offers significant research benefits, it also introduces new security and privacy risks, particularly when sensitive or regulated data is involved. This page provides the TC research community with the frameworks, best practices, and incident response guidance needed to use AI responsibly and securely.
On this page
Understanding the Risks of AI in Research
AI tools — including large language models, machine learning platforms, and generative AI applications — can expose sensitive research data in ways that traditional software does not. Researchers should be aware of the following risk categories before incorporating AI into their work.
Data Privacy & Exposure
Information entered into AI tools — including prompts — may be stored by the vendor, used to train future models, or exposed to unauthorized parties. Sensitive research data should never be entered into an AI tool without prior InfoSec review.
AI Hallucinations
AI tools can generate text, citations, or findings that appear credible but are inaccurate or fabricated. Researchers are responsible for verifying all AI-generated content before using it in research outputs.
Bias in AI Models
AI systems are trained on existing data and may reflect or amplify biases — including underrepresentation of particular groups. Researchers should critically evaluate AI outputs for bias, particularly in studies involving human subjects.
Phishing & Social Engineering
AI has made it easier for malicious actors to craft sophisticated phishing attacks targeting researchers. Be skeptical of unsolicited messages requesting login credentials, data access, or research materials.
Intellectual Property & Attribution
AI-generated content may reproduce the work of others without authorization or proper attribution. This is of particular concern in academic research, where citation integrity is essential.
Regulatory Compliance
Using AI to process HIPAA-covered data, FERPA-protected records, or federally controlled data without appropriate safeguards may violate compliance obligations and the terms of your grant or data use agreement.
Best Practices for AI Use in Research
-
Use only TC-approved AI tools for sensitive research data. Not all AI tools have undergone the security review required for use with sensitive or regulated data. Check the Applications Approved for Research list before using any AI tool with identifiable, health, or federally regulated data.
-
Never enter sensitive or regulated data into consumer AI tools. Consumer AI products (ChatGPT, Gemini, Claude.ai, Copilot in personal accounts, etc.) are not approved for use with HIPAA-covered data, FERPA-protected records, CUI, or non-public research data unless TC has entered into a data processing agreement with the vendor.
-
Review and verify all AI-generated content before use. You are responsible for the accuracy and integrity of research outputs — regardless of whether AI assisted in producing them. Always review, edit, and verify AI-generated text, citations, code, and analysis before inclusion in any research product.
-
Disclose AI use in your IRB protocol and research outputs. If AI tools are used in data collection, analysis, or writing, this should be disclosed in your IRB protocol and in any publications or presentations. Use our AI Use Disclosure Template to get started, or consult your IRB coordinator if you are unsure what disclosure is required.
-
Apply data minimization before using AI. Before submitting data to any AI tool, remove or de-identify all personally identifiable information (PII), protected health information (PHI), and other sensitive fields that are not necessary for the AI task. Use the minimum data required.
-
Secure your AI models and training datasets. If your research involves training or fine-tuning AI models, treat the model and its training data as sensitive research assets. Store them in approved research storage, apply access controls, and document who has access.
-
Review vendor terms before sharing data with an AI platform. Understand what the AI vendor does with your data — including whether prompts are stored, used for model training, or shared with third parties. The Information Security team can help you evaluate vendor terms and data processing agreements.
-
Apply extra caution when using AI in international or field research. AI tools accessed in the field or in international contexts may be subject to additional export control or data sovereignty considerations. Contact InfoSec before using AI tools outside the TC network or with data collected internationally.
Data Classification & AI Tool Suitability
TC classifies research data into four categories. The category of your data determines which AI tools, if any, may be used to process it. When in doubt, treat data as the most sensitive category that applies and consult InfoSec.
| Data Classification | Examples | AI Tool Guidance |
|---|---|---|
| Public | Published datasets, anonymized public records, open research data | Consumer and enterprise AI tools generally acceptable. Always verify outputs. |
| Internal | Unpublished research drafts, internal communications, aggregate institutional data | Use TC-approved enterprise tools only. Avoid consumer AI tools. Consult InfoSec if unsure. |
| Confidential | De-identified human subjects data, non-public grant data, proprietary research datasets | Only use AI tools with a signed data processing agreement. InfoSec review required before use. |
| Restricted | HIPAA-covered data, FERPA records, CUI, federally controlled research data | AI use requires explicit InfoSec and sponsor approval. Most consumer tools are prohibited. Contact InfoSec before proceeding. |
Roles & Responsibilities
| Role | Responsibilities |
|---|---|
| Researcher (Faculty / Student / Affiliate) | Use only approved AI tools for research data. Disclose AI use in IRB protocols and publications. Verify AI-generated content before use. Report suspected incidents to InfoSec immediately. |
| TCIT Information Security | Review and approve AI tools for research use. Provide consultation on AI data security practices. Conduct periodic reviews of approved tools. Lead incident response for AI-related security events. Maintain and update this guidance. |
| IRB Office | Review AI-related disclosures in research protocols. Advise on consent language related to AI data processing. Confirm protocol amendments when AI tools are added to existing studies. |
| Office of the General Counsel | Review vendor data processing agreements and terms of service for AI tools used with research data. Advise on intellectual property and compliance considerations related to AI-generated content. |
| Vice Dean of Research | Provide institutional oversight and guidance on AI use in TC research. Support adoption of AI security best practices across research teams. |
Incident Response: AI-Related Security Events
Examples of AI-related security incidents include: accidentally submitting restricted data to a consumer AI tool, discovering that an AI vendor has experienced a data breach, unauthorized access to an AI model or training dataset, or AI-generated outputs that appear to reproduce another researcher's unpublished work.
-
1Stop using the AI tool immediately.
Do not delete any data, close any accounts, or attempt to remediate the issue yourself. Preserve evidence.
-
2Report to the TCIT Information Security team.
Contact InfoSec as soon as possible. Describe what happened, what data may be involved, and which AI tool or platform was in use. Do not wait until you are certain — report suspected incidents too. Download our Research Data Security Incident Report to document the incident once you've made contact.
-
3Notify your faculty sponsor or PI.
If you are a student or research affiliate, notify your faculty supervisor. If you are a PI, notify your department. The InfoSec team will advise on any further notifications required.
-
4Work with the InfoSec team during the investigation.
InfoSec will lead the incident response process, including assessing the scope of exposure, notifying affected parties if required, and working with the IRB and OGC as needed.
-
5Document and review.
After the incident is resolved, InfoSec will work with you to document what occurred and implement changes to prevent recurrence.
Definitions
- Artificial Intelligence (AI) Tool
- Software that uses machine learning or related techniques to perform tasks such as generating text, analyzing data, recognizing patterns, or making predictions. Includes large language models (LLMs), generative AI applications, and ML platforms.
- Consumer AI Tool
- An AI product designed for general public use (e.g. ChatGPT, Gemini, Claude.ai) that does not include enterprise data protection agreements by default. These tools are generally not approved for use with TC research data beyond the Public classification.
- Data Processing Agreement (DPA)
- A legal contract between TC and an AI vendor that governs how the vendor may collect, store, use, and protect data submitted to their platform. Required before using any AI tool with Confidential or Restricted research data.
- AI Hallucination
- A term for AI-generated output that is factually incorrect, fabricated, or unsupported by evidence, but presented in a confident and plausible manner.
- Prompt
- Text or other input provided to an AI tool to generate a response or output. Prompts may be stored by the AI vendor and should never contain sensitive research data unless the tool is specifically approved for that data type.
- Controlled Unclassified Information (CUI)
- Government-created or government-handled information that requires safeguarding per federal law, regulation, or policy. Research funded by certain federal agencies may involve CUI and requires specific security controls.
- Data Minimization
- The practice of using only the minimum amount of data necessary to accomplish a task — removing or de-identifying sensitive fields before submitting data to an AI tool.
Questions about AI and research security?
TCIT Information Security · 212-678-3300, Opt. 2 · Submit a request