Lab Behavior & Industry

Quick Assessment

Key Links

Overview

This page tracks measurable indicators of AI laboratory behavior, safety practices, and industry transparency. These metrics help assess whether leading AI companies are following responsible development practices and honoring their public commitments.

Understanding lab behavior is critical because corporate practices directly influence AI safety outcomes. Even the best technical safety research is insufficient if labs are racing to deploy systems without adequate testing, suppressing internal safety concerns, or failing to disclose dangerous capabilities.

Lab Behavior Dynamics

1. Voluntary Commitment Compliance Rate

Current Status (2025): Mixed compliance, with significant variation across companies and commitment types.

Key Findings

A comprehensive study from August 2025↗📄 paper★★★☆☆arXivDo AI Companies Make Good on Voluntary Commitments to the White House?Wang, Jennifer, Huang, Kayla, Klyman, Kevin et al. (2025)Research analyzed 16 AI companies' compliance with White House voluntary AI commitments in 2023, finding wide disparities in performance with an average score of 53% and signifi...capabilitiescybersecuritySource ↗ examining companies' adherence to their White House voluntary AI commitments found significant variation:

Compliance by Commitment Type

Related Commitments

In May 2024, 16 AI companies joined the Frontier AI Safety Commitments, pledging to develop Responsible Scaling Policies (RSPs) by February 2025. Many companies did publish RSPs, though the quality and specificity varies significantly.

Concerning Developments

In April 2025, OpenAI removed a provision from its Preparedness Framework without noting the change in the changelog, raising transparency concerns.

Data Quality: Based on detailed public rubric scoring of disclosed company behavior. Limitations include reliance on public disclosure and potential for selective transparency.

2. RSP Capability Threshold Crossings

Current Status (2025): First ASL-3 activation announced for Claude Opus 4. No other publicly confirmed threshold crossings, though evaluation methodologies remain contested.

Anthropic's RSP Framework

Anthropic pioneered the Responsible Scaling Policy↗🔗 web★★★★☆AnthropicAnthropic pioneered the Responsible Scaling PolicygovernancecapabilitiesSource ↗ approach in September 2023, with the policy now at Version 2.2 (effective May 14, 2025):

ASL-3 Activation (2025)

Anthropic activated ASL-3 protections↗🔗 web★★★★☆Anthropicactivated ASL-3 protectionsSource ↗ for Claude Opus 4, representing the first publicly confirmed capability threshold crossing:

• ASL-3 Security Standard: Increased internal security measures to protect model weights
• ASL-3 Deployment Standard: Targeted measures limiting CBRN weapons misuse risk
• Precautionary basis: Anthropic has not definitively confirmed Claude Opus 4 crossed the capability threshold, but could not "clearly rule out ASL-3 risks"

Key Threshold Domains

Recent Changes and Concerns

Version 2 shift: Anthropic moved from quantitative benchmarks to qualitative descriptions of capability levels. Critics note this reduces verifiability.

Grade decline: According to SaferAI, Anthropic's grade dropped from 2.2 to 1.9, placing them in the "weak" category alongside OpenAI and DeepMind. The primary concern is the shift away from precisely defined thresholds.

Institute for AI Policy and Strategy recommendation: Companies should define verifiable risk thresholds informed by "societal risk" tolerances from other industries. Current thresholds may be too lenient.

Detection Challenges

Small improvements in elicitation methodology can dramatically increase scores on evaluation benchmarks. Naive elicitation strategies may significantly underreport risk profiles, potentially missing dangerous capabilities that sophisticated actors could unlock.

Data Quality: Limited public disclosure of evaluation results. Companies control both the design and disclosure of dangerous capability evaluations, creating incentives to underreport concerning findings.

3. Time Between Model Training and Safety Evaluation

Current Status (2025): Decreasing evaluation windows, with some tests compressed from months to days.

Shortened Timelines

The Financial Times reported↗🔗 webFinancial Times reportedSource ↗ that OpenAI has been slashing safety evaluation time, giving testers "just a few days for evaluations that had previously been allotted weeks or months to be completed."

This compression creates severe constraints on thorough safety testing:

• Complex evaluations require substantial time to design and execute
• Emergent capabilities may only become apparent through extended testing
• Red teams need adequate access to explore edge cases and failure modes
• One evaluator told FT: "We had more thorough safety testing when [the technology] was less important"

Pre-Deployment Testing Examples

OpenAI o1 evaluation (December 2024): US AISI and UK AISI conducted joint pre-deployment testing during a "limited period of pre-deployment access." Testing was conducted by expert engineers and scientists across three domains:

1. Cyber capabilities
2. Biological capabilities
3. Software and AI development

The evaluators noted that testing was "conducted in a limited time period with finite resources, which if extended could expand the scope of findings."

Evaluator Access Challenges

METR↗🔗 web★★★★☆METRAI models can be dangerous before public deploymentThe article argues that current AI safety frameworks focused solely on pre-deployment testing are inadequate, as internal AI model usage and development can pose significant ris...safetyevaluationSource ↗ and other evaluation organizations report that comprehensive risk assessments require:

• Substantial expertise and specialized knowledge
• Direct access to models and training data
• More time than companies typically provide
• Information about technical methodologies that companies often withhold

METR argues that powerful AI systems "are not ordinary products" and risks should be addressed throughout the whole AI development lifecycle. They advocate for "earlier evaluations for dangerous capabilities, better forecasting of AI capabilities prior to training, and more emphasis on security and safety throughout development."

AI Safety Index Findings

The 2025 AI Safety Index↗🔗 web★★★☆☆Future of Life InstituteFLI AI Safety Index Summer 2025The FLI AI Safety Index Summer 2025 assesses leading AI companies' safety efforts, finding widespread inadequacies in risk management and existential safety planning. Anthropic ...safetyx-risktool-useagentic+1Source ↗ from Future of Life Institute found:

• "AI companies are unlikely to make high-assurance safety cases if timelines are short"
• "AI developers control both the design and disclosure of dangerous capability evaluations, creating inherent incentives to underreport alarming results"
• "Naive elicitation strategies cause significant underreporting of risk profiles"
• The gap between capabilities acceleration and risk management practice is widening

Data Quality: Limited public data on specific evaluation timelines. Most information comes from investigative journalism, evaluator reports, and company transparency documents.

4. External Red-Team Engagement Rate

Current Status (2025): External red teaming is standard practice at major labs, but engagement scope and disclosure vary significantly.

Major Lab Practices

OpenAI: Conducts pre-deployment adversarial testing by vetted external experts. External red teamers identify alignment failures, injection vectors, tool misuse paths, and safety regressions. Findings inform mitigation strategies and deployment decisions.

Notable recent engagements: ControlPlane's Torin van den Bulk contributed to external red team testing on GPT-4o, Operator, o3-mini, and Deep Research, with live access to model checkpoints.

HackerOne Platform: Provides structured AI Red Teaming (AIRT) as 15 or 30-day engagements. Over 750 AI-focused researchers contribute to engagements for frontier labs including Anthropic, Snap, and Adobe. HackerOne has tested 1,700+ AI assets across customer scopes.

Key Vulnerabilities Found

From HackerOne's aggregated testing data across 1,700+ AI assets:

Anthropic Jailbreak Challenge Results (2025)

Anthropic partnered with HackerOne↗🔗 webAnthropic partnered with HackerOneSource ↗ to test Constitutional Classifiers on Claude 3.5 Sonnet:

• 300,000+ chat interactions from 339 participants
• $55,000 in bounties paid to four successful teams
• One team found a universal jailbreak passing all levels
• One team found a borderline-universal jailbreak
• Two teams passed all eight levels using multiple individual jailbreaks

Testing Metrics

Government Framework

CISA defines AI red teaming as a subset of AI Testing, Evaluation, Verification and Validation (TEVV). NIST has operationalized this through programs like Assessing Risks and Impacts of AI (ARIA) and the GenAI Challenge.

Limitations

While external red teaming is increasingly common, critical gaps remain:

• Limited disclosure of red team findings and remediation actions
• Selective engagement: Labs choose which red teamers to work with
• Short engagement windows: 15-30 days may be insufficient for complex systems
• Post-deployment gaps: Less emphasis on continuous adversarial testing after launch

Data Quality: Some public information from lab announcements and red team providers. Comprehensive engagement rates and detailed findings remain largely non-public.

5. Dangerous Capability Disclosure Delays

Current Status (2025): Significant and increasing delays, with some major model releases lacking safety documentation entirely.

Google Gemini 2.5 Pro (March 2025)

Google released Gemini 2.5 Pro↗🔗 web★★★☆☆FortuneGoogle's Gemini 2.5 Pro missing key safety report in violation of promisesGoogle launched Gemini 2.5 Pro without publishing a required safety report, contradicting previous commitments made to government and international bodies about model transparen...safetyevaluationllmSource ↗ without a model card, violating commitments made to the U.S. government and at international AI safety summits:

Government Response: 60 U.K. politicians signed an open letter↗🔗 web★★★☆☆FortuneBritish lawmakers accuse Google of 'breach of trust' over delayed Gemini 2.5 Pro safety reportA group of 60 U.K. lawmakers criticized Google DeepMind for not fully disclosing safety information about its Gemini 2.5 Pro AI model as previously committed. The letter argues ...safetyevaluationllmSource ↗ accusing Google DeepMind of "a troubling breach of trust with governments and the public" and a "failure to honour" international commitments.

Google's Defense: The company claimed Gemini 2.5 Pro was an "experimental" release, exempting it from normal documentation requirements.

OpenAI Documentation Gaps

• Deep Research model: Released without a system card, which was published weeks later
• GPT-4.1: OpenAI announced it would not publish a technical safety report, arguing the model is "not a frontier model"

Broader Industry Pattern

Meta Llama 4: Model card was similarly brief and limited in detail, drawing criticism from AI safety researchers.

Systemic Issues: The AI Safety Index found that "AI developers control both the design and disclosure of dangerous capability evaluations, creating inherent incentives to underreport alarming results or select lenient testing conditions that avoid costly deployment delays."

Transparency Requirements

While voluntary commitments emphasize transparency, actual disclosure practices show significant gaps:

• Limited disclosure of evaluation methodologies
• Weak evidence of systematic safety processes
• Uneven adoption of robust evaluation practices

New Legal Requirements

California's Transparency in Frontier AI Act (effective 2026) establishes:

• Transparency requirements for large AI developers
• Mandatory reporting of critical safety incidents to state attorney general
• Whistleblower protections for employees reporting risks

Data Quality: Based on public monitoring by AI governance organizations, investigative journalism, and government oversight. Actual capability evaluation results remain largely proprietary.

6. Pre-Deployment Safety Testing Duration

Current Status (2025): Highly variable and generally decreasing. No standardized minimum testing period exists.

Testing Approaches

Major frontier AI labs follow safety policies that include pre-deployment testing:

• OpenAI's Preparedness Framework↗🔗 web★★★★☆OpenAIPreparedness Frameworkbiosecuritydual-use-researchx-riskSource ↗ (Version 2, April 2025)
• Google DeepMind's Frontier Safety Framework
• Anthropic's Responsible Scaling Policy↗🔗 web★★★★☆AnthropicAnthropic pioneered the Responsible Scaling PolicygovernancecapabilitiesSource ↗ (Version 2.2, May 2025)

Third-party evaluators (UK AISI, US AISI, Apollo Research, METR↗🔗 web★★★★☆METRmetr.orgsoftware-engineeringcode-generationprogramming-aisocial-engineering+1Source ↗) also conduct pre-deployment assessments, though their access and time are limited. METR's analysis of 12 companies↗🔗 web★★★★☆METRMETR's analysis of 12 companiesevaluationsdangerous-capabilitiesautonomous-replicationSource ↗ with published frontier AI safety policies found variable commitment levels.

Known Testing Examples

OpenAI o1 (December 2024): Joint US AISI and UK AISI evaluation during a "limited period" before public release. Specific duration not disclosed publicly.

Safeguard Testing Benchmarks: Research examples show wide variation in time requirements:

• First vulnerability test: 10 minutes of expert red teamer time
• Second test (novel universal jailbreak): Over 7 hours of expert effort

Industry Trends

The 2025 AI Safety Index concluded that:

• Pre-deployment testing is "likely necessary but insufficient" for responsible AI development
• Testing is conducted with "limited time periods and finite resources"
• "If timelines are short, AI companies are unlikely to make high-assurance safety cases"

Comparison to Other Industries

Unlike pharmaceuticals (multi-year clinical trials) or aerospace (extensive certification processes), AI systems lack:

• Standardized testing protocols
• Minimum duration requirements
• Independent verification mandates
• Clear pass/fail criteria for deployment

Data Quality: Very limited public data. Specific testing durations are rarely disclosed. Assessment based on general industry reports and occasional third-party evaluator statements.

7. Model Release Velocity

Current Status (2025): Unprecedented acceleration, with major labs releasing frontier models within weeks of each other.

Release Frequency Trends

2024 Baseline: Major labs typically released frontier models annually or semi-annually.

2025 Acceleration: "Companies that typically released major models annually or semi-annually were now shipping frontier models within weeks of each other." Each release incorporated learnings from the previous week's competitive announcements.

November-December 2025: "Tit-for-Tat Arms Race"

In just 25 days, four major AI companies launched their most powerful models↗🔗 web25 days, four major AI companies launched their most powerful modelsSource ↗:

This concentration represented "a compression of innovation never before seen in technology history." OpenAI's Sam Altman issued an internal "code red" memo↗🔗 web★★★☆☆TechCrunchissued an internal "code red" memoSource ↗ after Gemini 3 topped leaderboards, with internal sources reporting that some employees asked for delays but "competitive pressure forced the accelerated timeline."

2025 Release Summary

OpenAI:

• GPT-5 with improved coding and "thinking" mode
• GPT-5.1 Codex Max (agentic coding model)
• GPT-5.2
• gpt-oss-120b and gpt-oss-20b (open-source models)
• Dozens of feature launches (GPT-4o Image, standalone Sora app, group chats)

Anthropic:

• Claude 4 family (Opus and Sonnet)
• Claude Opus 4.5 (November 24)
• Claude 4.5 Haiku

Google DeepMind:

• Gemini 2.5 (March)
• Gemini 3 (November)
• Gemini 2.5 Deep Think
• Genie 3.0 (world model)

Meta & Others:

• DeepSeek R1 (January 20, 2025) - major open model impact
• Qwen 3 and various Chinese lab releases

Tracking Data

• AI Flash Report: Tracked 43 model releases as of October 27, 2025
• Our World in Data: Tracks large-scale AI systems (>10²³ FLOP training compute)

Safety Implications

Rapid release velocity creates pressure that:

• Reduces time available for safety evaluation
• Encourages "shipping within weeks" competitive dynamics
• Creates feedback loops of rapid iteration
• May prioritize "shiny products" over safety culture

Data Quality: Good tracking of major model releases through multiple sources. Precise internal development timelines remain proprietary.

8. Open-Source vs Closed Model Capability Gap

Current Status (2025): Gap narrowing significantly, from approximately 16 months in 2024 to approximately 3 months in late 2025.

Current Gap Estimate

Epoch AI research from October 2025↗🔗 web★★★★☆Epoch AIEpoch AI research from October 2025Source ↗ found:

Specific Example: Meta's Llama 3.1 405B (released July 2024) took approximately 16 months to match the capabilities of GPT-4's first version.

Gap Narrowing Evidence

2024: Ecosystem relied primarily on Llama 3, with Qwen2.5 and DeepSeek known to specialists.

2025: DeepSeek R1 (January 20, 2025)↗🔗 web2025 Open Models Year in ReviewFlorian Brand, Substack, SubstackThe 2025 open model landscape saw dramatic capability increases, with models like DeepSeek R1 and Qwen 3 rivaling closed models across key benchmarks. Chinese and global open mo...capabilitiesevaluationopen-sourceSource ↗ achieved performance parity with OpenAI's o1 while operating at 15x reduced cost, training for just $5.6 million. The open ecosystem "immensely accelerated in terms of capabilities, rivaling closed models on most key benchmarks."

Meta's Claims

Meta described Llama 3.1 as "the first frontier-level open source AI model," claiming it "outperforms GPT-4.0 and Anthropic's Claude 3.5 on several benchmarks" in internal evaluations.

Remaining Closed Model Advantages

Highest-end performance: GPT-4 and newer models remain more capable on complex tasks requiring deep reasoning.

Enterprise benchmarks: On SWE-Bench Verified (real GitHub issue fixes):

• State-of-the-art closed models (GPT-5.2-Codex, Claude Opus 4.5): 80%+
• Top open-source models: 65%
• This gap is described as "critical for enterprise use"

Open Model Strengths

Niche verticals: Open-source models lead in biomedicine, law, and defense applications where institutional constraints (privacy, security, customization) matter more than raw performance.

Cost-effectiveness: Significantly cheaper for customization and fine-tuning.

Adoption trends: According to a16z research:

• 41% of surveyed enterprises will increase use of open-source models
• Another 41% will switch from closed to open if performance matches

Open Source Definition Debate

Meta's Llama models don't meet the Open Source Initiative's definition, which requires sharing:

• Model weights (Meta provides this)
• Training data (Meta does not provide this)
• Training code (Meta does not provide this)

Google Internal Assessment

A leaked 2023 Google memo warned: "we aren't positioned to win this arms race … I'm talking, of course, about open source. Plainly put, they are lapping us."

Data Quality: Based on benchmark comparisons, research organization analysis (Epoch AI), and industry reports. Benchmarks may not capture all relevant capability dimensions.

9. Lab Safety Team Turnover Rate

Current Status (2025): Specific turnover rates not publicly disclosed, but high-profile departures suggest significant retention challenges at leading labs.

OpenAI Safety Departures (2024-2025)

May 2024 - Superalignment Team Dissolution↗🔗 web★★★☆☆CNBCOpenAI dissolves Superalignment AI safety teamOpenAI has disbanded its Superalignment team, which was dedicated to controlling advanced AI systems. The move follows the departure of key team leaders Ilya Sutskever and Jan L...safetyresearch-agendasalignmentinterpretabilitySource ↗:

Jan Leike's criticism: "Over the past few months my team has been sailing against the wind. Sometimes we were struggling for [computing resources]" despite OpenAI's promise to allocate 20% of compute to Superalignment. He joined Anthropic.

Result: The entire Superalignment team was disbanded, with members reassigned to other teams.

September 2024 Leadership Exits:

• Mira Murati (CTO, 6 years at OpenAI)
• Bob McGrew (Chief Research Officer)
• Barret Zoph (VP of Research)
• Hannah Wong (Chief Communications Officer)
• Tom Cunningham (Economics Researcher)
• Miles Brundage (Policy Research Head)

Total documented senior departures: 25+ as of December 2024

June 2024 Open Letter

Nine current and former OpenAI employees wrote an open letter criticizing the company for "recklessly racing" to build AGI. Daniel Kokotaljo spoke out despite OpenAI initially conditioning his equity (worth ≈$1.7 million) on non-disparagement agreement compliance.

Anthropic

Anthropic has positioned itself as a safety-focused alternative and received several high-profile hires from OpenAI, including Jan Leike. Specific turnover data not publicly available.

Industry-Wide Context

General corporate AI employee retention challenges in 2025:

• High demand for AI talent creates strong external offers
• Burnout from rapid development pace
• Philosophical disagreements over safety prioritization

Whistleblower Issues

The 2025 AI Safety Index noted: "Public whistleblowing policies are a common best practice in safety-critical industries. Yet, among the assessed companies, only OpenAI has published its full policy, and it did so only after media reports revealed the policy's highly restrictive non-disparagement clauses."

Data Quality: Very limited. Based on public departure announcements, investigative journalism, and open letters. Internal turnover rates for safety-specific teams are not disclosed. No denominator data (total safety team size over time) publicly available.

10. Whistleblower Reports from AI Labs

Current Status (2025): Small but growing number of public whistleblower reports, primarily from OpenAI. Structural barriers remain significant.

"The OpenAI Files" (June 2025)

Comprehensive report compiled by the Midas Project and Tech Oversight Project tracking issues with governance, leadership, and safety culture at OpenAI. Drew on:

• Legal documents and complaints
• Social media posts
• Media reports
• Open letters and insider accounts

Described as "the most comprehensive collection to date of documented concerns with governance practices, leadership integrity, and organizational culture at OpenAI."

Notable Whistleblower Cases

Daniel Kokotajlo (2024): Spoke publicly despite non-disparagement agreement that initially conditioned ≈$1.7 million in equity on his silence. Resigned "in protest after losing confidence in the company."

Jan Leike (2024): While not technically a whistleblower (as he departed to Anthropic), publicly criticized OpenAI on X, stating safety "took a back seat to shiny products" and the team was under-resourced.

Nine-Person Open Letter (June 2024): Current and former OpenAI employees criticized the company for "recklessly racing" toward AGI.

Structural Barriers to Whistleblowing

Non-Disparagement Agreements: OpenAI initially used agreements that conditioned equity vesting on non-criticism. This practice was exposed and modified after public backlash.

Whistleblower Policy Gaps: The AI Safety Index found that only OpenAI published its full whistleblowing policy, and only after media scrutiny revealed restrictive clauses.

Industry Comparison: Most AI labs lack public whistleblower policies comparable to safety-critical industries like aviation or nuclear power.

Cross-Lab Safety Criticism

In 2025, AI safety researchers from OpenAI, Anthropic, and other organizations publicly criticized xAI's "reckless" and "completely irresponsible" safety culture following company scandals.

New Protections

California SB 53 (supported by Anthropic): Provides whistleblower protections for employees reporting AI-related risks or safety concerns to authorities. Effective January 1, 2026.

California AI Safety Act: Establishes protections from retaliation for reporting AI-related risks.

Cross-Lab Evaluation Initiative (2025)

In early summer 2025, Anthropic and OpenAI agreed to evaluate each other's public models using in-house misalignment evaluations and released findings in parallel. While not whistleblowing per se, this represents increased transparency.

Limitations

Actual number of whistleblower reports remains unknown because:

• Many concerns may be raised internally without public disclosure
• Non-disparagement agreements suppress some reports
• Fear of career consequences deters whistleblowing
• No centralized reporting or tracking mechanism exists

Data Quality: Very limited. Based on public letters, media investigations, and individual whistleblower accounts. Represents visible tip of potentially larger iceberg of internal concerns.

Summary of Data Availability

Key Takeaways

1. Compliance varies widely: Even leading labs struggle with certain commitments (especially model weight security and third-party reporting)

2. Evaluation timelines are shortening: Despite increasing capabilities, safety testing windows are compressed, raising concerns about thoroughness

3. Transparency gaps persist: Major model releases sometimes lack promised safety documentation, violating voluntary commitments

4. Release velocity is accelerating: Competitive pressure has created unprecedented model release density, particularly in late 2025

5. Open-source catching up: The capability gap between open and closed models is narrowing from ~16 months to potential parity in some domains

6. Safety team retention challenges: High-profile departures, particularly from OpenAI's Superalignment team, suggest cultural or resource allocation issues

7. Limited whistleblower infrastructure: Despite safety-critical nature of AI development, formal whistleblower protections and reporting mechanisms remain underdeveloped

8. Data quality challenges: Most metrics suffer from limited disclosure, creating information asymmetry between labs and external stakeholders

Sources

• Do AI Companies Make Good on Voluntary Commitments to the White House?↗📄 paper★★★☆☆arXivDo AI Companies Make Good on Voluntary Commitments to the White House?Wang, Jennifer, Huang, Kayla, Klyman, Kevin et al. (2025)Research analyzed 16 AI companies' compliance with White House voluntary AI commitments in 2023, finding wide disparities in performance with an average score of 53% and signifi...capabilitiescybersecuritySource ↗ - August 2025 compliance study
• Anthropic's Responsible Scaling Policy↗🔗 web★★★★☆AnthropicAnthropic's Responsible Scaling PolicyAnthropic introduces a systematic approach to managing AI risks by establishing AI Safety Level (ASL) Standards that dynamically adjust safety measures based on model capabiliti...governancecapabilitiessafetyx-risk+1Source ↗
• Anthropic's Responsible Scaling Policy Update Makes a Step Backwards↗🔗 webAnthropic's Responsible Scaling Policy Update Makes a Step BackwardsAnthropic's recent Responsible Scaling Policy update reduces specificity and concrete metrics for AI safety thresholds. The changes shift from quantitative benchmarks to more qu...governancecapabilitiessafetyevaluationSource ↗ - SaferAI analysis
• Responsible Scaling: Comparing Government Guidance and Company Policy↗🔗 webResponsible Scaling: Comparing Government Guidance and Company PolicyThe report critiques Anthropic's Responsible Scaling Policy and recommends more rigorous risk threshold definitions and external oversight for AI safety levels.governancecapabilitiessafetySource ↗ - Institute for AI Policy and Strategy
• 2025 AI Safety Index↗🔗 web★★★☆☆Future of Life InstituteFLI AI Safety Index Summer 2025The FLI AI Safety Index Summer 2025 assesses leading AI companies' safety efforts, finding widespread inadequacies in risk management and existential safety planning. Anthropic ...safetyx-risktool-useagentic+1Source ↗ - Future of Life Institute
• AI Safety Index Winter 2025↗🔗 web★★★☆☆Future of Life InstituteAI Safety Index Winter 2025The Future of Life Institute assessed eight AI companies on 35 safety indicators, revealing substantial gaps in risk management and existential safety practices. Top performers ...safetyx-riskdeceptionself-awareness+1Source ↗
• AI industry timelines to AGI getting shorter, but safety becoming less of a focus↗🔗 web★★★☆☆FortuneAI industry timelines to AGI getting shorter, but safety becoming less of a focusJeremy KahnLeading AI researchers predict AGI could arrive by 2027-2030, but companies are simultaneously reducing safety testing and evaluations. Competitive pressures are compromising re...safetyevaluationagiSource ↗
• OpenAI: Red Teaming GPT-4o, Operator, o3-mini, and Deep Research↗🔗 webOpenAI: Red Teaming GPT-4o, Operator, o3-mini, and Deep ResearchOpenAI employed external red team testing to systematically evaluate safety vulnerabilities in GPT-4o, Operator, o3-mini, and Deep Research models. The testing targeted alignmen...alignmentsafetyevaluationcybersecurity+1Source ↗ - ControlPlane
• Advancing red teaming with people and AI↗🔗 web★★★★☆OpenAIAdvancing red teaming with people and AIOpenAI explores external and automated red teaming approaches to systematically test AI model safety and potential risks. The research focuses on developing more diverse and eff...safetyeconomiccybersecuritySource ↗ - OpenAI
• AI Red Teaming | Offensive Testing for AI Models↗🔗 webAI Red Teaming | Offensive Testing for AI ModelsHackerOne offers AI red teaming services that use expert researchers to identify security risks, jailbreaks, and misalignments in AI models through targeted testing. The service...alignmentsafetyevaluationcybersecuritySource ↗ - HackerOne
• AI Red Teaming: Applying Software TEVV for AI Evaluations↗🏛️ government★★★★☆CISAAI Red Teaming: Applying Software TEVV for AI EvaluationsI apologize, but the provided text does not appear to be a substantive document about AI red teaming. Instead, it seems to be a collection of blog post titles related to cyberse...safetyevaluationcybersecuritybenchmarks+1Source ↗ - CISA
• AI models can be dangerous before public deployment↗🔗 web★★★★☆METRAI models can be dangerous before public deploymentThe article argues that current AI safety frameworks focused solely on pre-deployment testing are inadequate, as internal AI model usage and development can pose significant ris...safetyevaluationSource ↗ - METR
• Pre-Deployment Evaluation of OpenAI's o1 Model↗🏛️ government★★★★★NISTPre-Deployment Evaluation of OpenAI's o1 ModelJoint evaluation by US and UK AI Safety Institutes tested OpenAI's o1 model across three domains, comparing its performance to reference models and assessing potential capabilit...capabilitiessafetyevaluationx-risk+1Source ↗ - NIST
• Pre-Deployment evaluation of OpenAI's o1 model↗🏛️ government★★★★☆UK AI Safety InstitutePre-Deployment evaluation of OpenAI's o1 modelA comprehensive safety assessment of OpenAI's o1 model by US and UK AI Safety Institutes, testing capabilities across cyber, biological, and software development domains. The ev...capabilitiessafetyevaluationbiosecurity+1Source ↗ - UK AISI
• Google's Gemini 2.5 Pro missing key safety report in violation of promises↗🔗 web★★★☆☆FortuneGoogle's Gemini 2.5 Pro missing key safety report in violation of promisesGoogle launched Gemini 2.5 Pro without publishing a required safety report, contradicting previous commitments made to government and international bodies about model transparen...safetyevaluationllmSource ↗
• British lawmakers accuse Google of 'breach of trust' over delayed Gemini 2.5 Pro safety report↗🔗 web★★★☆☆FortuneBritish lawmakers accuse Google of 'breach of trust' over delayed Gemini 2.5 Pro safety reportA group of 60 U.K. lawmakers criticized Google DeepMind for not fully disclosing safety information about its Gemini 2.5 Pro AI model as previously committed. The letter argues ...safetyevaluationllmSource ↗
• Google is shipping Gemini models faster than its AI safety reports↗🔗 web★★★☆☆TechCrunchGoogle is shipping Gemini models faster than its AI safety reportsGoogle is accelerating its AI model releases, including Gemini 2.5 Pro and 2.0 Flash, but has not published required safety documentation. This raises concerns about transparenc...safetyopen-sourcellmSource ↗
• 2025 Open Models Year in Review↗🔗 web2025 Open Models Year in ReviewFlorian Brand, Substack, SubstackThe 2025 open model landscape saw dramatic capability increases, with models like DeepSeek R1 and Qwen 3 rivaling closed models across key benchmarks. Chinese and global open mo...capabilitiesevaluationopen-sourceSource ↗
• AI Model Release Timeline↗🔗 webAI Model Release TimelineAI Flash ReportA detailed chronological record of AI model releases from various companies, documenting their specifications, performance metrics, and key capabilities. Covers language models,...capabilitiesopen-sourcellmSource ↗ - AI Flash Report
• Timeline of AI model releases in 2024↗🔗 webTimeline of AI model releases in 2024VentureBeat created a detailed tracking of significant AI model releases in 2024, using data from the Artificial Intelligence Timeline project. The timeline covers both API and ...open-sourceSource ↗
• 2025 LLM Year in Review↗🔗 web2025 LLM Year in ReviewA review of 2025's LLM developments highlighting key paradigm shifts including Reinforcement Learning from Verifiable Rewards (RLVR), novel AI interaction models, and emerging A...llmSource ↗ - Andrej Karpathy
• AI Models Comparison 2025: Claude, Grok, GPT & More↗🔗 webAI Models Comparison 2025: Claude, Grok, GPT & MoreThe 2025 AI landscape features six prominent model families with specialized capabilities, including Claude 4's coding prowess, Grok 3's reasoning, and emerging trends in multim...interpretabilitycapabilitiesllmSource ↗
• The Gap Between Open and Closed AI Models Might Be Shrinking↗🔗 web★★★☆☆TIMEThe Gap Between Open and Closed AI Models Might Be ShrinkingEpoch AI research reveals that open AI models are approximately one year behind closed models in capabilities, with the gap potentially shrinking as open models advance.capabilitiesopen-sourceSource ↗ - Time, Epoch AI report
• Open vs. Closed LLMs in 2025: Strategic Tradeoffs for Enterprise AI↗✏️ blog★★☆☆☆MediumOpen vs. Closed LLMs in 2025: Strategic Tradeoffs for Enterprise AIThe landscape of large language models in 2025 is characterized by a nuanced approach to model selection, moving beyond binary open vs. closed debates. Organizations are increas...open-sourcellmSource ↗
• OpenAI's recent departures force leaders to reaffirm safety commitment↗🔗 webOpenAI's recent departures force leaders to reaffirm safety commitmentsafetySource ↗ - Axios
• Top OpenAI researcher resigns, saying company prioritized 'shiny products' over AI safety↗🔗 web★★★☆☆FortuneTop OpenAI researcher resigns, saying company prioritized 'shiny products' over AI safetyJan Leike resigned from OpenAI, citing concerns about the company's commitment to AI safety. His departure follows that of co-lead Ilya Sutskever, highlighting tensions within t...safetySource ↗
• OpenAI dissolves Superalignment AI safety team↗🔗 web★★★☆☆CNBCOpenAI dissolves Superalignment AI safety teamOpenAI has disbanded its Superalignment team, which was dedicated to controlling advanced AI systems. The move follows the departure of key team leaders Ilya Sutskever and Jan L...safetyresearch-agendasalignmentinterpretabilitySource ↗ - CNBC
• The OpenAI Safety Exodus: 25+ Senior Researchers Departed↗🔗 webThe OpenAI Safety Exodus: 25+ Senior Researchers DepartedOver 25 senior OpenAI researchers have departed, including key leadership in AI safety roles. The departures suggest a potential strategic realignment away from careful AI safet...safetySource ↗
• "The OpenAI Files" reveals deep leadership concerns about Sam Altman and safety failures↗🔗 web★★★☆☆Fortune"The OpenAI Files" reveals deep leadership concerns about Sam Altman and safety failuresBeatrice NolanThe 'OpenAI Files' examines internal issues at OpenAI, highlighting leadership challenges and potential risks in AI development. The report critiques Sam Altman's leadership and...safetySource ↗
• The Fight to Protect AI Whistleblowers↗🔗 webThe Fight to Protect AI WhistleblowersThe provided text appears to be a collection of labor law and union-related news articles with no coherent focus on AI whistleblowers.economicSource ↗
• Anthropic: Compliance framework for California SB 53↗🔗 web★★★★☆AnthropicAnthropic: Compliance framework for California SB 53Anthropic outlines its Frontier Compliance Framework (FCF) in response to California's Transparency in Frontier AI Act, detailing approaches to assess and mitigate potential cat...x-riskSource ↗