More than two years after Google was arrested perfectly for Openai’s ChatGPT release, the company has dramatically increased its pace.
In late March, Google launched the Gemini 2.5 Pro, an AI inference model. It leads the industry in several benchmarks measuring coding and mathematical function. The launch comes just three months after Tech Giant debuted another model, the Gemini 2.0 Flash. This was the cutting edge for the time being.
Tulsee Doshi, director of Google and head of Gemini’s product, told TechCrunch in an interview that the growing key points of the company’s model launches are part of a collaborative effort to keep up with the rapidly evolving AI industry.
“We’re still trying to figure out what the right way to output these models is. What’s the right way to get feedback,” Dosi said.
However, the ramped release time frame appears to be expensive. Google has raised concerns that the company prioritizes speed over transparency as it has not yet released safety reports for the latest models, including the Gemini 2.5 Pro and Gemini 2.0 Flash.
Today, it is fairly standard for Frontier AI Labs, including OpenAI, Humanity and Meta to report safety tests, performance ratings, and use cases every time they launch a new model. These reports, sometimes referred to as system cards or model cards, were proposed by industry and academia researchers a few years ago. Google was in fact one of the first to propose model cards in a 2019 research paper, calling it “an approach to responsible, transparent and accountable practices in machine learning.”
Doshi told TechCrunch that it has not released the model cards for the Gemini 2.5 Pro as the model is considered an “experimental” release. The goal of these experimental releases is to bring out AI models in a limited way, get feedback, and iterate through the models prior to production launches, she said.
Google plans to unveil the Gemini 2.5 Pro model card. Doshi adds that if the model becomes generally available, the company is already doing safety testing and hostile red teams.
In a follow-up message, a Google spokesperson told TechCrunch that safety remains a “first priority” for the company and plans to release more documentation on AI models, including Gemini 2.0 Flash. Generally available Gemini 2.0 Flash also does not have a model card. The last model card released by Google was for the Gemini 1.5 Pro, which came out over a year ago.
System cards and model cards provide information that companies don’t always promote AI widely, and sometimes tedious. For example, the release of System Card Openai for the O1 inference model revealed that its models tend to “plan” towards humans and secretly pursue their own goals.
Overall, the AI community recognizes these reports as good efforts to support independent research and safety assessments, but this report has become even more important in recent years. As Transformer previously pointed out, Google told the US government in 2023 that it would release a safety report for all “critical” public AI models “within scope.” The company has made similar commitments to other governments, pledging to “provide public transparency.”
There have been regulatory efforts to develop safety reporting standards for AI model developers at the federal and state levels in the US. However, they met with limited adoption and success. One more prominent attempt was California’s veto bill, SB 1047. Lawmakers have also issued legislation to approve the US Standards Setting Agency, the US Institute of AI Safety, to establish guidelines for model releases. However, the Safety Institute is currently facing the potential for cuts under the Trump administration.
From all the looks, Google is behind some of its promises to report on model testing, but at the same time it is shipping models faster than ever. Many experts argue that it is a bad precedent, especially as these models become more capable and refined.
Source link
