Login

Recent searches

No recent searches

Search options

Only available when logged in.
social.coop is one of the many independent Mastodon servers you can use to participate in the fediverse.
A Fediverse instance for people interested in cooperative and collective projects. If you are interested in joining our community, please apply at https://join.social.coop/registration-form.html.

Administered by:

Server stats:

488
active users

social.coop: AboutStatusProfiles directoryPrivacy policy

Mastodon: AboutGet the appKeyboard shortcutsView source codev4.3.4

#aisafety

3 posts3 participants0 posts today
Public
Miguel Afonso Caetano

"OpenAI has slashed the time and resources it spends on testing the safety of its powerful artificial intelligence models, raising concerns that its technology is being rushed out without sufficient safeguards.

Staff and third-party groups have recently been given just days to conduct “evaluations”, the term given to tests for assessing models’ risks and performance, on OpenAI’s latest large language models, compared to several months previously.

According to eight people familiar with OpenAI’s testing processes, the start-up’s tests have become less thorough, with insufficient time and resources dedicated to identifying and mitigating risks, as the $300bn start-up comes under pressure to release new models quickly and retain its competitive edge."

ft.com/content/8253b66e-ade7-4

Financial Times · OpenAI slashes AI model safety testing timeBy Cristina Criddle
#AI#GenerativeAI#OpenAI
Public
Calishat

#OpenAI #AI #TrainingAI #AISafety

"OpenAI has slashed the time and resources it spends on testing the safety of its powerful artificial intelligence models... Staff and third-party groups have recently been given just days to conduct 'evaluations', the term given to tests for assessing models’ risks and performance, on OpenAI’s latest large language models, compared to several months previously."

ft.com/content/8253b66e-ade7-4

Public
Dr Pen

Today's latest 'AI scandal', it's churning out porn and child abuse material. Not surprising and very likely happening across numerous models independent of any Bigtech safeguarding. Genai cannot be legislated, it's much more difficult than legislating the Internet and that's fairly impossible already. Do we teach this adequately? Or even at all?

#ai #genai #aisafety #ailegislation #academia

AI Image Generator’s Exposed Database Reveals What People Used It For
wired.com/story/genomis-ai-ima

WIRED · An AI Image Generator’s Exposed Database Reveals What People Really Used It ForBy Matt Burgess
Public *
MSvana

ai-2027.com - excellent blend of reality and fiction. The original intention may have been forecasting, but I read it more as a cautionary tale giving issues related to AI a more concrete form. This includes:

- Technical work on AI alignment
- Job loss
- Contentration of power and the question of who controls powerful AI systems
- Geopolitical tensions
- The consequences of Europe lagging behind

AI 2027
ai-2027.comAI 2027A research-backed AI scenario forecast.
#ai#aisafety#technology
Public
Miguel Afonso Caetano

"Backed by nine governments – including Finland, France, Germany, Chile, India, Kenya, Morocco, Nigeria, Slovenia and Switzerland – as well as an assortment of philanthropic bodies and private companies (including Google and Salesforce, which are listed as “core partners”), Current AI aims to “reshape” the AI landscape by expanding access to high-quality datasets; investing in open source tooling and infrastructure to improve transparency around AI; and measuring its social and environmental impact.

European governments and private companies also partnered to commit around €200bn to AI-related investments, which is currently the largest public-private investment in the world. In the run up to the summit, Macron announced the country would attract €109bn worth of private investment in datacentres and AI projects “in the coming years”.

The summit ended with 61 countries – including France, China, India, Japan, Australia and Canada – signing a Statement on Inclusive and Sustainable Artificial Intelligence for People and the Planet at the AI Action Summit in Paris, which affirmed a number of shared priorities.

This includes promoting AI accessibility to reduce digital divides between rich and developing countries; “ensuring AI is open, inclusive, transparent, ethical, safe, secure and trustworthy, taking into account international frameworks for all”; avoiding market concentrations around the technology; reinforcing international cooperation; making AI sustainable; and encouraging deployments that “positively” shape labour markets.

However, the UK and US governments refused to sign the joint declaration."

computerweekly.com/news/366620

ComputerWeekly.com · AI Action Summit review: Differing views cast doubt on AI’s ability to benefit whole of societyBy Sebastian Klovig Skelton
#AI#AIActionSummit#AISafety
Public
chribonn

We tested different AI models to identify the largest of three numbers with the fractional parts .11, .9, and .099999. You'll be surprised that some AI mistakenly identifying the number ending in .11 as the largest. We also test AI engines on the pronunciation of decimal numbers. #AI #ArtificialIntelligence #MachineLearning #DecimalComparison #MathError #AISafety #DataScience #Engineering #Science #Education #TTMO

youtu.be/TB_4FrWSBwU

youtu.be- YouTubeEnjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.
Public
Miguel Afonso Caetano

After all these recent episodes, I don't know how anyone can have the nerve to say out loud that the Trump administration and the Republican Party value freedom of expression and oppose any form of censorship. Bunch of hypocrites! United States of America: The New Land of SELF-CENSORSHIP.

"The National Institute of Standards and Technology (NIST) has issued new instructions to scientists that partner with the US Artificial Intelligence Safety Institute (AISI) that eliminate mention of “AI safety,” “responsible AI,” and “AI fairness” in the skills it expects of members and introduces a request to prioritize “reducing ideological bias, to enable human flourishing and economic competitiveness.”

The information comes as part of an updated cooperative research and development agreement for AI Safety Institute consortium members, sent in early March. Previously, that agreement encouraged researchers to contribute technical work that could help identify and fix discriminatory model behavior related to gender, race, age, or wealth inequality. Such biases are hugely important because they can directly affect end users and disproportionately harm minorities and economically disadvantaged groups.

The new agreement removes mention of developing tools “for authenticating content and tracking its provenance” as well as “labeling synthetic content,” signaling less interest in tracking misinformation and deep fakes. It also adds emphasis on putting America first, asking one working group to develop testing tools “to expand America’s global AI position.”"

wired.com/story/ai-safety-inst

WIRED · Under Trump, AI Scientists Are Told to Remove ‘Ideological Bias’ From Powerful ModelsBy Will Knight
#USA#Trump#ResponsibleAI
Public
Tiago F. R. Ribeiro

Superintelligent Agents Pose Catastrophic Risks (Bengio et al., 2025)

📎arxiv.org/pdf/2502.15657

Summary: “Leading AI firms are developing generalist agents that autonomously plan and act. These systems carry significant safety risks, such as misuse and loss of control. To address this, we propose Scientist AI—a non-agentic, explanation-based system that uses uncertainty to safeguard against overconfident, uncontrolled behavior while accelerating scientific progress.” #AISafety #AI #Governance

Public
Miguel Afonso Caetano

"A high volume of recent ML security literature focuses on attacks against aligned large language models (LLMs). These attacks may extract private information or coerce the model into producing harmful outputs. In real-world deployments, LLMs are often part of a larger agentic pipeline including memory systems, retrieval, web access, and API calling. Such additional components introduce vulnerabilities that make these LLM-powered agents much easier to attack than isolated LLMs, yet relatively little work focuses on the security of LLM agents. In this paper, we analyze security and privacy vulnerabilities that are unique to LLM agents. We first provide a taxonomy of attacks categorized by threat actors, objectives, entry points, attacker observability, attack strategies, and inherent vulnerabilities of agent pipelines. We then conduct a series of illustrative attacks on popular open-source and commercial agents, demonstrating the immediate practical implications of their vulnerabilities. Notably, our attacks are trivial to implement and require no understanding of machine learning."

arxiv.org/html/2502.08586v1

arxiv.orgCommercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
#AI#GenerativeAI#LLMs
Public
Kevin Thomas ✅

I know many engineers worry about LLMs replacing cybersecurity and reverse engineering roles, but history proves otherwise. Every major tech shift creates new vulnerabilities and demand for skilled engineers.

Reverse Engineering & AI Security
• AI-Powered Malware: LLMs generate polymorphic malware that evades detection. Engineers must use runtime analysis to dissect threats.
• Model Extraction: Proprietary AI models will be encrypted and obfuscated—reverse engineers must verify integrity using side-channel attacks and binary analysis.
• Embedded AI Risks: AI in IoT, drones, and industrial systems introduces security flaws that require firmware audits and adversarial testing.

AI Safety & Adversarial Defense
• Adversarial Attacks: Hackers use gradient-based perturbations to mislead AI. Engineers must build adversarial training to prevent manipulation.
• AI Supply Chain Security: Poisoned datasets introduce neural backdoors. Engineers need dataset audits and integrity verification to mitigate risk.
• Explainability & Model Hardening: AI must be transparent and resilient. Engineers must develop XAI (Explainable AI) tools for security validation.

AI Security & Reverse Engineering Are the Future

AI isn’t replacing cybersecurity—it’s making it more critical than ever. Engineers skilled in AI security, adversarial testing, and model extraction will be in high demand.

Now is the time to adapt. Master AI security and stay ahead.

#ReverseEngineering#AI#AISafety
Public
Miguel Afonso Caetano

"Vance came out swinging today, implying — exactly as the big companies might have hoped he might – that any regulation around AI was “excessive regulation” that would throttle innovation.

In reality, the phrase “excessive regulation” is sophistry. Of course in any domain there can be “excessive regulation”, by definition. What Vance doesn’t have is any evidence whatsoever that the US has excessive regulation around AI; arguably, in fact, it has almost none at all. His warning about a bogeyman is a tip-off, however, for how all this is going to go. The new administration will do everything in its power to protect businesses, and nothing to protect individuals.

As if all this wasn’t clear enough, the administration apparently told the AI Summit that they would not sign anything that mentioned environmental costs or “existential risks” of AI that could potentially going rogue.

If AI has significant negative externalities upon the world, we the citizens are screwed."

garymarcus.substack.com/p/ever

Marcus on AI · Everything I warned about in Taming Silicon Valley is rapidly becoming our realityBy Gary Marcus
#AI#GenerativeAI#AISafety
ExploreLive feeds
About