Quick take - A new AI reasoning model developed by a Chinese lab has garnered attention for its advanced capabilities, but it has also raised concerns about security vulnerabilities, particularly related to prompt injection and Cross-Site Scripting (XSS), prompting tests that revealed potential exploits and leading to a swift response from the application’s support team to address these issues.

Fast Facts

A new AI reasoning model from a Chinese lab has advanced capabilities but raises security concerns, particularly regarding prompt injection and Cross-Site Scripting (XSS).
XSS vulnerabilities can allow attackers to execute malicious JavaScript, potentially compromising user sessions and sensitive information.
Tests on the AI model revealed risks associated with document uploads, which could facilitate prompt injection attacks.
A specific exploit demonstrated how attackers could access session tokens stored in local storage or cookies, using both prompt injection and XSS techniques.
The vulnerability was reported and quickly addressed by the DeepSeek team, highlighting the importance of securing AI applications against such threats.

New AI Reasoning Model Raises Security Concerns

A new AI reasoning model has been released by a Chinese lab, drawing significant attention from the AI community. The model is noted for its advanced reasoning capabilities. However, concerns have arisen regarding potential security vulnerabilities in AI-powered web applications.

Security Vulnerabilities in AI Applications

These concerns focus on issues such as prompt injection and Cross-Site Scripting (XSS). XSS is a well-known vulnerability that allows attackers to inject malicious JavaScript into a webpage, leading to unauthorized code execution within a user’s browser. The consequences of such an attack can be severe, as attackers can gain access to sensitive information, including cookies and local storage, resulting in complete user compromise and account takeover.

In an effort to explore the security vulnerabilities of the new AI model, tests were conducted specifically targeting prompt injection angles. One notable feature of the AI model is its ability to allow users to upload documents for analysis, which could potentially create opportunities for prompt injection attacks.

Investigating Session Handling and Exploits

The research included an investigation into session handling in web applications, highlighting that session tokens are commonly stored in local storage or cookies. The author revealed that it is possible to access a user’s session by exploiting these stored session tokens. A specific prompt injection exploit was discussed, illustrating how an adversary could gain access to a compromised user’s session through both prompt injection and XSS tactics.

To exemplify the exploit, a basic JavaScript code snippet was provided, designed for loading and dumping session tokens. The author noted that further steps would be necessary to transmit the token to a third-party server. To enhance the effectiveness of the prompt injection, the final payload was base64 encoded, a technique employed to evade detection by web application firewalls (WAFs) and increase the likelihood of acceptance by the AI model.

Response and Mitigation

In response to these findings, the author reported the identified vulnerability to the application’s support team, and a commendable fix was implemented within a day. The article underscores the critical need to address vulnerabilities like prompt injection and XSS in AI applications to ensure user security. The author acknowledged the rapid response from the DeepSeek team in mitigating the reported vulnerability.

References to additional resources were provided, including the DeepSeek homepage and a training video on web application security fundamentals.

Original Source: Read the Full Article Here