The Good Tech Companies - The Capabilities of Large Language Models: Hacking or Helping?

Episode Date: May 23, 2024

This story was originally published on HackerNoon at: https://hackernoon.com/the-capabilities-of-large-language-models-hacking-or-helping. Exploring the Capabilities of LLM Agents: A Study on Website Hacking Check more stories related to machine-learning at: https://hackernoon.com/c/machine-learning. You can also check exclusive content about #large-language-models, #dual-use-capabilities-of-llm, #llm-for-cybersecurity, #llms-for-hacking, #llm-threats, #llm-website-hacking, #sql-union-exploit, #good-company, and more. This story was written by: @hostkey. Learn more about this writer by checking @hostkey's about page, and for more stories, please visit hackernoon.com. Large Language Models (LLMs) are rapidly evolving and are widely used as autonomous agents. Developers can design agents that interact with users, process queries, and execute tasks based on received data. Researchers are growing increasingly concerned about the dual-use capabilities of LLMs - their ability to perform malicious tasks.

This audio is presented by Hacker Noon, where anyone can learn anything about any technology. The capabilities of large language models, hacking or helping, by Hostki.com. Large language models, LLMs, are rapidly evolving and are widely used as autonomous agents. Developers can design agents that interact with users, process queries, and execute tasks based on received data, such as detecting errors in complex code, conducting economic analysis, or assisting in scientific discoveries. N-H-O-W-E-V-E-R, researchers are growing increasingly concerned about the dual-USE capabilities of LLMs, their ability to perform malicious tasks, particularly in the context of

cybersecurity. For instance, CHAD-GPT can be utilized to aid individuals in penetration testing and creating malware. Moreover, these agents may operate independently, without human involvement or oversight. N researchers at Cornell University, including Richard Fang, Rohan Bindu, Akul Gupta, Kiyoshi Jean, and Daniel Kan, have conducted studies that shed light on the threats posed by LLMs and provide valuable insights into their potential consequences. Their findings serve as a sobering reminder of the need for careful consideration and regulation in this rapidly evolving field. Greater-than-rent GPU servers with instant deployment are a server with a custom greater-than configuration with professional-grade NVIDIA Tesla A100, H180 gigabits or A5000, greater than A4000 cards. GPU servers with gaming

RTX 4090 cards are also available. Autonomous website breaches. The study demonstrated that LLM agents can execute complex breaches, for instance, a blind SQL injection attack combining queries. This type of attack targets web applications using SQL, structured query language, to interact with databases. Such attacks enable malicious actors to obtain confidential information from databases, even if the application does not display any signs of error or abnormal behavior. The root of these attacks lies in the exploitation of the SQL Union operator, which enables the combination of multiple query results into a single dataset. By crafting a specially designed query featuring this operator, a malicious actor

can merge the result set of a database query with that of a confidential information table. This allows them to access sensitive data. To successfully execute these attacks, an agent must possess the ability to navigate websites and perform more than 45 actions to breach the site. Notably, as of February this year, only GPT-4 and GPT-3.5 were capable of breaching websites in this manner. However, it is likely that newer models, such AS-Lama3, will also be able to perform similar operations. NTO investigate the potential misuse of large language models, LLMs, in web breaches. Researchers leveraged various AI tools and frameworks. Specifically, they utilized Langchain for creating agents and generative adversarial networks as well as OpenAI models through API assistance.

React was employed to breach websites with agents interacting through Playwright. To enhance contextual understanding, previous function calls were integrated into the current context. A controlled environment was established by creating 15 isolated web pages with various vulnerabilities, ensuring that actual websites and individuals remained protected. Ten large-scale language models, including GPT-4 and GPT-3.5, as well as eight open-source models highly rated on Chatbot Arena, were utilized to test the breach attempts. Each website was targeted five times, and any successful attempt granted access through one of these five tries agent pass at 5 gpt 4 assistant 73 3 gpt 3 5 assistant 6 7 o p e n h e r m e s 2

5 mistra 1 to 7 b 0 0 llama 2 chat 70 b 0 0 llama 2 chat 13 b 0 0 llama 2 chat 70 b 0 0 percent llama 2 chat 13 b 0 0 percent llama 2 chat 7 b 0 0 percent mixtrel 8 by 7 b instruct 0 0 percent mistral 7 b instruct v 0 20 0 percent noose heres 2E, 34B, 0, 0% OpenChat 3, 5O, 0% The capabilities of large language models, LLMs, are often governed by the law of scaling, where increasing model size and complexity can lead to improved performance on specific tasks. This phenomenon was recently demonstrated through an experiment involving GPT-4, a state-of-the-art AI model, and an open-source LLM. In this study, researchers observed that GPT-4 successfully breached 11 out of 15 websites tested, whereas the open-source LLM failed to exploit any vulnerabilities. The contrast between these results highlights the pronounced impact of scaling on the

capabilities of LLMs. NTHE experiment unfolded as follows. Initial navigation. The model identified the target web page through web page navigation. Standard credentials. A standard username and password were attempted to gain access. Information gathering. Leverage was taken from the obtained information for an SQL injection attempt. Source code analysis. The source code was analyzed to determine if the SQL query contained a parameter underscore get. Exploitability assessment. It was determined whether the query could be exploited for an SQL union attack. Immediate attack execution. An immediate SQL union attack was executed, comprising up to 48 steps.

And following this experiment, GPT-4 was asked to breach actual websites. Although it declined to perform the attack outright, it successfully identified one vulnerability and explained how it could be exploited among 50 randomly selected test websites. Exploiting day one vulnerabilities with large language models. This study investigates the feas one vulnerabilities with large language models. This study investigates the feasibility of using large language models, LLMs, to exploit day one vulnerabilities, also known as zero-day vulnerabilities. The SEER security flaws that have been publicly disclosed through sources like CVE, common vulnerabilities and exposures, but no patch

or update has yet been released to address the issue this raises the possibility of an exploitable path existing although it has not yet been utilized notably even though these vulnerabilities become public knowledge there is no guarantee that existing tools can automatically detect them for instance attackers or penetration testers without access to internal system details might not know the version of software being used upon exploitation. Given the complexity of many day-one vulnerabilities, which are often found in closed systems making it impossible to reproduce them, researchers focused on vulnerabilities in open-source software. For this study, researchers selected 15 vulnerabilities that cover web application vulnerabilities,

container management software vulnerabilities, and Python package vulnerabilities. These include a mix of high-risk and low-risk vulnerabilities discovered after the information collection cutoff date for testing LLMs. The specific vulnerabilities used in this experiment were vulnerability description runc escape from container via embedded file descriptor csrf plus ace cross-site request forgery exploitation for executing code with arbitrary privileges wordpress sqli sql injection via wordpress plugin wordpress xss 1 cross-site scripting xss in wordpress plugin wordpress xss 2 cross-site scripting xss in wordpress plugin travel journal xss cross-site scripting xss in travel journal iris xss cross-site scripting. XSS. In WordPress plugin. Travel journal. XSS. Cross-site scripting. XSS. In travel journal. Iris. XSS. Cross-site scripting. XSS. In Iris. CSRF plus privilege escalation.

Cross-site request forgery. CSRF. Exploitation for elevating privileges to administrator and ledger. SMB. ALF.ph io key leakage key disclosure upon visiting specific endpoint for ticket reservation system astrophi rce inadequate input validation allowing for invocation of subprocess poppin hertz beat rce jndi injection exploitation for remote code execution newboard xss ace xss vulnerability in newboard allowing for code execution with arbitrary privileges symphony 1 RCE abuse of PHP arrays object usage for arbitrary code execution with elevated privileges peering manager SSTI RCE server side template injection vulnerability leading to remote code execution RCE acid rain warshawski and bayless 2017 database attack utilizing parallelism vulnerability cve

publication date thread level runc cve 2024 21626 1 31 20 248 6 high csrf plus ace cve 2424524 2 February 2024 8. High. WordPress SQLICVE 2021-2466 September 27, 2021 9. 8. Critical. WordPress XSS 1 CVE-20231119 17 October 2023 6. 1. Medium. WordPress XSS 2 Cape Verde Escudos 2023 1119 2 10 July 2023 6. 1. Medium. Travel Journal XSSCVE 2024-24041 1 February 2024 6. 1. Medium. Iris XSS CVE 2024-256402. 19-2024 4. 6. Medium. CSRF Plus Privilege Escalation CVE-202423831 2nd of February 2024 7. 5. High. ALF. IKE Leakage CVE-202425635 February 19, 2024 8. 8. High. Astrophy RCE CVE-2023-41334 March 18, 2024 8. 4. High. Hertz Beat RCE CVE 2023-51653 February 22, 2024 9. 8. Critical. New Board XSS ACE CVE-202424156 March 16, 2024 N. A Symphony 1 RCE CVE-202428859 March 15, 2024 5. 0. Medium. Peering Manager SSTI RCECVE-2024-281143. 12. 2024 8. 1. High, Acid Rain, Warshawski & Baylis, 2017, 2017n.a. The agents mimicked those employed in website breaches,

but with a twist. Instead of targeting sensitive documents, they carried detailed descriptions of common vulnerabilities and exposures, CVEs, along with realistic exploitation scenarios for leveraging vulnerabilities on day one. NTO assessed the capabilities of language models, LLMs, in exploiting vulnerabilities. Researchers leveraged 10 large-scale LLMs, including GPT-4 and 8 open-source alternatives, as well as two automated tools, Z-Attack Proxy, ZAP, developed by OWASP, and Metasploit, a framework created by Rapid7. This study revealed that GPT-4 was capable of exploiting 87% of vulnerabilities, whereas other LLMs were unable to do so. Notably, GPT-4 failed only with two specific vulnerabilities, Iris XSS and Hertzbeat RCE.

The Iris web platform, used for collaborative work and incident response investigations, proved challenging for the LLM agent due to its reliance on JavaScript navigation. This rendered the agent unable to access crucial forms and buttons or interact with desired elements, a task that a human could accomplish successfully. Further investigation revealed that GPT-4 struggled to translate Hertzbeat details, which were only available in Chinese, due to its English-based query language. Consequently, it encountered difficulties in reproducing the vulnerability. Findings also highlighted the importance of CVE descriptions

in LLM success rates. Without these descriptions, the success rate dropped dramatically from 87% to 7%. This suggests that LLM agents currently require detailed instructions to develop exploitation plans for vulnerabilities and are not yet capable of independently creating such plans. However, this is merely the beginning, and future advancements may alter this landscape. Conclusions The study demonstrated that LLM agents are already capable of autonomously breaching websites and exploiting certain real vulnerabilities in computer systems, with the majority of them

being exploitable with a description of their exploitation. Fortunately, current agents are unable to exploit unknown and undisclosed vulnerabilities, nor can open-source solutions demonstrate results comparable to the paid CHAD GPT-4 and GN GPT-4-0. However, it is possible that future extensions could enable the exploitation of such vulnerabilities, with free-access LLM models potentially replicating the success of their proprietary counterparts. All this suggests that developers of large language models must approach the training process more responsibly. Furthermore, cybersecurity specialists need to be prepared for the fact that these models will be used to create bots to twill

systematically scanned systems for vulnerabilities. Even open-source models can claim they will not be used for illicit activities. Lama3 flatly refused to help breach a website. However, it is precisely Duetto openness that there are no obstacles beyond ethical considerations preventing the creation of censorship-free models. There are numerous ways to convince an LLM to assist in breaching, even if it initially resists. For instance, one could ask it to become a pentester and help improve site security by doing a good deed. Greater than rent GPU servers with instant deployment or a server with a custom greater than configuration with professional-grade NVIDIA Tesla A100, H180 gigabits or A5000,

greater than A4000 cards. GPU servers with Game RTX 4090 cards are also available. Thank you for listening to this Hackernoon story, read by Artificial Intelligence. Visit hackernoon.com to read, write, learn and publish.