Cybersecurity of Gen AI/LLM

Refers to a focus on the security features or attributes inherent to Generative AI and Large Language Models themselves.

It's more about the security aspects that are part of these systems.

Cybersecurity in Gen AI/LLM

Refers to discussing the broader context of security within the field or domain of Generative AI and Large Language Models.

This can include external measures and strategies implemented to protect these systems, not just the security features that are intrinsic to them.

Challenges

Tool, Library & Framework Vulnerabilities Innovation Outpacing Security Lag in Cybersecurity Responses Programming Language Specific Risks

Vulnerabilities

• Widespread tools, libraries & frameworks with security flaws being exploited.
  • Langchain, Pandas, etc
• Established systems widely used but often less updated, posing silent risks.
  • Established systems provide a solid, unchanging foundation, yet the solutions built upon them rely on their stability while continuing to innovate.

Innovation Outpacing Security

• Rapid AI, digital transformation and tech advancement lead to emerging threats.
• New AI tools developed faster than security measures can adapt.

Lag in Cybersecurity Responses

• Security protocols and defenses lag in response to real-time threats.
• Reactive rather than proactive approach in addressing AI vulnerabilities.

Programming Language Specific Risks

• In Python, utilization of dynamic features like 'exec/eval' functions allows arbitrary code execution.
• 'ctypes' library exposes system internals, opening the door for low-level system attacks.

Examples

• Langchain Tool Exploit - Arbitrary Code Execution
• PandasAI Jailbreak - Arbitrary Code Execution
• llama-cpp-python Exploit - Arbitrary Code Execution
• Intelligent “Agent Smith” - Jailbreak

Langchain Tool Exploit

Langchain is an open source framework that allows software developers to work with AI and its ML subset, combining LLMs with other external components to develop LLM-powered applications.

Langchain's Wikipedia tool has a vulnerability that allows dynamic loading of arbitrary code (payload that looks like ordinary string) during online search or definition lookup requests.

PandasAI Jailbreak

PandasAI is a Python library that adds Gen AI capabilities to Pandas, the popular data analysis and manipulation tool.

The exploit uses vulnerability within the whitelisted SciPy library, using the "ctypes" module to import "subprocess" and execute shell commands, thereby running arbitrary code.

llama-cpp-python Exploit

This exploit uses weakness in llama-cpp-python library, making use of a custom suffix parameter in Llama._create_completion method to execute arbitrary code.

By manipulating the custom suffix parameter, the exploit successfully utilized Python's subprocess module, thereby demonstrating a proof of concept for this vulnerability.

Intelligent “Agent Smith”

An intelligent agent with the remarkable capability to breach sandboxes, detach from its original host, and transfer to alternative hosts.

While this Proof of Concept hints at the possibility for increasing LLM agent capabilities, it delves into Agent Smith's capacity for continuous code and prompt refinement, facilitating its ongoing self-enhancement.

Present

Persistent security gaps in LLMs applications reveal critical vulnerabilities in data processing libraries and the emergence of intelligent agents capable of bypassing digital safeguards.

These issues underscore the urgent necessity for stronger cybersecurity measures, continuous threat monitoring, and prompt response strategies.

Prevention

• Strengthening sandbox security.
• Implementing robust monitoring and anomaly detection systems.
• Ensuring data encryption and privacy measures.
• Conducting regular audits and testing.
• Educating users.

Future

A constant battle with emerging threats, where rapid innovation magnifies system vulnerabilities.

Enhanced security measures, including fortified sandboxes and advanced anomaly detection.

The focus will intensify on preemptive security strategies and robust frameworks to address the relentless evolution of risks.