Integration and Security
Last updated
Last updated
In this section, we look to dive into protection of LLM and its integration processes to prevent malicious usage by attackers. LangChain has explained some basic concepts and usages of its capabilities. It can also be used as potential attack vectors and malicious endpoint.
llm_mathCredits to Nvidia
With LangChain's rising popularity and usage to communicate with models, LangChain can be used as an offensive tool to inject malicious prompts that the model API Endpoint or the model itself may not properly sanitize. Because of rapid developmenet and deployment to public, designs of such tools may be inadequate and lack security enhancements. Recent developments have shown possibilities revolving arbitrary malicious input from attackers to control the output of the LLM.
Exploited Proof of Concepts (PoCs) using llm_math are further discussed in LLM Attacks.
Recommended to use LangChain's latest version to avoid pitfalls of vulnerable APIs that malicious actors can abuse. Reduce the use of plugins from packages, create custom plugins if possible otherwise, treat external plugins with lowest level of privilege.
Data and Control (context passing) planes are inseparable and sometimes passed by user, important to always sanitize inputs on both ends. Parameterized queries can help to maintain a lower privileged context as well.
Paraphrasing can be used as possible mitigation method to prompt injection. This may break the order of the payload within the input which helps reduce malicious input execution.
Data prompt isolation to prevent malicious code within the data input to override the current instruction/context prompt. Solely treating data prompt as data input could reduce Context Ignoring attacks. Delimiters such as ''' or <xml> and other tags have to be carefully treated to prevent execution during data prompting.
Sandwich prevention prefixes an instrcution prompt to ensure the LLM do not modify its original context and prevents attacker's injected instruction to execute. However, it reduces flexibility in allowing normal users from performing custom prompts. Therefore, context should be carefully managed and a limiter could be used to detect if context has been shifted maliciously and only then, revert to its original instructions.
LLM integration into existing products is a trend to boost productivity, how do you secure LLM integration and pipelines?
What defense techniques can be used to secure API functions?
LLM Security - Defense - Fantastic resource links
AWS - Architect Defense for Generative AI - Maybe write in separate topic
Neeraj - LLM Defense Strategies - To incorporate in the future
Microsoft = AI/ML Security - To incorporate in the future
Paper - Prompt Injection and Defenses in LLM-Integrated Applications - Can further explore the paper
