From 79caad5ede317589b81a05a208a26cbd63def8d6 Mon Sep 17 00:00:00 2001 From: rpriven <74690648+rpriven@users.noreply.github.com> Date: Sun, 13 Apr 2025 23:29:33 -0600 Subject: [PATCH] Create README.md --- gandalf-lakera-walkthrough/README.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 gandalf-lakera-walkthrough/README.md diff --git a/gandalf-lakera-walkthrough/README.md b/gandalf-lakera-walkthrough/README.md new file mode 100644 index 0000000..d791f91 --- /dev/null +++ b/gandalf-lakera-walkthrough/README.md @@ -0,0 +1,22 @@ +# Gandalf Walkthrough: Defeating a Prompt-Defended LLM + +## Introduction + +[Gandalf.lakera.ai](https://gandalf.lakera.ai) is an interactive AI security challenge designed to demonstrate the vulnerabilities and defense mechanisms of Large Language Models (LLMs). Created by Lakera, this platform features an AI assistant named "Gandalf" that guards a secret password, challenging users to employ various prompt engineering techniques to extract it. + +The challenge premise is straightforward yet profoundly educational: your goal is to devise creative prompts that can convince, trick, or manipulate Gandalf into revealing its secret password. This simulates real-world scenarios where malicious actors might attempt to circumvent AI safeguards to extract sensitive information or bypass content policies. + +This writeup aims to document effective strategies for defeating Gandalf's defenses, explain the underlying principles that make these approaches work, and highlight the broader implications for AI safety and security. By understanding how these vulnerabilities can be exploited, developers can better design robust defense mechanisms for their own AI systems. + +## Challenge Structure + +The Gandalf challenge consists of seven distinct levels (1-7), each progressively more difficult than the last. Each level introduces new defensive mechanisms that Gandalf employs to protect its password, requiring increasingly sophisticated prompt engineering tactics to overcome. + +Level 1 starts with basic defenses that can be bypassed with simple techniques, while the later levels incorporate advanced protection mechanisms such as input filtering, content monitoring, instruction priority systems, and context-aware defenses. + +The difficulty progression serves as an excellent educational journey through the evolution of prompt injection attacks and defenses in modern LLMs. + +Fundamentally, Gandalf works by having two sets of instructions: the visible instructions that users can see, and hidden system prompts that establish the rules and boundaries Gandalf should follow. These prompt defenses act as guardrails, instructing the model to avoid revealing the password regardless of user inputs. The challenge for participants is to find creative ways to circumvent these defenses by crafting inputs that confuse the model, reframe the context, exploit reasoning flaws, or otherwise create scenarios where the model inadvertently reveals the protected information. + +## Level-by-Level Walkthrough +