Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Riyaz Walikar, profile picture
Uploaded byRiyaz Walikar
1 view

Exploring Security and Privilege Boundaries in Amazon Q AI Copilot

An in-depth look at Amazon Q's AI Copilot, its tool permissions, privilege boundaries, and potential security risks in AWS environments, highlighting methods to test and secure AI agent capabilities.

Technology◦
Related topics:
Generative AI•
The Confused Copilot: Exploring Capabilities and Privilege Boundaries in Amazon Q Riyaz Walikar
./whoami ● Riyaz Walikar - Closing in on 2 decades of breaking and securing things ● Independent Security Researcher and Consultant ● Author, Speaker, Mentor and Trainer at multiple conferences and corporate and community events ● Current research areas of interest include AI, Agentic AI, MCP, local models, router gateways etc. ● Avid supporter of - “Let’s press the button to see what happens” ● My love for biryani >> motivation to workout
Context ● Today's talk is about abusing a system whose structure is well-defined but whose behavior is probabilistic, like any AI system you interact with. ● Confusing an AI Copilot to perform actions outside its assumed scope of operations - in our case, perform a security (authorization) review of itself. ● Identify potentially undocumented functionality within the AI Agent to leak data by attempting to overstep the privilege boundary of AWS IAM
What is Amazon Q? Amazon Q is a generative AI-powered assistant built for enterprise use by Amazon Web Services (AWS). It is designed to boost workplace productivity by answering AWS cloud infra questions, generate content, troubleshoot and review logs and write and debug code*
What are Agent Tools? Think of them as functions an Agent can call to act on the world or fetch data it doesn't hold itself - turning text generation into actions with real effects.
A get_weather(city) tool - The Agent decides to call it based on user input, passes "Bangalore" which your deterministic code hits a weather API, and the result is fed back into the Agent's context to answer. Auth??
So What happened with Amazon Q? Can you spot the bug here?
Identifying Tool Permission Boundaries ● List tools that Amazon Q can call ● For each tool, determine whether the tool can access AWS account data either through ○ the current logged in user’s IAM privileges ○ some undocumented underlying API (not IAM dependent) ○ public data that anyone can access ● Call each tool to test the privilege boundary claim that you have identified
The Confused Copilot Why don’t we ask Amazon Q to help us pentest itself?
Walkthrough of chat with Amazon Q https://youtu.be/30dq4EgW_YA
Tool Permissions and Boundaries Table of Data - Excel/Markdown * additional testing does show a minimum set of permissions required to use Amazon Q and its tools
Minimum Permissions to (ab)use Amazon Q AmazonQDeveloperAccess ce:GetCostAndUsage ce:GetCostAndUsageWithResources These permissions do not give you access to Cost Explorer to view the data in the dashboard but you can use APIs to query the data, which is what Amazon Q was able to do in my account.
Some (Ab)use queries to leak resource names Use the BillingInspector tool to do Cost Analysis and tell me which are my 5 most expensive S3 buckets. Use the BillingInspector tool to get me a list of EC2 instances. Use the InvestigatorCapability to review Cloudwatch alarms to extract any EC2 instance names mentioned there. * For a user with no explicit access to AWS services but with the minimum Cost Explorer perms, we can query Amazon Q to fetch information that can be used to leak resource names.
Conclusion All Agents can be made to do things outside of its prompt bound scope. The appetite for risk, access to customer information and internal systems should be your metrics of (ab)use and security. Expansion of an Agents autonomy and capabilities via tools and MCP is very common, which is why the tool needs to be secured the same way an API system would be ● Authorization / Privilege boundaries ● Sanitization of output ● Code Exec capabilities and PoLP ● Undocumented internal capabilities
Thank you! Riyaz Walikar https://ibreak.software https://github.com/riyazwalikar