Responsible/Trustworthy AI in the Era of Foundation Models
•Download as PPTX, PDF•
0 likes•273 views
The document discusses responsible AI and challenges in developing AI systems. It summarizes CSIRO's work on closing gaps between principles and engineering practices, understanding increasingly complex AI systems, and designing systems that incorporate foundation models. Key points include the need to measure system-level impacts, develop engineering methods for explainability and oversight, and design tools that can evolve with AI capabilities rather than target specific functions.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Responsible/Trustworthy AI in the Era of Foundation Models
Similar to Responsible/Trustworthy AI in the Era of Foundation Models (20)
More from Liming Zhu
More from Liming Zhu (16)
Recently uploaded
Recently uploaded (20)
Responsible/Trustworthy AI in the Era of Foundation Models
- 1. Australia’s National Science Agency Liming Zhu Research Director, CSIRO’s Data61 Conjoint Professor, UNSW Responsible/Trustworthy AI in the Era of Foundation Models All pencil drawings in this presentation are created by AI
- 2. What’s Responsible AI? 2 | Responsible AI is the practice of developing and using AI systems in a way that provides benefits to individuals, groups, and wider society, while minimizing the risk of negative consequences. Not model/algorithm System requirements/quality linked to benefit/risk impact
- 3. What about the System/SE Level? 3 | 2014-2015 2020-2022 ICSE23 TechDebt Keynote - Technical Debt in AI-based Software Systems: Challenges and Approaches. CSIRO’s Data61, Sherry Xu ICSE23 DeepTest Keynote - Testing Generative Large Language Model: Mission Impossible or Where Lies the Path? CSIRO’s Data61, Zhenchang Xing Trust Debt Architecture Debt Explainability Debt Prompt Controllability/Testability Modular/Testable AI Chains Beyond Accuracy
- 4. Build/Evaluate -> Discover/Oversee 4 | intentions -> agents -> oversee • data foraging/synthesis • emerging capabilities • scalable (AI) oversights https://medium.com/@itamar_f/software-3-0-the-era-of-intelligent-software- development-acd3cafe6cd7 https://karpathy.medium.com/software-2-0-a64152b37c35 requirements -> build -> evaluate examples -> discover -> assess risk Future directions • (Learned) Guardrails • Radical observability • Understand rather than build at the system-level
- 5. Australia’s National Science Agency Challenges & Trends
- 6. Australia’s AI ethics framework OECD AI principles Principles Standards Frameworks NIST AI RMF ISO Standards Algorithms Models SE for RAI …… … 1. The Vertical Gap – Alignment & Practices Model Alignment != System Alignment Principles/Standards != Eng. Practices Lu, Q., Luo, Y., Zhu, L., Tang, M., Xu, X., Whittle, J., 2023. Operationalising Responsible AI Using a Pattern-Oriented Approach: A Case Study on Chatbots in Financial Services. IEEE Intelligent Systems. 6 |
- 7. 2. The Understanding Gap - Inscrutable Do we have to fully understand the AI model? Can system-level understanding help? 7 |
- 8. One More Thing – Here Come the LLMs 8 | Lu, Q., Zhu, L., Xu, X., Xing, Z., Whittle, J., 2023. Towards Responsible AI in the Era of ChatGPT: A Reference Architecture for Designing Foundation Model-based AI Systems. https://arxiv.org/abs/2304.11090
- 9. Australia’s National Science Agency Directions & Questions
- 10. 1. Close the Gaps – engineering practices 10 | Lu, Q., Zhu, L., Xu, X., Whittle, J., Xing, Z., 2022. Towards a Roadmap on Software Engineering for Responsible AI, in: 1st International Conference on AI Engineering (CAIN) Measurements/Metrics, Evaluation/Verification/Validation Methods
- 11. Close the Gaps – operationalisable 11 | Xia, B., Lu, Q., Perera, H., Zhu, L., Xing, Z., Liu, Y., Whittle, J., 2023. Towards Concrete and Connected AI Risk Assessment (C2AIRA). 2nd International Conference on AI Engineering (CAIN) Dozens of Frameworks Which methods & tools for which stakeholders?
- 12. Close the Gaps – Connected Patterns 12 | Lu, Q., Zhu, L., Xu, X., Whittle, J., 2023. Responsible-AI-by-Design: A Pattern Collection for Designing Responsible AI Systems. IEEE Software https://research.csiro.au/ss/science/projects/responsible-ai-pattern-catalogue/ Lee, S.U., Perera, H., Xia, B., Liu, Y., Lu, Q., Zhu, L., Salvado, O., Whittle, J., 2023. QB4AIRA: A Question Bank for AI Risk Assessment. https://doi.org/10.48550/arXiv.2305.09300
- 13. 2. Understand at the System Level Increasingly, the study of these trained (but un-designed) systems seems destined to become a kind of natural science… … they are similar to the grand goals of biology, which is to "figure out" while being content to get by without proofs or guarantees … “AI as (an Ersatz) Natural Science?” by Subbarao Kambhampati 13 |
- 14. Understanding via “Testing” Zhuo, T.Y., Huang, Y., Chen, C., Xing, Z., 2023. Exploring AI Ethics of ChatGPT: A Diagnostic Analysis https://arxiv.org/abs/2301.12867 14 | ICSE23 DeepTest Keynote - Testing Generative Large Language Model: Mission Impossible or Where Lies the Path? Zhenchang Xing, CSIRO’s Data61 Capability +/-/⊥ Alignment Waluigi Effect prevents model-level solution
- 15. Understanding via Accountability 15 | No Agreed Best Practices No Agreed Safety Test Verifiable investment in safety Accountability enforced by law/market
- 16. Understanding via Accountability 16 | Xu, X., Wang, C., Wang, Jeff, Lu, Q., Zhu, L., 2022. Dependency tracking for risk mitigation in machine learning systems, in: 44th ICSE Xia, B., Bi, T., Xing, Z., Lu, Q., Zhu, L., 2023. An Empirical Study on Software Bill of Materials: Where We Stand and the Road Ahead, in: 45th ICSE Software Bills of Materials (SBOM)/AIBOM
- 17. 3. Design Foundation Model-based Systems Lu, Q., Zhu, L., Xu, X., Xing, Z., Whittle, J., 2023. A Framework for Designing Foundation Model based Systems https://arxiv.org/abs/2305.05352v1 LLM eating the traditional system functions Moving boundaries ex emerging capabilities • Design with capabilities, not functionalities • Design for capability evolution and agility Tools being optimized for LLM/Agents • Selected/Used by both human and LLM/Agents • Trusted by human and LLM/Agents
- 18. Responsible AI for LLM-based Applications 18 | Lu, Q., Zhu, L., Xu, X., Xing, Z., Whittle, J., 2023. Towards Responsible AI in the Era of ChatGPT: A Reference Architecture for Designing Foundation Model-based AI Systems. http://arxiv.org/abs/2304.11090
- 19. RAI in the Era of Foundation Models AI Engineering Directions • (Learned) Guardrails • Radical observability • Understand rather than build Responsible AI Engineering • Close the principle-alg. gaps • Engineering practices/methods • Measurement/metrics • Connected patterns • Understand at the system level • AIBOM & accountability More info & Contact https://research.csiro.au/ss/ Liming.Zhu@data61.csiro.au Brendan.Omalley@data61.csiro.au Coming out late 2023 Foundation Models • Design with capabilities, not func. • Design for system evolution • Tools optimised for LLM/Agents • Special RAI patterns Collaborate with CSIRO’s Data61 on • RAI Engineering best practices & evaluation • LLM/Foundation model-based system design/eval For the latest, follow me on Twitter: @limingz LinkedIn: Liming Zhu
Editor's Notes
- Not AI algorithms and models Functional and non-functional requirements AI alignment + existential risks; AI safety; ethical/law risks
- Entanglements, Cascades, Dependency, Unstable Data Dependencies, Hidden Feedback Loops Debt: Abstraction, Reproducibility ”Federated data collection, storage, model, and infrastructure” Interaction with other teams “co-design and co-versioning”…
- Mechics/physics Bridges and buildings Fully understand the human brain to trust No Empirical software engineering and testing. Level of understanding ; I am not talking about you fully Why? My wife expecting, apology
- Governance to connect with management Process to connect with other practices
- The "science" suffix of computer science has sometimes been questioned and caricatured; perhaps not any longer, as AI becomes an ersatz natural science studying large learned artifacts. Likewise, LLMs are produced by a relatively simple training process (minimizing loss on next-token prediction, using a large training set from the internet, Github, Wikipedia etc.) but the resulting 175 billion parameter model is extremely inscrutable. This is the why the field of “AI interpretability” exists at all: to probe large models such as LLMs, and understand how they are producing the incredible results they are producing. Increasingly, the study of these large trained (but un-designed) systems seems destined to become a kind of natural science, even if an ersatz one: observing the capabilities they seem to have, doing a few ablation studies here and there, and trying to develop at least a qualitative understanding of the best practices for getting good performance out of them. Modulo the fact that these are going to be studies of in vitro rather than in vivo artifacts, they are similar to the grand goals of biology, which is to "figure out" while being content to get by without proofs or guarantees. Indeed, machine learning is replete with research efforts focused more on why the system is doing what it is doing (sort of "FMRI studies" of large learned systems, if you will), instead of proving that we designed the system to do so. The knowledge we glean from such studies might allow us to intervene in modulating the system's behavior a little (as medicine does). The in vitro part does, of course, allow for far more targeted interventions than in vivo settings do. AI's turn to natural science also has implications to computer science at large–given the outsized impact AI seems to be having on almost all areas of computing. The "science" suffix of computer science has sometimes been questioned and caricatured; perhaps not any longer, as AI becomes an ersatz natural science studying large learned artifacts. Of course, there might be significant methodological resistance and reservations to this shift. After all, CS has long been used to the "correct by construction" holy grail, and from there it is quite a shift to getting used to living with systems that are at best incentivized ("dog trained") to be sort of correct—sort of like us humans! Indeed, in a 2003 lecture, Turing laureate Leslie Lamport sounded alarms about the very possibility of the future of computing belonging to biology rather than logic, saying it will lead us to living in a world of homeopathy and faith healing! To think that his angst was mostly at complex software systems that were still human-coded, rather than about these even more mysterious large learned models!
- Everyone is a requirements engineering, architect and tester/verifier.