The document proposes an AI-compatible process for developing software that focuses on defining problems, researching use cases, and mapping skills upfront, then auditing data for quality and privacy before running parallel training experiments, benchmarking performance, and implementing live training with ongoing feedback in an agile manner overseen by roles like a data owner, coordinator, and ethical board.

1. AI-compatible process
within an ethical framework
for software development
Catalina Butnaru, City AI Ambassador in London
@katchja

2. How to Build A
Cognitive Business
TECHNOLOGY
AI = TD+ML+HITL

3. How to Build A
Cognitive Business
TECHNOLOGY
AI = TD+ML+HITL
F = T ∇ Sτ1
GOAL
1 Michael Scharf: A New Equation for Intelligence

4. *without breaking things
PROCESS
How to Build A
Cognitive Business
TECHNOLOGY
AI = TD+ML+HITL
F = T ∇ Sτ1
GOAL
1 Michael Scharf: A New Equation for Intelligence

5. *without breaking things
PROCESS
How to Build A
Cognitive Business
TECHNOLOGY
AI = TD+ML+HITL
F = T ∇ Sτ1
GOAL
1 Michael Scharf: A New Equation for Intelligence

6. How agile technology companies see AI
Tim Urban, Andrew Finn: WaitButWhy.com

7. How agile technology companies see AI
Tim Urban, Andrew Finn: WaitButWhy.com

8. Tim Urban, Andrew Finn: WaitButWhy.com
How soon-to-be cognitive companies see AI

9. Tim Urban, Andrew Finn: WaitButWhy.com
How soon-to-be cognitive companies see AI

10. Tim Urban, Andrew Finn: WaitButWhy.com
How soon-to-be cognitive companies see AI

11. Tim Urban, Andrew Finn: WaitButWhy.com
How soon-to-be cognitive companies see AI
KEEP CALM,
ADAPT
AND
STAY IN
BUSINESS

12. MULTI-AGENT
SYSTEMS
MAS
Agents with own goals in
the same environment.
“Watchdog” for
unusual events;
Automation.
INTEGRATED
INTELLIGENT
SYSTEMS
CBR, MAS, ANNs
FTW!
Complex, multi-faceted
business problems.
ARTIFICIAL NEURAL
NETWORKS
ANNs
Learn by example.
Supervised or unsupervised
Prediction,
forecasting.
CASE-BASED
REASONING
CBR
Given a specific problem,
what is the solution?
Rules / general recipe for
solving a peculiar
problem;
Decision-making.
Solving Complex Logistics Problems with Multi-Artificial Intelligent System, Y.K. Tse, T.M. Chan, R.H. Lie
International Journal of Engineering Business Management, First Published March 1, 2009

13. MULTI-AGENT
SYSTEMS
MAS
Agents with own goals in
the same environment.
“Watchdog” for
unusual events;
Automation
INTEGRATED
INTELLIGENT
SYSTEMS
CBR, MAS, ANNs
FTW!
Complex, multi-faceted
business problems.
Logistics
(Ocado)
ARTIFICIAL NEURAL
NETWORKS
ANNs
Learn by example.
Supervised or unsupervised
Prediction,
forecasting.
Traffic forecasting
CASE-BASED
REASONING
CBR
Given a specific problem,
what is the solution?
Rules / general recipe for
solving a peculiar
problem;
Decision-making
Solving Complex Logistics Problems with Multi-Artificial Intelligent System, Y.K. Tse, T.M. Chan, R.H. Lie
International Journal of Engineering Business Management, First Published March 1, 2009

14. Stacey
Matrix
for decision making
in organisations seen
as “complex responsive
processes of relating”

15. Adapted
Stacey
Matrix
in software
development,
where Agile is fit for
purpose in “Complicated”
and “Complex”.

16. Can we
agree?
AI is coming of age.
But there’s
push-back from
businesses,
regulators, and
skeptics
misunderstanding
what AI can do.

17. Process

18. Is Agile compatible with AI?
Is a change of process necessary, just
because of AI?
A new process.
Why a process?

19. PLAN
DESIGN
DEVELOP
TEST
FEEDBACK
hard to plan training time, data
quality and compatibility
if B2B integration with BI, ops, then
design is not necessary
skill mapping is!
Ml training is done in-situ (black box)
-> series of waterfall sessions, not
scalable
Sandbox testing VS different results
from live testing*
Direct customer feedback may
render training unusable (eg. Tay)
+AIAGILE =INCOMPATIBILITIES

20. SCOPE
Define problem, research applied AI use cases;
Define success and metrics;
Skill mapping and ethical standards.

21. SCOPE
Define problem, research applied AI use cases;
Define success and metrics;
Skill mapping and ethical standards.
DATA
AUDIT
Data sourcing;
Data quality (completeness) and compatibility;
Privacy and ownership
TRAIN Formulate and test a hypothesis;
Run several experiments in parallel
BENCHMARK
Error threshold against similar applications;
Compare against existing system performance;
Human intervention - when and why;
FEEDBACK

22. SCOPE
Define problem, research applied AI use cases;
Define success and metrics;
Skill mapping and ethical standards.
DATA
AUDIT
Data sourcing;
Data quality (completeness) and compatibility;
Privacy and ownership
TRAIN Formulate and test a hypothesis;
Run several experiments in parallel
BENCHMARK
Error treshhold against similar applications;
Compare against existing system performance;
Human intervention - when and why;
FINAL AUDIT
Wizard of oz experiments;
Level of judicial accountability, safety of AI integration;
Parallel live training;
AUGMENT Support workforce with utilising and adopting AI.
LIVE
TRAINING
FEEDBACK

23. DATA AUDIT
FINAL AUDIT AUGMENTSCOPE
TRAIN
BENCHMARK
LIVE TRAINING
HAI- a new process?
SANDBOX TRAINING

24. Agile HAI for applied AI
for greenfield
projects
➔ Wizard of Oz experiments➔ Customer feedback
➔ Skill mapping and augmentation➔ Equal knowledge of system
➔ Start small, build big ➔ Start big. Avoid cold start problems.
➔ Compatibility and compliance
(ethical, judicial)
➔ Less constrained
➔ Multiple parallel experiments➔ One idea at a time

25. Team roles

27. Data owner
Data Ops + Dev Ops + Data Ethics
Data management
internal and external
Hypothesis
Based on data
volume

28. Data owner
Data Ops + Dev Ops + Data Ethics
Hypothesis
Based on data
volume
Data management
internal and external

29. Data owner
Data Ops + Dev Ops + Data Ethics
Commissions data
*cold-start problem
Ensures Database
completeness
*reduce bias
Legacy systems
*compatibility or
migration
Hypothesis
Based on data
volume
Quality
*is data relevant?
Privacy & Ethics
*Fair, legal use of data for
human wellbeing
Data management
internal and external

30. Coordinator
Skill mapping + training
● safety of automation - to what
extent?
● skill mapping for targeted jobs
● set up parallel training
experiments to accurately
compare with human domain
knowledge
● deploy engineers to train
employees if needed

31. THE FUTURE OF EMPLOYMENT: HOW SUSCEPTIBLE ARE JOBS TO COMPUTERISATION?∗ Carl Benedikt Frey and Michael A. Osborne, September 17, 2013
Oxford University Engineering Sciences Department and the Oxford Martin Programme on the Impacts of Future Technology
Job computerisation

32. The Impact of AI in UK Constituencies: Where will automation hit hardest? - Future Advocacy, London, 2017
Risk of job
displacement
- 22% to over 39% of jobs
could be partially
displaced in the UK
- South Edinburgh
constituency is at lowest
risk of automation; Hayes
and Harlington (Heathrow)
- highest

34. Ethical board
- leading a transparent review
of application of AI
- conducting wizard of oz
experiments
- applying ethical standards
throughout the process
(transparency, responsibility,
human benefit, education)

36. Thank you :)
Your turn!
Catalina Butnaru
@katchja
butnaru.catalina@gmail.com
City AI London Ambassador @theCityai
IEEE Committee Member: Wellbeing and Ethics