RCODI presentation on open innovation and case studies.

1. Sabine Brunswicker, Professor for Digital Innovation
Teaching Engineers about Open Innovation
May 15, 2023

2. 1. Background
2. Introducing the Experimental Learning Framework
3. Cycle I: Simulated, case-based learning about
managing open innovation at the project level
4. Cycle II: Experimental learning as participant of
open innovation project
5. Q&A
2
Agenda

3. Click to edit Master title style
3
About me:
▪ Interdisciplinary trained (Engineering Sciences & management/behavioral
sciences)
▪ Full Professor for Digital Innovation
▪ Director of the Research Center on Open Digital Innovation (Discovery Park)
2014
▪ Visiting Professor at the Kellogg School of Management at Northwestern
University, additional visiting roles at TU Munich, University of
Southampton, Sheffield, Windracers.org
▪ Until 2013: Head of Open Innovation at Fraunhofer Institute for Industrial
Engineering, Stuttgart
Research Interests:
▪ Digital innovation, complex systems/systems engineering, network/data
science, AI/ML in socio-technical settings
▪ Understanding how digital technologies with a focus AI/ML and
autonomous systems
▪ Engineering mind
▪ Computational socio-technical scientists (big data, experiments, design
science, systems engineering, etc.)
I am excited to be here today …

4. Click to edit Master title style
4
Over the last years, digital technologies have been transforming the process and the
outcomes of innovation
I came to Purdue in Fall 2014; I am a computational scientist who studies digital innovation
Connected
everything
Analytics/AI
/ML
Cloud
Digital innovation &
AI-human systems
Social
media
Mobile
Digital technologies afford new ways of organizing innovation and operations/productions
across industries; they also form new AI-human systems

5. 5
We - the Research Center for Open Digital Innovation (RCODI) - are an interdisciplinary
team and network of scholars and thinkers who aspire to advance theories and
technologies for digital innovation in complex systems. We build, study, and practice
new forms of digital innovation, operations, and productions: open source software
(OSS) communities, crowdsourcing, cloud-based science infrastructures, blockchains,
peer production, data science challenges, conversational AI in healthcare, digital
operations in emergency response and autonomous aerial vehicles (AAVs). Our vision is
to leverage the power of distributed machine and human intelligence to augment
humans’ and businesses ability to perform complex problems.
www.rcodi.org

6. Click to edit Master title style
6
Recent publications
Brunswicker S, Mukherjee S (2023) The microstructure of modularity in design: a design motif view. Industrial and Corporate Change
32(1):234–261. (Special Issue: The Power of Modularity Today: 20 Years of "Design Rules" (honoring the work of Carliss Baldwin and
Kim Clark)
Brunswicker, S. and Rashidian, C. (2023). The Impact of Empathy in Conversational AI. 9th International Conference on Computational
Social Science.
Brunswicker, S. and S. Haefliger, Majchrzak (2023). The Impact of Feedback from Humans and Machines on Innovation in Online
Communities. SMS Conference 2023.
Brunswicker, S., de Torres, S., and Kasim, S. (2023). Modeling Complex OSS Supply-Chain Networks to Understand Cyber Security
Risks. 9th International Conference on Computational Social Science.
Brunswicker, S (2022) The Interplay between Platform Design and Complementors’ Design Strategies on Platform Innovation and
Evolution. Invited Panel Presentation at the Professional Development Workshop (PWD) on “Interdisciplinary Conversations
on Platforms: Design, Governance, and Evolution”,. 82nd Meeting of the Academy of Management (AOM), Seattle.
Brunswicker S, Cheoh JL (June, 2022) The Effect of Transparency on Innovation in Web Programming Contests: A Randomized Field
Experiment in the Cloud. International Conference for Computational Social Science (IC2S2). (University of Chicago Booth School of
Business).
Brunswicker, S. (2022) The Microstructure of Technology: A Micro-level View towards Modeling Technology Structure. (Invited talk at the
Institutes of Mathematics, New Economic Thinking, and Complexity at Oxford University, UK)
Brunswicker, S. (2022) The Microstructure of Technology: A Micro-level View towards Modeling Technology Structure. Invited talk at the
expert workshop on “The Structure of Technology” at the Santa Fe Institute, the World’s most famous research institute for complexity
sciences in Santa Fe, New Mexico.
Brunswicker, S. and Mukherjee S. (2021, August). Evolution of Coordination Structures in OSS Development: An Exponential Random
Graph Model. Best Paper Proceedings of the 80th Annual Meeting of the Academy of Management. 2021(1).

7. Click to edit Master title style
7
The experimental learning framework
Photo by Nick Fewings on Unsplash
Experimental framework and theoretical
foundation

8. Click to edit Master title style
8
Establish foundations
THINK
Chose a mode for an OI project
PLAN AND DO (1)
Solve a problem in an OI
project
PLAN AND DO (2)
Reflect upon learnings
OBSERVE
A B C
Kolb, David A. (1984). Experiential Learning: Experience as the Source of Learning and Development.

9. Click to edit Master title style
9
Before engaging in an experimental learning process, students will be exposed to the
theoretical foundations
Problem
Complexity
Hiddenness/L
ack of Access
to Know-how
High
Low
Low High
Type 1
Type 3 Type 4
Type 2
Bagherzadeh, Mehdi, & Sabine Brunswicker. (2018); Bagherzadeh, Mehdi, Andrei Gurca, & Sabine Brunswicker. (2022); Felin & Zenger (2014)

10. Click to edit Master title style
10
Engineering students easily grasp the concept of problem complexity when focusing on
engineering architectures and dependency structures
Baldwin and Clark (2000); Brunswicker and Mukherjee (2023); Photo by Hemanth Nirujogi on Unsplash
Dependency graph for D3 visualization modules
A B C D E F G H I
A 0 0 0 0 0 0 0 0 0
B 0 0 0 0 0 0 0 0 0
C 1 0 0 0 0 0 0 0 0
D 1 0 0 0 0 0 0 0 0
E 0 1 0 0 0 0 0 0 0
F 0 0 1 0 0 0 0 0 0
G 0 0 1 1 0 0 0 0 0
H 1 1 0 1 0 0 0 0 0
I 0 0 0 0 1 0 0 0 0
A formal design structure matrix
Problem
Complexity
Low High

11. Click to edit Master title style
11
In addition, I introduce students to a simple framework for classifying different
governance modes
Knowledge
sharing
Diversity/
number of
actors
involved
High
Low
Transactional Interactive/
Collaborative
Mode 3: Open
Innovation
Platform/
Contest
Mode 4: Open
Innovation
Community
Mode 2: Open
Innovation
Partnership
Mode 1:
Markets
/Contract
Bagherzadeh, Mehdi, & Sabine Brunswicker. (2018); Bagherzadeh, Mehdi, Andrei Gurca, & Sabine Brunswicker. (2022); Felin & Zenger (2014)

12. Click to edit Master title style
12
This modes differ in terms of knowledge sharing and diversity of knowledge
Markets/ Contracts
Open Innovation
Partnerships
Open Innovation
Platforms (Contest)
Open Innovation
Community
Medium to high-
powered incentives
High-powered,
cooperative
Moderate incentives
Low to moderate
incentives
Usually high (and
externally owned)
Negotiable Varied Varied
Limited Strong Limited (Problem)
Strong and
multidimensional
Low Low High High
Bilateral Multiple actors
Incentives
Control over IP
Knowledge sharing
Diversity of
sources/knowledge

13. 13
The experimental learning framework
Photo by Nick Fewings on Unsplash
Cycle I: “Simulated”, Case-based Experiences in
Choosing an Open Innovation Mode

14. Click to edit Master title style
14
Using real-world case studies, students move through simulated experiences about
choosing the right governance mode
High
Low
Transactional Interactive/
Collaborative
HP Dreamweaver
Rendering
Huawei IoT
Evonik Community
Ford OpenXC
El Lilly OIDD
Pfizer Design
Contest
Bosch Technology
Contest
Open
Innovation
Platforms
Open
Innovation
Community
Open Innovation
Partnerships
Samsung
ARTIK
Diversity/
number of
actors
involved
Knowledge
sharing

15. Click to edit Master title style
15
Cases overview: The cases cover different sectors and modes
Company Country Projects OI Mode Key insights
Bosch Germany
Innovation contest on non-electric
energy storage
Mode 3
The effects of limited knowledge sharing, and limited horizontal
interaction between actors; Anonymity as a solution seeker
Eli Lilly US
Drug discovery community focused
community focused on
bioactive components
Mode 4
Supporting bilateral and vertical information sharing between internal
and external teams; The importance of fostering a collaborative
environment and motivating sharing through the implementation of
and privacy mechanisms
Evonik Germany
Community on vacuum technology
in semiconductors
Mode 4
Expanding solution knowledge when internal knowledge base is out
of reach; Assembling partners and expertise to explore an emerging
technology; Supporting deep knowledge exchange and
integration from different actors across agencies; Positive effects
of community driven self-organizing practices
Ford US
OpenXC community for smart
mobility solutions
Mode 4
Facilitating the assembly of a developer community and platform
community and platform designs to support this process; Knowledge
process; Knowledge sharing in open innovation using a combination
a combination of open-source tools and licenses
Huawei China Partnership for IoT cellular services Mode 2
Partnership coordination activities required to ensure success;
Handling opportunistic behavior that prevents knowledge sharing
Pfizer US Design contest on prefilled syringe Mode 3
Knowledge sharing within and across teams; Overcoming internal
resistance among team members towards open innovation approach
Samsung US
Contest for add-on complements of
IoT platform ARTIK
Mode 4
Managing knowledge sharing between internal and external
and external partners/developers; Coordinated testing

16. Click to edit Master title style
Samsung’s ARTIK
Context of the project:
• Internet of Things (IoT) as a strategic field for innovation
• ARTIK as an embedded hardware and software platform in BETA version
Problem Type 3:
• Developing third-party applications for the ARTIK platform (an embedded
hardware and software platform in BETA version)
• Low to moderate complexity: Well-defined system interfaces (APIs, SDKs)
• High hiddenness of knowledge: unknown solution space
Open Innovation Mode: Mode 3 (Platform)
• Launch of an Innovation Contests with financial awards to tap into the ideas of
makers and citizens
• The contest is focused on a civic problem in California: the water crisis
• ~150 solutions were created; it was assumed that if 20% of those solutions
would be high quality; they could successfully launch the platform
Challenges
• Managing knowledge exchange
• Defining requirements for software and contests
• Incentive for collaborators

17. Click to edit Master title style
Ford’s OpenXC
Context of the project:
• Electronics and software as a strategic innovation area
• Open innovation is a strategy to implement this project
Problem Type 4:
• High complexity: Mobility as an complex problem (not just technical but also
social factors play an important role)
• High hiddenness: Sources of know-how not known to sources
Open Innovation Mode: Model 4 (Community)
• Open data movement: Release of machine-readable data (19 vehicle data)
• Establishment of own community with open source and open hardware license
and creative commons scheme
• Involvement of a variety of innovation ecosystem actors with the help of self-
organizing community principles (makers, citizens)
• Tripartite protection scheme: OSS, Open hardware, creative commons
Challenges
• Sharing knowledge under security constraints
• Fostering collaborative community that integrates technical & domain experts as
well as end-users
• High internal effort for managing the community; value capture (e.g. using
accelerator programs to move projects to maturity stage
Example of solutions
Bluetooth HUD Shifter with Haptic
Feedback

18. Click to edit Master title style
Ideally cycle 1 will move through six stages
Stage 1
Stage 2
Stage 3
Stage 4
• Assemble students into teams of 3 to 4 students
• Teams select case study based on summary and team discussions
• Students engage with case study and supplementary information
• Teams analyze problem complexity and hiddenness of knowledge
• Teams document their analysis and reflect upon it
• Students analyze the mode chosen in that particular case (Why was it chosen? Why does it fit that
particular problem? What were the challenges? What were challenges that are not reported?
• Students document their analysis and reflect upon it in team discussions
Stage 5
Stage 6
• Students perform what/if analysis (“simulation”) – they imagine the company had chosen a different
mode – and then compare the results in terms of solutions
• Students document their what/if analysis and reflect upon it in team discussions
• Students prepare board presentation to win the support for the project: They summarize: 1) Project
type 2) Open innovation mode 3) Implementation details 4) comparative analysis 5) potential
challenges and mitigation strategies
• Board will make go/no-go decision
• Final plenary discussion: Students’ reflect upon their learnings and instructor will summarize the
discussion (moderating role)
• Students write an individual report about their simulated case study and summarize their key
learnings

19. Click to edit Master title style
19
Through the experimental learning process, students learn how to best “match”
problems with governance modes
Problem
complexity
Hiddenness of
knowledge
High
Low
Low High
Mode 3: Open
Innovation
Platform/
Contest
Mode 4: Open
Innovation
Community
Mode 2: Open
Innovation
Partnership
Mode 1:
Markets
/Contract
Bagherzadeh, Mehdi, & Sabine Brunswicker. (2018); Bagherzadeh, Mehdi, Andrei Gurca, & Sabine Brunswicker. (2022); Felin & Zenger (2014)

20. Click to edit Master title style
20
The experimental learning framework
Photo by Nick Fewings on Unsplash
Cycle II: Experimental learning as participant of
open innovation project

21. Click to edit Master title style
21
Students gain additional insights when acting as solver of an open innovation project
Transactional Interactive/
Collaborative
Thingiverse
nanoHub
Kaggle
Ninesigma
Open
Innovation
Platforms
Open
Innovation
Community
Topcoder
R
Own platform
High
Low
Diversity/
number of
actors
involved
Knowledge
sharing

22. Click to edit Master title style
1.What was the characteristic of the problem I solved? (complexity,
hiddenness)
2. Did I contribute to solving the sponsor’s problem? If so how?
3. From my perspective, what challenges did I run into?
4. How did the contest/community support knowledge sharing?
5. How does this experience help me better understand how to best
manage open innovation?
After the participation in such projects, students should critically reflect upon their
experiences as solver

23. Connected
everything
Analytics/AI/
ML
Cloud
Open platforms for
collective innovation
Social media
Mobile
Digital technologies afford new ways to be innovative in a collective way!
Powered by the Purdue’s
Research Center for Open Digital
Innovation (RCODI)
+ We design technologies and study
their effect on groups and
collectives.
Background
www.rcodi.org

24. A challenge
IronHacks is a platform that supports open innovation
projects related to programming and data sciences
A data science
solution/ digital
innovation

25. Global Reach
To date, IronHacks has over 3,000
participants. Participants come from various
parts of the globe such as USA, China, India,
Colombia, Thailand, Indonesia, Malaysia,
Singapore, Spain
Participants include professional
software engineers, data
scientists, PhD students, Master
students and undergraduate students

26. Focused on societal challenges that matter to all of us.
Background
AFFORDABLE HOUSING
HEALTHY LIVING COVID-19 Challenge
Use open data & develop an app
that helps citizens find cheap &
seasonally fresh vegetables from
local markets.
Build a website with interactive
visualizations to help new students
find safe & affordable housing near
their university.
Build a statistical model using
social movement, distancing &
infection data to predict COVID-
19 impact.

27. *Note: This is a sample timeline
Background
Iterative Process: Single or multiple phase, with end-
points, where submissions are evaluated real-time.
Registration
opens
Release
task
*milestone
First Set of Data
*required
...
#1 #2 #4 14#
Week 1 Final Week
#3 #5
*milestone
6#
Second Set of Data
Week 2
Final release of data

28. Provides users with unique data science resources for data science
problem-solving
User
Features
Data Science
Workspace
(JupyterLab)
Secure BigQuery
database
integration
Pre-processed
datasets
Challenge-specific
tutorials/examples
Full workflow support
and integrated platform
Iterative
submission
feature
Real-time scoring/
dashboard
Solution sharing &
social interaction
Icon Sources: www.flaticon.com
Background

29. About IronHacks
Key technologies/programming languages:
Jupyter Notebook, BigQuery, R, Python, SQL.
Key skills/performance focus:
● Descriptive data analysis
● Data visualization for decision making
● Predictive modeling (Regressions,time
series, simulations)
JupyterLab 1.0: Jupyter’s Next-Generation Notebook
Interface.
A web-based interactive development environment for
Jupyter notebooks, code, & data. Supports many workflows
in data science, scientific computing, and machine learning.
Extensive & modular.
Offers a state of the art data science environment:
The workspace is a JupyterLab

30. Google BigQuery
100% Automated Data Flow from Data Provider to Participants
Providers
Google
Cloud Storage
Google
BigQuery
Participants
★ SafeGraph
★ Indiana Workforce Development.
★ Management Performance Hub.
★ Data imported from Storage.
★ Cleaned & staged
★ Transferred to user accessible Google
Cloud Project.
Once data is transferred, participants can access
data & create custom queries via their Jupyterhub in
IronHacks through their code.
★ Raw files imported from our Providers.
★ 100% automated via Virtual Machines.
★ Expandable bucket storage to store all data.

31. Background
PARTNERS

32. Example
The IronHacks 2022 focused on unemployment in the state of Indiana
The task “Predict Unemployment claims in the state of Indiana”
★ Predict the unemployment claims using the historical data provided by DWD, Indiana
using bigquery and IronHacks workspace.
★ Submission need to have a brief summary on how the model works.
★ Prediction task with ground truth solution (not revealed in task)
The
workspace
“JupyterLab hosted on Kubernetes Server”
★ Access to personalised instance using GitHub authentication
★ Auto Save feature so user never lose their work
★ Multi Language support such as Python, R, Julia.
★ GPU support so model runs fast and without any delays

33. Example
Task WorkSpace/JupyterHub

34. Example
We have ground truth information available to score the participants after each
round
The process “Multiple submission rounds with automated scoring”
★ Individuals were able to submit 3 times within a phase to get scored (real-
time).
★ Submitted via submission form on IronHacks platform; results immediately
published in personal & peers dashboard.
The evaluation “Two dimensions of evaluation”
★ Prediction accuracy scores: Mean & Absolute difference between the
ground truth values & prediction values.
★ Exploration scores:
○ Exploration Effort: How much effort a user has made, which
includes the different functions,models etc.
○ Exploration Uniqueness: How unique a user submission is from
their peers.
○ Exploration Progress: How much a user has explored from their
previous submission.
○ BigQuery Effort: Number of queries made to the bigquery
server.

35. Example
Via the platform it is possible to manipulate the “openness” and transparency of
information
Summary + notebooks of peers are
visible
1 Only notebooks of peers are visible
2

36. Provides researchers and admins with flexibility to run experiments
“Admin”
Features
Synchronous &
Asynchronous
process
Flexible scoring &
feedback
Information
Disclosure &
Collective
Feedback Design
Rich Data Collection
Randomization versus
purposive sampling
Multiple
treatment
Design
Variable Task Design
Survey Integration
Icon Sources: www.flaticon.com
Admin Features
Individual or
Group Goals &
Coding

37. Data Data Type Description
Survey Qualtrics Survey
Form
Pre hack, post hack and before each submission. Includes experience in
programming language through likert scale, personality traits,
demographics, education etc.
Workspace SQLite Database Includes execution history of each cell in jupyterhub with timestamp and
error logs.
Submission Notebook Python notebook All the notebooks are stored in firestorage transferred to automatic scoring
function.
Submission Summary Text User submit a paragraph of at least 500 characters which explains the
working of their model.
Forum posts Text Announcements, comments and queries during the competition.
User Data Firebase Data is fetched from qualtrics survey and stored into the user profile on the
platform.
Email thread Email Queries sent to us using “get in touch” button on platform.
Click data Firebase Each action of a user on the platform is recorded in firebase using eventID,
path and their userID.
Notebook Analysis JSON Participants notebook are analysed and broken down into variables like
number of functions, imports, etc.
Admin Features
Compared to other platforms, we offer very rich data collection

38. Admin features
Command Group Execution Count
IPython Command History - Function Count per Command
IPython Command History - Cumulative Function Count
Example - Participant JupyterHub Activity

39. Click to edit Master title style
1.What was the characteristic of the problem I solved? (complexity,
hiddenness)
2. Did I contribute to solving the sponsor’s problem? If so how?
3. From my perspective, what challenges did I run into?
4. How did the contest/community support knowledge sharing?
5. How does this experience help me better understand how to best
manage open innovation?
After the participation in such projects, students should critically reflect upon their
experiences as solver

40. • Brunswicker, S. (2023). Teaching Engineers about
Open Innovation. In H. Chesbrough, A. Radziwon, W.
Vanhaverbeke, & J. West (Eds.), Handbook for Open
Innovation. Oxford University Press
• Website:
• https://rcodi.org/education/
• https://rcodi.org/oxfordhbcases/
40
References

41. December 14, 2022
f
THANK YOU!