The document outlines insights from a seasoned researcher in software engineering, emphasizing the challenges and subjectivity of research in academic settings. It advocates for defining clear goals aligned with personal passion and institutional expectations while navigating publication processes and maintaining perseverance. The author shares lessons learned from experience, including the importance of collaboration with industry, effective communication, and the acceptance of inherent setbacks in academia.

1. .lusoftware verification & validation
VVS
Strategy?
Try Black Magic
Lionel Briand
NFS’17@ICSE’2017

2. About me
• 23 years of post-PhD research experience
• IEEE Fellow, Harlan Mills IEEE CS award
• Canada Research Chair, ERC Advanced grant
• ICSE PC co-chair in 2014
• EiC of Empirical Software Engineering (Springer) for 13 years
• Graduated 20 PhD students
• Worked with >30 industry partners (aerospace, automotive, health care, finance …)
• H-index = 69, around 22,000 citations (for those interested in the “number game”)
• Had lots of papers rejected
2

3. Remember why you do this job
• We want to advance software engineering knowledge
• We want our work to matter
• We want to take pride in our achievements
• We aspire to continuously learn, improve, and strengthen our
expertise and experience
• NO MATTER WHAT, NEVER COMPROMISE ON THIS
3

4. In an Ideal World …
• It would be easy to provide advice
• Only quality and impact would matter
• Subjectivity and arbitrariness would play a negligible role in the
review / publications process
• A publication record would be related to impact
• Research communities would welcome novel perspectives and
viewpoints
• Resources dedicated to a domain of research would be
proportional to its importance
4

5. In Reality …
• Software engineering research is a highly subjective realm,
disconnected from the reality of industrial software engineering –
varying opinions abound
• Little of the research has any practical impact anyway – no reality
check, hence the significant subjectivity
• Research communities are inherently conservative – incremental
work is much easier to publish
• Research trends are very much driven by fads and opportunism
5

6. Also…
• Journals (some more than others):
• Extremely slow response time
• Difficulties finding adequate reviewers
• Associate editors have little time and attention to dedicate to papers
• Conferences:
• High review load
• Some people are on many program committees
• Mismatch of expertise
6

7. Context Matters
• What are the expectations in your institution? E.g., publications,
grants …
• There is wide variation across institutions
• Hard reality: If you intend to stay there, you have to comply with
such constraints and expectations
• But not if it deprives you from any sense of purpose and
contentment
• I never worked in a top academic institution, it did not match what
I wanted to do
7

8. Carefully Define your Goals
• Such decisions will have highly significant consequences
• Pick a problem domain you can be passionate about, not just what
seems like a good opportunity
• Pick something that has a potential for significant impact, whether
theoretical or industrial
• Industrial problems are fascinating, as opposed to what you may
hear
• People claiming to work on “future SE problems” often work on
elusive problems that never materialize – it is anyway a good idea
to run your problem definitions by industry experts
8

9. Institution’s
Expectations Your Goals
Realities of SE Research
Strategy

10. Aim High but …
• Aim high, be ambitious in your ultimate goals
• However, define intermediary objectives (strategy) so as to
incrementally achieve them
• Submit to good quality venues only – there is a level below which it
is not worth it
• Adjust your target to how much risk you are willing to take, how
quickly you want to publish a piece of work
• Account for the needs of your students and postdocs
10

11. Be Persistent
• Listen, learn from reviewers and others, when possible
• But in the end, if you believe in what you do and you are not
given credible evidence or arguments to think otherwise, be
persistent, don’t give up or get discouraged when facing
opposition or disagreement
• Software engineering research is highly subjective, opinions
abound, perspectives widely differ, hype matters
11

12. Learn how to be Convincing
• Once you have made decisions and plans, use all
opportunities you have to discuss your work with close and
trustworthy colleagues
• Learn how to develop convincing arguments and prevent
arbitrary criticisms by clarifying and justifying all your
assumptions, placing carefully your work in the context of
existing work – recall that even when it is only remotely
related, reviewers will think it is highly relevant
12

13. What would I have done
differently?
• Spend more time on “paper engineering”, however boring
• Be a bit more disciplined in choosing where I invest my
energy – and not just be driven by curiosity
• Start working intensively with industry earlier
• Be slightly more diplomatic with some of my academic
colleagues ;-)
13

14. What would I have done the
same?
• Work with great people I enjoyed spending time with
• Work on industry-relevant problems, in collaboration with
partners
• Open my big mouth when needed
• Act to help improve the research community
14

15. About being vocal …
• “The Case for Context-Driven Software Engineering Research:
Generalizability is Overrated”, IEEE Software, forthcoming
Sept/Oct 2017
• “Embracing the Engineering Side of Software Engineering”,
IEEE Software 29(4): 96, 2012
• Keynotes … (SlideShare)
15

16. Every Job has Drawbacks
• When discouraged or feeling down, remember that every job –
academic or otherwise – has perks and drawbacks, and we must
acquire the ability to surmount the latter
• Remember the positives aspects of what we do: intellectual and
creative freedom, the privilege to work with students, and the big
money ;-)
• Accept once and for all the subjectivity and arbitrariness we have
to work with: if you listen, learn, and persist, you will prevail
• Keep your sense of humor – you’ll need it
16