The document outlines the components of software product engineering management, including project management, code management, and people management. It emphasizes a strategic, iterative approach to product design and development, highlighting the importance of understanding user needs and continuous improvement through methodologies like Scrum and MVP. Additionally, it addresses the complexities of managing code through automated testing and the need for engineering excellence in hiring and evaluating technical talent.

1. The Management of Software Product
Engineering
Michael Lei
Feb 28, 2023

2. Overview of Components
2
All
Management of Software
Product
Management of Projects/Tasks
Management
of Code
Manage
ment of
People
Software
Product
Software
Engineering
Management
of Tech

3. 3
All
Management of Products

4. Life Cycle of Software Product
4
Strategy &
Planning
Analysis of
Problem
Product
Design
Implementa
tion
Analysis of
Solution

5. Design Thinking – A Product Design Process
October 14, 2023
5
EMPATHIZ
E
DEFINE
IDEATE
PROTOTY
PE
TEST
Understanding
users’ context
and needs by:
observing,
interviewing,
and data
collection/anal
ysis to gain
insights into
their
behaviors,
motivations,
and pain
points.
Defining the problem
based on the insights
gathered in the
empathize stage, the
problem is framed as a
specific and actionable
challenge.
Generate a range of
potential solutions to
the defined problem,
through creative
thinking.
Represent the
solutions through
sketch, mock-up,
simulation, functional
prototype, etc.
Test the prototypes and gather
feedbacks, observe metrics, to
validate hypothesis, identify and
improve/refine solutions.
Multi-step Process
Iterative Process

6. Analysis is the Key of Product Success
6
All
• Analysis of the Problem
• Framing a business problem into solvable technical problems.
• Design business metrics.
• Pre-analysis of Solutions
• Prototype/POC/Simulation of implementations for verification and comparison (A/B Tests) of
alternative ideas.
• Fast search/filter of solution space before costly investments.
• Post-analysis of Solutions
• Online A/B tests
• Track changes of business metrics
• Feedback to next iteration of Product Planning & Design

7. MVP & Iterative Improvements
7
All
• Minimal Viable Product
• V0 of a product with minimal set of functionalities to be deemed useful.
Eg: MFA with SMS as 2nd factor as SMS is the only usable solution for most users.
• MVP requires substantial planning, design and implementation efforts to start a product on the
right track.
• Iterative Improvements
• Incremental efforts to improve metrics. Eg: Search relevance, user conversion on a website.
• Extend features from previous version.
• Small incremental investments with verifiable benefits.

8. Understand Users
8
All
• Consumer: need very intuitive UI and least friction.
Eg: No one wants to read a user guide to know how to use Google search or Amazon Market
Place.
• Professional:
• Has prior knowledge and experience  implicit expectation on usability patterns.
Eg: programmers will want sophisticated SDK and API, much more than a UI.
• Enterprise:
• Not only user manuals but also training is not a problem.
• Rich functionality and high customizability are favored over usability.
• Integration w. existing enterprise environment
• Multiple roles of users with very different interests.

9. Generalization: Configurability & Customizability
9
All
• Generalization over many use cases is the foundation of business software.
• Low-code redevelopment
Eg: Workflow engine, scripting, configuration

10. Problem Pattern 1: Business Process Automation
10
Users, Product Managers and
Engineers have the same high level
view of software that automates
business processes:
• An ontology of the business.
• Automation of the business
processes and workflows.
• Views and analytics over business
objects.
• Users/roles have different
accesses to data of business
objects and different functionalities.
Ontology
of
Business
Objects
Processes/
Workflows
Views/
Analytics
Users/
Roles

11. Problem Pattern 2: Optimization
11
All
Given a constraint C(x), find value of x to optimize function(x).
• Some optimization problem can be formulated mathematically with analytical or numerical
solutions.
Eg: Newton’s method and Deep Learning solves optimization problems with numerical methods.
• A software program is also a function.
Business_Metric = MetricComputation(program(p))
Some business metrics are computed over the output of software program with parameter p. P
may represent a simple parameter with alternative values or a software component with
alternative implementations. Eg:
• Google Ads: maximize click through rate of ad with a constraint on Ads relevance.
• Appen’s Fair Pay: balance between underpay for contributors and overcharge for customers

12. 12
All
Management of Projects/Tasks

13. 13
A Scrum Team
Enterprise,
Global BUs
Contributors
(crowd & in-house)
Project Managers
Enterprise Customers
SRE
Feature
spec ,
Prioritize/
track
bugs &
features

14. Scrum Process
14
All
Sprint
Planning
Daily Standup
& Dev
Sprint Review
Sprint
Retrospective
Backlog
Grooming
Product
Planning
UAT,
Product
Release

15. Organizing Tasks
15
All
• The Ontology of Projects/Tasks
• Initiative, Epic, Story are all derived from Issue.
• Epic models a stream of features
• Story models an individually testable/verifiable feature unit.
• A Story may have sub-stories that are Dev Stories, QA Stories and SRE Stories.
• Issue has the following property:
• “sub-task” that references another issue as a logical containment relationship.
• “depend on”
• Scrum-team
• Assignee
• State: new, triaged, in-progress, in-test, in-verification, closed
• Management visibility is through analytical reporting on those objects.

16. The Agile Nature of Scrum
16
All
• Epics may be planned quarterly, but Stories are added/updated each sprint.
• Stories are groomed for business priority and estimated with story points to measure Eng
workload.
• Completion time of an Epic is always dynamically updated throughout its life cycle through
multiple sprints.
• There is only rough estimate at the planning time of an Epic.
• The point of time estimate of an Epic is the aggregate over all its Stories.

17. Evaluate Eng Productivity in Scrum
17
All
• On-time Delivery of an Epic is hard to measure, because
• Stories are dynamically added/updated, hence points of an Epic are dynamically updated.
• Prioritization of stories is dynamic. There is usually no reservation of resources to work on an
epic from start to end.
• Can productivity be measured as points delivered per sprint?
• Such measure can easily be gamed because points of a story is determined by Eng, who is
also the party being evaluated.
• Independent auditing of points?
• It takes an expert to judge an expert.
• We will revisit this topic in people management.

18. 18
All
Management of Code
• Overview of the life cycle of code
• Git branch strategy of code
• Journey of Code
• Tests
• Design and Code Reviews

19. Journey of Code
19
Code/
UnitTests/
ServiceTes
ts Dev
Regression
Tests:
- unit tests
- service tests
Integrati
on Tests
Code
Reviews
automat
ed
FE tests
Binary
Release
has more PR
Developer’s Environment
Master/Integration
Code Br
Release
Candidate
Production
feature code
all deployed
has more binary
changes
Feature
Release:
- turn on behavior
- A/B Tests
Production
Monitoring
& Diagnosis
Design
&
Reviews

20. Code Branching Strategy w. Release Br.
20
release
master
feature
1
feature
2
PR
PR
PR
Hotfix
branch
1.1.
0
1.1.hotf
ix
Merge back
PR
1.1.inprog
1.2.inprog
PR
1.2
time
Ships code to Staging, Pre-PROD and PROD, Protected and alive till app is alive
Ships code to Feature Pools only, should be short lived
PR

21. From Dev to Master Br.
Dev box
Commit to local or
push to remote
feature branch
Send Pull
Request to
Master
Code Review
PR Validation
Merge to
Master
● Unit Tests are coded in tandem with features
● Unit Tests are runnable w/o network connectivity
● API/Service Tests are using mocks for dependencies
● Static Analysis performed
● All local testing passed
Automation Quality Gate:
● Unit Test 100% pass
● UT Code Coverage > 80%
● Static Analysis passed for
code compliance and security
compliance

22. From Release branch to Core Staging/Pre-Prod
Build Release
Candidate
Deploy to
Integration/Pre-
prod
Feature automation
+ regression
Deploy to Core
Staging/Pre-Prod
Automation:
● Unit Test 100% pass
● Static Analysis passed for
code compliance and security
compliance
Automation:
● Feature automation +
P1/P2 Regression
● Accessibility (for UI)
Fork Release Br
from Master

23. Code journey from Core Staging/Pre-Prod to Production
All functional automation
Speed testing
(Server side + Client side)
Security testing
Accessibility testing (for UI)
Deploy to Prod one
single instance (in
traffic)
Smoke test
10%
deployment
20%
deployment
Automation:
● All tests passed
● IT code coverage > 70%
Automation:
● P1/P2
tests
passed
Deployment
completed
Deploy to Core
Staging/Pre-Prod

24. Production Monitoring
Feature
enable/disable/AB Test
Production
Monitoring
● Ops related monitoring - CPU, Memory, errors,
exceptions, logs, software/data parity and etc.
● Functional Automation in Production to identify
issues and then take actions immediately
● Data Monitoring - Data correctness, Analytics data
(e.g. Tracking, Conversion and etc.)
● Release calendar for all prod changes to help
Production issues root cause narrow down
● make monitoring reports very simple and easy to
read, everybody could help monitor and
identify/report issue.
Deployment
completed

25. Production Code : Testing Code
1 : 1.5
● Primary burden of tests is on developers
○ otherwise we have to hire more “testers” than
“developers”!
○ force them to write “testable” code.
Testability == Modularity, Extendability
Test

26. Pyramid of
Coverage
Test: Classifications
26
Unit Tests
Service-level
Blackbox Tests
Cross-service
Integration
Tests
UI
Tests
PD: test framework, write tests
QE/PD:
Framework, mock service
dependencies
Write tests
QE: Framework, Maintain
staging/preprod env, write tests.
QE/PM: manual and automated
test

27. Test: Unit vs Service Tests
Difference
● sub-service modules vs. microservice
Commonalities
● Mock all dependencies
● Fully portable
○ Dev’s machine, build machine, no need for staging/preprod envs
● Data driven:
○ Test input
○ Expected output + side_effect

28. Design Reviews
• Requirements: reviewed against the project requirements.
• Architecture: modular, scalable, and meets design goals & SLAs.
• Data flow: efficient, secure, and reliable.
• Algorithms and logic: correctness and efficiency.
• Error handling and recovery.
• Security: security vulnerabilities and how they are addressed.
• Usability: user-friendly and meets the needs of the users.
• Performance: potential performance issues and how they are addressed.
• Maintainability & Extendability
• Standards & Guidelines:
Industry standards, guidelines, best practices and compliance with
regulations and requirements.

29. Code Reviews
● Coding styles. Naming/Commenting conventions, …
But why? Is 2-space indentation really better than 4-space?
The hidden agenda is to enforce standard and homogeneity.
● Enforce design patterns and programming models.

30. Code Reviews
● Check unit/service test coverage: code coverage, branch
coverage, corner cases, etc.
● Mandatory Reviewers: owners of code (by repo, by
directory, automated as owners files).
● Involve code reviewers as early as design review.
● Small check-ins (PRs) to ease reviews.

31. Binary vs. Behavior release
● Binaries are released at fixed interval with/without
features.
● Binary release are gated by Regressions (expected original
behavior) instead of new feature behavior.
● A feature implementation is usually chunked into multiple
PRs and binary releases.
● Behavior release happens after all binary changes are
released and is turned on and rolled out using A-B test
mechanism.

32. 32
All
Management of People
• Starts with Hiring
• Engineering Excellence
• Evaluation

33. Hiring: Characteristics of Good Engineers
33
All
• Curiosity, especially on how things work.
• Attention to Detail.
• Hands-on, especially the ability to handle complexity.
• Good learner.
• Analytical mindset.
• Logical & Abstract thinking.
• Collaboration & Communications are important but not specific to
engineering.

34. Hiring: Finding the right candidates
34
All
• Jira, Git
• Scrum process
• Kafka programming
• AWS experience
• Coding
• Data Structure & Algo
• Event driven architecture
• Experience on legacy code base
• Experience with Platform Services
• Most are specific skills/processes
that can be learned quickly.
• Scrum is not a universal standard
for agile (not really emphasized in
Google or Meta).
• Has the candidate learned well from
classroom?
• All are foundational knowledge and
skills for software professionals.
• A person with good foundation,
learning capability & coding skills
can pick up specific tools very
quickly.
VS

35. Hiring: Process
35
All
• Standardize engineering hiring process across teams.
• Each JD has:
• Mostly role (FE, BE Dev, SRE, QA, ML Eng, etc) specific
requirements.
• Some project specific requirements.
• Interviews are designed to cover all requirements, with emphasis on
hands-on.
• Fair and transparent decision process based on interview results.

36. Engineering Excellence
36
All
• Promote technical soundness outside business impacts.
• Eg: code and design have to pass reviews and satisfy standards
even if delivery is delayed.
• Under Engineering Excellence Culture
• Engineers will not cut corners to cause technical debts.
• Engineers will pay off technical debts much faster.

37. Evaluation
37
All
• No magic metrics to evaluate engineers’ productivity.
• Because all of them can be gamed by engineers.
• It takes an expert to evaluate an expert.
• We rely on meritocracy.
• Managers and leaders need to be technical enough to come up
with fair assessment on work complexity and result quality.
• A manager should potentially be able to do the jobs of his/her
reports.