1. 我的 Julia 軟體架構演進之旅
Yueh-Hua Tu Ph.D.
ML Engineer@Taiwan AI Labs
Saturday 29th July, 2023
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 1 / 28
3. Outline
1. Prototyping-Development Iteration
2. TDD and Test Coverage
3. Evolutionary Design
4. Principles of Software Engineering
5. Development Mindset
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 3 / 28
4. Developing from scratch
If we want a project for statistical analysis, where could we start?
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 4 / 28
5. Prototyping linear regression
Inference
y = Xβ
1 julia> predict(X, β)=X∗β
Fitting
β = (XT
X)−1
XT
y
1 julia> fit(X, y) = inv(X'*X) * X'*y
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 5 / 28
7. Prototyping-Development Iteration
Prototyping-Development Iteration (PDI)
Prototyping first then develop for flexibility and performance!
For prototyping and researching
Figure: Scripts
⇒
For development
Figure: Codebase
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 7 / 28
8. Prototyping-Development Iteration
Prototyping
Figure: Scripts
Implement a simple and standalone feature in a script
to ensure correctness.
1 julia> predict(X, β)=X∗β
2
3 julia> fit(X, y) = inv(X'*X) * X'*y
And try to wrap these features into functions.
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 8 / 28
9. Prototyping-Development Iteration
Development
Figure: Codebase
Puts functions or components into your codebase and design for
reusablility and flexibility.
1 struct LinearRegressionModel
2 β
3 end
4
5 coefs(m::LinearRegressionModel) = m.β
6
7 predict(m::LinearRegressionModel, X) = X*coefs(m)
8
9 fit(::Type{LinearRegressionModel}, X, y) =
LinearRegressionModel(inv(X'*X) * X'*y)
,
→
After putting components into codebase, ensure your scripts still work for you. Don't put to
much specifics into codebase (e.g. data, configs).
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 9 / 28
10. TDD and Test Coverage
TDD and Test Coverage
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 10 / 28
11. TDD and Test Coverage
Test-Driven Development (TDD)
Actually, PDI is a kind of TDD process.
We still need to add some test cases and ensure tests are passed.
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 11 / 28
12. TDD and Test Coverage
TDD in Julia
TDD is very simple in Julia.
test/runtests.jl
1 @testset "MyProject.jl" begin
2 @test coefs(model) == β
3 ...
4 end
1 (@v1.9) pkg> test
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 12 / 28
13. TDD and Test Coverage
Develop for flexibility
src/stats.jl
1 cov(X, Y) = innerprod(X, Y)
src/linalg.jl
1 innerprod(x, y) = x'*y
src/regression.jl
fit(::Type{LinearRegressionModel}, X, y) = LinearRegressionModel(inv(innerprod(X, X)) *
innerprod(X, y))
,
→
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 13 / 28
14. TDD and Test Coverage
Develop for performance
src/linalg.jl
function innerprod(X::AbstractMatrix, y::AbstractVector)
# acceleration techniques: SIMD, GPU...
end
function innerprod(X::AbstractMatrix, Y::AbstractMatrix)
# acceleration techniques: SIMD, GPU...
end
Specification for optimization and keep consistent interfaces.
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 14 / 28
15. TDD and Test Coverage
Test coverage
Keep high test coverage as possible to ensure your codebase provides desired features.
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 15 / 28
17. Evolutionary Design
Adding new features
If we want to add new approach to solve a linear system, how can we do?
Ax = b
1 using LinearSolve
2
3 A = rand(4, 4)
4 b = rand(4)
5 prob = LinearProblem(A, b)
6 sol = solve(prob)
Adding new features could be prototyped by
Adding types
Extending existing functions
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 17 / 28
18. Evolutionary Design
Adding types
Original
struct RegularSolver end
solve(m::RegularSolver, A, b) =
solve(LinearProblem(A, b))
,
→
⇒
Adding CholeskySolver
1 abstract type LinearSolver end
2
3 struct RegularSolver <: LinearSolver end
4 struct CholeskySolver <: LinearSolver end
5
6 solve(m::RegularSolver, A, b) =
solve(LinearProblem(A, b))
,
→
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 18 / 28
19. Evolutionary Design
Extending functions/methods
Original
abstract type LinearSolver end
struct RegularSolver <:
LinearSolver end
,
→
struct CholeskySolver <:
LinearSolver end
,
→
solve(m::RegularSolver, A, b) =
solve(LinearProblem(A, b))
,
→
⇒
Extending solve to CholeskySolver.
1 abstract type LinearSolver end
2
3 struct RegularSolver <: LinearSolver end
4 struct CholeskySolver <: LinearSolver end
5
6 solve(m::RegularSolver, A, b) =
solve(LinearProblem(A, b))
,
→
7 solve(m::CholeskySolver, A, b) = #
Cholesky decomposition...
,
→
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 19 / 28
20. Principles of Software Engineering
Principles of Software Engineering
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 20 / 28
21. Principles of Software Engineering
Single Responsibility Principle
A class or function should be responsible to only one actor.
LinearModel: represent a linear model
coefs: returns model coefficients
fit: fitting model
LinearSolver: approach to solve a linear system
solve: determines how to solve a linear system
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 21 / 28
22. Principles of Software Engineering
Open-Closed Principle
A module will be said to be open for extension and be said to be closed for modification.
struct LinearRegressionModel
β
end
coefs(m::LinearRegressionModel) =
m.β
⇒
1 abstract type LinearModel end
2
3 struct LinearRegressionModel <:
LinearModel
,
→
4 β
5 end
6
7 struct LogisticRegressionModel <:
LinearModel
,
→
8 β
9 end
10
11 coefs(m::LinearRegressionModel) = m.β
12 coefs(m::LogisticRegressionModel) = m.β
Benefitial from Julia's features of type and method separation, it's easy to add new types or
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 22 / 28
23. Principles of Software Engineering
Liskov Substitution Principle
Derived class can be able to replace its base class.
abstract type LinearModel end
struct LinearRegressionModel <: LinearModel
β
end
struct LogisticRegressionModel <: LinearModel
β
end
Since Julia's subtyping system doesn't allow composite types as super types, Julia always run
composite types in practice.
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 23 / 28
24. Principles of Software Engineering
Interface Segregation Principle
No code should be forced to depend on methods it does not use.
1 julia> model = fit(LinearRegressionModel,
X, y)
,
→
2
3 julia> coefs(model)
4
5 julia> model =
fit(LogisticRegressionModel, X, y)
,
→
6
7 julia> coefs(model)
User just need to know interfaces.
1 module MyPackage
2 abstract type LinearModel end
3 struct LinearRegressionModel <:
LinearModel
,
→
4 β
5 end
6 struct LogisticRegressionModel <:
LinearModel
,
→
7 β
8 end
9 coefs(m::LinearRegressionModel) = m.β
10 coefs(m::LogisticRegressionModel) =
m.β
11 end
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 24 / 28
25. Principles of Software Engineering
Dependency Inversion Principle
Dependency injection
1 abstract type LinearSolver end
2
3 struct RegularSolver <: LinearSolver end
4 struct CholeskySolver <: LinearSolver end
5 struct QRSolver <: LinearSolver end
1 abstract type LinearModel end
2
3 struct LinearRegressionModel <:
LinearModel end
,
→
4 struct LogisticRegressionModel <:
LinearModel end
,
→
5
6 fit(::Type{LinearRegressionModel}, X, y;
algo=CholeskySolver())
,
→
7 fit(::Type{LogisticRegressionModel}, X, y;
algo=CholeskySolver())
,
→
Y.H. Tu (Taiwan AI Labs) Evolutional Software Architecture Saturday 29th
July, 2023 25 / 28