My journey to became a backend engineer from system engineer.

1. From system engineer
to GOPHER
I-Fan@XRSPACE
2021/07/28
coscup.ifan.tw

2. About ME
Back-end engineer @ XRSPACE,
Newbie to Golang and back-end
Android system software engineer @ XRSPACE,
Developed world first 5G VR HMD

3. Outline
• System Software Engineer vs Back-end Engineer
• Why Golang
• Clean Architecture with Golang
• Scratch an API server

4. System Software Engineer
vs
Back-end Engineer

5. System Software Engineer

6. ID
Android Device(Phone, HMD)
Hardware
Bootloader
Android Framework
Tools
Linux & Device Driver
HAL
Android Libs
Android Runtime
Android APP
From device bring-up to
software tool for device,
The scope of my work.
What’s system software engineer
(android)

7. A “Working” System
New request
Tracing lot of source code
Add little piece of codes
Customized System
DRM Playback.
Plain android system.
Tracing android/kernel + DRM module + base non-hlos
source code.
Enable/Modify related source code to integrate DRM
module into system.
A android system capable of DRM playback

8. Back-end Engineer

9. What’s software engineer in my
imagination
Silicon Valley S01E05
A Mac with fancy tools that used to code at Starbucks

11. Discussing and Prototyping
New idea
Write some code
Test and validation
Developed a system
Back-end
Engineer
More time for discussing and prototyping.
More coding and I am building something
from scratch,
it’s delightful.

12. Why Golang

13. • One of the newest language
• Developer community
• Performance
• Statically typed
• Great for back-end development
• Effective Go
• Modern grammar
• Build-in test
• Go fmt
• Go vet
Why Golang

14. Clean Architecture

15. Clean Architecture with Golang
• Independent of Frameworks
• Testable
• Independent of UI
• Independent of Database
• Independent of any external agency

16. Clean Architecture with Golang
• The Dependency Rule
• Dependencies can only point inwards,
nothing in an inner circle can know
anything at all about something in an
outer circle.
• Dependency inversion principle

17. Clean Architecture with Golang
• The Dependency Rule
• Dependencies can only point inwards,
nothing in an inner circle can know
anything at all about something in an
outer circle.
• Dependency inversion principle

18. Clean Architecture with Golang
• Pros
• Same concept as DDD(Domain)
• Flexibility
• Testability
• Maintenance
• Cons
• Learning curve
• encapsulation, inheritance and
polymorphism everywhere!
• Heavy(compare to Monolithic)

19. Scratch an API server

20. Scratch an API server
User
API
SERVER
Database
AWS S3
File
Access URL
Save Meta
Save File
coscup.ifan.tw

21. Scratch an API server
Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository
coscup.ifan.tw

22. Scratch an API server
Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository

23. type AssetID string
type AssetType int
type Asset struct {
FileSize int64
Type AssetType
UploadedTime time.Time
TransferredTime time.Time
ID AssetID
Status AssetStatus
FileName string
Label string
Description string
hashedFileName string
}
Scratch an API server
Domain
- Define asset data structure
domain/asset.define.go

24. Scratch an API server
The order of fields in struct matters
type myStruct struct {
myInt bool // 1 byte
myFloat float64 // 8 bytes
myBool int32 // 4 bytes
// this takes 24 byte
}
type myStructOptimized struct {
myFloat float64 // 8 bytes
myInt int32 // 4 bytes
myBool bool // 1 byte
// this takes 16 byte
}
Memory
Allocation

25. type Repository interface {
Save(asset Asset) error
}
type StorageClient interface {
Save(ctx context.Context, fileName string)
(string, error)
}
Scratch an API server
Domain
- Define interfaces for external
layers to follow it
- Interfaces for repository
- Interface for storage
client
domain/asset.define.go

26. package application
package domain package repository
Scratch an API server
Dependency inversion principle? How?
type Repository {} struct
type Asset {} struct
package repository
type Repository struct {}
func (repo *Repository)Save(a Asset){}
package domain
type Asset struct {}
References
package application
asset := Asset{}
repo := Repository{}
repo.Save(asset)

27. package application
package domain package repository
Scratch an API server
Dependency inversion principle? How?
type Repository {} struct
type Asset {} struct
package repository
type Repository struct {}
func (repo *Repository)Save(a Asset){}
package domain
type Asset struct {}
type Repository interface{
Save(a Asset)
}
package application
asset := Asset{}
repo := domain.Repository
repo.Save(asset)
type Repository interface {
Save(a Asset)
}
References
Inherits

28. Scratch an API server
Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository

29. type Asset struct {
FileSize int64 `bson:"file_size,omitempty"`
Type int64 `bson:"type,omitempty"`
UploadedTime time.Time `bson:"uploaded_time,omitempty"`
ModifyTime time.Time `bson:"modify_time,omitempty"`
TransferredTime time.Time `bson:"transferred_time,omitempty"`
ID string `bson:"id,omitempty"`
Status string `bson:"status,omitempty"`
FileName string `bson:"file_name,omitempty"`
XRID string `bson:"xrid,omitempty"`
Label string `bson:"label,omitempty"`
Description string `bson:"description,omitempty"
HashedFileName string `bson:"hashed_file_name,omitempty"`
}
Scratch an API server
Repository
- Define data
structure to
save to
MongoDB
repository/mongodb/asset.go

30. type assetRepo struct {
collection *mongo.Collection
ctx context.Context
}
func (repo *assetRepo) Save(asset domain.Asset) error {
result := Asset{}
err := copier.Copy(&result, &asset)
if err != nil { … }
result.HashedFileName = asset.HashedFileName()
filter := bson.M{
"id": asset.ID,
}
update := bson.M{"$set": &result}
opts := options.Update().SetUpsert(true)
_, err = repo.collection.UpdateOne(repo.ctx, filter,
update, opts)
return err
}
Scratch an API server
Repository
repository/mongodb/asset.go
- Implement Save()

31. Scratch an API server
Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository

32. type S3StorageClient struct {
channel chan io.Reader
…
}
func (s S3StorageClient) Save(ctx context.Context, fileName string) (string,
error) {
…
file, ok := <-s.channel
if ok != true { … }
res, err := s.uploader.UploadWithContext(ctx, &s3manager.UploadInput{
Body: file,
Bucket: aws.String(s.bucket),
Key: aws.String(fileName),
})
if err != nil { … }
return res.Location, nil
}
Scratch an API server
Storage Client
storageclient/asset.go
- Implement Save()

33. Scratch an API server
Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository
coscup.ifan.tw

34. type UploadFileCommand struct {
FileSize int64
FileName string
Label string
Description string
TransferredTime time.Time
UploadedTime time.Time
}
type UploadFileResult struct {
HashedFileName string
AssertURL string
AssetStatus string
ID string
}
Scratch an API server
Application
application/command.go
- Define upload command and response

35. type AppService struct {
repo domain.AssetRepository
storageClient domain.AssetStorageClient
}
func NewAssetAppService(repo domain.Repository,
storageClient domain.StorageClient)
*AppService {
return &AppService{
repo: repo,
storageClient: storageClient,
}
}
Scratch an API server
Application
application/application.srv.go
- Define service struct and constructor

36. func (assetApp *AppService) UploadFile(command UploadFileCommand) (*UploadFileResult, error) {
ts := time.Now()
asset, err := domain.NewAsset(domain.NewAssetParam{
ID: assetApp.repo.NewID(),
FileName: command.FileName,
FileSize: command.FileSize,
Status: domain.AssetStatusUploaded,
UploadedTime: command.UploadedTime,
Description: command.Description,
Label: command.Label,
})
if err != nil {…}
err = assetApp.repo.Save(*asset)
if err != nil {…}
urlAsset, err := assetApp.storageClient.Save(context.Background(), asset.HashedFileName())
if err != nil {…}
asset.Transferred(asset.UploadedTime.Add(time.Now().Sub(ts)))
err = assetApp.repo.Save(*asset)
if err != nil {…}
return &UploadFileResult{
HashedFileName: asset.HashedFileName(),
AssertURL: urlAsset,
ID: string(asset.ID),
AssetStatus: string(asset.Status),
}, nil
}
Scratch an API server
Application
application/application.srv.go
- Implement
function for
upload usecase

37. Scratch an API server
Controller
Storage
Client
Domain
Service
REST
MongoDB
AWS S3
Repository
coscup.ifan.tw

38. func (a *Asset) Upload(ctx *gin.Context) {
label := ctx.PostForm("label")
description := ctx.PostForm("description")
fileAsset, headerAsset, err := parseUploadFiles(ctx,
"asset")
if err != nil {}
ch := make(chan io.Reader, 1)
defer close(ch)
ch <- fileAsset
client, err := storageclient.NewStorageClient(config,
ch, time.Second*5)
if err != nil {}
repoAsset := a.sys.GetRepository().Asset()
appSrv := application.NewAssetAppService(repoAsset,
client)
cmd := application.UploadFileCommand{
FileName: headerAsset.Filename,
Label: label,
FileSize: headerAsset.Size,
Description: description,
}
result, err := appSrv.UploadFile(cmd)
if err != nil {}
ctx.JSON(200, gin.H{"success": result.AssertURL
,})
}
Scratch an API server
Controller
- Translate input request to
internal data format
- Wake application up for work
domain/asset.define.go

39. Save
Repository
Upload
Application
Save
Storage Client
POST /upload
Controller
Asset
Domain
Used by
Used by
Used by
Used by Injected interface{}
Independent test to
database/storage
Independent test to
repository/storage
Independent test to
application
Top layer can be used
every where

40. Demo
Source code:
coscup.ifan.tw

41. What I have learned
• The working model as is totally different from system
engineer.
• Never hesitate to refactor or even abandon your code, always
refine it.
• It’s important to learn it with fine architecture or design
pattern, it might to tough but it’s worth.

42. Thank you.
linkedin.com/in/ifan-wang
w07fan@gmail.com