ORM in Go
Internals, tips & tricks
Dmytro Istratkin
Senior Software Engineer
SoftServe
Object-Relational Mapping frameworks history
ORM approaches
• Object to schema (code first)
• Schema to object (database first)
• Meet in the middle
ORM in Go
Name Stars ⭐️ DBMS support Approach Notes
gorm 15153
MySQL, PostgreSQL, Sqlite3,
MS SQL Server
Code first
xorm 5356
MySQL, PostgreSQL, Sqlite3,
MS SQL Server, Oracle, …
Code first, DB first Codegen, sync
go-pg 3106 PostgreSQL only Code first, DB first
gorp 3099
MySQL, PostgreSQL, Sqlite3,
MS SQL Server, Oracle
Code first
SQLBoiler 2336
MySQL, PostgreSQL, Sqlite3,
MS SQL Server, CockroachDB
DB first Full codegen
reform 815
MySQL, PostgreSQL, Sqlite3,
MS SQL Server
Code first Codegen
go-queryset 457
MySQL, PostgreSQL, Sqlite3,
MS SQL Server
Code first Codegen, on top of GORM
GORM
• Transactions
• Hooks
• Associations
• Logger
• Actively developed
GORP
• Transactions
• Hooks
• ORM-ish
• Logger
• Discontinued
Tips & tricks
Query result caching
• About GCache: LRU, LFU, ARC support
• Why GCache is used in current project
Caching in GORM: after query
db.Callback().Query().After("gorm:query").Register(”mykey:after_query", func(scope *gorm.Scope) {
key := fmt.Sprintf(“%s”, scope.DB().QueryExpr())
if gch.Has(key) {…}
if value, ok = scope.Get(“cachemarker”); !ok {
value = scope.Value
}
if value == nil {
scope.Log("nothing to cache")
}
if err := gch.SetWithExpire(key, value, 30 * time.Minute); err != nil {…}
})
Caching in GORM: before query
db.Callback().Query().Before("gorm:query").Register("my:before_query", func(scope *gorm.Scope) {
key := fmt.Sprintf(“%s”, scope.DB().QueryExpr())
value, err := gch.GetIFPresent(key)
if err == gcache.KeyNotFoundError {…}
dst, ok := scope.Get(“cachemarker”)
if !ok {…}
srcValue, dstValue := indirect(reflect.ValueOf(value)), indirect(reflect.ValueOf(dst))
if !dstValue.CanSet() {…}
dstValue.Set(srcValue)
scope.InstanceSet("gorm:skip_query_callback", nil)
})
Caching in GORM: caching marker
var target []struct {
FileName string `gorm:"column:file_name"`
}
db := c.DB().Table("files").
Where("file_id = ?", fileID).
Select("file_name").
Set(“cachemarker”, &target).
Scan(&target)
GORM: Explain query
func explainAnalyzeCallback(scope *gorm.Scope) {
vars := strings.TrimPrefix(fmt.Sprintf("%#v", scope.SQLVars), "[]interface {}")
result := []string{fmt.Sprintf("nn%s %sn", scope.SQL, vars)}
query := fmt.Sprintf("EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS) %s", scope.SQL)
rows, err := scope.SQLDB().Query(query, scope.SQLVars...)
if err != nil {…}
defer func() { _ = rows.Close() }()
… // concatenate result
scope.Log(strings.Join(result, "n"))
}
GORP: Explain query
func (m GorpExplainLogger) Printf(format string, v ...interface{}) {
query := strings.TrimLeftFunc(v[1].(string), unicode.IsSpace)
ok, values := parseArgs(v[2].(string))
if !ok {...}
var result []string
if _, err := m.executor.Select(&result, "EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS) ”
+ query, values...); err != nil {…}
for _, s := range result {
m.logger.Printf("%s", s)
}
}
ORM: Pros
• Only one place for model
• Use your favorite programming language
• SQL is hidden
• DB abstraction
ORM: Cons
• ORM has it’s own language
• Configuration
• Poor optimization capabilities
• DB abstraction
Links
Repositories:
• GORM: https://github.com/jinzhu/gorm
• XORM: https://github.com/go-xorm/xorm
• GCache: https://github.com/bluele/gcache
Media:
• https://github.com/MariaLetta/free-gophers-pack
• https://ih1.redbubble.net/image.581976886.8080/poster,840x830,f8
f8f8-pad,750x1000,f8f8f8.u2.jpg
Questions?

ORM in Go. Internals, tips & tricks

  • 1.
    ORM in Go Internals,tips & tricks Dmytro Istratkin Senior Software Engineer SoftServe
  • 2.
  • 3.
    ORM approaches • Objectto schema (code first) • Schema to object (database first) • Meet in the middle
  • 4.
    ORM in Go NameStars ⭐️ DBMS support Approach Notes gorm 15153 MySQL, PostgreSQL, Sqlite3, MS SQL Server Code first xorm 5356 MySQL, PostgreSQL, Sqlite3, MS SQL Server, Oracle, … Code first, DB first Codegen, sync go-pg 3106 PostgreSQL only Code first, DB first gorp 3099 MySQL, PostgreSQL, Sqlite3, MS SQL Server, Oracle Code first SQLBoiler 2336 MySQL, PostgreSQL, Sqlite3, MS SQL Server, CockroachDB DB first Full codegen reform 815 MySQL, PostgreSQL, Sqlite3, MS SQL Server Code first Codegen go-queryset 457 MySQL, PostgreSQL, Sqlite3, MS SQL Server Code first Codegen, on top of GORM
  • 5.
    GORM • Transactions • Hooks •Associations • Logger • Actively developed
  • 6.
    GORP • Transactions • Hooks •ORM-ish • Logger • Discontinued
  • 7.
  • 8.
    Query result caching •About GCache: LRU, LFU, ARC support • Why GCache is used in current project
  • 9.
    Caching in GORM:after query db.Callback().Query().After("gorm:query").Register(”mykey:after_query", func(scope *gorm.Scope) { key := fmt.Sprintf(“%s”, scope.DB().QueryExpr()) if gch.Has(key) {…} if value, ok = scope.Get(“cachemarker”); !ok { value = scope.Value } if value == nil { scope.Log("nothing to cache") } if err := gch.SetWithExpire(key, value, 30 * time.Minute); err != nil {…} })
  • 10.
    Caching in GORM:before query db.Callback().Query().Before("gorm:query").Register("my:before_query", func(scope *gorm.Scope) { key := fmt.Sprintf(“%s”, scope.DB().QueryExpr()) value, err := gch.GetIFPresent(key) if err == gcache.KeyNotFoundError {…} dst, ok := scope.Get(“cachemarker”) if !ok {…} srcValue, dstValue := indirect(reflect.ValueOf(value)), indirect(reflect.ValueOf(dst)) if !dstValue.CanSet() {…} dstValue.Set(srcValue) scope.InstanceSet("gorm:skip_query_callback", nil) })
  • 11.
    Caching in GORM:caching marker var target []struct { FileName string `gorm:"column:file_name"` } db := c.DB().Table("files"). Where("file_id = ?", fileID). Select("file_name"). Set(“cachemarker”, &target). Scan(&target)
  • 12.
    GORM: Explain query funcexplainAnalyzeCallback(scope *gorm.Scope) { vars := strings.TrimPrefix(fmt.Sprintf("%#v", scope.SQLVars), "[]interface {}") result := []string{fmt.Sprintf("nn%s %sn", scope.SQL, vars)} query := fmt.Sprintf("EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS) %s", scope.SQL) rows, err := scope.SQLDB().Query(query, scope.SQLVars...) if err != nil {…} defer func() { _ = rows.Close() }() … // concatenate result scope.Log(strings.Join(result, "n")) }
  • 13.
    GORP: Explain query func(m GorpExplainLogger) Printf(format string, v ...interface{}) { query := strings.TrimLeftFunc(v[1].(string), unicode.IsSpace) ok, values := parseArgs(v[2].(string)) if !ok {...} var result []string if _, err := m.executor.Select(&result, "EXPLAIN (ANALYZE, COSTS, VERBOSE, BUFFERS) ” + query, values...); err != nil {…} for _, s := range result { m.logger.Printf("%s", s) } }
  • 14.
    ORM: Pros • Onlyone place for model • Use your favorite programming language • SQL is hidden • DB abstraction
  • 15.
    ORM: Cons • ORMhas it’s own language • Configuration • Poor optimization capabilities • DB abstraction
  • 16.
    Links Repositories: • GORM: https://github.com/jinzhu/gorm •XORM: https://github.com/go-xorm/xorm • GCache: https://github.com/bluele/gcache Media: • https://github.com/MariaLetta/free-gophers-pack • https://ih1.redbubble.net/image.581976886.8080/poster,840x830,f8 f8f8-pad,750x1000,f8f8f8.u2.jpg
  • 17.

Editor's Notes

  • #3 TopLink: types mapping, N-M, row -> objects, caching, multi DB vendors EF: model first, database first, code first
  • #4 MitM – validation and binding
  • #8 Motivation in current project
  • #12 Pluck, Rows
  • #14 Gorp – check for caching possibility