SlideShare a Scribd company logo
1 of 55
Download to read offline
The integration for package ecosystem
Hiroshi SHIBATA / GMO Pepabo, Inc.
2019.07.26 RubyConf Taiwan 2019
The Future of library dependency
management of Ruby
self.introduce
Hiroshi SHIBATA @hsbt
https://www.hsbt.org
Executive Officer VP of Engineering
Technical Director
at GMO Pepabo, Inc. @pepabo
self.introduce
=> {
name: “SHIBATA Hiroshi”,
nickname: “hsbt”,
organizations: [“ruby”, “rubygems”, “asakusarb”, “pepabo”, …],
commit_bits: [“ruby”, “rake”, “rubygems”, “bundler”, “rdoc”,
“psych”, “ruby-build”, “railsgirls”, “railsgirls-jp”, …],
sites: [“hsbt.org”, “ruby-lang.org”, “rubyci.org”, “railsgirls.com”,
“railsgirls.jp”],
}
No.7
No.1
No.5
Agenda
•How to use libraries on the Ruby language?
•What are RubyGems and Bundler?
•What’s the Gamification project?
•The Challenge for Bundler Integration
•The future plans for RubyGems 4.0 and Bundler 2.1
•The Roadmap for Ruby 3.0
How to use libraries
On the Ruby language?
1.
3/40min
What’s the standard library?
What’s the Standard library?
• We called its “標準添付ライブラリ” in Japanese.
• It needs to `require` difference from embedded libraries like
String, Thread, etc.
• It can be used without Bundler or RubyGems
Classification of standard libraries
Standard
Libraries
Default
Gems
Bundled
Gems
Pure Ruby 44 22 7
C extensions 12 16 0
This matrix shows number of standard libraries and their
classifications in Ruby 2.6.
Inside Default gems
• The ruby core team can release default gems to the
rubygems.org. You can install them via RubyGems.
• Rubygems have a detection method for default gems.
• Default gems are openssl, psych, json, etc…
>> Gem.loaded_specs["did_you_mean"].default_gem?
=> false
>> require 'openssl'
=> true
>> Gem.loaded_specs["openssl"].default_gem?
=> true
Inside Bundled gems
• We bundled *.gem and unpacked files to tarball package for
Bundled gems with `gems/bundled_gems`.
• `make install` installed Bundled gem your box.
What are
RubyGems and Bundler?
2.
8/40min
What’s rubygems?
RubyGems is a package management framework for Ruby.
• rubygems/rubygems.org:
• The Ruby community's gem host.
• rubygems.org is maintain by infrastructure team of rubygems. It is different
team from rubygems cli team.
• rubygems/rubygems:
• Command line tool of rubygems
• Rubygems are created by Seattle.rb
What’s new in RubyGems 3
•I released RubyGems 3 at 19 Dec 2018
https://blog.rubygems.org/2018/12/19/3.0.0-released.html
•This version dropped to support the old Ruby versions like
1.8 and 1.9
•RubyGems 3 have a lot of features and bugfixes.
The Important Notification
Do protect your account of
RubyGems.org
with 2-factor auth.
What’s Bundler?
•The vendoring tool of Ruby.
•RubyGems couldn’t care dependency of Ruby libraries and
isolate version managing with ruby process.
•Bundler can do them with `Gemfile`
# frozen_string_literal: true
source "https://rubygems.org"
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec
# We need a newish Rake since Active Job sets its test tasks' descriptions.
gem "rake", ">= 11.1"
What’s new in Bundler 2?
•There is no incompatible feature from Bundler 1.17.x.
•We disabled the incompatible features like renaming
`gems.rb` from `Gemfile`
•They no longer support under the Ruby 2.2.
What’s the Gamification
project?
3.
18/40min
Gemification for standard library
https://bugs.ruby-lang.org/issues/5481
• We extracted stdlibs like net-telnet, xmlrpc, rake to bundled
gems.
• These are extracted under the https://github.com/ruby/ . And
shipped on rubygems.org
• Other gems are also extracted at the future.
Pros of Gemification
• Maintainers can release gem for bugfix, new feature
independent with Ruby core.
• Easily backport stable version from develop version. Ruby
users can use new feature on stable version.
• If upstream is available on GitHub, Ruby users easily send
patch via Pull request.
Cons of Gemification
• Abandoned and complex dependency on rubygems and
bundler.
• Maintainers need to maintain ruby core and GitHub
repositories both.
• It’s hard to maintain compatibility with old ruby version.
Default gems on Ruby 2.6
bigdecimal (default: 1.4.1)
bundler (default: 1.17.2)
cmath (default: 1.0.0)
csv (default: 3.0.9)
date (default: 2.0.0)
dbm (default: 1.0.0)
(snip)
strscan (default: 1.0.0)
sync (default: 0.5.0)
thwait (default: 0.1.0)
tracer (default: 0.1.0)
webrick (default: 1.4.2)
zlib (default: 1.0.0)
Current status of Default gems
on Ruby 2.6
I’m going to promote more the
standard libraries to default
gem at Ruby 2.7.0. after that
we promote it to bundled gems.
The Challenge for Bundler
Integration
4.
20/40min
Bundler Integration on RubyGems 2.7
• It disabled in Ruby
2.5 because
bundler is not part
of standard
library.
• You can enabled it
with only `gem
update --system`
if USE_BUNDLER_FOR_GEMDEPS
ENV["BUNDLE_GEMFILE"] ||= File.expand_path(path)
require 'rubygems/user_interaction'
Gem::DefaultUserInteraction.use_ui(ui) do
require "bundler"
@gemdeps = Bundler.setup
Bundler.ui = nil
@gemdeps.requested_specs.map(&:to_spec).sort_by(&:name)
end
else
rs = Gem::RequestSet.new
@gemdeps = rs.load_gemdeps path
rs.resolve_current.map do |s|
s.full_spec.tap(&:activate)
end
end
Merged Bundler into ruby core 😤
The bundler finder issue of Heroku
•https://github.com/heroku/heroku-buildpack-ruby/pull/850
•Heroku platform only uses version 1 of Bundler like 1.17.x. But Bundler
version finder of RubyGems detects Bundler 1 or 2 from your Gemfile.lock.
@schneems fixes this issue on heroku.
•When You use Gemfile.lock updated by Bundler 2 with `bundle update --
bundler`, Heroku reject your app. Now you can use Ruby 2.6 and Bundler 2
on heroku.
BLESSED_BUNDLER_VERSIONS = {}
BLESSED_BUNDLER_VERSIONS["1"] = "1.15.2"
BLESSED_BUNDLER_VERSIONS["2"] = "2.0.1"
The path injection for LOAD_PATH issue
•https://bugs.ruby-lang.org/issues/15469
•After that, You can’t use the specified version of gems like json or psych.
It activates the versions of default gems provided by ruby core.
- “/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib"
- “/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash”
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib"
- (snip)
- "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0"
- "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash"
- "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib"
- (snip)
The current behavior of the bundled bundler
•It integrates with default gems like
json, psych.
•The upstream is https://github.com/
bundler/bundler. I backport the
released/developed version to ruby
repository.
•Ruby 2.6 always enabled Bundler
gem_deps now(New!)
~ > gem list | rg default:
bigdecimal (1.4.3, default:
1.4.2)
bundler (2.0.1, default: 1.17.3)
cmath (default: 1.0.0)
csv (3.0.6, default: 3.0.4)
(snip)
thwait (default: 0.1.0)
tracer (default: 0.1.0)
webrick (default: 1.4.2)
zlib (default: 1.0.0)
Ruby 2.6.3 is the best version ever
The future plans
For RubyGems 4.0
And Bundler 2.1
5.
28/40min
RubyGems 4
• Make enable as default for conservative option: https://
github.com/rubygems/rubygems/pull/2233
• Removed duplicated code and files.
• Make ruby gem install to user-install by default: https://
github.com/rubygems/rubygems/issues/1394
• Activation issues with default gems.
Make conservative option as default
• We got the installation time when already installed gems.
• To use conservative is ignore re-install action.
~ > gem i rails
Successfully installed rails-5.2.0
1 gem installed
~ > gem i rails ——conservative
~ >
Dependency Resolver incompatible
• RubyGems 2.x and 3.x uses Molinillo-0.5.7
• Bundler 1.x and 2.x also uses Molinillo-0.6.4
• These are different versions and behavior of dependency
resolver.
~/D/g/r/rubygems (master) > ls lib/rubygems/resolver/molinillo/lib/molinillo
delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb
dependency_graph errors.rb modules resolver.rb
~/D/g/b/bundler (master) > ls lib/bundler/vendor/molinillo/lib/molinillo
compatibility.rb dependency_graph errors.rb modules resolver.rb
delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb
Make `--user-install` as default
• RubyGems 4 will install the all gems to `~/.gem` maybe.
• Pros: Ruby in linux distribution has many of FAQ for gem
installation for using `sudo`. This change resolve this issues.
• Cons: Ruby version manager like rbenv is not support it. And
RubyGems have a lot of issues related this.
Activation issues about default gems
•You couldn’t use the specified version of default gems like json when
RubyGems/Bundler activated them.
•When rubygems uses json-2.1.0, You couldn’t use json 1.8.x. Because ruby
gems and rubygems.org never uses JSON format.
•We can resolve it with `vendoring` approach. But json, psych, and openssl
is C extension library.
We always welcome your patch.
Roadmap for Ruby 3.0
6.
33/40min
RubyGems side
Support JRuby and TruffleRuby
•Surprisedly, RubyGems and Bundler never test JRuby and
TruffleRuby in CI.
•We try to add JRuby and TruffleRuby to Travis or other CI
environments.
•To JRuby and TruffleRuby tam: Please join us for this
support.
RubyGems/Bundler integration
•Now, We put the bundler as
submodule in rubygems
repository.
•We will move the canonical
repository of bundler to
rubygems org or rubygems/
rubygems.
Bump up RubyGems/Bundler
•We will merge into RubyGems 3.2 and Bundler 2.1 into
Ruby 2.7.0. After that, RubyGems 4.0 will be merge Ruby 3.
Ruby
Bundler
RubyGems
2.7.0 3.02.7-rcX
3.1
2.0
3.0
2.1
3.2
3.0?
4.0
?
Ruby side
Ruby 2.7.0-preview1
was released May 30.
The features of Ruby 2.7.0
•Compaction GC by tenderlove
•Pattern Matching by k_tsj
•Next generation IRB with reline by aycabta
Gamification on Ruby 3.0(TBD)
base64
benchmark
cgi
digest
English
erb
fileutils
find
io/console
monitor
net/http
net/https
openssl
optparse
pathname
pp
rbconfig
resolv
set
shellwords
socket
stringio
strscan
tempfile
thread
time
timeout
tmpdir
tsort
uri
webrick
Win32API
zlib
We will extract the standard libraries to the bundled gems.
git.ruby-lang.org
We have a many of contributors
Ruby is designed to make
programmers happy.
Yukihiro Matz Matsumoto

More Related Content

What's hot

How to distribute Ruby to the world
How to distribute Ruby to the worldHow to distribute Ruby to the world
How to distribute Ruby to the worldHiroshi SHIBATA
 
Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0Hiroshi SHIBATA
 
How to distribute Ruby to the world
How to distribute Ruby to the worldHow to distribute Ruby to the world
How to distribute Ruby to the worldHiroshi SHIBATA
 
Dependency Resolution with Standard Libraries
Dependency Resolution with Standard LibrariesDependency Resolution with Standard Libraries
Dependency Resolution with Standard LibrariesHiroshi SHIBATA
 
How to develop the Standard Libraries of Ruby?
How to develop the Standard Libraries of Ruby?How to develop the Standard Libraries of Ruby?
How to develop the Standard Libraries of Ruby?Hiroshi SHIBATA
 
20140425 ruby conftaiwan2014
20140425 ruby conftaiwan201420140425 ruby conftaiwan2014
20140425 ruby conftaiwan2014Hiroshi SHIBATA
 
20140419 oedo rubykaigi04
20140419 oedo rubykaigi0420140419 oedo rubykaigi04
20140419 oedo rubykaigi04Hiroshi SHIBATA
 
Middleware as Code with mruby
Middleware as Code with mrubyMiddleware as Code with mruby
Middleware as Code with mrubyHiroshi SHIBATA
 
Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0Hiroshi SHIBATA
 
tDiary annual report 2009 - Sapporo Ruby Kaigi02
tDiary annual report 2009 - Sapporo Ruby Kaigi02tDiary annual report 2009 - Sapporo Ruby Kaigi02
tDiary annual report 2009 - Sapporo Ruby Kaigi02Hiroshi SHIBATA
 
An introduction and future of Ruby coverage library
An introduction and future of Ruby coverage libraryAn introduction and future of Ruby coverage library
An introduction and future of Ruby coverage librarymametter
 
How to Begin to Develop Ruby Core
How to Begin to Develop Ruby CoreHow to Begin to Develop Ruby Core
How to Begin to Develop Ruby CoreHiroshi SHIBATA
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesHiroshi SHIBATA
 
Gemification plan of Standard Library on Ruby
Gemification plan of Standard Library on RubyGemification plan of Standard Library on Ruby
Gemification plan of Standard Library on RubyHiroshi SHIBATA
 

What's hot (20)

How to distribute Ruby to the world
How to distribute Ruby to the worldHow to distribute Ruby to the world
How to distribute Ruby to the world
 
Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0
 
How to distribute Ruby to the world
How to distribute Ruby to the worldHow to distribute Ruby to the world
How to distribute Ruby to the world
 
Dependency Resolution with Standard Libraries
Dependency Resolution with Standard LibrariesDependency Resolution with Standard Libraries
Dependency Resolution with Standard Libraries
 
RubyGems 3 & 4
RubyGems 3 & 4RubyGems 3 & 4
RubyGems 3 & 4
 
Gems on Ruby
Gems on RubyGems on Ruby
Gems on Ruby
 
How to develop the Standard Libraries of Ruby?
How to develop the Standard Libraries of Ruby?How to develop the Standard Libraries of Ruby?
How to develop the Standard Libraries of Ruby?
 
RubyGems 3 & 4
RubyGems 3 & 4RubyGems 3 & 4
RubyGems 3 & 4
 
20140425 ruby conftaiwan2014
20140425 ruby conftaiwan201420140425 ruby conftaiwan2014
20140425 ruby conftaiwan2014
 
20140419 oedo rubykaigi04
20140419 oedo rubykaigi0420140419 oedo rubykaigi04
20140419 oedo rubykaigi04
 
Middleware as Code with mruby
Middleware as Code with mrubyMiddleware as Code with mruby
Middleware as Code with mruby
 
20140918 ruby kaigi2014
20140918 ruby kaigi201420140918 ruby kaigi2014
20140918 ruby kaigi2014
 
20140925 rails pacific
20140925 rails pacific20140925 rails pacific
20140925 rails pacific
 
Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0Gemification for Ruby 2.5/3.0
Gemification for Ruby 2.5/3.0
 
tDiary annual report 2009 - Sapporo Ruby Kaigi02
tDiary annual report 2009 - Sapporo Ruby Kaigi02tDiary annual report 2009 - Sapporo Ruby Kaigi02
tDiary annual report 2009 - Sapporo Ruby Kaigi02
 
An introduction and future of Ruby coverage library
An introduction and future of Ruby coverage libraryAn introduction and future of Ruby coverage library
An introduction and future of Ruby coverage library
 
How to Begin to Develop Ruby Core
How to Begin to Develop Ruby CoreHow to Begin to Develop Ruby Core
How to Begin to Develop Ruby Core
 
From 'Legacy' to 'Edge'
From 'Legacy' to 'Edge'From 'Legacy' to 'Edge'
From 'Legacy' to 'Edge'
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 Minutes
 
Gemification plan of Standard Library on Ruby
Gemification plan of Standard Library on RubyGemification plan of Standard Library on Ruby
Gemification plan of Standard Library on Ruby
 

Similar to The integration for package ecosystem and future plans

The story of language development
The story of language developmentThe story of language development
The story of language developmentHiroshi SHIBATA
 
The secret of programming language development and future
The secret of programming  language development and futureThe secret of programming  language development and future
The secret of programming language development and futureHiroshi SHIBATA
 
Exploring Ruby on Rails and PostgreSQL
Exploring Ruby on Rails and PostgreSQLExploring Ruby on Rails and PostgreSQL
Exploring Ruby on Rails and PostgreSQLBarry Jones
 
Why Bundler 1.1 will be much faster
Why Bundler 1.1 will be much fasterWhy Bundler 1.1 will be much faster
Why Bundler 1.1 will be much fasterPat Shaughnessy
 
RubyStack: the easiest way to deploy Ruby on Rails
RubyStack: the easiest way to deploy Ruby on RailsRubyStack: the easiest way to deploy Ruby on Rails
RubyStack: the easiest way to deploy Ruby on Railselliando dias
 
Ruby on rails toolbox
Ruby on rails toolboxRuby on rails toolbox
Ruby on rails toolboxBlazing Cloud
 
JRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the CloudJRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the CloudHiro Asari
 
Dcjq node.js presentation
Dcjq node.js presentationDcjq node.js presentation
Dcjq node.js presentationasync_io
 
RubyConfBD 2013 decouple, bundle and share with ruby gems
RubyConfBD 2013   decouple, bundle and share with ruby gems RubyConfBD 2013   decouple, bundle and share with ruby gems
RubyConfBD 2013 decouple, bundle and share with ruby gems nhm taveer hossain khan
 
RVM and Ruby Interpreters @ RSC Roma 03/2011
RVM and Ruby Interpreters @ RSC Roma 03/2011RVM and Ruby Interpreters @ RSC Roma 03/2011
RVM and Ruby Interpreters @ RSC Roma 03/2011Marcello Barnaba
 
An introduction to Rails 3
An introduction to Rails 3An introduction to Rails 3
An introduction to Rails 3Blazing Cloud
 
RVM, Bundler and Ruby Tracker
RVM, Bundler and Ruby TrackerRVM, Bundler and Ruby Tracker
RVM, Bundler and Ruby TrackerKeith Pitty
 
Crate - ruby based standalone executables
Crate - ruby based standalone executablesCrate - ruby based standalone executables
Crate - ruby based standalone executablesJeremy Hinegardner
 
Getting started with Rails (1), Season 2
Getting started with Rails (1), Season 2Getting started with Rails (1), Season 2
Getting started with Rails (1), Season 2RORLAB
 
An introduction to the ruby ecosystem
An introduction to the ruby ecosystemAn introduction to the ruby ecosystem
An introduction to the ruby ecosystemGeison Goes
 

Similar to The integration for package ecosystem and future plans (20)

The story of language development
The story of language developmentThe story of language development
The story of language development
 
How DSL works on Ruby
How DSL works on RubyHow DSL works on Ruby
How DSL works on Ruby
 
The secret of programming language development and future
The secret of programming  language development and futureThe secret of programming  language development and future
The secret of programming language development and future
 
Exploring Ruby on Rails and PostgreSQL
Exploring Ruby on Rails and PostgreSQLExploring Ruby on Rails and PostgreSQL
Exploring Ruby on Rails and PostgreSQL
 
rubyonrails
rubyonrailsrubyonrails
rubyonrails
 
rubyonrails
rubyonrailsrubyonrails
rubyonrails
 
Why Bundler 1.1 will be much faster
Why Bundler 1.1 will be much fasterWhy Bundler 1.1 will be much faster
Why Bundler 1.1 will be much faster
 
RubyStack: the easiest way to deploy Ruby on Rails
RubyStack: the easiest way to deploy Ruby on RailsRubyStack: the easiest way to deploy Ruby on Rails
RubyStack: the easiest way to deploy Ruby on Rails
 
Ruby on rails toolbox
Ruby on rails toolboxRuby on rails toolbox
Ruby on rails toolbox
 
JRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the CloudJRuby, Ruby, Rails and You on the Cloud
JRuby, Ruby, Rails and You on the Cloud
 
11 Ruby Gems
11 Ruby Gems11 Ruby Gems
11 Ruby Gems
 
Dcjq node.js presentation
Dcjq node.js presentationDcjq node.js presentation
Dcjq node.js presentation
 
RubyConfBD 2013 decouple, bundle and share with ruby gems
RubyConfBD 2013   decouple, bundle and share with ruby gems RubyConfBD 2013   decouple, bundle and share with ruby gems
RubyConfBD 2013 decouple, bundle and share with ruby gems
 
RVM and Ruby Interpreters @ RSC Roma 03/2011
RVM and Ruby Interpreters @ RSC Roma 03/2011RVM and Ruby Interpreters @ RSC Roma 03/2011
RVM and Ruby Interpreters @ RSC Roma 03/2011
 
An introduction to Rails 3
An introduction to Rails 3An introduction to Rails 3
An introduction to Rails 3
 
RVM, Bundler and Ruby Tracker
RVM, Bundler and Ruby TrackerRVM, Bundler and Ruby Tracker
RVM, Bundler and Ruby Tracker
 
Crate - ruby based standalone executables
Crate - ruby based standalone executablesCrate - ruby based standalone executables
Crate - ruby based standalone executables
 
Getting started with Rails (1), Season 2
Getting started with Rails (1), Season 2Getting started with Rails (1), Season 2
Getting started with Rails (1), Season 2
 
An introduction to the ruby ecosystem
An introduction to the ruby ecosystemAn introduction to the ruby ecosystem
An introduction to the ruby ecosystem
 
Install Guide
Install GuideInstall Guide
Install Guide
 

More from Hiroshi SHIBATA

Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Deep dive into Ruby's require - RubyConf Taiwan 2023
Deep dive into Ruby's require - RubyConf Taiwan 2023Deep dive into Ruby's require - RubyConf Taiwan 2023
Deep dive into Ruby's require - RubyConf Taiwan 2023Hiroshi SHIBATA
 
How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?Hiroshi SHIBATA
 
How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?Hiroshi SHIBATA
 
Ruby コミッターと歩む Ruby を用いたプロダクト開発
Ruby コミッターと歩む Ruby を用いたプロダクト開発Ruby コミッターと歩む Ruby を用いたプロダクト開発
Ruby コミッターと歩む Ruby を用いたプロダクト開発Hiroshi SHIBATA
 
Why ANDPAD commit Ruby and RubyKaigi?
Why ANDPAD commit Ruby and RubyKaigi?Why ANDPAD commit Ruby and RubyKaigi?
Why ANDPAD commit Ruby and RubyKaigi?Hiroshi SHIBATA
 
RailsGirls から始める エンジニアリングはじめの一歩
RailsGirls から始める エンジニアリングはじめの一歩RailsGirls から始める エンジニアリングはじめの一歩
RailsGirls から始める エンジニアリングはじめの一歩Hiroshi SHIBATA
 
The details of CI/CD environment for Ruby
The details of CI/CD environment for RubyThe details of CI/CD environment for Ruby
The details of CI/CD environment for RubyHiroshi SHIBATA
 
Productive Organization with Ruby
Productive Organization with RubyProductive Organization with Ruby
Productive Organization with RubyHiroshi SHIBATA
 

More from Hiroshi SHIBATA (9)

Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Deep dive into Ruby's require - RubyConf Taiwan 2023
Deep dive into Ruby's require - RubyConf Taiwan 2023Deep dive into Ruby's require - RubyConf Taiwan 2023
Deep dive into Ruby's require - RubyConf Taiwan 2023
 
How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?
 
How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?How resolve Gem dependencies in your code?
How resolve Gem dependencies in your code?
 
Ruby コミッターと歩む Ruby を用いたプロダクト開発
Ruby コミッターと歩む Ruby を用いたプロダクト開発Ruby コミッターと歩む Ruby を用いたプロダクト開発
Ruby コミッターと歩む Ruby を用いたプロダクト開発
 
Why ANDPAD commit Ruby and RubyKaigi?
Why ANDPAD commit Ruby and RubyKaigi?Why ANDPAD commit Ruby and RubyKaigi?
Why ANDPAD commit Ruby and RubyKaigi?
 
RailsGirls から始める エンジニアリングはじめの一歩
RailsGirls から始める エンジニアリングはじめの一歩RailsGirls から始める エンジニアリングはじめの一歩
RailsGirls から始める エンジニアリングはじめの一歩
 
The details of CI/CD environment for Ruby
The details of CI/CD environment for RubyThe details of CI/CD environment for Ruby
The details of CI/CD environment for Ruby
 
Productive Organization with Ruby
Productive Organization with RubyProductive Organization with Ruby
Productive Organization with Ruby
 

Recently uploaded

Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessWSO2
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesManik S Magar
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsYoss Cohen
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Jeffrey Haguewood
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactivestartupro
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerAnchore
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024BookNet Canada
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationDianaGray10
 

Recently uploaded (20)

Accelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with PlatformlessAccelerating Enterprise Software Engineering with Platformless
Accelerating Enterprise Software Engineering with Platformless
 
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotesMuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
MuleSoft Online Meetup Group - B2B Crash Course: Release SparkNotes
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 
Infrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platformsInfrared simulation and processing on Nvidia platforms
Infrared simulation and processing on Nvidia platforms
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
Email Marketing Automation for Bonterra Impact Management (fka Social Solutio...
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactive
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey Hightower
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Women in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automationWomen in Automation 2024: Career session - explore career paths in automation
Women in Automation 2024: Career session - explore career paths in automation
 

The integration for package ecosystem and future plans

  • 1. The integration for package ecosystem Hiroshi SHIBATA / GMO Pepabo, Inc. 2019.07.26 RubyConf Taiwan 2019 The Future of library dependency management of Ruby
  • 3. Hiroshi SHIBATA @hsbt https://www.hsbt.org Executive Officer VP of Engineering Technical Director at GMO Pepabo, Inc. @pepabo
  • 4. self.introduce => { name: “SHIBATA Hiroshi”, nickname: “hsbt”, organizations: [“ruby”, “rubygems”, “asakusarb”, “pepabo”, …], commit_bits: [“ruby”, “rake”, “rubygems”, “bundler”, “rdoc”, “psych”, “ruby-build”, “railsgirls”, “railsgirls-jp”, …], sites: [“hsbt.org”, “ruby-lang.org”, “rubyci.org”, “railsgirls.com”, “railsgirls.jp”], }
  • 8.
  • 9. Agenda •How to use libraries on the Ruby language? •What are RubyGems and Bundler? •What’s the Gamification project? •The Challenge for Bundler Integration •The future plans for RubyGems 4.0 and Bundler 2.1 •The Roadmap for Ruby 3.0
  • 10. How to use libraries On the Ruby language? 1. 3/40min
  • 12. What’s the Standard library? • We called its “標準添付ライブラリ” in Japanese. • It needs to `require` difference from embedded libraries like String, Thread, etc. • It can be used without Bundler or RubyGems
  • 13. Classification of standard libraries Standard Libraries Default Gems Bundled Gems Pure Ruby 44 22 7 C extensions 12 16 0 This matrix shows number of standard libraries and their classifications in Ruby 2.6.
  • 14. Inside Default gems • The ruby core team can release default gems to the rubygems.org. You can install them via RubyGems. • Rubygems have a detection method for default gems. • Default gems are openssl, psych, json, etc… >> Gem.loaded_specs["did_you_mean"].default_gem? => false >> require 'openssl' => true >> Gem.loaded_specs["openssl"].default_gem? => true
  • 15. Inside Bundled gems • We bundled *.gem and unpacked files to tarball package for Bundled gems with `gems/bundled_gems`. • `make install` installed Bundled gem your box.
  • 16. What are RubyGems and Bundler? 2. 8/40min
  • 17. What’s rubygems? RubyGems is a package management framework for Ruby. • rubygems/rubygems.org: • The Ruby community's gem host. • rubygems.org is maintain by infrastructure team of rubygems. It is different team from rubygems cli team. • rubygems/rubygems: • Command line tool of rubygems • Rubygems are created by Seattle.rb
  • 18. What’s new in RubyGems 3 •I released RubyGems 3 at 19 Dec 2018 https://blog.rubygems.org/2018/12/19/3.0.0-released.html •This version dropped to support the old Ruby versions like 1.8 and 1.9 •RubyGems 3 have a lot of features and bugfixes.
  • 20. Do protect your account of RubyGems.org with 2-factor auth.
  • 21. What’s Bundler? •The vendoring tool of Ruby. •RubyGems couldn’t care dependency of Ruby libraries and isolate version managing with ruby process. •Bundler can do them with `Gemfile` # frozen_string_literal: true source "https://rubygems.org" git_source(:github) { |repo| "https://github.com/#{repo}.git" } gemspec # We need a newish Rake since Active Job sets its test tasks' descriptions. gem "rake", ">= 11.1"
  • 22. What’s new in Bundler 2? •There is no incompatible feature from Bundler 1.17.x. •We disabled the incompatible features like renaming `gems.rb` from `Gemfile` •They no longer support under the Ruby 2.2.
  • 24. Gemification for standard library https://bugs.ruby-lang.org/issues/5481 • We extracted stdlibs like net-telnet, xmlrpc, rake to bundled gems. • These are extracted under the https://github.com/ruby/ . And shipped on rubygems.org • Other gems are also extracted at the future.
  • 25. Pros of Gemification • Maintainers can release gem for bugfix, new feature independent with Ruby core. • Easily backport stable version from develop version. Ruby users can use new feature on stable version. • If upstream is available on GitHub, Ruby users easily send patch via Pull request.
  • 26. Cons of Gemification • Abandoned and complex dependency on rubygems and bundler. • Maintainers need to maintain ruby core and GitHub repositories both. • It’s hard to maintain compatibility with old ruby version.
  • 27. Default gems on Ruby 2.6 bigdecimal (default: 1.4.1) bundler (default: 1.17.2) cmath (default: 1.0.0) csv (default: 3.0.9) date (default: 2.0.0) dbm (default: 1.0.0) (snip) strscan (default: 1.0.0) sync (default: 0.5.0) thwait (default: 0.1.0) tracer (default: 0.1.0) webrick (default: 1.4.2) zlib (default: 1.0.0) Current status of Default gems on Ruby 2.6 I’m going to promote more the standard libraries to default gem at Ruby 2.7.0. after that we promote it to bundled gems.
  • 28. The Challenge for Bundler Integration 4. 20/40min
  • 29.
  • 30. Bundler Integration on RubyGems 2.7 • It disabled in Ruby 2.5 because bundler is not part of standard library. • You can enabled it with only `gem update --system` if USE_BUNDLER_FOR_GEMDEPS ENV["BUNDLE_GEMFILE"] ||= File.expand_path(path) require 'rubygems/user_interaction' Gem::DefaultUserInteraction.use_ui(ui) do require "bundler" @gemdeps = Bundler.setup Bundler.ui = nil @gemdeps.requested_specs.map(&:to_spec).sort_by(&:name) end else rs = Gem::RequestSet.new @gemdeps = rs.load_gemdeps path rs.resolve_current.map do |s| s.full_spec.tap(&:activate) end end
  • 31. Merged Bundler into ruby core 😤
  • 32. The bundler finder issue of Heroku •https://github.com/heroku/heroku-buildpack-ruby/pull/850 •Heroku platform only uses version 1 of Bundler like 1.17.x. But Bundler version finder of RubyGems detects Bundler 1 or 2 from your Gemfile.lock. @schneems fixes this issue on heroku. •When You use Gemfile.lock updated by Bundler 2 with `bundle update -- bundler`, Heroku reject your app. Now you can use Ruby 2.6 and Bundler 2 on heroku. BLESSED_BUNDLER_VERSIONS = {} BLESSED_BUNDLER_VERSIONS["1"] = "1.15.2" BLESSED_BUNDLER_VERSIONS["2"] = "2.0.1"
  • 33. The path injection for LOAD_PATH issue •https://bugs.ruby-lang.org/issues/15469 •After that, You can’t use the specified version of gems like json or psych. It activates the versions of default gems provided by ruby core. - “/Users/user-name/.rbenv/versions/2.5.3/lib/ruby/gems/2.5.0/gems/bundler-1.17.2/lib" - “/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash” - "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.5.0/gems/json-1.8.6/lib" - (snip) - "/Users/user-name/.rbenv/versions/2.6.0/lib/ruby/2.6.0" - "/Users/user-name/.rbenv/rbenv.d/exec/gem-rehash" - "/Users/user-name/temp/aiueo/vendor/bundle/ruby/2.6.0/gems/json-1.8.6/lib" - (snip)
  • 34. The current behavior of the bundled bundler •It integrates with default gems like json, psych. •The upstream is https://github.com/ bundler/bundler. I backport the released/developed version to ruby repository. •Ruby 2.6 always enabled Bundler gem_deps now(New!) ~ > gem list | rg default: bigdecimal (1.4.3, default: 1.4.2) bundler (2.0.1, default: 1.17.3) cmath (default: 1.0.0) csv (3.0.6, default: 3.0.4) (snip) thwait (default: 0.1.0) tracer (default: 0.1.0) webrick (default: 1.4.2) zlib (default: 1.0.0)
  • 35. Ruby 2.6.3 is the best version ever
  • 36. The future plans For RubyGems 4.0 And Bundler 2.1 5. 28/40min
  • 37. RubyGems 4 • Make enable as default for conservative option: https:// github.com/rubygems/rubygems/pull/2233 • Removed duplicated code and files. • Make ruby gem install to user-install by default: https:// github.com/rubygems/rubygems/issues/1394 • Activation issues with default gems.
  • 38. Make conservative option as default • We got the installation time when already installed gems. • To use conservative is ignore re-install action. ~ > gem i rails Successfully installed rails-5.2.0 1 gem installed ~ > gem i rails ——conservative ~ >
  • 39. Dependency Resolver incompatible • RubyGems 2.x and 3.x uses Molinillo-0.5.7 • Bundler 1.x and 2.x also uses Molinillo-0.6.4 • These are different versions and behavior of dependency resolver. ~/D/g/r/rubygems (master) > ls lib/rubygems/resolver/molinillo/lib/molinillo delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb dependency_graph errors.rb modules resolver.rb ~/D/g/b/bundler (master) > ls lib/bundler/vendor/molinillo/lib/molinillo compatibility.rb dependency_graph errors.rb modules resolver.rb delegates dependency_graph.rb gem_metadata.rb resolution.rb state.rb
  • 40. Make `--user-install` as default • RubyGems 4 will install the all gems to `~/.gem` maybe. • Pros: Ruby in linux distribution has many of FAQ for gem installation for using `sudo`. This change resolve this issues. • Cons: Ruby version manager like rbenv is not support it. And RubyGems have a lot of issues related this.
  • 41. Activation issues about default gems •You couldn’t use the specified version of default gems like json when RubyGems/Bundler activated them. •When rubygems uses json-2.1.0, You couldn’t use json 1.8.x. Because ruby gems and rubygems.org never uses JSON format. •We can resolve it with `vendoring` approach. But json, psych, and openssl is C extension library.
  • 42. We always welcome your patch.
  • 43. Roadmap for Ruby 3.0 6. 33/40min
  • 45. Support JRuby and TruffleRuby •Surprisedly, RubyGems and Bundler never test JRuby and TruffleRuby in CI. •We try to add JRuby and TruffleRuby to Travis or other CI environments. •To JRuby and TruffleRuby tam: Please join us for this support.
  • 46. RubyGems/Bundler integration •Now, We put the bundler as submodule in rubygems repository. •We will move the canonical repository of bundler to rubygems org or rubygems/ rubygems.
  • 47. Bump up RubyGems/Bundler •We will merge into RubyGems 3.2 and Bundler 2.1 into Ruby 2.7.0. After that, RubyGems 4.0 will be merge Ruby 3. Ruby Bundler RubyGems 2.7.0 3.02.7-rcX 3.1 2.0 3.0 2.1 3.2 3.0? 4.0 ?
  • 50. The features of Ruby 2.7.0 •Compaction GC by tenderlove •Pattern Matching by k_tsj •Next generation IRB with reline by aycabta
  • 51. Gamification on Ruby 3.0(TBD) base64 benchmark cgi digest English erb fileutils find io/console monitor net/http net/https openssl optparse pathname pp rbconfig resolv set shellwords socket stringio strscan tempfile thread time timeout tmpdir tsort uri webrick Win32API zlib We will extract the standard libraries to the bundled gems.
  • 53.
  • 54. We have a many of contributors
  • 55. Ruby is designed to make programmers happy. Yukihiro Matz Matsumoto