Oracle Linux is expanding its share in the enterprise market. Not only as a RHEL compatible OS, it has variety of functions that will interest enterprise system admins. Ksplice is one of its attractive function, in this session we will show you how life is going to be easier patching security vulnerability without rebooting system using Ksplice. This presentation was delivered at LinuxCon Japan 2016 by Fumiyasu Ishibashi

1. Introducing Oracle Linux and
Securing it with Ksplice
July 14 2016
Oracle Japan
Global Business Unit
Oracle Linux and Oracle VM Sales
Principal Sales Consultant
Fumiyasu Ishibashi

2. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Safe
Harbor
Statement
The
following
is
intended
to
outline
our
general
product
direcFon.
It
is
intended
for
informaFon
purposes
only,
and
may
not
be
incorporated
into
any
contract.
It
is
not
a
commitment
to
deliver
any
material,
code,
or
funcFonality,
and
should
not
be
relied
upon
in
making
purchasing
decisions.
The
development,
release,
and
Fming
of
any
features
or
funcFonality
described
for
Oracle’s
products
remains
at
the
sole
discreFon
of
Oracle.
2

3. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Agenda
• Summary
of
Oracle
Linux
• Live
patching
with
Ksplice
3

4. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Summary
of
Oracle
Linux

5. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Introducing Oracle Linux
5
Long history
Linux support from 1998
Oracle distro 2006
Live patching for Kernel and userpace
process
One stop
support
24x7
Supports exisiting
RHEL and CentOS
Dtrace,
OCFS2,
Clusterware・・・
Includes support
for many Oracle softwares
https://linux.oracle.com
Oracle
Linux
Premium
Support
Sustaining
Support
Oracle Linux
Extended Support
10年
1年
1年
1年
無期限
RedHat
Compatible
Kernel
Same glibc
UEK(Unbreakable
Enterprise Kernel)
Non-Oracle Hardware supported
Free to download
Free to use
Completely opensource
Oracle Standard
You can chose
the kernel
100%
Binary
compatible
Oracle Linux
Support
Endless support

6. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Oracle
Linux
Support
type
6
Ksplice
support
Oracle
Enterprise
Manager
free
of
use
and
support
Oracle
Clusterware
free
of
use
and
support
24x7
online
and
phone
support
Downloading
patch,
fixes,
erratas
Dtrace
support
Oracle
OpenStack
for
Oracle
Linux
support
LifeFme
Sustaining
Support
Oracle
Linux
Premier
Support
Oracle
Linux
Basic
Support
Login
account
for
ULN
Oracle
Linux
Network
Support
Spacewalk
support

7. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Oracle
Linux
SubscripFon
Pricing
• Buy
support
for
the
systems
you
need
–
use
the
same
soware
with
updates
on
everything!
• Oracle
only
counts
physical
sockets;
no
limit
on
cores
or
number
of
virtual
guests
Level
Price
Installable
binaries
and
errata
Free
Basic
Limited
(24x7,
unlimited
support)
(2
or
less
CPUs)
$499
Basic
(24x7,
unlimited
support)
(More
than
2
CPUs)
$1,199
Premier
Limited
(24x7,
unlimited
support)
(2
or
less
CPUs)
$1,399
Premier
(24x7,
unlimited
support)
(More
than
2
CPUs)
$2,299

8. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Unbreakable
Linux
Network
(ULN)
h7ps://linux.oracle.com
8
Oracle version of RHN
Portal site for Oracle Linux.
Download rpm packages.
Unbreakable Linux Network User‘s Guide
• How
to
register
your
server
to
ULN
• How
to
setup
a
ULN
mirror
site
(English)
https://docs.oracle.com/cd/E37670_01/E39381/html/index.html
（Japanese）
https://docs.oracle.com/cd/E39368_01/b72803/index.html
Switching from RHN to ULN
https://linux.oracle.com/switch.html
Free to use our public yum repo
http://public-yum.oracle.com/

9. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Oracle
Linux
security
informaFon
on
ULN
• Searching
Erratas,
CVEs
– hep://linux.oracle.com/errata/
– hep://linux.oracle.com/cve/
• New
erratas
announced
through
the
mailing
list
– heps://oss.oracle.com/mailman/lisFnfo/el-­‐errata
9

10. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Live
patching
with
Ksplice

11. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Zero downtime patching
Patching without rebooting the OS, services.
Not only the kernel but also the userspace
application like,
glibc and openssl
Rollback
If something goes wrong with the new patch, you can
rollback where the apps were fine!
Also used for support, putting the debug kernel
temporary.
Fast errata release
Since the patching data is complete under oracles
control we provide the fully tested patches as fast
as we can
Oracle
ConfidenFal
–
Internal/Restricted/Highly
Restricted
11
Proven history
Released from 2008
Joined Oracle from 2011

12. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Benefits from Ksplice
Vulnerability
Easier to patch vulnerability issues
Reducing administration work
No more maintenance plan for patching. It can
also automatically patch instead of you.
Easier to solve problems
In some case our support team will give you
a Ksplice debug kernel patch so our support
can collect more information to find the
problem you have.
Of course witout reboot
Security Compliant
It will be easier to be security compliant if you don’t
need wait for pathing security fixes
Oracle
ConfidenFal
–
Internal/Restricted/Highly
Restricted
12

13. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Using
Ksplice
on-­‐line
or
off-­‐line
• Need
Oracle
Linux
Premiumer
Support
13
Ksplice
Server
Ksplice
Client
internet
ULN
Ksplice
Server
Proxy
internet
ULN
Ksplice
Client
Ksplice
Server
ULN
Mirror
(local
yum)
internet
ULN
Ksplice
Client
Ksplice
Server
ULN
Mirror
(local
yum)
internet
ULN
Ksplice
Client
(local
yum)
copy
Connect your server to the ULN via Proxy
Offline from the local ULN repovia ULN mirror

14. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice Technology
memory
NewBug
memory
Before ksplice
Bug
After Ksplice patching
①
Insert jump to
②
③
④
⑤

15. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Inspector
• heps://ksplice.oracle.com/inspector
• Validate
the
patch
level
of
your
kernel;
Apply
the
patches
you
need
15

16. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Desktop
hep://ksplice.oracle.com/try/desktop
• Free
of
charge
• No
support
• Ubuntu 16.04 Xenial
• Ubuntu 15.10 Wily
• Ubuntu 15.04 Vivid
• Ubuntu 14.04 LTS Trusty
• Ubuntu 12.04 LTS Precise
• Fedora
22
• Fedora
23
• Fedora
24
16

17. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
GUI(Ubuntu
and
Fedora
only)
17

18. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
30
days
trial
hep://ksplice.oracle.com/try/trial
• Easy
register
and
use
it
for
30
days
• RHEL
5,6,7
and
Oracle
Linux
5,6,7
supported
18

19. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Easy
installaFon
• Get
ULN
account(trial
or
Premier
support)
• Register
your
server
to
ULN
• Add
ksplice
channel
subscripFon
to
your
server
from
the
ULN
web
site.
• Install
uptrack
• Done.
No
reboot.
19
# yum install -y uptrack
*
You
can
also
uninstall
it

20. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Command
Line
Tools
(1/4)
• uptrack-­‐show
command
• List
the
kernel
patches
that
is
applied
uptrack-­‐show
# uptrack-show
Installed updates:
[guclwyc2] CVE-2012-0957: Information leak in uname syscall.
[j4d07e02] Kernel panic in IPv4 ARP and IPv6 Neighbor Discovery.
[r8og1ec4] CVE-2013-1979: Privilege escalation with UNIX socket credentials.
#
# uptrack-show --available
Available updates:
[fiq04xbb] CVE-2013-2237: Information leak on IPSec key socket.
[9q4luou3] CVE-2014-3687: Remote denial-of-service in SCTP stack.
#
• With
the
–available opFon,
you
can
find
the
patches
that
are
available.
Ksplice
ID

21. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Command
Line
Tools
(2/4)
• uptrack-­‐upgrade
command
Command
to
apply
all
patches
that
are
available.
uptrack-­‐upgrade/uptrack-­‐install
# uptrack-upgrade -y
The following steps will be taken:
Install [guclwyc2] CVE-2012-0957: Information leak in uname syscall.
Install [j4d07e02] Kernel panic in IPv4 ARP and IPv6 Neighbor Discovery.
Install [r8og1ec4] CVE-2013-1979: Privilege escalation with UNIX socket credentials.
Install [fiq04xbb] CVE-2013-2237: Information leak on IPSec key socket.
Install [9q4luou3] CVE-2014-3687: Remote denial-of-service in SCTP stack.
Installing [guclwyc2] CVE-2012-0957: Information leak in uname syscall.
Installing [j4d07e02] Kernel panic in IPv4 ARP and IPv6 Neighbor Discovery.
Installing [r8og1ec4] CVE-2013-1979: Privilege escalation with UNIX socket credentials.
Installing [fiq04xbb] CVE-2013-2237: Information leak on IPSec key socket.
Installing [9q4luou3] CVE-2014-3687: Remote denial-of-service in SCTP stack.
Your kernel is fully up to date.
Effective kernel version is 2.6.39-400.215.13.el6uek
#
# uptrack-upgrade guclwyc2 -y
The following steps will be taken:
Install [guclwyc2] CVE-2012-0957: Information leak in uname syscall.
Installing [guclwyc2] CVE-2012-0957: Information leak in uname syscall.
Your kernel is fully up to date.
#
• uptrack-install <Ksplice ID> will
make
you
apply
to
a
specific
patch
level

22. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Command
Line
Tools
(3/4)
• uptrack-remove command
You
can
remove
all
the
patches
applied
by
Ksplice.
uptrack-­‐remove
# uptrack-remove -y
The following steps will be taken:
Remove [guclwyc2] CVE-2012-0957: Information leak in uname syscall.
Remove [j4d07e02] Kernel panic in IPv4 ARP and IPv6 Neighbor Discovery.
Remove [r8og1ec4] CVE-2013-1979: Privilege escalation with UNIX socket credentials.
Remove [fiq04xbb] CVE-2013-2237: Information leak on IPSec key socket.
Remove [9q4luou3] CVE-2014-3687: Remote denial-of-service in SCTP stack.
#
# uptrack-show
Installed updates:
None
#
• uptrack-remove <Ksplice ID>
You
can
also
can
rollback
to
the
level
you
want
# uptrack-remove –y 9q4luou3
The following steps will be taken:
Remove [9q4luou3] CVE-2014-3687: Remote denial-of-service in SCTP stack.
#

23. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Command
Line
Tools
(4/4)
• The
uname
command
will
output
the
kernel
version
that
is
on
the
disk.
To
see
the
Ksplice
kernel
patch
level
use
uptrack-uname.
uptrack-­‐uname
# uptrack-show
Installed updates:
None
# uname -r
2.6.39-300.26.1.el6uek.x86_64
# uptrack-uname -r
2.6.39-300.26.1.el6uek.x86_64
# uptrack-upgrade -y
The following steps will be taken:
Install [guclwyc2] CVE-2012-0957: Information leak in uname syscall.
Install [j4d07e02] Kernel panic in IPv4 ARP and IPv6 Neighbor Discovery.
...
Installing [fiq04xbb] CVE-2013-2237: Information leak on IPSec key socket.
Installing [9q4luou3] CVE-2014-3687: Remote denial-of-service in SCTP stack.
Your kernel is fully up to date.
Effective kernel version is 2.6.39-400.215.13.el6uek
# uname -r
2.6.39-300.26.1.el6uek.x86_64
# uptrack-uname -r
2.6.39-400.215.13.el6uek.x86_64
Before
appling
any
Ksplice
patch
ADer
appling
Ksplice
patch

24. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
configuraFon
file
• /etc/uptrack/uptrack.conf
24
• You can set proxy server,
https_proxy = https://proxy_URL:https_port
• If you want the patches automatically set yes,(default no). Ran by cron.
autoinstall = yes
• If you set “yes” the kernel will be on the same patch level before the reboot(default yes)
install_on_reboot = yes
• If you also want the new patches automatically applied after reboot set yes,(default no)
upgrade_on_reboot = yes

25. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Uptrack
API
Tools
• RESTful
web
API
• The
command
line
API
tools
are
included
with
the
Python
bindings
for
the
API
in
the
python-ksplice-uptrack
package.
• The
details
are
describe
in
our
sites.
– hep://ksplice.oracle.com/uptrack/api
– heps://docs.oracle.com/cd/E37670_01/E39380/html/ol_kspapi.html
25

26. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Enhanced
Client
• New
feature
from
2015
• Ksplice
Enhanced
client
can
patch
in-­‐memory
pages
of
Ksplice-­‐aware
shared
libraries.
• Currently
for
glibc
and
openssl
user-­‐space
processes
• Need
addiFon
packages
to
enable
Ksplice
Enhanced
client.
• Also
need
to
update
the
system
to
install
the
Ksplice-­‐aware
versions
of
the
user-­‐space
libraries:
26
# yum install -y ksplice
# yum update *glibc *openssl*

27. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Enhanced
Client
command
(1/3)
27
# ksplice all list-targets
User-space targets:
glibc-ISO8859-1-2.17.78.0.1.1.ksplice25.el7
└─ gnome-shell (3783)
glibc-libutil-2.17.78.0.1.1.ksplice25.el7
├─ firewalld (680)
├─ tuned (695)
├─ libvirtd (1492)
├─ sshd (1497)
├─ httpd (1503)
├─ httpd (1706)
├─ httpd (1707)
├─ abrt-applet (3980)
├─ tracker-miner-f (4040)
├─ gvfsd-trash (4062)
├─ sshd (29328)
├─ packagekitd (29465)
└─ python (29679)
...
Kernel version: Linux/x86_64/3.10.0-229.el7.x86_64/#1 SMP Fri Mar 6 04:05:24 PST 2015
ksplice all list-targets command
display the running user-space processes that the client can patch
・・・

28. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Enhanced
Client
command
(2/3)
28
# ksplice all show
httpd (1706)
httpd (1708)
httpd (1707)
rsyslogd (689)
chronyd (705)
httpd (1503)
├─ [h73qvumn]: CVE-2014-7817: Command execution in wordexp().
└─ [ml55ngz4]: CVE-2015-1781: Privilege escalation in gethostbyname_r().
Ksplice kernel updates installed:
Installed updates:
[rfywob9d] Clear garbage data on the kernel stack when handling signals.
[6w5ho5e2] Provide an interface to freeze tasks.
ksplice all show command:
[89yjgn50] CVE-2015-3636: Memory corruption when unhashing IPv4 ping sockets.
[g327jyvw] CVE-2015-2922: Denial-of-service of IPv6 networks when handling router
advertisements.
・・・

29. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Ksplice
Enhanced
Client
command
(3/3)
29
# ksplice all show
httpd (1706)
httpd (1708)
httpd (1707)
rsyslogd (689)
chronyd (705)
httpd (1503)
├─ [h73qvumn]: CVE-2014-7817: Command execution in wordexp().
└─ [ml55ngz4]: CVE-2015-1781: Privilege escalation in gethostbyname_r().
Ksplice kernel updates installed:
Installed updates:
[rfywob9d] Clear garbage data on the kernel stack when handling signals.
[6w5ho5e2] Provide an interface to freeze tasks.
ksplice all show command:
[89yjgn50] CVE-2015-3636: Memory corruption when unhashing IPv4 ping sockets.
[g327jyvw] CVE-2015-2922: Denial-of-service of IPv6 networks when handling router
advertisements.
・・・

30. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
Demo
• Demo
environment
30
Ksplice
Server
Ksplice
Client
internet
ULN
VM
guest
（Virtual
Box）
This
PC
Oracle
Linux
6.2
vCPU
x1
RAM
4GB
Linux
Kernel
2.6.32-­‐220.el6.x86_64

31. Copyright
©
2015
Oracle
and/or
its
affiliates.
All
rights
reserved.
|
@ORCL_Linux
Facebook.com/
OracleLinux
Blogs.oracle.com/
linux
Oracle
Linux
Experts
Group
YouTube.com/
oraclelinuxchannel
Home
page:
oracle.com/linux
Ksplice
info:
ksplice.oracle.com
Download:
edelivery.oracle.com/linux
Learn
More
about
Oracle
Linux
Join
Our
CommuniPes,
Visit
Websites
For
More
InformaPon
ksplice-­‐support_ww@oracle.com