Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
KUBERNETES +
DOCKER +
ELIXIR =
KUBERNETES CAN'T TERMINATE POD
FOR 22 DAYS
sudo systemctl restart docker
did nothing
ktl delete pod ... --force --grace-period=0
kept containers running behind the scenes
docker stop COINTAINER_ID
docker kill COINTAINER_ID
did not stop containers either. (Docker bug?)
NOT ENOUGH
RESOURCES?
TOP
CPU UTILIZATION IS LOW TOO
2019-08-19T01:27:19.462500954Z [ERROR] Failed to restore
container ... with containerd: error getting containerd
task for ...
MEASURE YOUR PIDS
# This probe is used once a while to check that container is ready to join the cluster
readinessProbe:
initialDelaySeconds...
$ ps -ef | awk '{ print $8 $9 $10; }' | sort | uniq -c
...
4845 [epmd]<defunct>
4840 [erl_child_setup]<defunct>
3 [ext4-rs...
PID 1
USER PID PPID STAT COMMAND COMMAND
default 1 0 S beam.smp {beam.smp} /opt/imaginator/releases/0.4.13/imaginator.sh -...
@@ -70,6 +70,10 @@ ENV REPLACE_OS_VARS=true 
# Bash is required by Distillery
RUN apk add --update --no-cache bash
+ # Tin...
USER PID PPID STAT COMMAND
default 1 0 S /sbin/tini -- /bin/sh -c bin/${APPLICATION_NAME} foreground
default 6 1 S {beam.s...
THANK YOU!
QUESTIONS?
ALEXEI SHOLIK
ANDREW DRYGA
Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga | Elixir Club Ukraine
Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga | Elixir Club Ukraine
Upcoming SlideShare
Loading in …5
×
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

Download to read offline

Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga | Elixir Club Ukraine

Download to read offline


Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga

Slides of Alexei Sholik, Andrew Dryga, at Lightning Talk session at Elixir Club Ukraine, Kyiv, 28.09.2019
Next conference - http://www.elixirkyiv.club/


Follow us on social networks @ElixirClubUA and #ElixirClubUA
Announce and materials from conf - https://www.fb.me/ElixirClubUA
News - https://twitter.com/ElixirClubUA
Photo and free atmosphere - https://www.instagram.com/ElixirClubUA
*Organizer’s channel - https://t.me/incredevly

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga | Elixir Club Ukraine

  1. 1. KUBERNETES + DOCKER + ELIXIR =
  2. 2. KUBERNETES CAN'T TERMINATE POD FOR 22 DAYS
  3. 3. sudo systemctl restart docker did nothing
  4. 4. ktl delete pod ... --force --grace-period=0 kept containers running behind the scenes
  5. 5. docker stop COINTAINER_ID docker kill COINTAINER_ID did not stop containers either. (Docker bug?)
  6. 6. NOT ENOUGH RESOURCES?
  7. 7. TOP
  8. 8. CPU UTILIZATION IS LOW TOO
  9. 9. 2019-08-19T01:27:19.462500954Z [ERROR] Failed to restore container ... with containerd: error getting containerd task for container: OCI runtime state failed: runc did not terminate sucessfully: runtime/cgo: pthread_create failed: Resource temporarily unavailablenSIGABRT: abort
  10. 10. MEASURE YOUR PIDS
  11. 11. # This probe is used once a while to check that container is ready to join the cluster readinessProbe: initialDelaySeconds: 40 periodSeconds: 15 timeoutSeconds: 30 failureThreshold: 6 exec: command: - /opt/talkinto_domain/bin/talkinto_domain - ping # This probe is used once a while to check that container is healthy livenessProbe: initialDelaySeconds: 30 periodSeconds: 30 timeoutSeconds: 30 failureThreshold: 2 exec: command: - /opt/talkinto_domain/bin/talkinto_domain - ping
  12. 12. $ ps -ef | awk '{ print $8 $9 $10; }' | sort | uniq -c ... 4845 [epmd]<defunct> 4840 [erl_child_setup]<defunct> 3 [ext4-rsv-conver] 7200 [inet_gethost]<defunct> ...
  13. 13. PID 1 USER PID PPID STAT COMMAND COMMAND default 1 0 S beam.smp {beam.smp} /opt/imaginator/releases/0.4.13/imaginator.sh -Bd -SDio 100 -- ... default 137 1 Z bash [bash] default 144 1 Z epmd [epmd] default 145 1 S epmd /usr/local/lib/erlang/erts-10.4.4/bin/epmd -daemon default 149 1 S erl_child_setup erl_child_setup 1048576 default 268 0 S bash bash default 273 268 R ps ps -o user,pid,ppid,stat,comm,args
  14. 14. @@ -70,6 +70,10 @@ ENV REPLACE_OS_VARS=true # Bash is required by Distillery RUN apk add --update --no-cache bash + # Tini is used as the entrypoint to reap zombie processes. + # This is the same as using `docker run --init`. + RUN apk add --no-cache tini + @@ -85,6 +89,8 @@ WORKDIR /opt/${APPLICATION_NAME} # Change user to "default" to limit runtime privileges USER default + ENTRYPOINT ["/sbin/tini", "--"]
  15. 15. USER PID PPID STAT COMMAND default 1 0 S /sbin/tini -- /bin/sh -c bin/${APPLICATION_NAME} foreground default 6 1 S {beam.smp} /opt/talkinto_domain/releases/0.8.173/talkinto_domain.sh -Bd -SDio 100 -- ... default 146 1 S /usr/local/lib/erlang/erts-10.4.4/bin/epmd -daemon default 150 6 S erl_child_setup 1048576 default 259 150 S inet_gethost 4 default 260 259 S inet_gethost 4 default 261 259 S inet_gethost 4 default 262 259 S inet_gethost 4 default 10802 0 R ps -o user,pid,ppid,stat,args
  16. 16. THANK YOU! QUESTIONS? ALEXEI SHOLIK ANDREW DRYGA

Kubernetes + Docker + Elixir - Alexei Sholik, Andrew Dryga Slides of Alexei Sholik, Andrew Dryga, at Lightning Talk session at Elixir Club Ukraine, Kyiv, 28.09.2019 Next conference - http://www.elixirkyiv.club/ Follow us on social networks @ElixirClubUA and #ElixirClubUA Announce and materials from conf - https://www.fb.me/ElixirClubUA News - https://twitter.com/ElixirClubUA Photo and free atmosphere - https://www.instagram.com/ElixirClubUA *Organizer’s channel - https://t.me/incredevly

Views

Total views

529

On Slideshare

0

From embeds

0

Number of embeds

7

Actions

Downloads

3

Shares

0

Comments

0

Likes

0

×