20160520 暨南大學 QNAP應用程式設計授課教材 Develop QNAP NAS App by Docker

1. Develop QNAP NAS App
by Docker
Terry Chen 陳彥村
Engineer @ QNAP

2. Who am I?
https://tw.linkedin.com/in/seterrychen
https://github.com/seterrychen
https://twitter.com/seterrychen
● Backend engineer : Java, Python, Linux
● QNAP Container Station/Linux Station

3. Outline
● How to develop QNAP NAS App
● Base knowledge of Docker
● Create Docker App on QNAP NAS in QPKG format

4. NAS － Network attached storage
From : http://www.slideshare.net/QNAP_Inc/general-tech-trainingstorageabcqnap20140302compressed-copy
Network

5. QNAP AppCenter
● Customize your QNAP NAS with Apps

6. QPKG
● QNAP package, like deb(Debian/Ubuntu),
rpm(Red Hat/CentOS)
● QNAP tool-chain:
https://sourceforge.net/projects/qosgpl/files/QNAP%20NAS%20Tool%20Chains/
● QPKG format:
[/share/CACHEDEV3_DATA/.qpkg/QDK/tester] # ls
arm-x09/ arm-x31/ config/ package_routines shared/
x86_64/ arm-x19/ arm-x41/ icons/
qpkg.cfg x86/ x86_ce53xx/

7. Package application by QDK/QDK2
● QDK
http://www.qnap.com.tw/event/dev/cht/p_qdk.php
● QDK2
https://github.com/qnap-dev/qdk2
● QDK2 with Docker
https://github.com/fcwu/docker-qdk2

8. QDK
http://download.qnap.com/dev/QDK_Quick_Start_Guide_v4_eng.pdf

9. QDK2
● https://github.com/qnap-dev/qdk2
● Debian/Ubuntu base tool
● Support various source

10. Docker review

11. What is Docker?
● Docker allows you to package an application
with all of its dependencies into a standardized
unit for software development.
Others
App
image

12. Images center - Docker Hub

13. How to ship application by Docker
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
nginx:1.10
Image
Layer
$ docker pull nginx:1.10

14. Docker storage (version: 1.9.1)
vagrant@test:~$ sudo docker info
Containers: 0
Images: 2
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
………
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Nginx:1.10
vagrant@test:~$ sudo ls -l /var/lib/docker/aufs/diff
total 32
drwxr-xr-x 2 root root 4096 May 1 07:40 0b10aa9d5e8a3dd8de176539bd...
drwxr-xr-x 2 root root 4096 May 1 07:39 2a6ec6bae79235ea27adcceff6...
drwxr-xr-x 2 root root 4096 May 1 07:39 2eda4505e518d08147e4fab915...
drwxr-xr-x 2 root root 4096 May 1 07:15 32f2a4cccab87ff519e17c9cf4...
drwxr-xr-x 21 root root 4096 May 1 07:15 70e9a6907f1068b3e08a9e6357...
drwxr-xr-x 2 root root 4096 May 1 07:40 892607a8d00ba2ad196a7d8a43...
drwxr-xr-x 7 root root 4096 May 1 07:40 9c6eed7ce0e9c7f8ab8e95d2f7...
drwxr-xr-x 3 root root 4096 May 1 07:40 f4e6d72dbf272d7e2ba0ef48f7...

15. Docker storage (version: 1.9.1)
vagrant@test:~$ sudo docker info
Containers: 0
Images: 2
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
………
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Nginx:1.10
root@test:/var/lib/docker/aufs/diff/70e9a6907f1068b3e08a9e...# ls
bin boot dev etc home lib lib64 media mnt opt
proc root run sbin srv sys tmp usr var
root@test:/var/lib/docker/aufs/diff/9c6eed7ce0e9c7f8ab8e95...# ls
etc lib tmp usr var

16. AUFS – Another Union File System
From : https://docs.docker.com/engine/userguide/storagedriver/aufs-driver/

17. Run container (version: 1.9.1)
$ docker run -d -p 80:80 nginx:1.10
Docker
Client
Docker
Daemon
Send Request
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Read/Writer Container Layer
Nginx:1.10
Send Response
Image read only
Simplified diagram

18. copy-on-write strategy
892607a8d00b 0 B
0b10aa9d5e8a 0 B
f4e6d72dbf27 22 B
9c6eed7ce0e9 57.66 MB
2eda4505e518 0 B
2a6ec6bae792 0 B
32f2a4cccab8 0 B
70e9a6907f10 125.1 MB
Read/Writer
Image read only
Nginx:1.10
Read/Writer
Read/Writer
Container-1
Container-2
Container-3

19. Basic isolation of Linux
● chroot – change root directory
root@test:/var/lib/docker/aufs/diff/70e9a6907f1068b3e08a9e...# ls
bin boot dev etc home lib lib64 media mnt opt
proc root run sbin srv sys tmp usr var
root@test:/var/lib/docker/aufs/diff/70e9a6907f1068b3e08a9e...# chroot .
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
root@test:/# ls -al
bin boot dev etc home lib lib64 media mnt opt
proc root run sbin srv sys tmp usr var
root@test:/# ls -al ../
bin boot dev etc home lib lib64 media mnt opt
proc root run sbin srv sys tmp usr var

20. Basic isolation of Linux
● chroot – change root directory
Filesystem Isolation
root@test:/# mount -t proc proc /proc
vagrant@test:~$ ps -ef
root 2192 906 0 02:54 ? 00:00:00 docker-proxy -proto ...
root 2215 906 0 02:54 ? 00:00:00 docker-proxy -proto ...
......
vagrant 2275 1915 0 03:03 pts/5 00:00:00 ps -ef
# using chroot
root@test:/# ps -ef
root 2192 906 0 02:54 ? 00:00:00 docker-proxy -proto ...
root 2215 906 0 02:54 ? 00:00:00 docker-proxy -proto ...
......
root 2277 2269 0 03:03 ? 00:00:00 ps -ef

21. How to isolate by Docker
● Linux Kernel feature:
– Cgroups
– Namespaces
● Mount namespaces
● UTS namespaces
● IPC namespaces
● PID namespaces
● Network namespaces
● User namespaces

22. Process namespace
From : http://dockone.io/article/783

23. Stop container
From : http://dockone.io/article/783 From:
https://twitter.com/turnoff_us/status/710736561676132352/photo/1

24. Docker on Linux
From : https://en.wikipedia.org/wiki/Docker_(software)
libcontainer
libvirt
systemd-
nspawnLXC
cgroups namespaces
SELinux
AppArmor
Netfilter
Netlink
capabilities
Docker
Linux kernel

25. VM vs Container
From : https://www.docker.com/what-docker

26. The power of Docker
From:https://yq.aliyun.com/articles/32071

27. Back to docker-qdk2
https://github.com/fcwu/docker-qdk2

28. Dockerize App by Dockerfile
From : https://github.com/nginxinc/docker-nginx/blob/41aa13f7d2c24407e483c40fb1e8b33e73462ff1/mainline/jessie/Dockerfile

29. Build and Ship
● Build
– docker build -t your_account/app_name .
● Ship
– docker login --username=yourname
--email=youremail@company.com
– docker push your_account/app_name

30. Start/Stop script
#!/bin/sh
...
case "$1" in
start)
ENABLED=$(/sbin/getcfg $QPKG_NAME Enable -u -d FALSE -f $CONF)
if [ "$ENABLED" != "TRUE" ]; then
echo "$QPKG_NAME is disabled."
exit 1
fi
: ADD START ACTIONS HERE
;;
stop)
: ADD STOP ACTIONS HERE
;;
restart)
$0 stop
$0 start
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit 0

31. Example: ghost
● Just a blogging platform https://ghost.org
Build qpkg
$ git clone https://github.com/fcwu/docker-qdk2.git
$ cd docker-qdk2
$ docker run -it --rm
-v ${PWD}/example:/example
dorowu/qdk2 bash -c "cd /example; make"
$ ls -l example/ghost/build/ghost_0.7.4_x86_64.qpkg
-rw-r--r-- 1 u u 24242 Dec 30 13:10 example/ghost/build/ghost_0.7.4_x86_64.qpkg

32. Container Station API
● Version 1.6
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "install"}'
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "start"}'
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "stop"}'
- qbus post com.qnap.dqpkg/qpkg '{"qpkg": "online-document", "action": "remove"}'

33. docker-compose
● docker-compose.yml of Online-document
app:
image: dorowu/online-document.amd64:0.8.111
ports:
- "127.0.0.1:10001:5000"
volumes:
- /share:/share
- /mnt/rf/cd:/mnt/rf/cd
- /var/run/qbus.sock:/var/run/qbus.sock
● Read more: https://docs.docker.com/compose/

34. Apache config
● online-document.conf
ProxyPass /online-document/ws ws://127.0.0.1:10001/online-document/ws retry=0
ProxyPass /online-document/ http://127.0.0.1:10001/online-document/ retry=0

35. Create Issues/Pull Request
● https://github.com/fcwu/docker-qdk2
● https://github.com/qnap-dev/container-station-api

36. How to debug (In my case)
From : http://9gag.com/gag/anYZ9Eo/my-code-works-but-i-don-t-know-why

37. How to Learn (In my case)
“Try to learn something about everything and
everything about something.”
Thomas Huxley Quotes
Read more : 陈天：如何快速掌握一门技术？ (http://blog.jobbole.com/77712/)

38. Thank you