SatRday talk on running Rstudio servers inside Docker and Singularity on HPC, Cloud and on local machine.

1. HOW TO RUN RSTUDIO
SERVERS ANYWHERE WITH
CONTAINERS - HPC, CLOUD,
AND LOCALLY
Wendy Wong

2. Outline ▫ Bioinformatics at Inova
Translational Medicine Institute
▫ RStudio Server
▫ Container Solutions
▫ Running Rstudio Server
2

3. Bioinformatics at Inova
Translational Medicine
Institute
What do we do?
3

4. Bioinformatics
at Inova
Translational
Medicine
Institute
▫ Mostly Trio-based Whole Genome
Sequenced studies
▫ Preterm Birth study
▫ Childhood longitudinal study
▫ Congenital diseases study
▫ >8,000 whole genome sequences
integrated with clinical electronic
health record and survey data
▫ Other omics on mother: RNASeq,
miRNASeq, methylation
4

5. R and
Bioinformatics
5

6. RStudio Server
6

7. ● Provides a web browser
based interface to the version
of R running on the server
● Enables “chromebook data
science”
Place your screenshot here
7
Rstudio Server

8. Limitation of
RStudio
Server on
shared
computing
environment
▫ The RStudio server needs to be
initiated by root
▫ The user cannot change the version
of R it points to
▫ Each machine can only have one
server running
▫ Competing resources among users
8

9. Container solutions
Docker and Singularity
9

10. Containers
What and
Why?
What?
▫ Virtualization of the underlying
operating system
▫ Sharing the kernel of the host OS to
access the hardware.
Why?
▫ Lightweight
▫ Consistent Environment
▫ Platform independence
▫ Cost savings on resources
10

11. Docker
and Rocker
“Dockerfiles for different Docker
containers of interest to R users.”
11
https://github.com/rocker-org/rocker rocker

12. Singularity
▫ Designed for HPC
▫ No root owned daemon
processes
12
You don’t have to ask
your system administrator
to install anything for you.

13. Docker vs
Singularity
13
Docker Singularity
Isolation:
User inside container
cannot see hosts’
processes
Integration:
User has the same user id
inside the container
Image as layers - diffs from
the parent image
Image is a single file
Docker hub for portability:
Separate nodes cannot
share same images
Singularity hub or share
the same singularity file for
portability
Containers starts with
privileged access
Containers can be started
as normal user

14. Running RStudio Server
Examples with HPC (PBSPro), Google Cloud Shell and locally
14

15. Getting the
RStudio
Server
image
Docker (Rocker):
$ docker pull rocker/tidyverse
Singularity:
$ singularity pull docker://rocker/tidyverse
$ singularity pull
shub://nickjer/singularity-rstudio:3.5.1
$ sudo singularity build rstudio.sif rstudio.def
https://www.sylabs.io/2018/10/rstudio-ide-and-server/
15

16. Running
RStudio
Server on
HPC
With job
scheduler
(PBSPro)
Why?
▫ Reserve resources for R to run
efficiently
▫ Keeps compute nodes being
overloaded
▫ Run different R versions concurrently
16

17. RStudio server on PBSPro: A small example
https://www.sylabs.io/2018/10/rstudio-ide-and-server/
17
[user@remotehost] $cat runRStudio.sh
module load singularity/3.0
singularity instance start -B
<path>/rserver.conf.8790:/etc/rstudio/rserver.conf
<path>/rstudio.sif rs
[user@remotehost] $cat <path>/rserver.conf.8790
www-port=8790
[user@remotehost] $qsub -l select=1:ncpus=16:mem=128gb
runRStudio.sh

18. RStudio server on PBSPro: A small example
https://www.sylabs.io/2018/10/rstudio-ide-and-server/
18
[user@localhost]$ ssh wongwe@remotehost -L8790:localhost:8790

19. 19
RStudio
RServer on
the cloud
for free

20. 20
Web preview

21. Commands on Google cloud shell
21
wendy_wong@cloudshell:~
(ivory-period-88317)$ docker pull rocker/rstudio
wendy_wong@cloudshell:~
(ivory-period-88317)$ docker run -d -p
8080:8787 -e PASSWORD=<mypasswd>
--name rstudio rocker/rstudio

22. 22

23. Running
RStudio
Server
locally
Singularity
▫ Use the same singularity images for
reproducibility
Docker
▫ docker commit
▫ docker push
▫ docker pull
23

24. Tips and
lessons
learned
Reproducibility
Running Rstudio Server on
containers helps with
reproducibility. Make sure tag
info is included (otherwise
latest is pulled).
Security
User does need to have root
privilege to run Singularity
containers - much easier to
convince your sys admins to
install Singularity!
Debug
Bind the log directory of
RStudio Server to the host
system in case it crashes.
24
Home directory
sharing
By default, Singularity shares
home directory. Be aware if
RStudio .Rprofile is there.
Environmental
Variables
By default, Singularity shares
the host environmental
variables.
Creating Images
Might be easier to create a
docker image and bootstrap
from it to create the
Singularity image

25. 25
Thanks!
Questions and comments?
You can find me at
wendy.wong@gmail.com
https://www.linkedin.com/in/wendyswwong/
Resources:
https://www.docker.com
https://www.sylabs.io
https://www.rstudio.com/products/rstudio/#Server