Presentation from a workshop delivered by Piotr Chaberski during PyData Warsaw on Oct. 18, 2017. Description Machine Learning is not only about algorithms. Machine learning is about value and this can be achieved only after proper deployment of Machine Learning solutions. I will present best practices regarding managing R based ML projects. Abstract Agenda: I will use our open-source tool R Suite (http://rsuite.io/). During the workshop I will talk about: project structure development cycle repository management deployment During the workshop you will learn about our best practices (e.g. loggers, version control, etc.) we have developed for 12 years of using R. Requirements: Basic R knowledge. Basic ML/DS knowledge. Software installed: R in version 4.3.2 R Suite (latest) R Studio Desktop Operating system: I will be using Windows 10 and this is recommended but Linux should also work.

1. Copyright (c) WLOG Solutions 1
Managing large scale
projects in R with
PyData workshop
Warsaw, 2017-10-18
Piotr Chaberski, WLOG Solutions

2. Copyright (c) WLOG Solutions 2
Historical
data
Traffic
forecasting
Workforce
optimiser
Business rules
Human intervention
Results
Efficiency
curves
Workforce optimisation
Example of a large scale project

3. Copyright (c) WLOG Solutions 3
Dev
Version control
Continuous
Integration &
Deployment
Prod
Software development process

4. Copyright (c) WLOG Solutions 4
Where does R have its rough edges?

5. Copyright (c) WLOG Solutions 5
CRAN (MRAN) Github Other
Dev environment
Installed packages
Local CRAN
Source code
repo
install.packages(“data.table”)

6. Copyright (c) WLOG Solutions 6
data.table

7. Copyright (c) WLOG Solutions 7
How to maintain dev/prod
consistency?

8. Copyright (c) WLOG Solutions 8
Stand-alone R solution
An example re-built using RSuite

9. Copyright (c) WLOG SolutionsCopyright (c) WLOG Solutions 9
Instruction (Windows only!)
1. Install vanilla R 3.4.2
2. Download package
(https://goo.gl/RnZAQg)
3. Unzip
4. Open CMD
5. Rscript R/master.R
--port=7137
6. In browser open
http://localhost:7137
http://www.sumsar.net/blog/2014/07/estimate-your-bac-using-drinkr/

10. Copyright (c) WLOG Solutions 10
Deployable zip contains all needed
packages and dependencies

11. Copyright (c) WLOG Solutions 11
R Software Development
R Suite walk-through

12. Copyright (c) WLOG Solutions 12
Preliminaries

13. Copyright (c) WLOG Solutions 13
Install R
https://cran.r-project.org/bin/windows/base/R-3.4.2-win.exe

14. Copyright (c) WLOG Solutions 14
Install RStudio
https://www.rstudio.com/products/rstudio/download/
Optional, but you really won’t regret it

15. Copyright (c) WLOG Solutions
Install R Suite CLI
15
http://rsuite.io/RSuite_Download.php

16. Copyright (c) WLOG Solutions
Development cycle with R Suite
16
1. Start a project
2. Add a package to the project
a. Develop the package (using devtools)
3. Add dependencies to the project
4. Build the package
5. Build a deployment package

17. Copyright (c) WLOG Solutions 17
R Suite CLI commands
S3, Dir

18. Copyright (c) WLOG Solutions

19. Copyright (c) WLOG Solutions

20. Copyright (c) WLOG Solutions
Project structure
20

21. Copyright (c) WLOG Solutions 21
deployment
logs
packages
R
tests
import
export
work
Where all the
packages are
installed.

22. Copyright (c) WLOG Solutions 22
deployment
logs
packages
R
tests
import
export
work
Where all the logs
are stored.

23. Copyright (c) WLOG Solutions 23
deployment
logs
packages
R
tests
import
export
work
Where all the
project packages
are stored.

24. Copyright (c) WLOG Solutions 24
deployment
logs
packages
R
tests
import
export
work
Where master
scripts (~ main) are
stored.

25. Copyright (c) WLOG Solutions 25
deployment
logs
packages
R
tests
import
export
work
Where project tests
are stored.

26. Copyright (c) WLOG Solutions 26
deployment
logs
packages
R
tests
import
export
work
(Optional)
Raw data to be
imported.

27. Copyright (c) WLOG Solutions 27
deployment
logs
packages
R
tests
import
export
work
(Optional)
Results generated
by our R program.

28. Copyright (c) WLOG Solutions 28
deployment
logs
packages
R
tests
import
export
work
(Optional)
Temporary results.

29. Copyright (c) WLOG Solutions 29
Configuration files

30. Copyright (c) WLOG Solutions 30
PARAMETERS
● RVersion - R version for this
project
● Project - project name
● Repositories
○ MRAN[YYYY-MM-DD],
○ S3[URL]
○ URL[URL]
○ Dir
● Artifacts - what else should be
added to deployment
package.

31. Copyright (c) WLOG Solutions 31
Example from another project:
config_templ.txt (config.txt)
● LogLevel - level for loggers
● _templ.txt - template for config
● config.txt - deployment version

32. Copyright (c) WLOG Solutions
3 ways to manage RSuite project
32
1. Command Line Interface
2. Using R functions
3. Using RStudio Add-in

33. Copyright (c) WLOG Solutions

34. Copyright (c) WLOG Solutions
Package structure for development
34

35. Copyright (c) WLOG Solutions

36. Copyright (c) WLOG Solutions

37. Copyright (c) WLOG Solutions
Built binary package is added to libs
37

38. Copyright (c) WLOG Solutions

39. Copyright (c) WLOG Solutions

40. Copyright (c) WLOG Solutions

41. Copyright (c) WLOG Solutions

42. Copyright (c) WLOG Solutions

43. Copyright (c) WLOG Solutions

44. Copyright (c) WLOG Solutions

45. Copyright (c) WLOG Solutions

46. Copyright (c) WLOG Solutions

47. Copyright (c) WLOG Solutions 47
Different types of logs
loginfo("Phase 1 passed")
logdebug("Iter %d done", i)
logwarning("Are you sure?")
logerror("I failed :(")

48. Copyright (c) WLOG Solutions 48
Control master script log level
config_templ.txt (config.txt)
LogLevel: INFO - will log info, errors and warnings
LogLevel: DEBUG - will log info, errors, warnings and debug messages

49. Copyright (c) WLOG Solutions

50. Copyright (c) WLOG Solutions

51. Copyright (c) WLOG Solutions

52. Copyright (c) WLOG Solutions

53. Copyright (c) WLOG Solutions

54. Copyright (c) WLOG Solutions

55. Copyright (c) WLOG Solutions
Piotr Chaberski
Thank You!
pchaberski@wlogsolutions.com
http://rsuite.io