The document discusses the problem of managing multiple Scala/SBT projects (MBMH) and proposes a solution of using SBT archetype plugins. The key points are:
- MBMH refers to the difficulty of maintaining consistency across many SBT projects with different dependencies, plugins, and versions.
- The proposed solution is to define SBT archetype plugins that project share common build settings, dependencies, and provide a way to upgrade all projects by upgrading the archetype version.
- This centralizes configuration, promotes consistency, and simplifies upgrading whole suites of projects at once.
OSDC 2013 | Introduction into Chef by Andy HawkinsNETWAYS
This presentation will give an overview about what Chef is and how to access it. It will describe the typical use cases and architecture as well as Cookbooks, data bags and other concepts and will explain how to implement your CM solution. Finally it will show how to drive a successful Chef project.
The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
Getting a Neural Network Up and Running with OpenLabMelvin Hillsman
Access to hardware for AI/ML for the everyday developer wanting to explore this field can be challenging to obtain and maintain for even the most rudimentary applications and testing. Needing to go beyond a single development machine running locally only increases this. OpenLab is curated infrastructure accessible to open source projects and individuals working within and on open source projects designed to help address this use case. Access to GPU, FPGA, IoT, and more, allows HPC, AI/ML, Deep Learning, or other testing and applications. In this presentation, we will walk through getting an account with OpenLab, obtaining resources, and getting a neural network up and running with an app that will bring back great childhood memories.
This presentation is for enterprises that are considering adopting Scala. The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
Making Glance tasks work for you - OpenStack Summit May 2015 VancouverBrian Rosmaita
It's not widely known that the OpenStack Images API v2 contains an implementation of a "tasks" API that can be customized by operators to enable asynchronous processing of long-running operations. For example, a deployer might want to enable end users to upload their own custom images ... but only after such images have been approved by some thorough, computation-intensive validation process. The Glance tasks API provides a common interface across OpenStack installations, but allows the implementation of tasks to be customizable to a particular cloud environment. Join Brian Rosmaita, Compute Control Plane Product Manager at Rackspace to see how Glance tasks are being used at Rackspace and to learn how you can use Glance tasks in your OpenStack cloud.
OSDC 2013 | Introduction into Chef by Andy HawkinsNETWAYS
This presentation will give an overview about what Chef is and how to access it. It will describe the typical use cases and architecture as well as Cookbooks, data bags and other concepts and will explain how to implement your CM solution. Finally it will show how to drive a successful Chef project.
The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
Getting a Neural Network Up and Running with OpenLabMelvin Hillsman
Access to hardware for AI/ML for the everyday developer wanting to explore this field can be challenging to obtain and maintain for even the most rudimentary applications and testing. Needing to go beyond a single development machine running locally only increases this. OpenLab is curated infrastructure accessible to open source projects and individuals working within and on open source projects designed to help address this use case. Access to GPU, FPGA, IoT, and more, allows HPC, AI/ML, Deep Learning, or other testing and applications. In this presentation, we will walk through getting an account with OpenLab, obtaining resources, and getting a neural network up and running with an app that will bring back great childhood memories.
This presentation is for enterprises that are considering adopting Scala. The author is managing editor of http://scalacourses.com, which offers self-paced online courses that teach Introductory and Intermediate Scala and Play Framework.
Making Glance tasks work for you - OpenStack Summit May 2015 VancouverBrian Rosmaita
It's not widely known that the OpenStack Images API v2 contains an implementation of a "tasks" API that can be customized by operators to enable asynchronous processing of long-running operations. For example, a deployer might want to enable end users to upload their own custom images ... but only after such images have been approved by some thorough, computation-intensive validation process. The Glance tasks API provides a common interface across OpenStack installations, but allows the implementation of tasks to be customizable to a particular cloud environment. Join Brian Rosmaita, Compute Control Plane Product Manager at Rackspace to see how Glance tasks are being used at Rackspace and to learn how you can use Glance tasks in your OpenStack cloud.
CI/CD and Asset Serving for Single Page AppsMike North
A journey through best practices and technology for the modern build/deploy pipeline that your modern web app deserves! Also, an introduction to the concept and architecture of a new open source turnkey asset serving layer "Banker".
BP-10 Keeping Your Sanity – Rapid Development & Deployment ToolsAlfresco Software
There are many variations for deploying customizations into an Alfresco installation. Working as a global team on multiple projects, Blue Fish Development Group has created a set of processes, using standard tools such as Maven and Ant, to keep the development and deployment cycle sane. Working as a team presents challenges for maintaining source control and being able to quickly roll out the latest version of changes, and managing multiple projects requires the ability to quickly stand up a development environment and begin testing changes. You’ll learn Blue Fish’s procedures and walk away with a basic set of tools that will let you build your own development/ deployment framework that will reduce cycle time, improve repeatability and revert to a known/clean state.
Kubernetes and AWS Lambda can play nicely togetherEdward Wilde
Vendor lock-in is a major worry for many engineers . A new innovative approach, will for the first time, allow open-source serverless to run on AWS Lambda or Kubernetes using the same deployment artifact, packaged using the tools we love: containers.
The talk was given on local JUG meetup back in 2014. The purpose of the talk was to give an overview of the Akka library for the Java-programmers audience who have never used it before. My first appearance as a public speaker
Combining R With Java For Data Analysis (Devoxx UK 2015 Session)Ryan Cuprak
Java is a general-purpose language and is not particularly well suited for performing statistical analysis. Special languages and software environments have been created by and for statisticians to use. Statisticians think about programming and data analysis much different from Java programmers. These languages and tools make it easy to perform very sophisticated analyses on large data sets easily. Tools, such as R and SAS, contain a large toolbox of statistical tools that are well tested, documented and validated. For data analysis you want to use these tools.
In this session we will provide an overview of how to leverage the power of R from Java. R is the leading open source statistical package/language/environment. The first part of the presentation will provide an overview of R focusing on the differences between R and Java at the language level. We’ll also look at some of the basic and more advanced tests to illustrate the power of R. The second half of the presentation will cover how to integrate R and Java using rJava. We’ll look at leverage R from the new Java EE Batching (JSR 352) to provide robust statistical analysis for enterprise applications.
The slides for my UBC Alumni talk on programming for the Cloud. I show Cloud Foundry as an example of an open cloud platform and how easy it is to create modular, scalable applications using it.
This is my noob recap of KubeCon 2019, which I transformed into a kubernetes bootcamp. I walked away with a bunch of learnings, so here they are for you :)
Software Design Patterns in Laravel by Phill SparksPhill Sparks
Laravel makes use of quite a few well-established design patterns that promote reusable object-oriented code. Together, we will investigate the design patterns used in the core of Laravel 4 and discuss how they encourage reusable software.
We often employ the "build-once-run-everywhere" principle to our application binaries. Our build server builds an artifact and puts it in a repository, this same artifact is then promoted from environment to environment, from test to production, to make sure that what ends up in production is the very same thing as what we have thoroughly tested before.
Now, in a world of virtualization, what if we were to do the same thing with our complete infrastructure? In stead of just building our application and promote it from environment to environment, what if we would build a complete virtual machine image and do the same with that? Could we?
This is what immutable infrastructure is about. Boxfuse can help you get there.
Trent Hornibrook gave a recent talk at the Infracoders meet-up playing a thought experiment with the audience on 'what would be your tech decisions if you were given a blank cheque at at startup'.
Trent, recently working for a start-up then shared what decisions he made, and why
CI/CD and Asset Serving for Single Page AppsMike North
A journey through best practices and technology for the modern build/deploy pipeline that your modern web app deserves! Also, an introduction to the concept and architecture of a new open source turnkey asset serving layer "Banker".
BP-10 Keeping Your Sanity – Rapid Development & Deployment ToolsAlfresco Software
There are many variations for deploying customizations into an Alfresco installation. Working as a global team on multiple projects, Blue Fish Development Group has created a set of processes, using standard tools such as Maven and Ant, to keep the development and deployment cycle sane. Working as a team presents challenges for maintaining source control and being able to quickly roll out the latest version of changes, and managing multiple projects requires the ability to quickly stand up a development environment and begin testing changes. You’ll learn Blue Fish’s procedures and walk away with a basic set of tools that will let you build your own development/ deployment framework that will reduce cycle time, improve repeatability and revert to a known/clean state.
Kubernetes and AWS Lambda can play nicely togetherEdward Wilde
Vendor lock-in is a major worry for many engineers . A new innovative approach, will for the first time, allow open-source serverless to run on AWS Lambda or Kubernetes using the same deployment artifact, packaged using the tools we love: containers.
The talk was given on local JUG meetup back in 2014. The purpose of the talk was to give an overview of the Akka library for the Java-programmers audience who have never used it before. My first appearance as a public speaker
Combining R With Java For Data Analysis (Devoxx UK 2015 Session)Ryan Cuprak
Java is a general-purpose language and is not particularly well suited for performing statistical analysis. Special languages and software environments have been created by and for statisticians to use. Statisticians think about programming and data analysis much different from Java programmers. These languages and tools make it easy to perform very sophisticated analyses on large data sets easily. Tools, such as R and SAS, contain a large toolbox of statistical tools that are well tested, documented and validated. For data analysis you want to use these tools.
In this session we will provide an overview of how to leverage the power of R from Java. R is the leading open source statistical package/language/environment. The first part of the presentation will provide an overview of R focusing on the differences between R and Java at the language level. We’ll also look at some of the basic and more advanced tests to illustrate the power of R. The second half of the presentation will cover how to integrate R and Java using rJava. We’ll look at leverage R from the new Java EE Batching (JSR 352) to provide robust statistical analysis for enterprise applications.
The slides for my UBC Alumni talk on programming for the Cloud. I show Cloud Foundry as an example of an open cloud platform and how easy it is to create modular, scalable applications using it.
This is my noob recap of KubeCon 2019, which I transformed into a kubernetes bootcamp. I walked away with a bunch of learnings, so here they are for you :)
Software Design Patterns in Laravel by Phill SparksPhill Sparks
Laravel makes use of quite a few well-established design patterns that promote reusable object-oriented code. Together, we will investigate the design patterns used in the core of Laravel 4 and discuss how they encourage reusable software.
We often employ the "build-once-run-everywhere" principle to our application binaries. Our build server builds an artifact and puts it in a repository, this same artifact is then promoted from environment to environment, from test to production, to make sure that what ends up in production is the very same thing as what we have thoroughly tested before.
Now, in a world of virtualization, what if we were to do the same thing with our complete infrastructure? In stead of just building our application and promote it from environment to environment, what if we would build a complete virtual machine image and do the same with that? Could we?
This is what immutable infrastructure is about. Boxfuse can help you get there.
Trent Hornibrook gave a recent talk at the Infracoders meet-up playing a thought experiment with the audience on 'what would be your tech decisions if you were given a blank cheque at at startup'.
Trent, recently working for a start-up then shared what decisions he made, and why
Composing Project Archetyps with SBT AutoPluginsMark Schaake
A talk give at PNW Scala 2014 about how the Allen Institute for Artificial Intelligence is using the SBT 0.13.5 AutoPlugin architecture to compose plugins for archetypical projects.
Nowadays, it is quite common to have build infrastructure that, on every change in a repository, builds your software and runs all your tests. However this is where most development teams stop. This talk demonstrates how you can consistently deploy systems from development to testing, staging and production.
With Bndtools we semantically version bundles in the Eclipse IDE. The continuous build is setup to automatically deploy to a bundle repository and create snapshots of changed bundles compared to a baselined version that is available in a release repository. By hooking up these repositories to Apache ACE, such updates can instantly be deployed to systems in different configurations, allowing you to deploy new features to QA systems, beta users and finally roll them out to all other customers.
This presentation was given at ApacheCon NA 2014, Denver.
Boilerplates: Step up your Web Development ProcessFibonalabs
Having a boilerplate for your project, whether you are a developer or a designer, is always a good idea. It saves a lot of time from going back and forth between setting up the scaffolding and getting your hands dirty with code; it also helps in sharing your code with others. You can use it to get a head start on new projects and work on the core functionality. Using custom-designed React boilerplates shortened our initial sprints from 2 weeks to 3 days!
Measure and Increase Developer Productivity with Help of Serverless at JCON 2...Vadym Kazulkin
The goal of Serverless is to focus on writing the code that delivers business value and offload everything else to your trusted partners (like Cloud providers or SaaS vendors). You want to iterate quickly and today’s code quickly becomes tomorrow’s technical debt. In this talk we will show why Serverless adoption increases the developer productivity and how to measure it. We will also go through AWS Serverless architectures where you only glue together different Serverless managed services relying solely on configuration, minimizing the amount of the code written.
Apache Spark™ + IBM Watson + Twitter DataPalooza SF 2015Mike Broberg
Use Apache Spark Streaming in with IBM Watson on Bluemix to perform sentiment analysis and track how a conversation is trending on Twitter.
By David Taieb: https://twitter.com/DTAIEB55
Video: https://youtu.be/KLc_wazud3s
Tutorial: https://developer.ibm.com/clouddataservices/sentiment-analysis-of-twitter-hashtags/
Kubecon 2019 - Promoting Kubernetes CI/CD to the Next LevelTim Pouyer
Watch the presentation on YouTube: https://youtu.be/_DvFbN-VR3I
Many companies and organizations have adopted CI/CD processes in order to help deliver applications running on Kubernetes quickly, transparently, and with automated tests. While this is a desirable goal, it gets more complex when developing a management layer on top of k8s, especially when both images and helm charts are involved. In developing IBM Cloud Private, we have implemented a CI/CD process that automates promotion through a series of quality gates where we 1) Ensure that all charts (43) and images (135) are in sync across three supported architectures 2) Deploy instances of ICP clusters with different topologies, management services, and infrastructure 3) Run automated and manual functional and security regression tests against those clusters. Since implementation, have we have been able to iterate more quickly by discovering issues earlier in the development process.
My talk from Dockercon EU in Amsterdam, Dec 2014. Original abstract:
The ModCloth Platform team has been building a Docker-based continuous delivery pipeline. This presentation discusses that project and how we build containers at ModCloth. The topics include what goes into our containers; how to optimize builds to use the Docker build cache effectively; useful development workflows (including using fig); and the key decision to treat containers as processes instead of mini-vms. This presentation will also discuss (and demo!) the workflow we’ve adopted for building containers and how we’ve integrated container builds with our CI.
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...Docker, Inc.
The ModCloth Platform team has been building a Docker-based continuous delivery pipeline. This presentation discusses that project and how we build containers at ModCloth. The topics include what goes into our containers; how to optimize builds to use the Docker build cache effectively; useful development workflows (including using fig); and the key decision to treat containers as processes instead of mini-vms. This presentation will also discuss (and demo!) the workflow we’ve adopted for building containers and how we’ve integrated container builds with our CI.
5. Archetype | ˈärk(i)ˌtīp |
• a very typical example of a certain person or thing (Apple Dictionary)
• a statement, pattern of behavior, or prototype which other statements,
patterns of behavior, and objects copy or emulate (Wikipedia)
6. Archetype | ˈärk(i)ˌtīp |
• a very typical example of a certain person or thing (Apple Dictionary)
• a statement, pattern of behavior, or prototype which other statements,
patterns of behavior, and objects copy or emulate (Wikipedia)
7. Composing Project
Archetypes with SBT
AutoPlugins
Mark Schaake
allenai.org
“contribute to humanity through high-impact AI research and engineering”
8. Composing Project
Archetypes with SBT
AutoPlugins
Mark Schaake
allenai.org
“contribute to humanity through high-impact AI research and engineering”
9. Combatting Multiple Build
Maintenance Hell with
Archetype AutoPlugins
Mark Schaake
allenai.org
“contribute to humanity through high-impact AI research and engineering”
10. “When you have very limited visibility and
control over many project builds to the point
where you feel paralyzed to maintaining cross-
project consistency.”
(Multiple Build Maintenance Hell)
MBMH
• visibility: what libraries, plugins, versions?
• control: how to enforce team standards, upgrades?
• consistency: minimize context switching
33. Other Considerations
• Archetype settings shared by similar projects
• Core settings common to all projects
• Formatting and Style
34. Other Considerations
• Archetype settings shared by similar projects
• Core settings common to all projects
• Formatting and Style
• Generate Git version resource
43. web-service 1.0 Svc A
Archetype Plugins Projects
Svc B
Svc C
Svc D
web-service 2.0
web-service 2.1
library 1.0
library 1.1
cli 1.0
Lib X
Lib Y
CLI Z
…
…
…
web-app 1.0 App W
…
44. web-service 1.0 Svc A
Archetype Plugins Projects
Svc B
Svc C
Svc D
web-service 2.0
web-service 2.1
library 1.0
library 1.1
cli 1.0
Lib X
Lib Y
CLI Z
…
…
…
web-app 1.0 App W
…
53. Adding Popular SBT Plugin
Settings
“To add the default plugin settings, add the
following to your build.sbt…”
54. Adding Popular SBT Plugin
Settings
“To add the default plugin settings, add the
following to your build.sbt…”
import com.typesafe.sbt.SbtNativePackager._
import NativePackagerKeys._
packageArchetype.java_application
sbt-native-packager
org.scalastyle.sbt.ScalastylePlugin.Settings
sbt-scalastyle
scalariformSettings
sbt-scalariform
Revolver.settings
sbt-revolver
56. SBT AutoPlugins
• The new Plugin standard (since SBT 0.13.5)
• Well-defined plugin API
57. SBT AutoPlugins
• The new Plugin standard (since SBT 0.13.5)
• Well-defined plugin API
• Less need for conventions
58. SBT AutoPlugins
• The new Plugin standard (since SBT 0.13.5)
• Well-defined plugin API
• Less need for conventions
• Killer feature: can compose plugins via `requires`
59. abstract class AutoPlugin {
…
/** When this AutoPlugin is enabled, all required
* plugins will also be enabled automatically prior
* to enabling this AutoPlugin. */
def requires: Plugins = empty
/** The [[Setting]]s to add in the scope of each
* project that activates this AutoPlugin. */
def projectSettings: Seq[Setting[_]] = Nil
…
}
SBT AutoPlugins
https://github.com/sbt/sbt/blob/0.13.7/main/src/main/scala/sbt/Plugins.scala
78. Project Archetype Recipe
• Create a company “sbt-plugins” project
• Wrap third-party plugins (scalariform, scalastyle, etc.)
• Define a core settings AutoPlugin that enforces team
standards
• Define archetype plugins that require core settings
and add in archetype-specific settings
• Publish to Bintray and serve!