This is a talk given by Jason Hoffman at a workshop given by Joyent called "Scale With Rails" in 2006. There's quite a bit of prescience in this presentation, including the first documented use of ZFS in production ("Fsck you if you think ZFS isn't production") and of OS-based virtualization (zones) in the cloud (which, it must be said, was not called "cloud" in 2006).
My (very brief!) presentation at Interzone.io on March 11, 2015. A more in depth exploration of these ideas can be found at http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production video: https://www.joyent.com/developers/videos/docker-and-the-future-of-containers-in-production
My (very brief!) presentation at Interzone.io on March 11, 2015. A more in depth exploration of these ideas can be found at http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production video: https://www.joyent.com/developers/videos/docker-and-the-future-of-containers-in-production
server to cloud: converting a legacy platform to an open source paasTodd Fritz
This session discusses the process to move legacy applications "into the cloud". It is intended for a diverse audience including developers, architects, and managers. We will discuss techniques, methodologies, and thought processes used to analyze, design, and execute a migration strategy and implementation plan -- from planning through rollout and operational.
An important aspect of this is the necessity for technical staff to effectively communicate to mid-level management how these design decisions and strategies translate into cost, complexity and schedule.
Commonly used migration strategies, cloud technologies, architecture options, and low level technologies will be discussed.
The case will be made that investing in strategic refactoring and decomposition during the migration will reap the benefits of a modern, decoupled and simplified system.
The end game being alignment and adoption of current best practices around PaaS, Saas, SOA, event-driven architectures, and message-oriented middleware, at scale in the cloud, to provide quantifiable business value.
This talk will focus more on the big picture, at times delving into technical architectures and discussion of certain technologies and service providers.
Use of Containers (Docker) is evangelized for decoupling and decomposing legacy systems.
node.js in production: Reflections on three years of riding the unicornbcantrill
My presentation at #NodeSummit, December 3, 2013. Video is at http://www.joyent.com/developers/videos/reflections-on-three-years-of-nodejs-in-production
The Reactive Principles: Design Principles For Cloud Native ApplicationsJonas Bonér
Reactive Summit Keynote 2020: https://www.youtube.com/watch?v=e5kek8vx2ws
Abstract: Building applications for the cloud means embracing a radically different architecture than that of a traditional single-machine monolith, requiring new tools, practices, and design patterns. The cloud’s distributed nature brings its own set of concerns–building a Cloud Native, Edge Native, or Internet of Things (IoT) application means building and running a distributed system on unreliable hardware and across unreliable networks. In this keynote session by Jonas Bonér, creator of Akka, founder/CTO of Lightbend, and Chair of the Reactive Foundation, we’ll review a set of Reactive Principles that enable the design and implementation of Cloud Native applications–applications that are highly concurrent, distributed, performant, scalable, and resilient, while at the same time conserving resources when deploying, operating, and maintaining them.
How Events Are Reshaping Modern SystemsJonas Bonér
Event-driven architecture and design have been getting a lot of attention in recent years. It’s an old concept that has been around for decades, so why this sudden peak of interest?
In this talk, we will explore the nature of events, what it means to be event-driven, and how we can unleash the power of events. The goal is to arm you with a solid theoretical understanding of how to design an event-driven system, what tools and techniques you can use to reap the most benefit from its design, and perhaps most importantly, what to avoid.
We'll discuss how an event-driven design can help:
- drive autonomy
- reduce risk
- increase certainty
- increase loose coupling
- increase scalability
- increase resilience
- increase traceability
Skeptics should definitely attend.
InteropNY/CloudConnect 2014 - Quick Crash Course in Open Source Cloud ComputingMark Hinkle
This will be an overview of the open source software that can be used to deploy and manage a cloud computing environment. The session will include information on storage, networking(e.g. OpenDaylight) and compute virtualization (Xen, KVM, LXC) and the orchestration(Apache CloudStack, OpenStack) of the three to build their own cloud services.
OSCON 2014 - Crash Course in Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
Talk originally given at FISL 2012 in Porto Alegre, Brazil. Video was on YouTube but regrettably taken down. Fortunately, I gave a slightly updated (and frankly, tighter and better produced) version of this at the Liferay Symposium in the fall of 2012: https://www.youtube.com/watch?v=Pm8P4oCIY3g
server to cloud: converting a legacy platform to an open source paasTodd Fritz
This session discusses the process to move legacy applications "into the cloud". It is intended for a diverse audience including developers, architects, and managers. We will discuss techniques, methodologies, and thought processes used to analyze, design, and execute a migration strategy and implementation plan -- from planning through rollout and operational.
An important aspect of this is the necessity for technical staff to effectively communicate to mid-level management how these design decisions and strategies translate into cost, complexity and schedule.
Commonly used migration strategies, cloud technologies, architecture options, and low level technologies will be discussed.
The case will be made that investing in strategic refactoring and decomposition during the migration will reap the benefits of a modern, decoupled and simplified system.
The end game being alignment and adoption of current best practices around PaaS, Saas, SOA, event-driven architectures, and message-oriented middleware, at scale in the cloud, to provide quantifiable business value.
This talk will focus more on the big picture, at times delving into technical architectures and discussion of certain technologies and service providers.
Use of Containers (Docker) is evangelized for decoupling and decomposing legacy systems.
node.js in production: Reflections on three years of riding the unicornbcantrill
My presentation at #NodeSummit, December 3, 2013. Video is at http://www.joyent.com/developers/videos/reflections-on-three-years-of-nodejs-in-production
The Reactive Principles: Design Principles For Cloud Native ApplicationsJonas Bonér
Reactive Summit Keynote 2020: https://www.youtube.com/watch?v=e5kek8vx2ws
Abstract: Building applications for the cloud means embracing a radically different architecture than that of a traditional single-machine monolith, requiring new tools, practices, and design patterns. The cloud’s distributed nature brings its own set of concerns–building a Cloud Native, Edge Native, or Internet of Things (IoT) application means building and running a distributed system on unreliable hardware and across unreliable networks. In this keynote session by Jonas Bonér, creator of Akka, founder/CTO of Lightbend, and Chair of the Reactive Foundation, we’ll review a set of Reactive Principles that enable the design and implementation of Cloud Native applications–applications that are highly concurrent, distributed, performant, scalable, and resilient, while at the same time conserving resources when deploying, operating, and maintaining them.
How Events Are Reshaping Modern SystemsJonas Bonér
Event-driven architecture and design have been getting a lot of attention in recent years. It’s an old concept that has been around for decades, so why this sudden peak of interest?
In this talk, we will explore the nature of events, what it means to be event-driven, and how we can unleash the power of events. The goal is to arm you with a solid theoretical understanding of how to design an event-driven system, what tools and techniques you can use to reap the most benefit from its design, and perhaps most importantly, what to avoid.
We'll discuss how an event-driven design can help:
- drive autonomy
- reduce risk
- increase certainty
- increase loose coupling
- increase scalability
- increase resilience
- increase traceability
Skeptics should definitely attend.
InteropNY/CloudConnect 2014 - Quick Crash Course in Open Source Cloud ComputingMark Hinkle
This will be an overview of the open source software that can be used to deploy and manage a cloud computing environment. The session will include information on storage, networking(e.g. OpenDaylight) and compute virtualization (Xen, KVM, LXC) and the orchestration(Apache CloudStack, OpenStack) of the three to build their own cloud services.
OSCON 2014 - Crash Course in Open Source Cloud ComputingMark Hinkle
This crash course is designed to give an overview of cloud computing architecture and the open source software that can be used to deploy and manage a cloud computing environment.
Topics to be discussed in this session will include virtualization (KVM, LXC, and Xen Project), orchestration (Apache CloudStack, Eucalyptus, Open Nebula, and OpenStack), and storage (GlusterFS, Ceph, and others). The talk will also provide insight into how to deliver Platform-as-a-Service (PaaS) and what technologies can be used to compliment this evolving cloud computing paradigm.
Systems administrators and IT generalists will leave the discussion with a general overview of the options at their disposal to effectively build and manage their own cloud computing environments using free and open source software and understand the capabilities and benefits of a host of technologies.
Talk originally given at FISL 2012 in Porto Alegre, Brazil. Video was on YouTube but regrettably taken down. Fortunately, I gave a slightly updated (and frankly, tighter and better produced) version of this at the Liferay Symposium in the fall of 2012: https://www.youtube.com/watch?v=Pm8P4oCIY3g
Cgroups, namespaces and beyond: what are containers made from?Docker, Inc.
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
도커 무작정 따라하기: 도커가 처음인 사람도 60분이면 웹 서버를 올릴 수 있습니다!pyrasis
도커 무작정 따라하기
- 도커가 처음인 사람도 60분이면 웹 서버를 올릴 수 있습니다!
도커의 기본 개념부터 설치와 사용 방법까지 설명합니다.
더 자세한 내용은 가장 빨리 만나는 도커(Docker)를 참조해주세요~
http://www.pyrasis.com/private/2014/11/30/publish-docker-for-the-really-impatient-book
Web Performance tuning presentation given at http://www.chippewavalleycodecamp.com/
Covers basic http flow, measuring performance, common changes to improve performance now, and several tools and techniques you can use now.
Rich, Real-time Mobile User Experiences @Devoxx UKAndy Piper
Rich, Real-time Mobile User Experiences @Devoxx UK
My presentation at Devoxx UK on some ways to increase interactivty in mobile applications. Many of these techniques are embodied in Diffusion, Push Technology's premier data-distribution engine.
BD Conf: Visit speed - Page speed is only the beginningPeter McLachlan
How can we beat the speed of light and make visits faster? Pre-fetching is one way we can make resources available before they're needed. This talk explores challenges in mobile visit performance and discusses the design of a generic pre-fetching system.
Linked Data: The Real Web 2.0 (from 2008)Uche Ogbuji
"Linking Open Data (LOD) is a community initiative moving the Web from the idea of separated documents to a wide information space of data. The key principles of LOD are that it is simple, readily adaptable by Web developers, and complements many other popular Web trends. Linked, open data is the real substance of Web 2.0, and not flashy AJAX effects. Learn how to make your data more widely used by making its components easier to discover, more valuable, and easier for people to reuse—in ways you might not anticipate."
Systems Monitoring with Prometheus (Devops Ireland April 2015)Brian Brazil
Monitoring means many things to many people. This talk looks at Systems Monitoring, that is how to keep an eye on a given system and use this as part of overall management of a system. This talk will cover Why one monitors, What to monitor, How to monitor, the general design of a monitoring system and how Prometheus is a good fit for this in terms of instrumentation, consoles, alerts, general system health and sanity.
Prometheus is a next-generation monitoring system publicly announced earlier this year, developed by companies including SoundCloud, locals Boxever and Docker. Since launch there has been wide-spread interest, and many community contributions.
For more information see http://prometheus.io or http://www.boxever.com/tag/monitoring
How to be Successful with Responsive Sites (Koombea & NGINX) - EnglishKoombea
Can't decide if your organization should build a mobile app or responsive website? Do you interact with consumer-facing products or large scale developments?
This guide gives you an idea of what Responsive is, why you should use it, and then DIGS deep into the technical aspect and how to optimize for performance.
By: David Bohorquez & Rick Nelson
I was invited by the Hatchery+ to give a presentation and workshop on building products - a brief overview on modern web apps, tech stacks, languages, frameworks, services, APIs and more.
Talk given at the OCP Open System Firmware engineering workshop on 5/17/22. Talk was recorded; video at https://www.youtube.com/watch?v=eNI0wFgBNmY#t=7044s
Hardware/software Co-design: The Coming Golden Agebcantrill
Talk I gave as a keynote at RailsConf 2021. There is no Rails in the talk, though; this is all about the revolutions in open source firmware and hardware that are changing the way we build systems. Video to come!
Tockilator: Deducing Tock execution flows from Ibex Verilator tracesbcantrill
Talk given on March 20, 2020 at Oxidize 1K, a virtual conference that went from first idea to 300+ person conference in a week during the COVID-19 pandemic.
Platform values, Rust, and the implications for system softwarebcantrill
Talk given at Scale By The Bay 2018. Video is at https://www.youtube.com/watch?v=2wZ1pCpJUIM. If you are interested in this talk, you might also be interested in my talk on Platform as a Reflection of Values from Node Summit 2017: https://www.slideshare.net/bcantrill/platform-as-reflection-of-values-joyent-nodejs-and-beyond
My Papers We Love talk in San Francisco on October 12, 2017 on "ARC: A self-tuning, low overhead replacement cache." Video at https://www.youtube.com/watch?v=F8sZRBdmqc0
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
2. The Guys {Thanks}
David Young, Dean Allen, Jason Hoffman, Matt
Imbach, Michael Koziarski, Scott Barron, Chris
Morris, Luke Crawford, Justin French, Johan
Sörensen, Ben Myles, Marten Veldthuis, Bryan
Bell, Ryan Schwartz, Jan Isley, Florian Munz,
Filip Hajný, Daniel Crowell, Christopher Horrell,
Terrell Russell, Peter Watridge, Josh Roebuck
3. Today
• Please humor me => a 5 minute story with too
many slides about how I came to stand here in
front of you
• Somethings about the Ruby on Rails site
• An accounting what’s buzzing in my ears all the
time
• Some examples with numbers
• How to think about “scaling”
• To talk about where we’ve taken our
infrastructure
9. So we thought let’s involve some
friends
• Dean Allen (Textpattern)
• Matt Mullenweg (Wordpress)
• Brad Choate (doing MT hosting + plugins)
• David Heinemeier Hansson (Instiki and “Rails”)
• Rickard of PunBB
• Allan => Textmate
19. But sometimes it’s gotten “bad”
• Ruby on Rails
• Turbogears
• Textmate
• All have videos (”screencasts”)
• They’ve been in the same paragraph on
Slashdot
• Then we’ve pushed 150-200 Mbps
• Open Source is free?
21. OK, So what?
• We found ourselves needing an infrastructure
for “diverse” things
• But fundamentally we’re talking about web
stuff, mail stuff, database stuff, storing and
moving around files stuff
22. And why are we here?
• One indication: because I get, on average, 2
emails/day asking the same questions
• That’s just “me”
• Support system and “sales” gets them as well
23. Examples from last week
• I have yet to find any examples of websites that have heavy traffic and
stream media that run from a Ruby on Rails platform, can you suggest
any sites that will demonstrate that the ruby platform is stable and
reliable enough to use on a commercial level?
• We are concerned about the long-term viability of Ruby on Rails as a
development language/environment.
• How easily can a ruby site be converted to another language? (If for any
reason we were forced to abandon ruby at some point in the future or I
can’t find someone to work with our code?).
• My company has some concerns on whether or not Ruby on Rails is the
right platform to deploy on if we have a very large scale app.
I could go on for a while.
24. So what is?
• A “large scale” application?
• Do many of us really have one? I mean a BIG
ONE?
• What is an “intensive” application?
• What is an “enterprise” application?
25. But we’ve been doing “it” with
• Perl
• PHP
• Python
• Java
• Does the language really really matter to the
system’s folks?
26. Ah-ha!
• System’s Folks you say?
• What are these “system administrators” that you
speak of?
• Does the network really matter?
• You mean there’s “networking people”?
• You mean it takes more than just having my
designer(s) learns The Rails?
27. If I had a nickel for every time ...
• “We’re going to need to scale this up to Flickr-
sized proportions!”
• “This could go very very fast!”
• “The market is HUGE”
28. Reality Check
• “OK”, I say
• “Not a problem”, I say
• “I can do that”, I say
• “For what you’re asking, I’ll need $325,000
tomorrow to start, it’ll take $18,500 a month to
run and you can expect that to go up along with
your growth.”
• “It’s a good rule of thumb to try and keep it
about 10% of your revenue once you’re going
because people ended being the most
expensive”
29. Oh. But we don’t have that kind
of money. And the app is free to
start.
31. What a minute!
• Why are you going on and on about this not
worry about scaling til you need to scale stuff?
• I can read this same thing on what’s their face’s
weblog.
• It’s not what I’m asking about!
• I need to know if I can “scale” with ruby!
• We’re not a start-up, why are you talking about
this from a “start-up’s” perspective?!
• I need a ...
32. A What?
• A framework for thinking about the entire
system?
• You mean it’s more than just the language?
• It’s more than about a readily available IDE?
• You mean one’s “development” framework is
just one small part?
• My system’s guys can scale anything.
34. And oh yes ... you should worry
still worry about scaling
35. Because what does “scaling” actually
mean?
• For a little software application company?
• Can you do the Start-up => Mid-sized, “small
caps” company transition without going out of
business?
• In a big company?
• Can you do what you gotta do without having
your program cut?
• Will your app run on a phone? Scale down.
38. The slowest part ?
• The fast ethernet or gigabit network port
(assuming there’s more than one drive)
• Transactions of something/second
39. • 1 Gbps = 125 MB/sec (100 Mbps = 12.5 MB/sec)
• The question is can your OS and our CPUs
push it?
• And let’s say you can, just how much is a Gbps
in some kind of other thing?
40. We used to use FreeBSD
Moral of the story? Even restricted to a single processor with a
single core, Solaris Nevada Build 31 can push 60% of a gigabit
connection. FreeBSD can’t.
41. OK, Solaris Good.
What’s a 100 Mbps in normal web traffic?
• Textdrive.com => 122 KB and has 20 objects
(~125KB for an uncached page view).
• 125KB page => 100-1000 unique visitors per
second
• 20 objects per page, that is 2000 requests per
second that could pump out of that system.
• Maximum.
42. • What is the ability to do 2000 requests/second
then?
• ((2000 requests/sec)*(20 requests per page)) *
0.125MB per page = 12.5 MB/sec (100Mbps
constant).
• 86400 seconds/day on 100Mbps => 8,640,000
uniques in a day with 172,800,00 hits.
43. • $8000/month
• Give or take a couple of grand depending
How much does just a 100 Mbps
commit cost?
44. Can you do 2000 requests/second?
• Sure an Apache, Lighttpd or Litespeed can do
1000-15,000 static or proxy requests/second
45. How do you do 2000 requests/
second?
• Alistapart.com bursts to that with Zeldman’s
“Web 3.0” for over an hour
• 10 Lighttpds => 200 proxy requests/second
each
• 1 Lighttpd => 40 request/sec x 5 Rails-FCGI
each
• We cached everything but writing to MySQL for
comments and that kept it on a 3.2 Ghz P4 with
4GB of RAM
46. The Shared Hosting
• Is this odd heterogenous largely ruby “application”
• Users received 8,641,866 emails, they sent 5,860,769 emails and had
14,984,680 pieces of spam blocked. Mail through the system averages
300 emails/minute with bursts up to 60,000 emails/minute.
• Websites cumulatively did ~400,000,000 page views.
47.
48. What is it running now?
• There’s 22 TBs of strongspace
• Fiber-attached EMC storage
• Migrated to Solaris “11”
• One big ZFS pool with LUNs
• Migrated down to one “hot” server: Dual 3.0
Ghz Xeon with 8GB of RAM
• Apache, Lighttpd, static FCGI
51. “Logical” servers for the hosted
connector
1) Jumpstart/Boot & Administrative servers (x2 per setup)
2) DHCP/LDAP for server identification/authentication and control (x2)
3) DNS: DNSCache for outside resolution (DNSCache as resolver) and a private DNS system
4) DNS MySQL (x2, master/slave, innodb tables)
5) SPAM filtering servers (running DSPAM -> files to NFS store and tracking to postgresql) (x4)
6) SPAM database setup (running postgresql) (x2)
7) SPAM NFS store (x2 heads clustered)
8) SMTP gateway out (x2)
9) SMTP gateway in (x4 -> delivery to Maildir over NFS)
10) Mail NFS store (x2 heads cluster) *main user file store
11) IMAP proxy servers (x2)
12) IMAP servers (x6)
13) User LDAP servers (x2 with hitting postgresql DB backend)
14) User LDAP-postgresql db (x2)
15) User postgresql DB servers (x4)
16) User Web/Application servers (running with kernel SSL accelerators) (x6)
17) User File Storage (NFS dual cluster heads) (x2)
18) Joyent Organization Provisioning server
62. Back to simple
Console server
One kind of switch
One kind of server
One kind of storage
One kind of interconnect
One kind of power plug
One kind of power strip
72. DIY: How do you start?
20amp 208V AC power
L6-20R
Street price: ~$50,000 co-lo’ed
73. Why DIY? Or at least a visit?
Outsourcing!You can whois yourself?
74. Some final things
• Keep it simple even if you’re starting out big
• Keep it simple even if you’re starting out small
• Our “stacks” are currently:
• Solaris, Mongrel, Apache 2.2 event, kssl
• Solaris, FCGI, Apache 2.2 event, kssl
• Solaris, FCGI, Lighttpd, kssl
• Solaris, FCGI, Lighttpd, Big-IP
• Zones, zones, zones