HA & DRS Gotchas that will Kill your Infrastructure Greg Shields Partner and Principal Technologist Concentrated Technolog...
This slide deck was used in one of our many conference presentations. We hope you enjoy it, and invite you to use it withi...
Class Discussion <ul><li>Two Questions: </li></ul><ul><li>So……How often do you  actually  vMotion your virtual machines? <...
Class Discussion <ul><li>Two Questions: </li></ul><ul><li>So……How often do you  actually  vMotion your virtual machines? <...
vMotion Solves Two Problems Problem #1:  Protection from Host Failures or Scheduled Host Downtime (These are relatively ra...
vMotion Solves Two Problems Problem #2: Load Balancing of VM & Host Resources (Much more common, where turned on)
Costs vs. Benefits <ul><li>High-availability adds dramatically greater uptime for virtual machines. </li></ul><ul><ul><li>...
Contrary to Popular Belief… <ul><li>… seeing the actual vMotion process occur isn ’t all that sexy . </li></ul><ul><li>DEM...
vMotion:  What Really Happens? <ul><li>Sexy  is recognizing what ’s going on  under the hood . </li></ul>
vMotion:  What Really Happens? <ul><li>Sexy  is recognizing what ’s going on  under the hood . </li></ul><ul><li>Let ’s Co...
vMotion:  What Really Happens? <ul><li>Sexy  is recognizing what ’s going on under the hood. </li></ul><ul><li>Let ’s Comp...
vMotion:  What Really Happens? <ul><li>So…How does it work? </li></ul><ul><ul><li>As you invoked a Hyper-V  “Quick Migrati...
vMotion:  What Really Happens?
vMotion:  What Really Happens? <ul><li>This saving of virtual machine state and transferring memory contents  consumed dow...
vMotion:  What Really Happens? <ul><li>This saving of virtual machine state and transferring memory contents  consumed dow...
vMotion:  Gotcha ’s <ul><li>Successful vMotion requires similar processors. </li></ul><ul><ul><li>Processors must be from ...
vMotion:  Gotcha ’s
vMotion:  Gotcha ’s <ul><li>Big problem:  As a virtual environment ages, hardware is refreshed and new hardware is added. ...
vMotion:  Gotcha ’s <ul><li>Big problem:  As a virtual environment ages, hardware is refreshed and new hardware is added. ...
vMotion:  Gotcha ’s <ul><li>Big problem:  As a virtual environment ages, hardware is refreshed and new hardware is added. ...
vMotion:  Gotcha ’s <ul><li>Big problem:  As a virtual environment ages, hardware is refreshed and new hardware is added. ...
Class Discussion <ul><li>Blades versus Servers has been a common battle in virtualization spaces. </li></ul><ul><ul><li>Ev...
Class Discussion <ul><li>Blades versus Servers has been a common battle in virtualization spaces. </li></ul><ul><ul><li>Ev...
New Topic! vMotion vs. Storage vMotion
vMotion vs. Storage vMotion
vMotion vs. Storage vMotion
vMotion vs. Storage vMotion <ul><li>Three options for migrating machines and disks. </li></ul><ul><ul><li>Online & Powered...
vMotion vs. Storage vMotion <ul><li>Requirements: </li></ul><ul><ul><li>Virtual machines with snapshots cannot be svMotion...
HA & DRS: Combining vMotion + Math
vMotion Itself Might Not be Sexy…
vMotion Itself Might Not be Sexy… <ul><li>… but what-you-can-do-with-it-once-you-combine-it-with-monitoring-and-a-set-of-s...
vMotion Itself Might Not be Sexy… <ul><li>… but what-you-can-do-with-it-once-you-combine-it-with-monitoring-and-a-set-of-s...
vCenter Constructs <ul><li>Datacenter </li></ul><ul><ul><li>The boundary of a virtual infrastructure (& vMotion) </li></ul...
Shares, Reservations, & Limits <ul><li>Shares </li></ul><ul><ul><li>Identifies the ratio of resources a VM can consume </l...
Understanding Shares & Resource Pools TechMentor Cluster PowerShell Deep Dive Resource Pool 1000 CPU & RAM Shares 2048 RAM...
Abstracting Resources <ul><li>EXTENDED DEMO: </li></ul><ul><ul><li>Reservations </li></ul></ul><ul><ul><li>Limits </li></u...
What Role Does DRS Play? TechMentor Cluster HOT! Chassis 1 Chassis 2 Chassis 3
What Role Does DRS Play? Administrator trust is often the deciding factor in choosing the automation level. Automation Lev...
How Does DRS Do its Doo-Doo? <ul><li>Five recommendation levels </li></ul><ul><ul><li>1 = DO THIS NOW! </li></ul></ul><ul>...
How Does DRS Do its Doo-Doo? <ul><li>Five recommendation levels </li></ul><ul><ul><li>1 = DO THIS NOW! </li></ul></ul><ul>...
How Does DRS Do its Doo-Doo? <ul><li>Great, but how is this calculated? </li></ul><ul><ul><li>Anybody wanna ’ get  super g...
What About HA? TechMentor Cluster CRASH ! RESTART RESTART RESTART Chassis 1 Chassis 2 Chassis 3
What About HA? <ul><li>VMware HA must be configured for each Virtual Machine. </li></ul><ul><li>HA is a component of DRS. ...
Affinity / Anti-Affinity <ul><li>Within a DRS cluster, certain machines should remain on the same chassis. </li></ul><ul><...
Building in Cluster Reserve <ul><li>All of these calculations highlight the notion that you ’re always going to need  some...
Easter Egg:  Change DRS Invocation Frequency <ul><li>You can customize how often DRS will automatically take its own advic...
Final Thoughts <ul><li>For the love of gosh, turn on HA/DRS. </li></ul><ul><ul><li>But only if you have enough hardware! <...
HA & DRS Gotchas that will Kill your Infrastructure Greg Shields Partner and Principal Technologist Concentrated Technolog...
This slide deck was used in one of our many conference presentations. We hope you enjoy it, and invite you to use it withi...
Upcoming SlideShare
Loading in...5
×

Ha & drs gotcha's

440

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
440
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
26
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • MGB 2003 © 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • Greg Shields
  • MGB 2003 © 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
  • Ha & drs gotcha's

    1. 1. HA & DRS Gotchas that will Kill your Infrastructure Greg Shields Partner and Principal Technologist Concentrated Technology www.ConcentratedTech.com
    2. 2. This slide deck was used in one of our many conference presentations. We hope you enjoy it, and invite you to use it within your own organization however you like. For more information on our company, including information on private classes and upcoming conference appearances, please visit our Web site, www.ConcentratedTech.com . For links to newly-posted decks, follow us on Twitter: @concentrateddon or @concentratdgreg This work is copyright ©Concentrated Technology, LLC
    3. 3. Class Discussion <ul><li>Two Questions: </li></ul><ul><li>So……How often do you actually vMotion your virtual machines? </li></ul><ul><ul><li>Or, would you if you haven ’t yet deployed ESX. </li></ul></ul>
    4. 4. Class Discussion <ul><li>Two Questions: </li></ul><ul><li>So……How often do you actually vMotion your virtual machines? </li></ul><ul><ul><li>Or, would you if you haven ’t yet deployed ESX. </li></ul></ul><ul><li>What failure states does vMotion provide protection? </li></ul><ul><ul><li>Hint: They ’re fewer than you’d think… </li></ul></ul>
    5. 5. vMotion Solves Two Problems Problem #1: Protection from Host Failures or Scheduled Host Downtime (These are relatively rare) (They will get even more rare as we migrate to ESXi)
    6. 6. vMotion Solves Two Problems Problem #2: Load Balancing of VM & Host Resources (Much more common, where turned on)
    7. 7. Costs vs. Benefits <ul><li>High-availability adds dramatically greater uptime for virtual machines. </li></ul><ul><ul><li>Protection against host failures </li></ul></ul><ul><ul><li>Protection against resource overuse </li></ul></ul><ul><ul><li>Protection against scheduled/unscheduled downtime </li></ul></ul><ul><li>High-availability also adds much greater cost… </li></ul><ul><ul><li>Shared storage between hosts </li></ul></ul><ul><ul><li>Connectivity </li></ul></ul><ul><ul><li>Higher (and more expensive) software editions </li></ul></ul><ul><li>Not every environment needs HA! </li></ul><ul><ul><li>Does anyone want to argue this point? </li></ul></ul>
    8. 8. Contrary to Popular Belief… <ul><li>… seeing the actual vMotion process occur isn ’t all that sexy . </li></ul><ul><li>DEMO: Watching a vMotion occur… </li></ul>
    9. 9. vMotion: What Really Happens? <ul><li>Sexy is recognizing what ’s going on under the hood . </li></ul>
    10. 10. vMotion: What Really Happens? <ul><li>Sexy is recognizing what ’s going on under the hood . </li></ul><ul><li>Let ’s Compare to Hyper-V’s First Release </li></ul><ul><ul><li>Remember those days? A Hyper-V VM could be relocated “with a minimum of downtime”. </li></ul></ul>
    11. 11. vMotion: What Really Happens? <ul><li>Sexy is recognizing what ’s going on under the hood. </li></ul><ul><li>Let ’s Compare to Hyper-V’s First Release </li></ul><ul><ul><li>Remember those days? A Hyper-V VM could be relocated “with a minimum of downtime”. </li></ul></ul><ul><ul><li>This downtime was directly related to the amount of memory assigned to the virtual machine and the connection speed between virtual hosts and shared storage. </li></ul></ul><ul><ul><li>A VM with 2G of vRAM could take 32 seconds or longer to migrate! </li></ul></ul><ul><ul><li>Virtual machines with more assigned virtual memory and/or slow networks take longer to complete. </li></ul></ul><ul><ul><li>Those with less complete the migration faster. </li></ul></ul>
    12. 12. vMotion: What Really Happens? <ul><li>So…How does it work? </li></ul><ul><ul><li>As you invoked a Hyper-V “Quick Migration”, the virtual machine was immediately put into a saved state. </li></ul></ul><ul><ul><li>That state was not a power down, nor was it the same as pausing. </li></ul></ul><ul><ul><li>In the saved state – and unlike pausing – the virtual machine released its memory reservation on the host machine and stored the contents of its memory pages to disk. </li></ul></ul><ul><ul><li>Once this completed, a target host could take over the ownership of the virtual machine and bring it back to operations. </li></ul></ul>
    13. 13. vMotion: What Really Happens?
    14. 14. vMotion: What Really Happens? <ul><li>This saving of virtual machine state and transferring memory contents consumed downtime . </li></ul><ul><li>So, how is this different from vMotion (& HV today)? </li></ul>
    15. 15. vMotion: What Really Happens? <ul><li>This saving of virtual machine state and transferring memory contents consumed downtime . </li></ul><ul><li>So, how is this different from vMotion (& HV today)? </li></ul><ul><ul><li>vMotion and today ’s Hyper-V use a pre-copy mechanism. </li></ul></ul><ul><ul><li>Transfers VM memory pages from source to target host prior to starting the migration and while it is still running. </li></ul></ul><ul><ul><li>During the pre-copy memory changes are logged. </li></ul></ul><ul><ul><li>These tend to be relatively small in size. </li></ul></ul><ul><ul><li>Once the initial copy has completed, vMotion then… </li></ul></ul><ul><ul><ul><li>… pauses the virtual machine </li></ul></ul></ul><ul><ul><ul><li>… copies the memory deltas </li></ul></ul></ul><ul><ul><ul><li>… notifies network switches (RARP) & fibre fabric </li></ul></ul></ul><ul><ul><ul><li>… transfers ownership to the target host </li></ul></ul></ul><ul><li>The result: Effectively “zero” downtime. </li></ul>
    16. 16. vMotion: Gotcha ’s <ul><li>Successful vMotion requires similar processors. </li></ul><ul><ul><li>Processors must be from the same manufacturer. No Intel-to-AMD or AMD-to-Intel vMotioning. </li></ul></ul><ul><ul><li>Processors must be of a proximate families. </li></ul></ul><ul><ul><li>This bites people a few years down the road all the time ! </li></ul></ul>
    17. 17. vMotion: Gotcha ’s
    18. 18. vMotion: Gotcha ’s <ul><li>Big problem: As a virtual environment ages, hardware is refreshed and new hardware is added. </li></ul><ul><ul><li>New servers sometimes create “islands” of vMotion capability </li></ul></ul><ul><li>How can we always vMotion between computers? </li></ul>
    19. 19. vMotion: Gotcha ’s <ul><li>Big problem: As a virtual environment ages, hardware is refreshed and new hardware is added. </li></ul><ul><ul><li>New servers sometimes create “islands” of vMotion capability </li></ul></ul><ul><li>How can we always vMotion between computers? </li></ul><ul><ul><li>You can always refresh all hardware at the same time (Har!) </li></ul></ul>
    20. 20. vMotion: Gotcha ’s <ul><li>Big problem: As a virtual environment ages, hardware is refreshed and new hardware is added. </li></ul><ul><ul><li>New servers sometimes create “islands” of vMotion capability </li></ul></ul><ul><li>How can we always vMotion between computers? </li></ul><ul><ul><li>You can always refresh all hardware at the same time (Har!) </li></ul></ul><ul><ul><li>You can cold migrate, with the machine powered down. This always works, but ain ’t all that friendly. </li></ul></ul>
    21. 21. vMotion: Gotcha ’s <ul><li>Big problem: As a virtual environment ages, hardware is refreshed and new hardware is added. </li></ul><ul><ul><li>New servers sometimes create “islands” of vMotion capability </li></ul></ul><ul><li>How can we always vMotion between computers? </li></ul><ul><ul><li>You can always refresh all hardware at the same time (Har!) </li></ul></ul><ul><ul><li>You can cold migrate, with the machine powered down. This always works, but ain ’t all that friendly. </li></ul></ul><ul><ul><li>You can use vMotion Enhanced Compatibility Mode to manage your vMotion-ability. Create islands as individual clusters. </li></ul></ul><ul><li>DEMO: vMotion EVC </li></ul>
    22. 22. Class Discussion <ul><li>Blades versus Servers has been a common battle in virtualization spaces. </li></ul><ul><ul><li>Even I hated them for years. </li></ul></ul><ul><ul><li>Then, I saw the light. <Cue angels singing> </li></ul></ul><ul><li>Why are blades and EVC perfect for each other? </li></ul>
    23. 23. Class Discussion <ul><li>Blades versus Servers has been a common battle in virtualization spaces. </li></ul><ul><ul><li>Even I hated them for years. </li></ul></ul><ul><ul><li>Then, I saw the light. <Cue angels singing> </li></ul></ul><ul><li>Why are blades and EVC perfect for each other? </li></ul><ul><li>How are blades and Private Clouds perfect for each other? </li></ul>
    24. 24. New Topic! vMotion vs. Storage vMotion
    25. 25. vMotion vs. Storage vMotion
    26. 26. vMotion vs. Storage vMotion
    27. 27. vMotion vs. Storage vMotion <ul><li>Three options for migrating machines and disks. </li></ul><ul><ul><li>Online & Powered On: vMotion </li></ul></ul><ul><ul><li>Online & Powered On: Storage vMotion </li></ul></ul><ul><ul><li>Offline & Powered Off: vMotion + Storage vMotion at the same time. </li></ul></ul>
    28. 28. vMotion vs. Storage vMotion <ul><li>Requirements: </li></ul><ul><ul><li>Virtual machines with snapshots cannot be svMotioned. </li></ul></ul><ul><ul><li>Virtual machine disks must be persistent mode or RDMs. </li></ul></ul><ul><ul><li>The host must have sufficient resources to support two instances of the VM running concurrently for a brief time. </li></ul></ul><ul><ul><li>The host must have a vMotion license, and be correctly configured for vMotion. </li></ul></ul><ul><ul><li>The host must have access to both the source and target datastores. </li></ul></ul>
    29. 29. HA & DRS: Combining vMotion + Math
    30. 30. vMotion Itself Might Not be Sexy…
    31. 31. vMotion Itself Might Not be Sexy… <ul><li>… but what-you-can-do-with-it-once-you-combine-it-with-monitoring-and-a-set-of-smart-calculations… is ! Rawrrr… </li></ul>
    32. 32. vMotion Itself Might Not be Sexy… <ul><li>… but what-you-can-do-with-it-once-you-combine-it-with-monitoring-and-a-set-of-smart-calculations… is ! Rawrrr… </li></ul><ul><li>HA </li></ul><ul><ul><li>A host goes down, to where do I relocate VMs? </li></ul></ul><ul><li>DRS </li></ul><ul><ul><li>A host is overloaded, how should I re-balance VMs to optimize resource capacity to resource demand? </li></ul></ul><ul><li>These aren ’t trivial questions to answer! </li></ul>
    33. 33. vCenter Constructs <ul><li>Datacenter </li></ul><ul><ul><li>The boundary of a virtual infrastructure (& vMotion) </li></ul></ul><ul><li>Cluster </li></ul><ul><ul><li>Collection of ESX hosts for centralized management </li></ul></ul><ul><li>Resource Pool </li></ul><ul><ul><li>Logical abstraction of processing and memory capacity, which can be distributed to VMs as necessary. </li></ul></ul><ul><ul><li>Sub-collections of resource capacity for distribution to VMs based on business rules. </li></ul></ul><ul><li>DEMO: Properties of a Cluster </li></ul>
    34. 34. Shares, Reservations, & Limits <ul><li>Shares </li></ul><ul><ul><li>Identifies the ratio of resources a VM can consume </li></ul></ul><ul><li>Reservations </li></ul><ul><ul><li>Identifies minimum resources a VM is guaranteed </li></ul></ul><ul><ul><li>Ensures minimum service level during resource contention </li></ul></ul><ul><li>Limits </li></ul><ul><ul><li>Identifies maximum resources a VM may have </li></ul></ul><ul><ul><li>Protects against resource overuse (spiking) </li></ul></ul>
    35. 35. Understanding Shares & Resource Pools TechMentor Cluster PowerShell Deep Dive Resource Pool 1000 CPU & RAM Shares 2048 RAM ESXpert Deep Dive Resource Pool 2000 CPU & RAM Shares 4096 RAM WMI Class 200 CPU Shares 512M RAM AD Class 600 CPU Shares 784M RAM Storage Class 1200 CPU Shares 3G RAM vMotion Class 800 CPU Shares 512M RAM
    36. 36. Abstracting Resources <ul><li>EXTENDED DEMO: </li></ul><ul><ul><li>Reservations </li></ul></ul><ul><ul><li>Limits </li></ul></ul><ul><ul><li>Shares </li></ul></ul><ul><ul><li>Resource Pools </li></ul></ul><ul><li>Remember: These all factor into HA/DRS cluster automation calculations </li></ul>
    37. 37. What Role Does DRS Play? TechMentor Cluster HOT! Chassis 1 Chassis 2 Chassis 3
    38. 38. What Role Does DRS Play? Administrator trust is often the deciding factor in choosing the automation level. Automation Level Initial VM Placement Dynamic Balancing Administrator Involvement Manual Manual Manual High – All actions Partially-Automated Automatic Manual Moderate – Approval Actions Fully- Automated Automatic Automatic Low – Monitoring
    39. 39. How Does DRS Do its Doo-Doo? <ul><li>Five recommendation levels </li></ul><ul><ul><li>1 = DO THIS NOW! </li></ul></ul><ul><ul><li>2 = DO THIS (SORT OF) NOW </li></ul></ul><ul><ul><li>3 = Do this very soon-ish </li></ul></ul><ul><ul><li>4 = You know, if you have the time, you might consider… </li></ul></ul><ul><ul><li>5 = Meh. Get around to it when you have time. </li></ul></ul>
    40. 40. How Does DRS Do its Doo-Doo? <ul><li>Five recommendation levels </li></ul><ul><ul><li>1 = DO THIS NOW! </li></ul></ul><ul><ul><li>2 = DO THIS (SORT OF) NOW </li></ul></ul><ul><ul><li>3 = Do this very soon-ish </li></ul></ul><ul><ul><li>4 = You know, if you have the time, you might consider… </li></ul></ul><ul><ul><li>5 = Meh. Get around to it when you have time. </li></ul></ul>
    41. 41. How Does DRS Do its Doo-Doo? <ul><li>Great, but how is this calculated? </li></ul><ul><ul><li>Anybody wanna ’ get super geeky ? </li></ul></ul>Uh, duh... “ Ceiling” Operator
    42. 42. What About HA? TechMentor Cluster CRASH ! RESTART RESTART RESTART Chassis 1 Chassis 2 Chassis 3
    43. 43. What About HA? <ul><li>VMware HA must be configured for each Virtual Machine. </li></ul><ul><li>HA is a component of DRS. </li></ul><ul><li>DRS will analyze the resource load of the system at an HA event and decide where to restart the failed server. </li></ul><ul><li>Crashed system will restart on new chassis. Will incur an outage, but that outage will be short. </li></ul><ul><li>Both HA and DRS require DNS . </li></ul><ul><ul><li>This is exceptionally important! </li></ul></ul>
    44. 44. Affinity / Anti-Affinity <ul><li>Within a DRS cluster, certain machines should remain on the same chassis. </li></ul><ul><ul><li>E.g., an application server and it ’s database server </li></ul></ul><ul><li>Others should never </li></ul><ul><ul><li>E.g., two domain controllers </li></ul></ul><ul><li>Use affinity rules and anti-affinity rules to ensure correct placement of systems during DRS load balancing. </li></ul><ul><ul><li>Ensure systems aren ’t given conflicting rules </li></ul></ul><ul><li>DEMO: Affinity </li></ul><ul><ul><li>&quot;Serve the public trust. Protect the innocent. Uphold the law. ” </li></ul></ul>
    45. 45. Building in Cluster Reserve <ul><li>All of these calculations highlight the notion that you ’re always going to need somewhere to go . </li></ul><ul><ul><li>Hosts that die need to send VMs somewhere. </li></ul></ul><ul><ul><li>DRS needs a resource buffer if its to do its job. </li></ul></ul><ul><ul><li>You need expansion potential. </li></ul></ul><ul><li>That ’s why maintaining a cluster reserve is important. </li></ul><ul><li>LIVE DRAW: Cluster Reserve </li></ul>
    46. 46. Easter Egg: Change DRS Invocation Frequency <ul><li>You can customize how often DRS will automatically take its own advice. </li></ul><ul><ul><li>I wish my wife had this setting… </li></ul></ul><ul><li>On your vCenter Server, locate C:UsersAll UsersApplication DataVMwareVMware VirtualCenter vpxd.cfg </li></ul><ul><li>Add in the following lines (appropriately!): </li></ul>
    47. 47. Final Thoughts <ul><li>For the love of gosh, turn on HA/DRS. </li></ul><ul><ul><li>But only if you have enough hardware! </li></ul></ul><ul><ul><li>You ’ve already paid for it. </li></ul></ul><ul><ul><li>It is smarter than you. </li></ul></ul><ul><li>Understand why your VMs move around. </li></ul><ul><ul><li>… and always make sure that you ’ve got the correct connected resources that they need on every host! </li></ul></ul><ul><li>Save some cluster resources in reserve. </li></ul><ul><ul><li>You ’ll thank me for it! </li></ul></ul>
    48. 48. HA & DRS Gotchas that will Kill your Infrastructure Greg Shields Partner and Principal Technologist Concentrated Technology www.ConcentratedTech.com Please fill out evaluations, or your karma will suffer ! !!!
    49. 49. This slide deck was used in one of our many conference presentations. We hope you enjoy it, and invite you to use it within your own organization however you like. For more information on our company, including information on private classes and upcoming conference appearances, please visit our Web site, www.ConcentratedTech.com . For links to newly-posted decks, follow us on Twitter: @concentrateddon or @concentratdgreg This work is copyright ©Concentrated Technology, LLC
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×