Build Your Own Middleware Machine

Build Your Own Middleware Machine Simon Haslam Veriton Limited1 (1.0h)

Simon Haslam / Veriton Specialised consultant & Oracle Partner, established for 15 years Demanding web & call-centre applications Architecture & development strategy; health-checks; disaster recovery; tuning Oracle Fusion Middleware (JavaEE, SSO, OAM, OID, clustering) ADF Applications (esp. strategy & admin) Database & related technologies (Solaris/Linux, load balancers, firewalls, …)2 (1.0h)

Agenda1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Non-functional Factors5. Case Study6. Summary

Background to this presentation • Julian Dyke & I came up with ‘RAC-in-a- BOX’ in 2006 – 2 * HP DL365 (8 cores each) – 1 * MSA1000 SAN (HA controllers) – 2 * Cisco 3750 switches • Standard OFA style installation of RAC, config of ASM etc • Ready to plug into network & power4 (1.0h)

From the ‘Old’ Days to Today • Was physical hosts with DAS • Storage arrays for consolidating storage • Now: most mid-size sites – Production: physical hosts for production databases – Dev & Test: virtualised environments very common • (leaving discussions about public cloud / hybrid models)5 (1.0h)

From now to future Public Cloud Remote On Premise Dedicated Hardware for Application Compute platform Storage platform Private Appliance Cloud Appliance Or heavy consolidation Physical Virtualised6 (1.0h)

Trends • Green IT – Carbon costing – Power costs – SCR • Convergence – FCoE – iSCSI maturing • Technology – Speed of commodity hardware – Virtualisation – Networks VLANs: now traffic management, LACP etc. – 10GbE more common – Not the bandwidth/latency of InfiniBand but may be good enough for most, certainly mid-size, sites7 (1.0h)

Exalogic is BIG! • Even ¼ rack Exalogic is: – 8 compute nodes @ 12 = 96 cores – WebLogic Suite list price = £1.4M • Even if ½ processor & ½ NUP = £870k – Add: WLS Management Pack, Cloud Management Pack..? – List price of hardware $475k (~ £320k) • Many of my customers only have two or three big web/call centre apps: – modern processors => production can run on 2 compute nodes (for HA) – with virtualisation 1 compute nodes (12 cores) can run multiple test envs – many still have Forms & Reports – expansion of Java EE apps likely over next 5 years but not from Day 1 – Web app to database traffic is modest • Nothing wrong with Exalogic… it is just very BIG Oracle OpenWorld 2011: announcement 1/8 rack Exalogic (4 x 12 core 96GB compute nodes, same NAS and IB)8 (1.0h)

My ideal ♥ Middleware Machine! • Hardware – Compute: fast processors & ‘plenty’ of memory – Stateless compute nodes – Networks for public, private, management – Power management – Totally managed remotely – (note: I’ve not mentioned storage) • Software – WebLogic – Web Tier – Load Balancing – Management – Layered products (old and new)9 (1.0h)

Confession: I am a blade convert!10 (1.0h)

Agenda1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Non-functional Factors5. Case Study6. Summary

Ingredients: 1. Compute12 (1.0h)

Ingredients: 2. Power • Single or 3 phase • Note: health and safety implications13 (1.0h)

Ingredients: 3. Storage14 (1.0h)

Ingredients: 4. Network • Core switches • Load balancers15 (1.0h)

Bake & Leave to Cool16 (1.0h)

Traditional Rackmount vs Blade? • Power consumption http://www.veriton.co.uk/roller/fmw/entry/... hp_blade_vs_rack_servers • Costs • Note some chassis-wide costs: – Converged networking – Administration modules • Vendor lock-in17 (1.0h)

Changes from adding fans/ power supplies Size of error bars unknown!18 (1.0h)

Exalogic 1/4 rack 8n (inc storage) 5.3-7.2kW* Exalogic 1/8th rack 4n (inc storage) 2.4-4.3kW* Exalogic 1/2 rack 16n: 8-11kW 1 rack 32n: 13-18kW Note: Exalogic has 96GB RAM per node cf 24GB, SSD boot disks and slightly faster procs (3.0GHz cf 2.8GHz)19 (1.0h) * Exalogic “typical” to “max” (from Oracle datasheet 14-NOV-11) – rounded to 2 sig. fig.

Compute Node Storage • What would you put on a server if you had shared storage? – Operating system files? – Swap space – Crash dumps – Application log files – Transaction logs?! • Do you need SSD for this? • As soon as you add storage to a server you make it stateful20 (1.0h)

Hardware Summary • Very fast processors • No local storage; boot from SAN • Shared 10GbE networking (max 20GbE per blade) • Centralised hardware management • Centralised power management21 (1.0h)

Agenda1. My Ideal Middleware Machine2. Hardware3. Software & Configuration4. Case Study5. Non-functional Factors6. Summary

Provisioning • Stateless blades (hardware) – No disk (or SDcard!) – No MAC addresses – No WWNs – No Lights-Out Management addresses = ~virtualisation of the hardware23 (1.0h)

Hardware/OS Configuration • Pre-assign IP addresses – Management – Public Prod – Test – Private – Cluster24 (1.0h)

Software Installation - today • Centralised Deployment – Scripted (e.g. WLST & response files) – Enterprise Manager – Deployment Pack (still some work to be done)25 (1.0h)

Virtualisation - notes • VMs – build from kickstart/jumpstart etc • There’s a lot to learn from AWS • Oracle Virtual Assembly Builder is a very interesting product for the future – VM template libraries • OVAB probably the future if you’re on OVM • OVAB apparently underpinning Java Service of Oracle Public Cloud26 (1.0h)

Middleware Installation • Shared middleware homes – Pros and Cons – Great for support ☺ • E.g. Exalogic EDG27 (1.0h)

Case Study • Mid-sized UK customer • ~800 agents in 4 call centres + web site • Previously 10g iAS on 6/8 2U rack servers (production, primary site) • Migration to WebLogic Suite 11g (plus OID 11g & Reports 11g) • Moving from Red Hat to Oracle Linux 5.6* • Virtualised • Centralised Oracle homes and management29 (1.0h)

How did it go? • Power was a challenge – you need to size for full capacity • Cooling – fortunately able to free up space, but could have been tricky due to heat density • Boot from SAN – Not really a problem – much more common these days – I suspect (I) will get blamed for anything bad • Networking – Forced through a rethink of some of the VLANs and subnets – We did have some problems with LACP/trunking30 (1.0h)

Now? • A project appeared from blue, sneaked in and went live on the blade system before Oracle project out of testing! • Chassis will probably be full within a year31 (1.0h)

Remember… Licences and support Hardware33 (1.0h)

Politics • Multiple teams are often responsible for support: – Application Development – Oracle Administration – System (e.g. Unix) Administration – Storage Management – Data Centre Management34 (1.0h)

Real-World Example in 2011 • Oracle team had bought lots of SOA Suite licences (sizing method a bit unknown – partly historical, partly reseller) • Production hardware still to be purchased • Licences only efficiently “fitted” on certain combinations of hardware at primary and DR site • System Administration team different to Oracle/project team • Cost of Oracle licences > 15x cost of hardware • Cost centre for hardware/OS/VMware seemed to be different from cost centre for Oracle => smallest amount of hardware bought* Oracle perpetual licences plus Support/Updates unused • This is not unusual behaviour! * I do have some sympathy here – the licences should have been right-sized in the first place35 (1.0h)

Real-World Example: What if they had chosen Exalogic? • 2 Exalogic ¼ racks (production and DR, would have probably had to run dev/test on DR) • Presumably Oracle licences would be bought at same time so match One decision to make 1 Purchase Order Choice cannot be ‘unbalanced’36 (1.0h)

Exalogic • Comes assembled & cabled up from Oracle • All decisions have been taken for you – you can only choose the size • Oracle love you & you’ll probably get a better deal on your licences • Oracle remote diagnostics & single point of support • You’ll get all the best Middleware features & super performance, especially if you have Exadata too • OVAB etc means it may be more “plug & go” • Oracle “Red Stack Administrator” is King/Queen! • But: – May have to buy more hardware than you need up-front – You’ll need at least 2 & possibly 3 – all are same (top) spec – Your admins may not be used to Sun hardware, LOM etc – (Vendor lock-in)38 (1.0h)

Build Your Own (Blade) • Flexibility to run other stuff on same infrastructure • Very neat cabling • Can buy the fastest processors and migrate slower ones to test envs • Great automation options • Hardware will be consistent with other non-Oracle stuff • Fits in with virtualised compute layer, storage layer, network layer => very clean architecture • But: – You’ll probably need to hire a consultant to set up to get this degree of automation (costly) – You may have internal bickering over storage, network, boot from SAN, spec of servers etc – Oracle hold back best features for Exalogic – With blades you will still have vendor lock-in39 (1.0h)

• Exalogic is bigger than many organisations need • Reasons to buy Exalogic may be as much political as technical • Modern alternative ways to build a “middleware machine” include stateless compute nodes booting directly from centralised storage • If building your own - careful configuration required with scripted management – don’t reinvent the wheel. Standardise!41 (1.0h)

Thanks for listening! Twitter: @simon_haslam Oracle Fusion Middleware Admin blog: http://simonhaslam.co.uk42 (1.0h)

Build Your Own Middleware Machine

by Veriton Limited on Dec 11, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 96

Statistics

Build Your Own Middleware Machine — Presentation Transcript