Data is being generated at rates never before encountered. The explosion of data threatens to consume all of our IT resources: People, budget, power, cooling and data center floor space. Are your systems coping with your data now? Will they continue to deliver as the stress on data centers increases and IT budgets dwindle?
Imagine if you could be ahead of the data explosion by being proactive about your storage instead of reactive. Now you can be, with NetApp's approach to the designs and deployment of storage systems. With it, you can take advantage of NetApp's latest storage enhancements and take control of your storage. This will allow you to focus on gathering more insights from your data and deliver more value to your business.
NetApp's most advanced storage solutions are NetApp Virtualization & scale out. By taking control of your existing storage platform with either solution, you get:
• Immortal Storage system
• Infinite scalability
• Best possible ROI from existing environment
Data is being generated at rates never before encountered. The explosion of data threatens to consume all of our IT resources: People, budget, power, cooling and data center floor space. Are your systems coping with your data now? Will they continue to deliver as the stress on data centers increases and IT budgets dwindle?
Imagine if you could be ahead of the data explosion by being proactive about your storage instead of reactive. Now you can be, with NetApp's approach to the designs and deployment of storage systems. With it, you can take advantage of NetApp's latest storage enhancements and take control of your storage. This will allow you to focus on gathering more insights from your data and deliver more value to your business.
NetApp's most advanced storage solutions are NetApp Virtualization & scale out. By taking control of your existing storage platform with either solution, you get:
• Immortal Storage system
• Infinite scalability
• Best possible ROI from existing environment
Basic knowledge of Storage technology and complete understanding on DAS, NAS & SAN with advantages and disadvantages. A quick understanding on storage will help you make the best decision in terms of cost and need.
IBM DS8880 and IBM Z - Integrated by DesignStefan Lein
This Presentation shows the strength of the IBM DS8880 Enterprise Storage Platform with special emphasis on the System Z integration capabilities. December 2017
Complete configuration of SAN using ESXI Environment and Installation guide. Now you will be able to configure storage area network with the help of these slides.
This configuration helps user to configure ESXI 4, ESXI 3.0 Servers
VxRail Appliance - Modernize your infrastructure and accelerate IT transforma...Maichino Sepede
An overview of the VxRail Appliance, including what’s new with VxRail on the 14th generation PowerEdge server, and advancements in the VxRail 4.5 software.
Deploy Storage Spaces
The Contoso Ltd. contains an Active Directory domain named contoso.com. There is a server named Server1 that runs Windows Server 2012 R2.
The company currently looking for a storage solution that must meet the following requirements:
a. Provide fault tolerance if a single disk fails.
b. Maximize the amount of files that can be stored in the storage space.
c. New storage can be added anytime as per requirement.
In the year 1984, a young Steve Jobs unveiled the first Apple Macintosh computer with a verse from Bob Dylan’s famous song - “The times they are a changing”.
Basic knowledge of Storage technology and complete understanding on DAS, NAS & SAN with advantages and disadvantages. A quick understanding on storage will help you make the best decision in terms of cost and need.
IBM DS8880 and IBM Z - Integrated by DesignStefan Lein
This Presentation shows the strength of the IBM DS8880 Enterprise Storage Platform with special emphasis on the System Z integration capabilities. December 2017
Complete configuration of SAN using ESXI Environment and Installation guide. Now you will be able to configure storage area network with the help of these slides.
This configuration helps user to configure ESXI 4, ESXI 3.0 Servers
VxRail Appliance - Modernize your infrastructure and accelerate IT transforma...Maichino Sepede
An overview of the VxRail Appliance, including what’s new with VxRail on the 14th generation PowerEdge server, and advancements in the VxRail 4.5 software.
Deploy Storage Spaces
The Contoso Ltd. contains an Active Directory domain named contoso.com. There is a server named Server1 that runs Windows Server 2012 R2.
The company currently looking for a storage solution that must meet the following requirements:
a. Provide fault tolerance if a single disk fails.
b. Maximize the amount of files that can be stored in the storage space.
c. New storage can be added anytime as per requirement.
In the year 1984, a young Steve Jobs unveiled the first Apple Macintosh computer with a verse from Bob Dylan’s famous song - “The times they are a changing”.
FC SAN vs iSCSI SAN: Which One is Right for Your Business?
.pdfMaryJWilliams2
FC SANs and iSCSI SANs are two of the most popular storage area network (SAN) technologies available. Each has its own advantages and disadvantages, so it's important to choose the right one for your specific needs. In this blog post, we compare FC SANs and iSCSI SANs to help you make an informed decision.
For more information visit : https://stonefly.com/storage/nas-storage
• Storage Basics and Terms
• Introduction to SANs
• SAN Connectivity—Fibre Channel
• SAN Applications
• SAN—Current Developments
• Cisco Storage Networking Partnerships
Sage Weil, Ceph Principal Architect at Red Hat discusses open source versus open standards and why open source software will be the key driver in the next revolution of storage.
It's the End of Data Storage As We Know It (And I Feel Fine)Stephen Foskett
Technological change is finally coming to storage, and it will wipe away the architecture we've come to know over the last few decades. Say goodbye to the "do it all" Fibre Channel SAN storage array and get ready for converged infrastructure, distributed storage, alternative attachments like PCIe, and top-of-rack flash! In this session, Stephen Foskett will explain why this change is inevitable and how it will shake out. You won't recognize what's coming, but it will be faster, cheaper, and more integrated than ever! Delivered at
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
2. Objectives for this UnitObjectives for this Unit
Understand Storage BasicsUnderstand Storage Basics
Introduce Direct Attached StorageIntroduce Direct Attached Storage
Differentiate between types of storageDifferentiate between types of storage
DASDAS
NASNAS
ISCSIISCSI
SANSAN
Introduce Network Attached StorageIntroduce Network Attached Storage
Introduce Fibre Channel SANIntroduce Fibre Channel SAN
3. A Few Storage Basics….A Few Storage Basics….
Where will data finally end up?Where will data finally end up?
How will it get there?How will it get there?
What will it pass through?What will it pass through?
4. Direct Attached StorageDirect Attached Storage
(Internal)(Internal)
Computer System
CPU
Memory
Bus
I/O - RAID
Controller
Disk Drives
5. Direct Attached StorageDirect Attached Storage
(Internal)(Internal)
Computer System
CPU
Memory
Bus
I/O - RAID
Controller
Disk Drives
12345
John
Sm
ith
512-555-1212
1424 Main Street
Data
6. Direct Attached StorageDirect Attached Storage
(Internal)(Internal)
Computer System
CPU
Memory
Bus
I/O - RAID
Controller
Disk Drives
12345
John
Sm
ith
512-555-1212
1424 Main Street
7. DAS w/ internal controller andDAS w/ internal controller and
external storageexternal storage
CPU
Memory
Bus
I/O - RAID
Controller
Computer System
Disk Drives
Disk Drives
Disk Drives
Disk Enclosure
12345
John
Sm
ith
512-555-1212
1424 Main Street
8. Comparing Internal and ExternalComparing Internal and External
StorageStorage
Internal Storage
Server
Storage
RAID controllers
and disk drives
are internal to the
server
SCSI, ATA, or
SATA protocol
between controller
and disks
SCSI Bus w/ external storage
Server
RAID Controller
Storage
RAID Controller
Disk Drives
RAID controller is
internal
SCSI or SATA
protocol between
controller and
disks
Disk drives are
external
Disk Drives
9. DAS w/ external controller andDAS w/ external controller and
external storageexternal storage
Computer System
CPU
Memory
Bus
HBA
RAID
Controller
Storage System
Disk Drives
Disk Drives
Disk Drives
Disk Enclosure12345
John
Sm
ith
512-555-1212
1424 Main Street
10. DAS over Fibre ChannelDAS over Fibre Channel
Server
HBA
Storage
Disk drives and
RAID controller
are external
Disk Drives
RAID Controller
HBA is internal
Fibre Channel
protocol
between HBAs
and external
RAID controller
External SAN Array
11. I/O TransferI/O Transfer
RAID ControllerRAID Controller
Contains the “smarts”Contains the “smarts”
Determines how the data will be written (striping,Determines how the data will be written (striping,
mirroring, RAID 10, RAID 5, etc.)mirroring, RAID 10, RAID 5, etc.)
Host Bus Adapter (HBA)Host Bus Adapter (HBA)
Simply transfers the data to the RAID controller.Simply transfers the data to the RAID controller.
Doesn’t do any RAID or striping calculations.Doesn’t do any RAID or striping calculations.
““Dumb” for speed.Dumb” for speed.
Required for external storage.Required for external storage.
12. Storage typesStorage types
Single Disk DriveSingle Disk Drive
JBODJBOD
VolumeVolume
Storage ArrayStorage Array
SCSI deviceSCSI device
DASDAS
NASNAS
SANSAN
iSCSIiSCSI
13. NAS: What is it?NAS: What is it?
Network Attached StorageNetwork Attached Storage
Utilizes a TCP/IP network to “share” dataUtilizes a TCP/IP network to “share” data
Uses file sharing protocols like Unix NFSUses file sharing protocols like Unix NFS
and Windows CIFSand Windows CIFS
Storage “Appliances” utilize a stripped-Storage “Appliances” utilize a stripped-
down OS that optimizes file protocoldown OS that optimizes file protocol
performanceperformance
14. Networked Attached StorageNetworked Attached Storage
NAS Server
Storage
Server has a
Network Interface
Card
No RAID
Controller or HBA
in the server
Public or Private Ethernet
network
RAID Controller
Disk Drives
All data converted
to file protocol for
transmission (may
slow down
database
transactions)
Server
NIC NIC
15. iSCSI: What is it?iSCSI: What is it?
An alternate form of networked storageAn alternate form of networked storage
Like NAS, also utilizes a TCP/IP networkLike NAS, also utilizes a TCP/IP network
Encapsulates native SCSI commands in TCP/IPEncapsulates native SCSI commands in TCP/IP
packetspackets
Supported in Windows 2003 Server and LinuxSupported in Windows 2003 Server and Linux
TCP/IP Offload Engines (TOEs) on NICs speedTCP/IP Offload Engines (TOEs) on NICs speed
up packet encapsulationup packet encapsulation
16. iSCSI StorageiSCSI Storage
iSCSI Storage
Server has a
Network Interface
Card or iSCSI
HBA
iSCSI HBAs use
TCP/IP Offload
Engine (TOE)
Public or Private Ethernet
network
RAID Controller
Disk Drives
SCSI commands
are encapsulated
in TCP/IP packets
Server
NIC or iSCSI HBA NIC or iSCSI HBA
17. Fibre Channel: What is it?Fibre Channel: What is it?
Fibre Channel is a network protocolFibre Channel is a network protocol
implemented specifically for dedicatedimplemented specifically for dedicated
storage networksstorage networks
Fibre Channel utilizes specializedFibre Channel utilizes specialized
SwitchesSwitches
Host Bus AdaptersHost Bus Adapters
RAID controllersRAID controllers
CablesCables
18. switch
Fibre Channel ComponentsFibre Channel Components
RAID Controller RAID Controller
switch
ServersServers
Host Bus AdaptersHost Bus Adapters
CablesCables
Fiber optic or copperFiber optic or copper
Fibre Channel SwitchesFibre Channel Switches
Two switches for redundancyTwo switches for redundancy
Fibre Channel Storage ArrayFibre Channel Storage Array
Two RAID Controllers forTwo RAID Controllers for
redundancyredundancy
4–100+ disk drives per array4–100+ disk drives per array
A true storage networkA true storage network
Multiple serversMultiple servers
Multiple switchesMultiple switches
Multiple Storage ArraysMultiple Storage Arrays
Server A
HBA
Server B
HBA
Server C
HBA
Disk Drives
FC Storage Array
HBA HBA HBA
19. SAN: What is it?SAN: What is it?
Storage Area NetworkStorage Area Network
A network whose primary purpose is the transferA network whose primary purpose is the transfer
of data between storage systems and computerof data between storage systems and computer
systemssystems
Fibre Channel is the primary technology utilizedFibre Channel is the primary technology utilized
for SANsfor SANs
Recently, SANs have been implemented withRecently, SANs have been implemented with
dedicated iSCSI networksdedicated iSCSI networks
20. Benefits of SAN/ConsolidatedBenefits of SAN/Consolidated
StorageStorage
Reduce cost of external storageReduce cost of external storage
Increase performanceIncrease performance
Centralized and improved tape backupCentralized and improved tape backup
LAN-less backupLAN-less backup
High-speed, no single-point-of-failureHigh-speed, no single-point-of-failure
clustering solutionsclustering solutions
Consolidation with > 70TB of storageConsolidation with > 70TB of storage
21. Fibre Channel TechnologyFibre Channel Technology
Provides concurrent communications between servers,Provides concurrent communications between servers,
storage devices, and other peripheralsstorage devices, and other peripherals
A gigabit interconnect technologyA gigabit interconnect technology
FC1: Over 1,000,000,000 bits per secondFC1: Over 1,000,000,000 bits per second
FC2: Over 2,000,000,000 bits per secondFC2: Over 2,000,000,000 bits per second
A highly reliable interconnectA highly reliable interconnect
Up to 127 devices (SCSI: 15)Up to 127 devices (SCSI: 15)
Up to 10 km of cabling (3-15 ft. for SCSI)Up to 10 km of cabling (3-15 ft. for SCSI)
Physical interconnect can be copper or fiber opticPhysical interconnect can be copper or fiber optic
22. Fibre Channel – (continued)Fibre Channel – (continued)
Hot-pluggable - Devices can be removedHot-pluggable - Devices can be removed
or added at will with no ill effects to dataor added at will with no ill effects to data
communicationscommunications
Provides a data link layer above theProvides a data link layer above the
physical interconnect, analogous tophysical interconnect, analogous to
EthernetEthernet
Sophisticated error detection at the frameSophisticated error detection at the frame
levellevel
Data is checked and resent if necessaryData is checked and resent if necessary
23. Fibre Channel – FrameFibre Channel – Frame
DissectionDissection
Up to 2048 byteUp to 2048 byte
payloadpayload
4 byte checksum4 byte checksum
for each framefor each frame
24. Fibre ChannelFibre Channel
What’s with the funny name?What’s with the funny name?
Some background history requiredSome background history required
Originally developed to only support fiber opticOriginally developed to only support fiber optic
cablingcabling
When copper cabling support was added, ISOWhen copper cabling support was added, ISO
decided not to rename the technologydecided not to rename the technology
ISO changed to the French spelling to reduceISO changed to the French spelling to reduce
association with fiber optics only mediumassociation with fiber optics only medium
25. Fibre ChannelFibre Channel
How does it work?How does it work?
Serial interfaceSerial interface
Data is transferred across a single piece ofData is transferred across a single piece of
medium at the fastest speed supportedmedium at the fastest speed supported
No complex signaling requiredNo complex signaling required
27. SCSI vs. Fibre ChannelSCSI vs. Fibre Channel
ProtocolProtocol
SCSISCSI
SCSI protocol vs. SCSI deviceSCSI protocol vs. SCSI device
SCSI is an established, tried and true protocolSCSI is an established, tried and true protocol
Provides services analogous to TCP/IPProvides services analogous to TCP/IP
Supported in every major OS on marketSupported in every major OS on market
Fibre ChannelFibre Channel
Fibre Channel runs on top of SCSIFibre Channel runs on top of SCSI
No re-inventing the wheelNo re-inventing the wheel
Immediate OS supportImmediate OS support
28. SCSI vs. FC TransmissionSCSI vs. FC Transmission
RAID Controller Disk Drive
Fibre Channel
SCSI
RAID Controller Disk Drive
29. SCSISCSI vs.vs. Fibre ChannelFibre Channel
Interface for internal storage toInterface for internal storage to
external disksexternal disks
Potential down time w/ SCSIPotential down time w/ SCSI
Single busSingle bus
RAID controller is SCSI hardwareRAID controller is SCSI hardware
Standards:Standards:
Ultra2 (80 MB/sec)Ultra2 (80 MB/sec)
Ultra 160 (160 MB/sec)Ultra 160 (160 MB/sec)
Ultra 320 (320 MB/sec)Ultra 320 (320 MB/sec)
Media specific (copper only)Media specific (copper only)
SCSI Limitations:SCSI Limitations:
Cables can’t be any longerCables can’t be any longer
than 3 feet for single ended;than 3 feet for single ended;
15 feet for LVD (low voltage15 feet for LVD (low voltage
differential)differential)
No more than 15 devices onNo more than 15 devices on
a SCSI busa SCSI bus
# of disk drives# of disk drives
Used with SANUsed with SAN
Lots of built-in redundancy with connectionsLots of built-in redundancy with connections
Redundant networkRedundant network
HBA is fibre channel hardwareHBA is fibre channel hardware
Standards:Standards:
FC1: 100 MB/secFC1: 100 MB/sec
FC2: 200 MB/secFC2: 200 MB/sec
Provides a data link layer above the physicalProvides a data link layer above the physical
interconnectinterconnect
Analogous to EthernetAnalogous to Ethernet
FC is a network of devicesFC is a network of devices
It can be media independent- copper orIt can be media independent- copper or
fibre opticfibre optic
Fibre Channel limitations:Fibre Channel limitations:
Cable length: Up to 10 kilometers (moreCable length: Up to 10 kilometers (more
a limitation of cable than FC itself)a limitation of cable than FC itself)
Up to 127 devicesUp to 127 devices
# of disk drives# of disk drives
30. Fibre Channel vs. iSCSIFibre Channel vs. iSCSI
Fibre ChannelFibre Channel
The current market leader for shared storage technologiesThe current market leader for shared storage technologies
Provides the highest performance levelsProvides the highest performance levels
Designed for mission-critical applicationsDesigned for mission-critical applications
Cost of components is relatively high, particularly per serverCost of components is relatively high, particularly per server
HBA costsHBA costs
Relatively difficult to implement and manageRelatively difficult to implement and manage
iSCSIiSCSI
Relatively new, but usage is increasing rapidlyRelatively new, but usage is increasing rapidly
Performance can approach Fibre Channel speedsPerformance can approach Fibre Channel speeds
A better fit for databases than NASA better fit for databases than NAS
A good fit for Small to Medium Size BusinessesA good fit for Small to Medium Size Businesses
Relatively inexpensive, compared to Fibre ChannelRelatively inexpensive, compared to Fibre Channel
Relatively easy to implement and manageRelatively easy to implement and manage
31. Microsoft Simple SAN InitiativeMicrosoft Simple SAN Initiative
Make operating system aware of SAN and SANMake operating system aware of SAN and SAN
capabilitiescapabilities
Shift integration burden from IT staff or services backShift integration burden from IT staff or services back
to VENDORS products:to VENDORS products:
MicrosoftMicrosoft
Storage hardware and softwareStorage hardware and software
Application developersApplication developers
Key storage technologies:Key storage technologies:
Volume Shadow Copy Service (VSS)Volume Shadow Copy Service (VSS)
Virtual Disk Service (VDS)Virtual Disk Service (VDS)
Microsoft Multipath Input/Output (MPIO)Microsoft Multipath Input/Output (MPIO)
Microsoft iSCSI driverMicrosoft iSCSI driver
Software Initiator (client)Software Initiator (client)
Software Target (attached to disk subsystem)Software Target (attached to disk subsystem)
32. ReviewReview
What is the difference between a RAIDWhat is the difference between a RAID
Controller and an HBA?Controller and an HBA?
How many protocols may be used for DAS?How many protocols may be used for DAS?
Name two types of storage that rely on ethernetName two types of storage that rely on ethernet
cablescables
Name two benefits of SANsName two benefits of SANs
Describe the four interface layers of the FibreDescribe the four interface layers of the Fibre
Channel protocolChannel protocol
Describe a scenario where an iSCSI SAN mayDescribe a scenario where an iSCSI SAN may
be preferred over a Fibre Channel SANbe preferred over a Fibre Channel SAN
33. SummarySummary
How data is routed through a server to I/OHow data is routed through a server to I/O
Types of storageTypes of storage
DASDAS
NASNAS
iSCSIiSCSI
SANSAN
Benefits of SAN technologyBenefits of SAN technology
Storage consolidationStorage consolidation
Reduced costsReduced costs
Centralized, LAN-free backup and restoreCentralized, LAN-free backup and restore
The Fibre Channel protocolThe Fibre Channel protocol
How it worksHow it works
Fibre Channel protocol vs. SCSI protocolFibre Channel protocol vs. SCSI protocol
Comparing Fibre Channel SANs and iSCSI SANsComparing Fibre Channel SANs and iSCSI SANs
Fibre Channel SANs offer mission-critical performance, with relatively highFibre Channel SANs offer mission-critical performance, with relatively high
costs and high complexitycosts and high complexity
iSCSI SANs offer moderate to high performance at an attractiveiSCSI SANs offer moderate to high performance at an attractive
price/performance ration and are relatively easy to administerprice/performance ration and are relatively easy to administer
Editor's Notes
These are the topics that will be covered in this unit.
Information (data) is constantly flowing in and out of computers. Data coming in is either stored in memory for quick access (but not permanent storage) or written to disk so that it can be retrieved later. How this information is stored depends on a couple of things: - How will it be stored (what will it be stored ON)? - What is the protocol by which this data will be transferred? - What devices will it pass through on the way to being stored?
This is an example of traditional internal data storage. (Similar to what you might have on your laptop or desktop computer.)
Data from outside the system is entered via keyboard or some other interface…
… and goes through the CPU, memory, bus, RAID controller and into internal storage – i.e. a hard disk drive.
Again, data from outside the system passes through the CPU, memory, bus, and RAID controller. In this case, it then is passed outside the system to some type of external disk enclosure where it is written to disk drives. Even though the data is stored outside the computer system, this is still referred to as Direct Attached Storage since the RAID controller is internal to the computer system, and the disk drives are attached to the controller via a single path.
Regardless of whether storage is internal to the server or external, there must be a connection between the RAID controller and the disk drives, and a communication protocol must be used to communicate across this connection. In the case of external storage, a cable connects the RAID controller and the external array. Even for internal storage, a short “ribbon” cable is commonly used to connect the RAID controller and disks. Most desktop systems with internal storage utilize the ATA or Serial ATA (SATA) protocols to communicate with disks. For desktop systems, internal disks are either ATA disks or SATA disks, depending on the protocol. ATA is a parallel communication protocol that can be used to communicate with a limited number of devices over a short distance. SATA is a serial form of the ATA protocol, which allows a true cable to be used, and allows a greater distance between the controller and disks. SCSI is a parallel protocol that allows increased distance between the controller and disks. SCSI is often used for internal storage on servers. When SCSI is used for internal storage, SCSI disks are used as well. Although SCSI may be used for internal storage on servers, its main use is with external storage. A single RAID controller may attach to up to 14 disks in an external cabinet. Most commonly, external disks attached to a SCSI RAID controller are SCSI disks. However, protocol converters may be used within the external cabinet to allow ATA or SATA drives to be attached to the SCSI bus. This is commonly done to lower disk costs. In addition, the SATA protocol allows “pure” SATA external arrays to be built. The RAID controller utilizes SATA to communicate with SATA disks, with similar characteristics to external SCSI or Fibre Channel arrays (there will be more about Fibre Channel in the next section). However, SATA arrays are currently slower and less reliable than SCSI arrays. Due to their low cost, they are ideal for backup-to-disk strategies and nearline storage. Evolving specifications for SATA promise competitive characteristics to SCSI or even Fibre Channel in the future.
A true external storage system is defined as external by the fact that the disk enclosure and “smarts” of the system (RAID controller) are outside the computer that is receiving the data. In this case, the flow of data on the computer system is again through the CPU, memory and a bus, but then it is transferred to the external storage system through a Host Bus Adapter (HBA). From the RAID controller on the external system, the data is then passed into storage on the disk drives.
To allow an external storage cabinet with an external RAID controller, a more flexible communication protocol is needed than SCSI. The Fibre channel protocol allows the “smart” RAID controller to reside outside the server, connected through a relatively “dumb” Host Bus Adapter (HBA). A single HBA (or two HBAs) may be directly attached to the external array, in a manner similar to SCSI or SATA Direct Attached Storage. However, the external array may contain many more disks than a SCSI array. (Disks may be Fibre, SCSI, SATA, or ATA with appropriate converters). In addition, Fibre Channel connections are not limited to a single direct path. Storage communication paths may split and merge, to form a true network. We will discuss this more under the Storage Area Network section.
From the perspective of the server, the main difference between storage attached via Fibre Channel and storage attached via SCSI is the nature of the interface hardware. SCSI controllers contain most of the intelligence needed to manage the array “onboard” the server. With Fibre Channel, the RAID controller is actually outside the server, in the external storage cabinet. To communicate with the outside world, the server utilizes a Host Bus Adapter (HBA). An HBA is a simpler device than a RAID controller, simply transferring the data over a cable to an external RAID controller. Although the use of an HBA offloads storage processing to the external RAID controller, it does not guarantee faster performance for any given IO. The same amount of processing for the IO must be done in either SCSI or Fibre Channel scenarios. An external SCSI array may actually be faster for some types of IO. Fibre Channel does enable greater flexibility on how storage is networked with servers. Fibre Channel may also offer greater throughput for multiple simultaneous IOs on one or more servers attached to the same external storage array.
How data is stored once it gets to the storage disk drive(s) depends on the type of storage selected. Data storage comes in many different formats. We’re all familiar with what it’s like save a file to our hard drive or to a floppy or CD. Those are all forms of storage. Obviously, it can get a lot more complicated than that. Following is a list of the most common types of data storage: Single disk drive (self explanatory) JBOD – just a bunch of disks. This is collection of disk drives pooled together for storage, but without any RAID, striping, etc. Volume – a “logical” disk drive. A concatenation of drives. When one fills up, it goes to the next one. No RAID, no striping. To the OS, a logical volume looks like one disk drive. Storage Array – Also a group of more than one disk joined together – but can do striping and/or redundancy. Implies some type of RAID (whatever the level). SCSI - SCSI stands for Small Computer System Interface. It is a means of attaching additional storage to a computer. For example, a typical RAID Controller is a SCSI device that allows connection to an external storage enclosure with multiple drives. NAS – Network Attached Storage. Sometimes rather than simply attaching storage to one machine, it is attached to the computer network. That way, multiple machines can access the storage. A file protocol must be used to communicate across the network. ISCSI – Internet/SCSI protocol. Another approach to offering storage on a network. Rather then using file protocols to communicate across a TCP/IP network, native SCSI commands are “encapsulated” in TCP/IP packets. An evolving standard that has already been adopted in Windows 2003 Server. SAN - Storage Area Network. Whereas a NAS is storage that is attached to a network, SAN is a storage network in and of itself that can be attached to multiple machines. SAN is an industry-wide term for both the storage and the switching network. A SAN does not have the protocol conversion overhead of NAS or ISCSI, and tends to offer better performance. However, a SAN may require a higher initial investment in infrastructure.
A NAS (Network Attached Storage) is designed to provide shared access to storage across a standard TCP/IP network. Sharing data across TCP/IP is accomplished by converting block-level SCSI commands to file sharing protocols. Common file sharing protocols include the UNIX Network File System (NFS) and the Windows Common Internet File System (CIFS). Linux or Windows servers may be used to share network files. However, “Appliance” servers are becoming readily available that offer better performance. These servers utilize a stripped-down operating system that is built to optimize file protocol management, and commonly support multiple file sharing protocols.
Unlike DAS or SAN, there is no RAID Controller or HBA on the server. Instead, a Network Interface Card is used to communicate with the NAS “server” across the TCP/IP network. The NAS server also utilizes a TCP/IP card. The ethernet network can be either a private or public network. Due to the data traffic and security concerns, a VLAN is preferred when using the public network. Native SCSI commands address storage at the block level. However, native TCP/IP can only communicate storage information at a higher logical level – the file protocol level. This means that a server must send file level requests over TCP/IP to the NAS “server”, which must convert file protocol information to block level SCSI information in order to talk to the disks. Returning data must be converted from block level disk information to file protocol once again and send it across the network cable in TCP/IP packets. Although a Gigabit Ethernet network is fast, all of this protocol conversion incurs significant overhead. The situation is even worse for database requests, because the database “talks” only in block level format to the database server, so protocol conversion must occur coming and going. Because of this, NAS may not be appropriate for all databases. Read-only databases may offer acceptable performance on NAS, as well as relatively small transactional databases. However, large transactional databases are rarely placed on NAS, due to perceived performance reasons. Despite the potential drawbacks, a NAS system may offer good performance at a good price, depending on your situation. A high end NAS appliance over a 1 Gigabit ethernet network can offer performance similar to a SAN. The advent of 10 Gigabit ethernet should alleviate any performance concerns.
iSCSI (Internet SCSI) storage systems are similar to NAS in that communication between servers and storage is accomplished over standard TCP/IP networks. However, iSCSI does not utilize file protocols for data transport. Instead, SCSI commands are encapsulated in TCP/IP packets and sent over the network (encryption may also be performed). iSCSI is supported through the operating system. Both Windows 2003 Server and Linux support iSCSI.
iSCSI communication can occur through standard Network Interface cards. However, the OS then incurs substantial overhead in managing TCP/IP encapsulation. A new type of NIC for storage is arriving on the market, sort of an iSCSI HBA. These cards use an onboard TCP/IP Offload Engine (TOE). TOEs perform encapsulation at the hardware level, freeing processor cycles on the server. Although the performance of iSCSI is not yet up to the speed of SCSI or SAN storage, the pace of improvement is rapid. The performance is perfectly acceptable for Small to Medium sized Businesses, and works well with non-mission critical databases. With the adoption of 10 Gigabit networks, iSCSI will become increasingly attractive, even for mission critical applications. Recently, high-performance iSCSI systems have been benchmarked at 90% of the performance of Fibre Channel SANs, at an attractive price/oerformance ratio.
A Fibre Channel SAN is a true network for storage. A SAN consists of one or more servers, one or more switches, and one or more Fibre Channel Storage Arrays. Components are connected by fiber optic cables (copper cabling may also be used). For redundancy and fail-over capabilities, many components may be doubled. This includes HBAs, switches, and RAID Controllers.
A SAN (Storage Area Network) is designed to provide access to storage over a dedicated private fibre channel network. Fibre Channel networks can connect multiple hosts and multiple storage devices, and can even incorporate fibre channel switches for enhanced connectivity. SAN solutions are implemented more widely than any other networked storage solution.
The key benefit of a SAN is storage consolidation. Because of the networked nature of a SAN, multiple servers can be attached to one storage array. This leads to significant cost savings. In addition, higher performance can be achieved through increased overall throughput. Multiple paths from each server to the shared SAN can be implemented. This not only enhances performance, but also adds redundancy to the data path, allowing for failover from one data path to another. Redundancy can be added at every level of the data path, offering a storage solution with no single point of failure. This is particularly valuable for cluster implementations. The current generation of mid-tier storage offers 70TB of storage or more in a fully configured RAID array.
The most widely implemented SAN technology is currently based on the Fibre Channel protocol. Fibre channel features up to 127 devices within a network, with up to 10 KM of cabling. This is substantially more than SCSI offers. FC2 can operate at two gigabit speed, making it faster than the ethernet networks implemented at most sites.
As quoted from the Garner Group in October of 1999, the benefits of using Fibre Channel storage include: Greater distance between servers and storage (as much as 10km) Links are simpler and more reliable than SCSI Disk storage consolidation is facilitated Tape resources can be shared on loops and switched fabrics in conjunction with resource serialization storage management software FC-connected disk storage and tape libraries, combined with FC switches and appropriate software in a SAN will allow Windows NT servers to support outboard backup. Data moves form disk to tape over the SAN, unloading the servers and eliminating server bus bandwidth as a constraint on backup speed.
This shows the actual FC frame. Inside the FC frame, a SCSI command is embedded. Also within the FC frame are some check sums and a header that has source and destination information. A Fibre Channel SAN often contains multiple HBAs and multiple processors. Within the FC frame, the source and destination information provide the addressing required to route the correct packet to the right location or device (similar to Ethernet).
Fiber is the cable; Fibre Channel is the protocol. Fibre Channel does not require fiber optic cable. It can be implemented on copper as well. Some variations of Fibre Channel over IP allow Fibre Channel over TCP/IP networks, utilizing common ethernet cable.
Fibre Channel is efficient at transmitting data because it is a serial interface. Serial interfaces use a separate send and receive wire. This allows data to be transferred full speed in both directions simultaneously. Because of the separation of send and receive data, no complex signal handling is required.
Internal to the Fibre Channel protocol, the SCSI command set is used. The device driver talks SCSI. Each SCSI command ride on top of fibre channel. (Similar to how IP rides on top of Ethernet.) The FC command itself goes across the fibre optic or copper cabling. Note the similarity to TCP/IP interface layers.
Sometimes the term “SCSI” refers to the SCSI protocol and sometimes it is used to refer to a SCSI device. We’ll discuss the protocol first. SCSI is a protocol and command set that provides services similar to TCP/IP. It’s been in use for a long time and when Fibre Channel was introduced, there was no reason to change the protocol that is used to communicate between devices. (Why re-invent the wheel? SCSI was already tested and accepted w/ every major operating system on the market.) So the protocol that Fibre Channel uses is SCSI.
A FC transmission is sent across only two “wires” (or two fiber optic strands, thus the slim FC cable). A SCSI transmission would send an 8 bit transmission across 8 different wires (thus the thick SCSI cable). There has to be additional signaling to let the receiving device know what is being transmitted – i.e. 8 bits being transmitted “now.”
The biggest difference between SCSI and Fibre Channel is availability. Fibre Channel was designed w/ all sorts of built-in redundancy. Part of the reason Fibre Channel came about was that people were looking to overcome downtime associated with SCSI. For example, with SCSI, if you break a cable, SCSI is going to die. Unlike SCSI, Fibre Channel is not limited to a single physical loop. A break on one cable in an FC network will not automatically disable devices connected to other cables. In addition, multiple Fibre Channel connections may be utilized between FC devices. This allows for virtually unlimited redundancy, so that the loss of one cable need not disrupt communications at all. SCSI is single bus – FC is redundant network for all components to talk to each other. SCSI hardware: Disk drives and controller – e.g. raid controller is SCSI hardware With fibre channel, you still have a RAID controller, but, by definition, it will be on the external storage. It will do the logical to physical translation of data, but the physical transfer from the internal system to the external storage will be handled by an HBA. 3 predominant SCSI standards: – Ultra2 (80 megabytes/sec) – Ultra 160 – 160 MB per sec – Ultra 320 – 320 MB per sec Fibre Channel standards: - FC1 – 100 MB/sec - FC2 – 200 MB/sec So how important is this difference in bandwidth? It depends on what you’re doing. When you’re talking about database applications, you usually never come close to using the entire bandwidth – it’s hardly ever the limiting factor. Fibre Channel provides a data link layer above the physical interconnect; analogous to Ethernet. This is completely different from SCSI. FC is network of devices that can be media independent – that’s why it can run on copper or fibre optic. SCSI has to run on copper Fibre channel is closer to a network. Devices talk either fiber optic or copper. Fibre channel changes the network – the physical layer - that things run across. There’s no way to run on a different medium on SCSI. SCSI is media specific. The limitations with SCSI and FC are also different. Cables have to be certain length: With SCSI, no more than 3 feet. No more than 15 devices on a SCSI bus (bus sort of analogous to FC loop – which can have 127 devices). SCSI pins have to be a certain way. IF SCSI were to change, then have to change every device to accommodate. With FC, if ran diff media – would have to change cables, but not devices. The number of disk drives that you can connect with SCSI is going to be less than with Fibre Channel. Lets say that you have 4 SCSI buses with 15 devices (a controller or a disk) on each, then you can have 60 devices. Whereas you can get 127 devices on a fibre channel loop. (Assuming no other devices on that loop.) SCSI Storage Characteristics The average SCSI clustered solution allows at most two RAID controllers for clustered disks. For most SCSI systems on the market, this results in a maximum of 24 - 48 disk drives for the cluster’s shared disks. If that amount of storage is not sufficient for the customer’s current and future growth needs, then Fibre Channel solutions should be considered. Fibre Channel Like SCSI, Fibre Channel is another means of attaching storage to a computer. The big differences between Fiber Channel and SCSI is the redundancy that it allows you and the higher availability (don’t know if we want to get into needing terminal on SCSI, etc.) A common misconception about Fiber Channel is that it is faster than SCSI. In reality, it’s not necessarily faster, but may allow more I/Os (and the increased redundancy & higher availability). Fibre Channel Characteristics Fibre Channel provides high availability through redundant Fibre Channel host bus adapters (HBAs) in the servers and dual storage processors in the storage system. Fibre Channel also requires dual standby power supplies (SPS) in the storage system, which provides integrity of the storage processor write-cache in case one power supply fails. Thus write-caching may be enabled, which will improve disk write performance. With SCSI, write-caching may not be used with a failover cluster. Fibre Channel allows for easy storage subsystem growth. It is relatively easy to add disks to Fibre Channel storage system without having to shutdown the system. A directly connected Fibre Channel solution can support greater than 200 disks on one array. For a Storage Attached Network (SAN) solution, it is possible to have greater than 900 disks on one cluster connecting through a Fibre Channel switch. The same switch can also support additional stand-alone systems and clusters. Although Fibre Channel systems are typically more expensive than SCSI, the expandability and flexibility is much greater than SCSI.
.
Microsoft’s Simple San Approach is designed to benefit both Fibre Channel SANs and iSCSI SANs.