This document provides an overview of data storage concepts and NetApp technologies. It begins with an introduction to different storage solutions like disks, disk arrays, and intelligent storage systems. It then covers storage types including DAS, NAS, and SAN. The document discusses virtualization, data networks, and protocols like SCSI, iSCSI, SAS, and Fibre Channel. It provides details on flash storage, solid state technologies, and flash endurance. The remainder covers NetApp products, technologies, and configuration topics.
M.Sc Dissertation: Simple Digital LibrariesLighton Phiri
My M.Sc. dissertation... it took me a total of 2 years and 61 days to finish--I LOVE TO COUNT! There are a few publications [1] based on this work---there is even a book chapter on the way.
You will notice from the structure of the manuscript that I used Information Mapping [2] principles. The content on the other hand is structured chronologically---based on the sequence of activities I undertook during my research.
I typeset the entire manuscript using LaTeX [3] and I am VERY proud of myself for doing that :p You would have to see the TeX source files [4] to see all corresponding packages I used. Block diagrams were rendered using PSTricks [5] and plots using R ggplot2 [6] package.
[1] http://scholar.google.co.za/citations?user=UIb4aEsAAAAJ&hl=en
[2] http://en.wikipedia.org/wiki/Information_mapping
[3] http://en.wikipedia.org/wiki/LaTeX
[4] https://github.com/lightonphiri/open-msc-thesis
[5] http://en.wikipedia.org/wiki/PSTricks
[6] http://en.wikipedia.org/wiki/Ggplot2
M.Sc Dissertation: Simple Digital LibrariesLighton Phiri
My M.Sc. dissertation... it took me a total of 2 years and 61 days to finish--I LOVE TO COUNT! There are a few publications [1] based on this work---there is even a book chapter on the way.
You will notice from the structure of the manuscript that I used Information Mapping [2] principles. The content on the other hand is structured chronologically---based on the sequence of activities I undertook during my research.
I typeset the entire manuscript using LaTeX [3] and I am VERY proud of myself for doing that :p You would have to see the TeX source files [4] to see all corresponding packages I used. Block diagrams were rendered using PSTricks [5] and plots using R ggplot2 [6] package.
[1] http://scholar.google.co.za/citations?user=UIb4aEsAAAAJ&hl=en
[2] http://en.wikipedia.org/wiki/Information_mapping
[3] http://en.wikipedia.org/wiki/LaTeX
[4] https://github.com/lightonphiri/open-msc-thesis
[5] http://en.wikipedia.org/wiki/PSTricks
[6] http://en.wikipedia.org/wiki/Ggplot2
A mixed method (quantitative and qualitative) approach was employed to investigate possible associations between all four experiences. The survey provided Junior players, who returned to play in the 2017 RL season, with the opportunity to record and voice their ‘lived experience’. Findings from this study provide a comprehensive ‘snapshot’ of the current Junior RL landscape, presenting useful findings pertaining to players’ experiences (2017).
This book is intended for an international audience with an interest in education and the Internet. It will be of special value to students, teachers, course designers, administrators, decision makers and educational officers in public and private sectors. The book discusses major developments in online education, and provides illustrating examples from the author's Scandinavian and international research. It includes articles outlining a theory of online education, teaching techniques, learning management systems, recommendations for decision makers and terminology.
Summary of the PMI PMBOK 6th edition to help PMP / CAPM exam practitioners to understand the guide much better and focus on its most important contents.Also it is very helpful for practicing project mangers to stay up to date with the recent changes made on the PMBOK guide
Predicting lead poisoning levels in chicago neighborhoods capstoneCarlos Ardila
This capstone project examines health indicators and demographic data aggregated by the community from 2006 to 2013 and compares the percent of children with elevated blood lead levels (BLL) with building permits, code violations, and HUD-Section 8 subsidized housing data from the same period. The project uses various statistical techniques and was produced as part of the MS Predictive Analytics Capstone Requirement.
We thought that we had contained the outbreak; we were wrong. The outbreak is back with a vengeance and society as we knew it has fallen. Fortunately, Mr. Billionaire planned for this contingency. His space tourism company has built a large lunar base, designed to support what is left of the human race until the first colony can be established on Mars.
A mixed method (quantitative and qualitative) approach was employed to investigate possible associations between all four experiences. The survey provided Junior players, who returned to play in the 2017 RL season, with the opportunity to record and voice their ‘lived experience’. Findings from this study provide a comprehensive ‘snapshot’ of the current Junior RL landscape, presenting useful findings pertaining to players’ experiences (2017).
This book is intended for an international audience with an interest in education and the Internet. It will be of special value to students, teachers, course designers, administrators, decision makers and educational officers in public and private sectors. The book discusses major developments in online education, and provides illustrating examples from the author's Scandinavian and international research. It includes articles outlining a theory of online education, teaching techniques, learning management systems, recommendations for decision makers and terminology.
Summary of the PMI PMBOK 6th edition to help PMP / CAPM exam practitioners to understand the guide much better and focus on its most important contents.Also it is very helpful for practicing project mangers to stay up to date with the recent changes made on the PMBOK guide
Predicting lead poisoning levels in chicago neighborhoods capstoneCarlos Ardila
This capstone project examines health indicators and demographic data aggregated by the community from 2006 to 2013 and compares the percent of children with elevated blood lead levels (BLL) with building permits, code violations, and HUD-Section 8 subsidized housing data from the same period. The project uses various statistical techniques and was produced as part of the MS Predictive Analytics Capstone Requirement.
We thought that we had contained the outbreak; we were wrong. The outbreak is back with a vengeance and society as we knew it has fallen. Fortunately, Mr. Billionaire planned for this contingency. His space tourism company has built a large lunar base, designed to support what is left of the human race until the first colony can be established on Mars.
This is a report detailing my industrial placement year at Tomo Motor Parts Ltd. This report was submitted to Brunel University and formed the majority of my A+ result for the year.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
2. ENG. AHMED GAMIL 1
Table of Contents
Course Goal:.......................................................................................................................................7
Introduction to Data Storage...............................................................................................................7
Storage solution:.............................................................................................................................7
Disk............................................................................................................................................7
Disk array....................................................................................................................................8
Just a bunch of disks (J-B-O-D) solution.........................................................................................8
Intelligent storage systems ..........................................................................................................8
Storage Types:..................................................................................................................................10
DAS..............................................................................................................................................10
Protocols ..................................................................................................................................10
Advantages...............................................................................................................................10
Disadvantages...........................................................................................................................10
NAS..............................................................................................................................................11
Protocols ..................................................................................................................................11
Advantages...............................................................................................................................11
SAN..............................................................................................................................................11
Protocols ..................................................................................................................................11
Advantages...............................................................................................................................11
Disadvantages...........................................................................................................................11
Virtualization....................................................................................................................................12
Server virtualization......................................................................................................................12
Server virtualization benefits......................................................................................................12
Storage virtualization....................................................................................................................12
Storage virtualization benefits:...................................................................................................12
Data Networks Fundamentals ...........................................................................................................13
Small Computer System Interface (SCSI).........................................................................................13
Internet SCSI (ISCSI) ......................................................................................................................13
Serial-Attached SCSI (SAS) .............................................................................................................13
Fiber Channel (FC).........................................................................................................................14
FC topological implementations.................................................................................................14
Fiber Channel switched infrastructure............................................................................................15
3. ENG. AHMED GAMIL 2
Zoning Types:............................................................................................................................15
Storage types................................................................................................................................16
Block storage ............................................................................................................................16
File storage...............................................................................................................................16
Object storage...........................................................................................................................16
Some acronyms................................................................................................................................16
Traditional Storage .......................................................................................................................16
Storage terms...............................................................................................................................16
Flash storage....................................................................................................................................17
Flash storage contains:..................................................................................................................18
Types of flash storage:..................................................................................................................18
Solid State Technologies ...................................................................................................................18
SSD Types:....................................................................................................................................18
Flash Endurance...............................................................................................................................19
Endurance metrics:.......................................................................................................................19
Endurance levels(categories)........................................................................................................20
Flash translationlayer...................................................................................................................20
Write amplification (WA)...............................................................................................................20
Garbage collection(GC).................................................................................................................21
Performance curve .......................................................................................................................22
Fresh out of the box (FOB).........................................................................................................22
Write Cliff.................................................................................................................................22
Steady State..............................................................................................................................22
Over-provisioning.........................................................................................................................22
Major Industry Trends per NetApp ....................................................................................................23
OnCommand Management Console ..................................................................................................24
System Manager GUI ....................................................................................................................24
Unified Manager GUI ....................................................................................................................25
Workflow Automation...................................................................................................................25
Performance Manager..................................................................................................................26
OnCommand Insight .....................................................................................................................26
ONTAP OS........................................................................................................................................26
4. ENG. AHMED GAMIL 3
Clustered Data ONTAP ..................................................................................................................27
ONTAP OS support....................................................................................................................27
Data ONTAP 7-Mode.........................................................................................................................28
NetApp Hardware Basics...................................................................................................................28
Slots and ports..............................................................................................................................29
Slots.........................................................................................................................................29
Ports.........................................................................................................................................29
Disk shelves..................................................................................................................................30
Disk classification ......................................................................................................................30
Diskidentification system..........................................................................................................31
Disk ownership..........................................................................................................................31
Internal components.....................................................................................................................32
Flash Cache 2............................................................................................................................32
FAS 2552 model................................................................................................................................32
Another Model.............................................................................................................................33
Using the CLI ....................................................................................................................................33
Date.............................................................................................................................................33
Version.........................................................................................................................................33
Help “?”.......................................................................................................................................33
System console.............................................................................................................................34
Man page.....................................................................................................................................34
Create volume..............................................................................................................................34
NETAPP technology ..........................................................................................................................34
Disks............................................................................................................................................36
Aggregates(RAID options).............................................................................................................36
Aggregates structure:................................................................................................................36
Snaplock...................................................................................................................................37
Creating Aggregates ..................................................................................................................37
FlashPools ...................................................................................................................................40
Activate flash pools...................................................................................................................40
FlashPools support ...................................................................................................................40
Flash pool Limitation .................................................................................................................41
5. ENG. AHMED GAMIL 4
FlexArray......................................................................................................................................41
Volume ........................................................................................................................................42
Volume modes..........................................................................................................................42
Volume types:...........................................................................................................................42
Qtrees..............................................................................................................................................47
Create Qtree.................................................................................................................................48
Calculating Usable Disk Space............................................................................................................49
Review.............................................................................................................................................50
SnapShot..........................................................................................................................................50
SnapRestore.....................................................................................................................................52
SnapMirror.......................................................................................................................................52
SnapVault........................................................................................................................................53
Network Administration....................................................................................................................53
Name resolution...........................................................................................................................53
Create the VLANs..........................................................................................................................55
Create VIF....................................................................................................................................57
Interface groups / aggregation...................................................................................................57
Using CLI...................................................................................................................................60
Creating LUNs...................................................................................................................................62
Create LUN GUI ............................................................................................................................62
Delete LUN GUI.............................................................................................................................66
Create LUN CLI..............................................................................................................................67
Using SnapDrive ...............................................................................................................................68
NFS Exports and CIFS Shares..............................................................................................................68
Delete Export ...............................................................................................................................69
Create Export...............................................................................................................................69
Create CIFS share..........................................................................................................................71
Stop Sharing.................................................................................................................................73
Using BranchCache...........................................................................................................................73
Distributed caching.......................................................................................................................74
Hosted Casing...............................................................................................................................74
Branch cache configurations..........................................................................................................74
6. ENG. AHMED GAMIL 5
Using Quotas....................................................................................................................................75
Create Qtree.................................................................................................................................76
Set Quota.....................................................................................................................................76
RBAC................................................................................................................................................80
Create User..................................................................................................................................80
Review.........................................................................................................................................81
Storage System Maintenance............................................................................................................81
Auto support................................................................................................................................81
Logs.............................................................................................................................................82
Degraded disk...............................................................................................................................82
Disk scrubbing..............................................................................................................................82
sysstat..........................................................................................................................................82
Storage Space Management..............................................................................................................83
Provisioning..................................................................................................................................83
Space allocation............................................................................................................................84
CLI ...............................................................................................................................................84
Storage Efficiency (Deduplication)..................................................................................................85
Clustered Data ONTAP Concepts........................................................................................................86
Clustered Data ONTAP User Interfaces...............................................................................................87
Clustered Volumes and Namespaces .................................................................................................87
Namespace ..................................................................................................................................88
Junction....................................................................................................................................88
Infinite volume .............................................................................................................................88
Clustered File Access.........................................................................................................................89
NFSv4...........................................................................................................................................89
PNFS(Parallel NFS)........................................................................................................................89
LIF............................................................................................................................................89
Common Internet File System (CIFS) ..............................................................................................90
Netapp Networks .............................................................................................................................90
Clustered Load Balancing..................................................................................................................91
Load Sharing Mirrors(LSM) ...........................................................................................................91
SAN Infrastructure............................................................................................................................91
7. ENG. AHMED GAMIL 6
Unified storage.............................................................................................................................92
Steps for the establishment of the SAN:.........................................................................................92
FC Connectivity.................................................................................................................................92
Fibre Channel connection:.............................................................................................................93
Commands...................................................................................................................................93
sysconfig command...................................................................................................................93
fcadmin command.....................................................................................................................93
License command......................................................................................................................94
To view and add license.........................................................................................................94
References.......................................................................................................................................95
8. ENG. AHMED GAMIL 7
NCSA
NetApp has a lot of great products:
unified storage
high performance san storage
enterprise all-flash storage
Course Goal:
Understand key storage topics
Understand the NetApp product portfolio
Describe and configure key NetApp elements
Introduction to Data Storage
Storage solution:
Disk
Is a hard disk that would be a spinning platter
Manufacturers would make the platter spin faster and faster and faster “RPM speeds”
It cause latency due to mechanical parts.
Solid statedrive is even more efficient than the mechanical spinning disk but it is still a traditional
type of disks
9. ENG. AHMED GAMIL 8
Disk array
Can do redundant array of inexpensive disks inside the array
It Contains:
Firmware of the array
RAID
Power supply
Fans for cooling
Just a bunch of disks (J-B-O-D) solution
Where we have disks out there that we just keep adding to the network and then the operating system
Each disk works as an independent disk in the array
Intelligent storage systems
A server that needs to access the data. And it does so through a storage area network (SAN)
10. ENG. AHMED GAMIL 9
Then we connect to this storage system
Storage system component:
front end
cache : for fast access to data since data that's frequently accessed will be cached
back end
actual disks
Factors before purchase:
Applications :
Like Email, DB, etc.
How much data do these application store
How quickly does the application able to retrieve data from the store
What about the requirements like latency?
Data protection :
Like back up and disaster recovery
Availability
How much downtime can we survive with?
How much is acceptable? 99.999 is acceptable
Availability could be accomplished by redundancy and disaster recovery systems
Security
Scalability
How fast, how easily your storage infrastructure can grow as you need it to
Performance
The throughput of a system, the response time, the capacity, the reliability of a system
11. ENG. AHMED GAMIL 10
Storage Types:
DAS : direct-attached storage
NAS : network-attached storage
SAN : storage area networks
DAS
Is an external disk system or an internal disk system “Blade Servers”
Storage appeared as a block storage which can be formatted by OS
Protocols
ATA, SATA, eSATA, SCSI, SAS, USB, USB 3.0, and Fiber Channel.
Advantages
Not complicated (storage network between the device and the storage itself)
Affordable (good price)
Disadvantages
Lack of scalability
12. ENG. AHMED GAMIL 11
NAS
It is storage attached to the network (Server must connect through a network)
NAS contains filesystem, CPU and resources
Storage appeared on the remote server as a shared folder
Protocols
Network file system (NFS), or common internet file system (CIFS)
Advantages
More scalable (easy to build out the amount of storagethat's connected to the LAN)
Servers worry only about its applications only not storing data to NAS
Easy to install, deploy and then manage.
Cost effective solution
Nice utilization rate
SAN
It is storage attached to the network (Server must connect through a network)
Storage appeared as a block storage which can be formatted by OS
Protocols
Fiber Channel, Fiber Channel over Ethernet, or iSCSI
FCOE: 10-gigabit-per-second Ethernet
Advantages
Redundancy “redundant NICs and RAID”
Highest performance
Most reliable infrastructure
Disadvantages
The most complex
13. ENG. AHMED GAMIL 12
Virtualization
Server virtualization
VMware make server virtualization and what it is called “Server sprawl”
Server sprawl: the need of more and more storage “server expansion”
Server virtualization benefits
buy fewer servers
better utilization with high flexibility
lower operating costs (like power, human resource)
lower management costs (easy to administer)
support more technologies like “ VMotion, backup and snapshotting technology ”
Storage virtualization
We can add as much physical storage as we want and it can be located anywhere.
Then we will represent it to the end users as one logical entity.
End user don’t know the location of the physical storage
NetApp tries to do for storage what VMwarehas done for servers (Storage sprawl)
Storage sprawl: the need of more and more storage “storage expansion”
Storage virtualization benefits:
Resources are pooled
Support multiple protocols
High storage utilization
Create products that lower the total cost of ownership (TCO) when it comes to provisioning and
maintaining storage for the data center
RAID-DPwhich is It is a RAID technology that is efficient like RAID-10 but cheap like RAID-5
Snapshot copies which is like a backup with smallest window
reduced space for the backup process and near instantaneous restoration of data
deduplication : in case of many VM uses same files, it will use only one copy for all VMs instead of
createa copy for each VM
14. ENG. AHMED GAMIL 13
Data Networks Fundamentals
We will talking about cabling and protocols that are going to run over that cabling inside SAN
SCSI : Small Computer System Interface
ISCSI : internet SCSI
SAS : Serial-Attached SCSI
FC : fiber channel
Small Computer System Interface (SCSI)
Hook up like printers and scanners to devices.
The SCSI system is a client server type of system (initiator - client)
Internet SCSI (ISCSI)
Used to run SCSI protocol commands for moving data to and from storage over IP networks
Uses Ethernet switches
Less expensive than FC
Support longer distances
Client server type of system (initiator - client)
Serial-AttachedSCSI (SAS)
Used to run SCSI protocol commands over a serial cable
Simple and inexpensive
Very limited (distance = 8-meter, number of devices that we can attachto the SAS network)
15. ENG. AHMED GAMIL 14
Fiber Channel(FC)
Flexible
Reliable
Scalable
Optical or copper: copper and optics can be utilized for Fiber Channel as well
Supercomputers
FC topological implementations
Fiber Channelpoint-to-point
Connect host to storage directly using HBA
Fiber ChannelArbitrated Loop
Devices being connected in like a ring
Fiber Channelswitched infrastructure
Device connects to two independent storage area networks
If one of the SAN completely fails, the other redundant fabric can still carry the information.
16. ENG. AHMED GAMIL 15
Fiber Channelswitched infrastructure
What is zoning?
Zoning means what devices in the fabric can communicate with what other devices.
H1 can access LUN2 of S1 and LUN0 of S2
H2 can access LUN1 of S2 and LUN1 and LUN 3 of S1
Zoning Types:
Hard zoning
Soft zoning
Hard zoning
Assign a certain switch port number to a particular HBA. If the port is down, so we need to assign another
port
Softzoning
If this HBAwith WWPN is connected to any switch port, it will add this HBA to a certain zone
17. ENG. AHMED GAMIL 16
Storage types
Block storage
File storage
Store things in terms of files on these devices
Object storage
Identify each thing stored as an object
It allows object re-usability (the object could be available for many server)
Some acronyms
High Performance Computing (HPC): amounts of memory “99.999%” and storage capacity
TraditionalStorage
Combination between flash and mechanical storage
It provide low latency and low cost
Storage terms
Seek time (rotational disk)
The amount of time it takes to move that mechanical head into the correct position over the platter.
Rotational latency (rotational disk)
The amount of time that it takes to spin the correct sector on the platter into the right position
Transfer time (flash / rotational disk)
The amount of time it takes to actually read information from the platter or write information to the platter
Overhead (flash / rotational disk)
How much disk logic stuff has to go on? And what does that do to our latency?
18. ENG. AHMED GAMIL 17
Longevity/Endurance (flash disk)
A lifetime of the flash media that it can endure this process of block writes over the top of other data.
Pages: (flash disk)
It could be small container of data (512 bytes) or large (8 kilobytes)
Erase Block (flash disk)
Container of memory pages (32 to 128 pages) that can be buffered
Flash storage
Is non-volatile storage and it is re-writable memory.
Require erasing blocks of data before they can be written to
Supports only a finite number of rights and that number is vary by the technology we use
SSD drive mean flash drive which provide
High performance and better than mechanical disk drive due to there is no moving parts
lower write performance than a read performance
Electricity is utilized to store and retrieve data storage very quickly
Low latency
High cost/GB
19. ENG. AHMED GAMIL 18
Flash storage contains:
Memory unit (DRAM): Dynamic holding location for the data
Access controller (Flash controller):
Managethe placement and access of data into the flash storage system
Move data between the DRAM and flash memory chip
Flash memory chip: The actualstorage of information
Types of flash storage:
Flash memory
SD card
SSD drive
Flash based PCI card
Solid State Technologies
Solid statedrive, or SSD is often used in enterprises today
Uses non-moving flash memory technology
Increase the reliability of the particular storage media. (Eliminating the moving mechanical parts)
Flash memory is non-volatile and it's re-writable memory
SSD Types:
Negative AND (NAND) SSDs:
20. ENG. AHMED GAMIL 19
Commonly used because it's more durable, it's less expensive.
Has denser cells, and the writeerase operations are faster than NOR SSD
Negative OR (NOR) SSDs:
Designed to store the binary code of programs
Has high performance in read operations
Single level cell Flash technology (SLC)
Store one bit per cell
Has high endurance
Multi-level cell Flash technology (MLC)
Store two bits per cell.
Provide higher capacity
Approximately 1/10 the endurance all of single level cell Flash memory.
3-bit per cell and even 4-bit per cell technologies
High capacity
Very limited number of writes to the memory.
Flash Endurance
Quality of the flash storage depending on:
How much data we are able to write to that flash media over its lifetime?
How long will the flash storage last us (life time)?
Endurancemetrics:
1- Total bytes written (TBW)/ data written per day.
2- Program/erase cycles that are capable for the particular flash media (P/E value)
We can categorize endurance based on:
What you need to store?
How much manipulation of that data needs to be done on an ongoing basis?
21. ENG. AHMED GAMIL 20
E.g.
Write intensive storage:
High endurance: 25 plus data writes per day
Medium endurance: 10 data rights per day
Read intensive storage:
One to three data rights per day.
Endurance levels (categories)
We can categorizethe endurance in order to balance between the flash technology and its cost
Single level cell NAND technology very high endurance very cost
Multi-level cell NAND + wear leveling provide balance between affordability and endurance
Over-provisioning there is a wasted capacity old technique (we will discuss)
Flash translation layer
FTL is a layer that introduce wear leveling techniques in order to improve the endurance of the media
FTL is a layer is located in the SW layer between the file system part and HW level
Write amplification (WA)
Flash memory must be erased before it can store new data
Data cannot be overwritten directly as it is in a hard disk drive,
22. ENG. AHMED GAMIL 21
In order to rewrite the data directly to the flash disk, Flash make a process called Write amplification
Write amplification (WA) move user data and metadata more than once to the buffer then to the media then
to the buffer then to the media again (read, updated and written to a new location)
SSDs use a process called garbagecollection (GC) to overcome this behavior
Garbage collection(GC)
Garbagecollection is a big part of write amplification on the SSD
Data is written to the flash memory in units called pages (Multiple cells)
The memory can be erased block by block, each block is made up of multiple pages
SSDs use garbagecollection to reclaim the space by the flowing steps
1- You write a data to pages (A-D) in Block “X”
2- Then write new data in new pages (E-G)
3- If you need to replace old (A-D) pages with new (A’-D’)pages, you will write new pages normally to
the same block “X”
4- Then read and rewrite pages (E-D’) from block “X” to Block “Y”
23. ENG. AHMED GAMIL 22
5- Then Block “X” is considered as empty and available to use
All SSDs include some level of garbagecollection, but they differ in when and how fast they perform the
process
Performance curve
It’sa graph of performance over your gigabits that arebeing stored
Fresh out of the box (FOB)
Describe a flash memory device that has few or no program/erase (P/E) cycles since the device was
manufactured
New data written to the flash directly before program/erase (P/E) cycles
Write Cliff
Performance of an SSD drive drops dramatically
It occurs when all of the cells within an individual drive have been written to at least once.
So any incoming data is buffered until the cell “flashed” and the new data re-written back to the cell ((P/E)
cycles)
Steady State
It is the state of the SSD over its life time
Over-provisioning
It is a technique that's used with a solid state drive.
It you bought SSD with reported capacity 1TB, the actual capacity is more than 1TB (over-provisioned)
This additional space is used as over-provisioned storagespace for wear leveling techniques, buffering and
garbagecollection and any other techniques that can improve the endurance performance
24. ENG. AHMED GAMIL 23
Major Industry Trends per NetApp
1- Flash-based storagetechnologies
If you're implementing flash smartly, it will lead to lower costs.
2- Cloud-based technologies
Reduce energy cost, space saving, enhance services
3- Software-defined data centers
An extension of cloud-based technologies
It is based on “ONTAP software” which creates a storagevirtual machine (SVM) from fabric-
attached storage(FAS) and provide logical views of that hardware for our end users
4- Converged infrastructures
Easier to manage
Cost savings
Example for converged infrastructure technology:
Fiber Channel over Ethernet (FCOE) to carry 10 gbps Ethernet.
LAN technology that's utilizing Ethernet (eth) + SAN technology using Fiber Channel (FC)
5- Mobility
Is to be able to access the data regardless its location or size or the device we're using
6- big data
Term for these huge data sets that is difficult to process using traditional applications
25. ENG. AHMED GAMIL 24
OnCommandManagement Console
NetApp has its own operating system which called ONTAP operating system
ONTAP operating system can be managed by GUI or CLI
NetApp named their graphical user interface stuff OnCommand
OnCommand is the umbrella software for a bunch of packages
System Manager GUI
This is a day-to-day management and monitoring tool
Interacting with the clustered Data ONTAP operating system.
Manageyour particular system
Manipulating the volumes
Manipulating the shares
Getting a snapshot of the usage of a particular device.
26. ENG. AHMED GAMIL 25
Unified Manager GUI
Monitoring and alerting tool
Give a dashboard to get any required information about the system
Support 3rd party APIs to provide multi-vendor dashboard
Workflow Automation
Allows to createa workflow to automate a process like (create a basic cluster mode volume)
Workflow Automation permits
27. ENG. AHMED GAMIL 26
Performance Manager
Biggest level of monitoring in detail
OnCommand Insight
It is a tool that designed for hybrid cloud environments
Monitor and managehybrid cloud
ONTAP OS
ONTAP OS has 2 different versions:
1- Clustered Data ONTAP
2- Data ONTAP 7-mode
28. ENG. AHMED GAMIL 27
Clustered Data ONTAP
Open Network Technology for Appliance Products (ONTAP)
It is an operating system from NetApp, for the management, maintenance, initial set-up, and configuration
ONTAP use RAID Dual Parity (RAID-DP) with 2 parity disks
ONTAP operating system goal:
1- to be able to do high availability (in case of maintenance or upgrade)
2- to be efficient (apply compression to save capacity, thin replication of data, data deduplication to get
rid of redundant copies )
3- scalability (Through a clustered solution that's virtualized with our storage virtual machines)
ONTAP OS support
1- Unified StorageArchitecture (USA):
NFS, CIFS, Fiber Channel, iSCSI, Fiber Channel over Ethernet,
2- Unify Fabric Attached Storage:
SAS, SATA, Fiber Channel, SSD
3- Storage Virtual Machines (SVM)
It is a logical representations of the storage that might have a lot of physical hardware.
Its own Logical interfaces, logical volumes and LUNs
4- Non-volatile RAM (NV-RAM):
There's a batteryconnected to this RAM so that we can preserve the data in case of a power failure
5- Write Anywhere File Layout File System (WAFL)
Is a file layout (like file system) that supports large, high-performance RAID arrays
29. ENG. AHMED GAMIL 28
In case of a crash or power failure, it will restart without long consistency checks
6- Snapshots
Is a pointer for the original data
Data ONTAP 7-Mode
Data ONTAP 7-Mode Support
Multi-protocol capabilities (NFS, CIFS, Fiber Channel, iSCSI, Fiber Channel over Ethernet)
NAS/SAN
RAID-DP
Snapshot
Improve Compression
Improve Encryption
64-bit aggregation
Move the running 32-bit aggregates, toa 64-bit aggregationnon-disruptively.
IPv6
Flash Pools: we can add Flash to the system, and the Data ONTAP will use it for caching
BranchCache: provide caching features to company branches which has slow WAN
Data ONTAP 7-Mode contains:
1- Disk Blade (D-Blade):
a. to manage WAFL, RAID and Storage (SAS, NFS, FC)
2- Management host (M-host) :
a. Managenodes by using
i. Command line interface “ng shell” or “ngsh” to managethe nodes
ii. Element Manager User interface from the browser
NetApp Hardware Basics
Let’s talk about some of the basic hardware components we should be familiar with
30. ENG. AHMED GAMIL 29
All hardware components can be break down into three general categories.
1- slots and ports
2- disk shelves
3- internal components
Slots and ports
Slots
Used to plug stuff, taking ports out, putting new ports in or configuring what the port will do in the software
Ports
ManagementPort“e0M”
Connected to a separate network (management network)
Remote LAN Module (RLM) interface
We can use this port to manage the node remotely, even if the NetApp device itself is down or ONTAP OS is
down
It provides remote access, monitoring, troubleshooting, logging, and alerting features
We can interact with RLMusing:
SSH (default)
31. ENG. AHMED GAMIL 30
Telnet (disabled by default)
Remote Shell (disabled by default)
Data ports “e0a, e0b,…..”
Connected to data network
Data interfaces support 802.3ad technology to aggregatelinks together
So you can aggregate“e0a” and “e0b” to “group1” to take advantageof link aggregation(high performance
and availability)
Unified port/Unified targetadapter (UTA2)
Is the port that can act as Fiber Channel or 10-gigabit Ethernet port
Disk shelves
The physical disks where we're storing information
We can put different drives in, and going from SSD technology to mechanical back to SSD technology again
Clients see volumes or aggregatesinstead of physical disks
Physical disks support Fiber Channel, ATA, Serial ATA, SAS, and SSD
FC and ATA use the Fiber Channel Arbitrated Loop (FC-AL) connection-type
SATA, SAS, and SSD use a SAS connection-type
Disk aggregationand RAID improve the performance and availability of the disk
Disk classification
Data disk: contain all data
32. ENG. AHMED GAMIL 31
Hot Spare disk: used in case of a disk failure
Parity disk: to store the parity information
Disk identification system
The disk identification name is based on its location in the NetApp appliance
The disk location is based on “slot no.”, “port no.” and “device_ID”
For disks that used FC-AL: the location is based on “shelf identifier”, “bay number” and “device ID”
Disk ownership
Disk ownership is controlled by a controller and disk pool
For single system set up
There is one controller that is in charge of that disk,
There is only one pool (pool 0) that the disk is a part of
For high availability set up
We have 2 controllers and 2 pools
Disk ownership is controlled by software settings (system manager)
33. ENG. AHMED GAMIL 32
Internal components
Like CPU, RAM, non-volatile RAMand Flash Cache 2.
Flash Cache 2
It is a technology that allow the mechanical drives that can act as SSD performance by caching the
frequently information in a Flash Cache on the device
It provide lower cost per TB, less power, more capacity storage, more IOPS and improve response time
FAS 2552 model
34. ENG. AHMED GAMIL 33
Another Model
Using the CLI
Date
Version
For firmware version
Help “?”
To get help bout the available commands
To get help about a certain command
35. ENG. AHMED GAMIL 34
System console
To go to the system console
CTRL + D to exit system console
Man page
Then “q” to quit
Create volume
To create new volume with name “my_cli_test_ncsa” and size “2g” on aggregate“aggr0”
NETAPP technology
37. ENG. AHMED GAMIL 36
Disks
This is the physical disk reside in a disk enclosure
Disk has a pathname like 2a.17, where 2a = SCSI adapter and 17 = disk SCSI ID
Spare will be used in any group or any aggregatetoreplace failed disks.
There are 4 types of disks:
1- Data : holds data stored within the RAID group
2- Spare: Does not hold usable data but is available to be added to a RAID group in an aggregate, also
known as a hot spare
3- Parity: Store data reconstruction information within the RAID group
4- dParity: Stores double-parity information within the RAID group, if RAID-DPis enabled
Aggregates (RAID options)
An aggregateisa component in NetApp that contains disks, RAID groups, etc.
Disks are grouped together in aggregatestoprovide storage volumes
All disks in the same aggregatehave to be same type
Each aggregatehas its own RAID configuration, and disks and luns
When using RAID4 or RAID-DPthe largest disks will be used as the parity disk/s
You can createtraditional volumes or NetApp's FlexVol volumes
We are using aggregate to createvolumes, specifically “FlexVols” which can be used by the end user
To create the aggregate wewill:
1- give aggregate a name
2- type of RAID
3- define a size
4- select no. of disks
Aggregates structure:
32-bit: low performance, limited size (16 terabytes)
64-bit: support up to 100 terabytes
38. ENG. AHMED GAMIL 37
32-bit and 64-bit aggregatescan co-exist now on the same device.
Fewer aggregatesreduces the overall management workload
Snaplock
It is a technology to write once, read many" (WORM) data
Administer non-rewritable storage of data
Creating Aggregates
Using system manager:
41. ENG. AHMED GAMIL 40
Flash Pools
It is a cash memory storage for NETAPP appliances
Flash pools used to createa flash cash which contains the frequently accessed data
The controller is ensuring that frequently accessed data is put on the flash, and less frequently accessed data
put on the mechanical disk drive
Flash Pools are differ from flash cache,
Flash cache is used for read operation
Flash pools are used for read, write, metadata operations
Fusion Drive is a type of drives that is contains a mechanical drive included with flash
Activate flash pools
From CLI:
1- select our aggregate
2- run command “aggregateoptions hybrid enabled on”
Flash Pools support
SnapMirror
aggregatesnapshotting
data compression
42. ENG. AHMED GAMIL 41
sync mirroring
snap blocking
metro clustering,
Flash pool Limitation
Flash Pool aggregatescannot be used in the following configurations:
flash pool is not supported on ONTAP7.2 and earlier
32-bit aggregates
Aggregatescomposed of arrayLUNs
SnapLock aggregates
Traditional volumes
A storage system that uses StorageEncryption
FlexShare is not supported for volumes associated with Flash Pool aggregates.
Read-only volumes, such as SnapMirror destinations, are not cached in the Flash Pool cache.
FlexArray
V-Series runs behind FAS arrays and other storage from other major vendors.
FlexArray is a software that will replace NetApp's V-Series (virtualization gateways).
FlexArray is built into Ontap. The first version supports NetApp FAS, EMC and Hitachi storage.
FlexArray also supports NetApp's E-Series high-performance arrays
Aggrigatewill use RAID type “Hyprid”
43. ENG. AHMED GAMIL 42
Volume
Volumes contain file systems that hold user data
User can access the data using NFS, CIFS, HTTP, FTP, FC, and iSCSI.
Volume modes
Online Read and write access to this volume is allowed.
Restricted Some operations, such as parity reconstruction, are allowed, but data access is not allowed.
Offline No access to the volume is allowed.
Volume types:
1- Traditional volume
2- FlexVol
3- Flex clone
4- Flex cache
5- Root volume
Traditionalvolume
Traditional volume tightly tied to the aggregate
No other volumes can get their storage from this aggregate.
To increase the traditional volume, we have to increase the size of the aggregateby adding physical disks.
MAX traditional volume size is 16 TB
44. ENG. AHMED GAMIL 43
Traditional volume cannot be decreased/shrink
The smallest traditional volume contains two disks (for RAID4) or three disks (for RAID-DP)
It is available in 32bit only
Traditional Volumes cannot be cloned, but we can create a copy of the volume with space equivalent to the
amount of the original storagespace
FlexVol Volume (flexible volume)
It is a volume that is not depend on the aggregate:
We can decoupling the flex volume from the aggregate
Every FlexVol can be administer independently, grow independently, set up particular features
independently
Choose independently the flexible volumes that are sharing the aggregate.
You can increase or decrease the size of FlexVol volumes online
We can clone the volume (FlexClone)
As mentioned in the image,
Each set of Hard Disks grouped in a RAID groups
All RAID groups aggregatedtoone aggregate.
Then, we can divide the aggregateto“flex volumes”,
Each flex volume can be used as a lun or NFS volume,
5% of the aggregatecapacityreserved for snapshot
20% of the flexvol size reserved for snapshot
0.5% of the flexvol size reserved for volume metadata (space guarantee)
45. ENG. AHMED GAMIL 44
There are two types of FlexVolume
32-bit
64-bit
To use Data ONTAP to move data between a 32-bit volume and a 64-bit volume, you must use “ndmpcopy”
or “qtree SnapMirror”. (vol copy command or volume SnapMirror cannot be used)
Create FlixVol volume
Creating FlexVol requires:
Name
Type of FlexVol (NAS/SAN)
Related aggregate
The size of the FlexVol
Snapshot reserve: what will be reserved in that FlexVol for snapshotting technology? 20%
Space guarantee: free space in the aggregatefor creating FlexVol, if it set to none, then at a certaintime,
users couldn’t write to the FlexVol, and the aggregatecould run out of space.
46. ENG. AHMED GAMIL 45
FlexClone volumes
Copies (point-in-time copies) of a parent flex volume.
FlexVol use its snapshot copy to create FlexClone,
47. ENG. AHMED GAMIL 46
FlexVol is Writable volumes
We can managethem separately as a regularFlexVol volume
After createFlexClone, any changes in the parent FlexVol will not reflected in the FlexClone volume.
Any LUNs present in the parent volume are present in the FlexClone volume but are unmapped and offline.
FlexClone volumes always exist in the same aggregateastheir parent volumes.
FlexClone volumes and parent volumes share the same disk space for any common data (no additional disk
space) until changes are made to the FlexClone volume or its parent
We can Split the FlexClone volume connection with the parent volume, so the FlexClone volume will use its
own disk space instead of sharing it with its parent
FlexClone volumes also can be cloned
The space guarantee and space reservation of the FlexClone volume is created as its parent if the aggregate
has enough space
Root volume
Every aggregatehaveroot volume
Root Volume contains system files, log files, core files, home directory and /etc directory
We can use system manager and the CLI in order to interact with those files
Root volume contain files that can be edit in order to change NetApp system behaviors.
FlexCache Volumes
It is a local volume (in our storage) that is used to cache data for other remote volume (remote storage) to
speeds up data access to remote data
When a client requests data from the FlexCache volume, the data is read from the origin system and cached
on the FlexCache volume, this data is then served directly from the local FlexCache volume.
48. ENG. AHMED GAMIL 47
FlexShare:
Is a features that applied to the volumes to provide:
1- Volume Priority: A higher priority gives a volume a greater percentage of available resources when a
system is fully loaded.
2- User versus system priority : Prioritize user workloads (application and end-user traffic) versus
system work (backup, replication, and so on)
3- Cache utilization : Optimizing cache usage by Configuring the cache to retain data in cache or reuse
the cache depending on workload characteristics
Qtrees
Qtrees enable you to partition your volumes into smaller segments that you can manage individually
Qtrees used to organize the data and apply the following:
Quotas: to limit the size of the data
Backup: to keep backups more modular, flexible backup schedules
Security style: for Windows files and applications, you can group the data in a qtree and set its
security style to NTFS
CIFS oplocks (opportunistic locks): opportunistic locks is a database concept that may require to be
“OFF” or “ON”, So you can assign Qtree to a certain type
We can't do snapshot copies of a Qtree,
49. ENG. AHMED GAMIL 48
We can't do space reservations or guarantees within a Qtree structure.
Create Qtree
50. ENG. AHMED GAMIL 49
Calculating Usable Disk Space
In earlier versions of the ONTAP software,
When NetApp prepare a physical disk, it reserve:
10% for the WAFL file system and 90% for aggregate
5% of the aggregatespace was reserved for the snapshot copy
We createour FlexVol inside the aggregate, 20%of the FlexVol is reserved for snapshot
In new versions
10% for the WAFL file system and 90% for aggregate
0% of the aggregatespace was reserved for the snapshot copy but it is configurable
51. ENG. AHMED GAMIL 50
We createour FlexVol inside the aggregate, 20%of the FlexVol is reserved for snapshot
Review
All physical disks in NetApp are part of what type of grouping?
RAID
Aggregate
FlexVol
Qtree
Which statement is false?
You can increase the size of a FlexVol.
You can decrease the size of a FlexVol.
A FlexVol is tied to a physical disk.
A FlexVol permits the reduction in administrative scope
SnapShot
Snapshot copies are frozen read-only views of our particular volume
It is the first line of defense for backup and recovery of software
We can have up to 255 snapshot per volume
We can do automatic snapshot scheduling, or manually using CLI or GUI (system manager)
We can see
52. ENG. AHMED GAMIL 51
We can take snapshot in hourly basis
We can reserve a space in “%” for snapshot copies
We can check the space allocation for snapshot
53. ENG. AHMED GAMIL 52
SnapRestore
Very easy automated restoration of data
SnapMirror
Technology for replication and disaster recovery
SnapMirror is a DR application that make a replication from primary controller and secondary controller
Then use the SnapUpdate to performing Data Migrations to apply any changes
54. ENG. AHMED GAMIL 53
SnapVault
SnapVault is a backup application
Moving SnapShot copy data to other locations
SV can be scheduled at multiple intervals
Network Administration
How you can set up that NetApp device to easily interact with the network around it.
Name resolution
As Linux machines, network setting contains hosts file, resolve.conf and NIS which contains IP-Name
resolution or DNS settings
55. ENG. AHMED GAMIL 54
And
And we can configure the storageto use ordered setting from “nsswitch.conf” file
56. ENG. AHMED GAMIL 55
Create the VLANs
Network interface tap contains all NIC types and configuration (Physical, Virtual, VLANs)
58. ENG. AHMED GAMIL 57
Create VIF
VIF is a virtual interface that linked to multible physical interfaces to provide FT
Interface groups / aggregation
If one physical interface is down, no effect on the group
Aggregationof links is part of the 802.3ad standard
Single mode:
One interface is active
Multimode staticconfig:
All interfaces are active
Provide LB for the traffic
Multimode dynamic config:
A dynamic protocol (LACP) is used to determine which interface will be used
Take advantage of link aggregationcontrol protocol (LACP) to support LB
Connected switches should support (LACP)
61. ENG. AHMED GAMIL 60
Using CLI
version
Get the firmware on this particular service processor
system
See all of the system commands
62. ENG. AHMED GAMIL 61
system console
connect to the system console
Ifconfig
Configure and monitor interfaces
Ifgrp
Configure and monitor Interfacegroup
63. ENG. AHMED GAMIL 62
setup
Will reconfigure basic configuration files like “/etc/rc”, “/etc/exports”, “/etc/nsswitch.conf”, etc…….
Creating LUNs
There are many ways to createLUNs
Create LUN GUI
Or
65. ENG. AHMED GAMIL 64
Then choose the relatedvolume that will createthe LUN inside
66. ENG. AHMED GAMIL 65
Then createthe igroup: the initiator group that's allowed to access this LUN (just a table of who can access
the particular LUN)
67. ENG. AHMED GAMIL 66
Delete LUN GUI
We cannot delete online LUN, we have to make it offline first
69. ENG. AHMED GAMIL 68
Using SnapDrive
Snapdrive is a software addons for WIN and Linux desktop
Used to expand the storage, managesnapshot copies, cloning disks
NFS Exports and CIFS Shares
In System Manager:
Create Share for a Windows based CIFS Share
Create Export for NFS export
70. ENG. AHMED GAMIL 69
For CIFS, WIN users has to Map network drive to share the location for all users
For NFS export, Linux host has to createdirectory and mount the exported NFS in order to use the storage
If we go to export TAP, we will find our previous FlixVol we had created“East”
In order to createnew export, lets delete “/vol/East” and createit again
Delete Export
Create Export
Choose the volume you need to export
72. ENG. AHMED GAMIL 71
Create CIFS share
No CIFS share automatically created, System Manager contains only hidden shares in windows env.
We will create new CIFS share with same steps as Export
73. ENG. AHMED GAMIL 72
Also you can “Edit” and change permissions and other options
74. ENG. AHMED GAMIL 73
Stop Sharing
Using BranchCache
BranchCache is a feature used when your remote office is connected with a low-bandwidth WAN connection,
and you have many devices in remote site needs the same file, so using BranchCache will send the file to only
one remote device then cache it to others
75. ENG. AHMED GAMIL 74
Distributed caching
Any remote device can cache the file and provide it to other devices like “P2P”
Hosted Casing
Use a server instead of normal devices to cache files and provide them to devices like “client-server”
Branch cache configurations
Most branch cache configurations done by CLI
Branch cache should be enabled per service
1- Go to system Console
2- Check Branch cache for CIFS share
3- Create Branch cache for CIFS share
cifs shares -add share_name path –branchcache
76. ENG. AHMED GAMIL 75
cifs shares -add projects /vol/East –branchcache
Using Quotas
We have many types of quotas: Qtree quota, user quota, group quota
Qtree is seems to be as flexvol volumes but there aresome differences
You can enable/disable snapshot feature
Space guarantee
Support quotas
Support backup
Security styles (NTFS, UNIX, MIXED)
CIFS oplock
77. ENG. AHMED GAMIL 76
Create Qtree
Set Quota
You can set quota based on the used space or no. of files
79. ENG. AHMED GAMIL 78
Then set hard limits, soft limit “warning” and threshold
80. ENG. AHMED GAMIL 79
To get information about the quota:
/etc/quotas to see raw storageinformation about the quota
81. ENG. AHMED GAMIL 80
RBAC
Role-Based Access Control used to enhance the security of your organization by:
Provide differing levels of access for different users or groups
Apply Least Privilege concept: createaccount that has the least level of privileges
Able to log what one administrator is doing from another
Provide
Capabilities, Roles,
Create User
Create new user “gamil” and assign it to Administrator group
82. ENG. AHMED GAMIL 81
Review
Which statement regarding RBAC in NetApp is incorrect?
Capabilities refer to specific actions
User accounts are placed in groups
Capabilities areassigned users
NetApp includes an Admin and Power Users group
Storage System Maintenance
Auto support
83. ENG. AHMED GAMIL 82
Logs
Through Event Management System (EMS) that generatethe logs, you can filter the logs
Degraded disk
It is a situation that we have RAID-DPwith 5 disks, then 2 disks failed, and no other spare disks
Disks will stay in this degraded mode for 24 hours by default, then shutdown
So data is still exist but there's no spares, this situation called degraded disk
If the situation occurred, you will receive auto support e-mails and
Disk scrubbing
NetApp check media for errors, check the parity information
Start checking Manually CLI:
disk scrub start
disk scrub stop
We can start checking automatically
sysstat
sysstat is a CLI tool that doing a certain number of queries of the system statistics, or set the interval
84. ENG. AHMED GAMIL 83
Storage Space Management
How you aredoing space guaranteeor disk provisioning
Provisioning
Full provisioning: full space guarantee
Thin provisioning: allocate space as it is stored
File provisioning: need space guarantee for a certain file
85. ENG. AHMED GAMIL 84
Space allocation
You can check space summary from “space allocation tap”
CLI
df –h ncsa_class
aggrshow_space –h aggr0
86. ENG. AHMED GAMIL 85
Storage Efficiency (Deduplication)
Take fingerprint of the block, then looking for blocks with identical fingerprints, then erase that blocks and
add pointer that points to the other block that has the identical fingerprint.
So that there's no identical storage.
87. ENG. AHMED GAMIL 86
Clustered Data ONTAP Concepts
In 2011, we had ONTAP 8.1 referring to two different modes of operation:
The 7-mode
Clustered mode
o For NAS, we can have 24 nodes inside of our clustered environment.
o For SAN, we can have 8 nodes inside of our clustered environment
Clustering = HA pair = multiple node is working together with 10 GB connection between each other
88. ENG. AHMED GAMIL 87
ONTAP 8.2 support single node cluster, 2 node switchless cluster
Virtual storage servers (VSS)
Data Vserver is presenting a particular namespace to clients
Clustered Data ONTAP User Interfaces
There are 2 user interfaces:
OnCommand system manager (GUI)
Cluster shell (CLI)
For CLI, you can use “tab” to auto complete
“..” go back one step
“top” go to the top of hierarchy
st get statistics or storage
Clustered Volumes and Namespaces
In a clustered environment we deal with only FlexVol
89. ENG. AHMED GAMIL 88
FlexVol can move through the cluster
FlexVol owned by Vservers
Namespace
Volumes from all over the cluster become a Namespace
Access to the root of the Namespace allow access to volumes information that exists below it
Junction
Bind the volumes together in the Namespace.
In order to createa namespace, we have to create a junction for all volumes
Junction looks like a directory path “/eng/p7/source”
Infinite volume
Starting from clustered data ontap 8.1.1
Provides a single mount point that can scale to 20PB and 2 billion files
Infinite volume integrates with deduplication, compression, and NetApp SnapMirror replication technology
From clustered data ontap 8.1.1, they started to add disaster recovery features
Infinite volume doesn’t support Qtrees or its quota
To create Infinite volume, you have to activate“IS repository switch”
90. ENG. AHMED GAMIL 89
Clustered File Access
Enabled NFS on the Storage Virtual Machine (SVM) to managefile access using NFS.
NFS v4 and later are stateful technology
Clustered Data ONTAP 8.1 support NFS4 and 4.1, PNFS (Parallel NFS) and CIFS
NFS4 use referrals concept that allow users to obtain the information from the storage system
If the user request for information comes into node1, but the information is actually stored in node2.or, if
volume is moved from one node to another, SO:
NFSv4
Clients have to unmount and remount the file system manually
PNFS (Parallel NFS)
Does not have to unmount or remount to access the new location of the volume
PNFS uses logical interface (LIF) to communicate with the cluster
LIF
There is LIF Inside clustered Data ONTAP, which is logical interface owned by storage virtual machine
(SVM)
91. ENG. AHMED GAMIL 90
LIF Is an IP address associated with a physical port. If there is any component failure, a LIF can fail over to or
be migrated to a different physical port, thereby continuing to communicate with the cluster.
PNFS allows us to separate the Metadata Path from the actual Data Path, so the client can reach the data
wherever it is stored in any node
PNFS does not support FlexCache volumes, coral volumes, and load-sharing volumes.
Common Internet File System (CIFS)
Used SMP protocol version 2, 2.1 and 3, it is stateful technology since SMB v2
SMB v2.1 Support Oplock leasing
SMB v3 support BranchCache 2, Witness Protocol and Hyper-V over Server Message Block
Netapp Networks
Management Network
Admin use LIF to access the management network and managethe infrastructure
Cluster Interconnect
There's logical interfaces for the cluster interconnecting for the inter-cluster communications
Data network
There's LIF for end user who is accessing the data that's stored in the cluster
92. ENG. AHMED GAMIL 91
Clustered Load Balancing
The ability to smooth out load, to load balance in your storage enterprise.
Natapp allow LB through DNS
You can createvolumes across NetApp cluster in a round-robin fashion, and we can move volumes within the
vserver between nodes or aggregates
In a CIFS environment, the CIFS clients might see a slight degradation in performance
Load Sharing Mirrors (LSM)
We can take a particular FlexVol, then createa load-sharing mirror and copy a read-only copy of that FlexVol.
The read-only copy exist in the same location in the namespace, but it's help with our load-balancing
implementation.
There is automatic and manual sync between original FlexVol and the read-only copy
SAN Infrastructure
There are 2 operating systems:
data ontap
SANtricity used in big data environments (E-Series NetApp devices)
93. ENG. AHMED GAMIL 92
Unified storage
NetApp can easily address both NAS and SAN and at the same time
NAS : file-based access to storageusing CIFS or NFS
SAN : Block Level access using iSCSI or FC or FCOE
For FC, the connection terms are:
WWNN: Worldwide Node Names, a unique identifier for the device itself.
WWPN: Worldwide Port Names that areidentifiers for the particular portal
For ISCSI:
IQN style
EUI style
Steps for the establishment of the SAN:
1- Disc system use as our Target device
2- Create a session from the Initiator to this Target
3- Create an igroup controls access to the LUNs. (Like Zoning)
4- Create a LUN
5- Map a LUN to igroup
6- Find a LUN on the OS by the user
7- Prepare the LUN
FC Connectivity
What is NetApp going to plug in to?
94. ENG. AHMED GAMIL 93
What is end host systems going to plug in to?
How's that all going to work?
The parallel SCSI has scalability issues with limited speed
Fibre Channel connectivity is faster and can accommodate 12 million addresses
Fibre Channel connection:
P2P : legacy
Arbitrated Loop (AL): used when connecting Fibre Channel disks inside of the NetApp
Fabric Config : used today
FC support Dual Fabric (switches) to provide redundancy to prevent single point of failure
We can do dual path with single fabric and dual path with dual fabrics
The initiator for the fabric can be Linux, WIN, and VMwareand the endpoint for the Initiators is a Host Bus
Adapter (HBA).
The target device is like Brocade, Cisco
So the initiator connect to HBAof the FC switch and the switch is connected to the NETAPP target FC
interface (fc0b)
Commands
sysconfig command
Allow you to identify your onboard Fibre Channel adapters on the particular NetApp device
Allow you to do the Fibre Channel Admin command
fcadmin command
Let you view the current configuration in Fibre Channel config,
configure particular Fibre Channel
95. ENG. AHMED GAMIL 94
License command
To view and add license
FCOE is not a Host Bus Adapter “HBA”, it is Converged Network Adapter
FCOE converge our network traffic, it take FC traffic and lay it through the LAN (encapsulating it in Ethernet
frames)
FCOE requires 10 gigabit per second connectivity
96. ENG. AHMED GAMIL 95
References
https://www.cbtnuggets.com/it-training/netapp-certified-storage-associate-ncsa-ns0-145
http://www.datadisk.co.uk/html_docs/netapp/netapp_disk.htm