The document discusses non-uniform cache architectures (NUCA), cache coherence, and different implementations of directories in multicore systems. It describes NUCA designs that map data to banks based on distance from the controller to exploit non-uniform access times. Cache coherence is maintained using directory-based protocols that track copies of cache blocks. Directories can be implemented off-chip in DRAM or on-chip using duplicate tag stores or distributing the directory among cache banks. Examples of systems like SGI Origin2000 and Tilera Tile64 that use these techniques are also outlined.
OSDC 2012 | Extremes Wolken Dateisystem!? by Dr. Udo SeidelNETWAYS
So-Called shared file systems have been known in the Unix / Linux-environment for a long time. The network-based and cluster data systems are thereby almost an old hat. The project XtreemFS– a part of the grid operating system XtreemOS – is a quite new but budded alternative in this field. The current version finally has some important features and is worth taking a closer look.
This presentation gives an insight into the data systems approaches, explains the architecture and describes the first steps of a XtreemFS-Cluster set-up.
Content repositories allow you to store simple files, documents with metadata, or even complex documents with structured metadata, and to work with them in various ways. This talk will describe the Nuxeo Core content management engine, and how it builds on standards like JCR 2 and CMIS to provide high-level features to end-users. We will show how Nuxeo Core uses the above standards to provide efficient storage of high volumes of structured or unstructured documents in a flexible manner, using either a JCR or a transparent SQL backend. The architecture and benefits of this new backend will also be presented in an interactive manner.
This is the presentation given during CentOS Dojo Bangalore.
The slides are originally authored by Vijay Bellur, Lalatendu Mohanty. Added a few slides at the end for CentOS setup.
OSDC 2012 | Extremes Wolken Dateisystem!? by Dr. Udo SeidelNETWAYS
So-Called shared file systems have been known in the Unix / Linux-environment for a long time. The network-based and cluster data systems are thereby almost an old hat. The project XtreemFS– a part of the grid operating system XtreemOS – is a quite new but budded alternative in this field. The current version finally has some important features and is worth taking a closer look.
This presentation gives an insight into the data systems approaches, explains the architecture and describes the first steps of a XtreemFS-Cluster set-up.
Content repositories allow you to store simple files, documents with metadata, or even complex documents with structured metadata, and to work with them in various ways. This talk will describe the Nuxeo Core content management engine, and how it builds on standards like JCR 2 and CMIS to provide high-level features to end-users. We will show how Nuxeo Core uses the above standards to provide efficient storage of high volumes of structured or unstructured documents in a flexible manner, using either a JCR or a transparent SQL backend. The architecture and benefits of this new backend will also be presented in an interactive manner.
This is the presentation given during CentOS Dojo Bangalore.
The slides are originally authored by Vijay Bellur, Lalatendu Mohanty. Added a few slides at the end for CentOS setup.
Block Level Storage Vs File Level StoragePradeep Jagan
Video Management System is responsible for accessing, controlling and managing the video content management environment across an Internet Protocol Network.
There are different dimensions for scalability of a distributed storage system: more data, more stored objects, more nodes, more load, additional data centers, etc. This presentation addresses the geographic scalability of HDFS. It describes unique techniques implemented at WANdisco, which allow scaling HDFS over multiple geographically distributed data centers for continuous availability. The distinguished principle of our approach is that metadata is replicated synchronously between data centers using a coordination engine, while the data is copied over the WAN asynchronously. This allows strict consistency of the namespace on the one hand and fast LAN-speed data ingestion on the other. In this approach geographically separated parts of the system operate as a single HDFS cluster, where data can be actively accessed and updated from any data center. The presentation also cover advanced features such as selective data replication.
Extended version of presentation at Strata + Hadoop World. November 20, 2014. Barcelona, Spain.
http://strataconf.com/strataeu2014/public/schedule/detail/39174
OSBConf 2015 | Scale out backups with bareos and gluster by niels de vosNETWAYS
During this talk, Niels will explain the basics of Gluster and show how Bareos integrates with it. Gluster provides a Software Defined Storage environment that can scale-out when the backup storage needs to grow. With a live demonstration Niels shows how simple it is to setup a small Gluster environment and configure Bareos to use the native Gluster protocol.
Block Level Storage Vs File Level StoragePradeep Jagan
Video Management System is responsible for accessing, controlling and managing the video content management environment across an Internet Protocol Network.
There are different dimensions for scalability of a distributed storage system: more data, more stored objects, more nodes, more load, additional data centers, etc. This presentation addresses the geographic scalability of HDFS. It describes unique techniques implemented at WANdisco, which allow scaling HDFS over multiple geographically distributed data centers for continuous availability. The distinguished principle of our approach is that metadata is replicated synchronously between data centers using a coordination engine, while the data is copied over the WAN asynchronously. This allows strict consistency of the namespace on the one hand and fast LAN-speed data ingestion on the other. In this approach geographically separated parts of the system operate as a single HDFS cluster, where data can be actively accessed and updated from any data center. The presentation also cover advanced features such as selective data replication.
Extended version of presentation at Strata + Hadoop World. November 20, 2014. Barcelona, Spain.
http://strataconf.com/strataeu2014/public/schedule/detail/39174
OSBConf 2015 | Scale out backups with bareos and gluster by niels de vosNETWAYS
During this talk, Niels will explain the basics of Gluster and show how Bareos integrates with it. Gluster provides a Software Defined Storage environment that can scale-out when the backup storage needs to grow. With a live demonstration Niels shows how simple it is to setup a small Gluster environment and configure Bareos to use the native Gluster protocol.
Εξεταστέα ύλη μαθημάτων της Γ΄τάξης του Τομέα Δομικών Εργων σχ.έτους 2015-16John Tzortzakis
Απόκομμα του Φ3/65958/Δ4 19-04-2016 εγγράφου του ΥΠΠΕΘ που αφορά στην ενδεικτική εξεταστέα ύλη μαθημάτων όλων των τομέων του ΕΠΑΛ. Το απόκομμα περιέχει μόνο όσα αναφέρονται στον Τομέα Δομικών Έργων.
Σημ Στο αρχικό έγγραφο δεν περιλαμβάνεται ύλη μαθημάτων της Β τάξης
ΦΕΚ 770 22-3-2016 ΑΠΣ Αναλυτικά Προγράμματα Σπουδών των μαθημάτων ειδικοτήτων του Τομέα Δομικών Έργων της Ομάδας Προσανατολισμού Τεχνολογικών Εφαρμογών των Β ́ και Γ ́ τάξεων ΕΠΑΛ
NoSQL – Data Center Centric Application EnablementDATAVERSITY
The growth of Datacenter infrastructure is trending out of bounds, along with the pace in user activity and data generation in this digital era. However, the nature of the typical application deployment within the data center is changing to accommodate new business needs. Those changes introduce complexities in application deployment architecture and design, which cascade into requirements for a new generation of database technology (NoSQL) destined to ease that complexity. This webcast will discuss the modern data centers data centric application, the complexities that must be dealt with and common architectures found to describe and prescribe new data center aware services. Well look at the practical issues in implementation and overview current state of art in NoSQL database technology solving the problems of data center awareness in application development.
Apache Cassandra is a non-relational database which is given by the Apache. Initially, Cassandra was open sourced by Facebook in 2008, and is now developed by Apache Group.
In the normal relational databases data stores in the format of rows, but in Cassandra the data will stored in columns format as key value pairs. Due to this column based data storage its giving the high performance while comparing the relational databases.
Cassandra can handle many terabytes of data if need be and can easily handle millions of rows, even on a smaller cluster. Cassandra can get around 20K inserts per second.
The performance of Cassandra is high and keeping the performance up while reading mostly depends on the hardware, configuration and number of nodes in your cluster. It can be done in Cassandra without much trouble.
Data Lake and the rise of the microservicesBigstep
By simply looking at structured and unstructured data, Data Lakes enable companies to understand correlations between existing and new external data - such as social media - in ways traditional Business Intelligence tools cannot.
For this you need to find out the most efficient way to store and access structured or unstructured petabyte-sized data across your entire infrastructure.
In this meetup we’ll give answers on the next questions:
1. Why would someone use a Data Lake?
2. Is it hard to build a Data Lake?
3. What are the main features that a Data Lake should bring in?
4. What’s the role of the microservices in the big data world?
Basic Introduction to Cassandra with Architecture and strategies.
with big data challenge. What is NoSQL Database.
The Big Data Challenge
The Cassandra Solution
The CAP Theorem
The Architecture of Cassandra
The Data Partition and Replication
PromCon EU 2022 - Centralized vs Decentralized Prometheus Scraping Architectu...Eric D. Schabell
There are two primary approaches to scrape and collect metrics using Prometheus - using a centralized set of dedicated scrapers or decentralized scrapers that run as an agent. With centralized scraping, Prometheus is deployed as a central scraper to pull metrics from all discoverable endpoints and sometimes can be split across multiple centralized instances using a few different approaches. However, with a decentralized approach, Prometheus runs as an agent and in Kubernetes is deployed as a DaemonSet on each node in a cluster and only collects metrics from the node it runs on. Each model has pros and cons - especially when operating at large scale - which can make it difficult when deciding which model to use.
In this session, the speaakers will provide an overview of Prometheus metrics collection at DoorDash, where having highly reliable resources, easy endpoint discovery, and real-time insights is critical. They will share insights and best practices into DoorDash’s decision to implement a decentralized model by offering pros and cons of each approach. Leave with a better understanding of the “right” model for your use case(s).
Factored Operating Systems (fos) - The Case for a Scalable Operating System for Multicores - Designing a new operating system targeting manycore
systems with scalability as the primary design constraint,
where space sharing replaces time sharing to increase
scalability.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
1. Outline
• Non-Uniform Cache Architecture (NUCA)
• Cache Coherence
• Implementation of directories in multicore
architecture
1
2. Non-Uniform Cache Architecture [1]
• Uniform Cache Architecture
▫ Multi-level cache hierarchies
Organized into a few discrete levels
Each level reduces access to the lower level
Inclusion overhead
Internal wire delays
Restricted number of ports
▫ Large on-chip cache
Single and discrete hit latency
Undesirable due to increasing wire delays
2
3. Non-Uniform Cache Architecture [1]
• Non-uniform cache architecture (NUCA)
▫ Exploit non-uniformity
Data in large cache closer to processor is accessed
faster than data residing physically farther
Level 2 caches architectures, 16MB with 50nm technology (taken from [1])
3
4. Non-Uniform Cache Architecture [1]
• Static NUCA
▫ Each bank can be accessed at different speeds
Proportional to the distance from the controller
Lower latency when closer to controller
▫ Mapping of data into banks based on block index
▫ Banks are independently addressable
▫ Access to banks may proceed in parallel
Banks have private channels
▫ Large number of wires
▫ Access time and routing delay increase with time
Best organization at smaller technologies uses larger
banks
4
6. Non-Uniform Cache Architecture [1]
• Switched Static NUCA
▫ 2D Mesh, point-to-point links
▫ Removes most of the large number of wires
▫ Allows a large number of faster, smaller banks
• Dynamic NUCA
▫ Allows data to be mapped to many banks
▫ Allows data to migrate among the banks
▫ Frequently used data can be promoted to faster
banks
6
8. Non-Uniform Cache Architecture [2]
• Policies
▫ Bank placement policy
Where is data placed in the NUCA cache memory
▫ Bank access policy
Determines bank-searching algorithm
▫ Bank migration policy
Determines if a data element is allowed to change its
placement from one bank to another
Regulates migration of data
▫ Bank replacement policy
How NUCA behaves when there is a data eviction from
one of the banks
8
10. Cache Coherence
• Cache-coherence problem
• Support for large number of processors
▫ Need for high bandwidth
▫ Bus architecture insufficient
• Point-to-Point networks
▫ No broadcast mechanism
▫ Snooping protocol unusable
• Directory
▫ Solution for point-to-point networks
▫ Stores location of cache copies of blocks of data
▫ Centralized or distributed
10
11. Implementation of directories in
multicore architectures [3]
• DRAM (off-chip) directory
▫ Stores directory information in DRAM
Ex: full-map protocol
▫ Does not exploit distance locality
▫ Treats each tile as a potential sharer of data
▫ Directory can be cached in on-chip SRAM
Do not need to access off-chip memory each time
11
13. Implementation of directories in
multicore architecture [4]
• DRAM (off-chip) directory with directory caches
▫ Private cache
▫ Directory is cached in each tile
Do not need to access off-chip memory each time
Non-coherent caches
Home node for any given cache line
Different range of memory address for each tile
▫ Directory controller in each tile
Controls coherency between private caches
13
15. Implementation of directories in
multicore architectures [3]
• Duplicate tag directory
▫ Directory centrally located in SRAM
▫ Connected to individual cores
▫ Exact duplicate tag store
Directory state for a block is determined by examining
copy of tags of every possible cache that can hold the
block
Keep copied tags up-to-date
▫ No more need to read states from DRAM memory
▫ Challenging as the number of cores increases
64 cores, 16-way associative cache = 1024 aggregate
associativity of all tiles
15
17. Implementation of directories in
multicore architecture [5]
Directory memory, 4-way associative caches (taken from [5])
17
18. Implementation of directories in
multicore architectures [3]
• Static cache bank directory
▫ Distributed directory among the tiles
Mapping block address to a tile (called the home tile)
Home tiles selected by simple interleaving
Location can be sub-optimal (see next slide)
Tile’s cache extended to contain directory
information
Integrates directory states with cache tags
Avoids SRAM or DRAM separate directory
18
20. Implementation of directories in
multicore architecture [7]
• SGI Origin2000 multiprocessor system
▫ Directory memory connected to on-chip memory
Shared L2 cache
Directory memory distributed over multiple tiles
Cache coherence controller
Home tile sends appropriate messages to cores
20
21. Implementation of directories in
multicore architecture [7]
SGI Origin2000 multiprocessor system (taken from [7])
21
22. Implementation of directories in
multicore architecture [8]
• Tilera Tile64 architecture
▫ 2d mesh network (8X8)
▫ Provides coherent shared-memory environment
▫ Uses neighborhood caching
Provides on-chip distributed shared cache
▫ Coherency is maintained at the home tile
Data is not cached at non-home tiles
▫ Communication over a Tile Dynamic Network
22
24. References
• [1] C. Kim, D. Burger, S.W. Keckler, “An Adaptative, Non-Uniform Cache Structure for Wire-Delay Dominated On-Chip
Caches”, in Proc. 10th Int. Conf. ASPLOS, San Jose, CA, 2002, pp. 1-12
• [2] J. Lira, C. Molina, A. Gonzalez, “Analysis of Non-Uniform Cache Architecture Policies for Chip-Multiprocessors Using
the Parsec Benchmark Suite”, MMCS’09, Mar. 2009, pp. 1-8
• [3] M.R. Marty, M.D. Hill, “Virtual Hierarchies to Support Server Consolidation”, ISCA’07, June 2007, pp. 1-11
• [4] J.A. Brown, R. Kumar, D. Tullsen, “Proximity-Aware Directory-based Coherence for Multi-core Processor Architectures”,
SPAA’07, June 2007, pp. 1-9
• [5] J. Chang, G.S. Sophi, “Cooperative Caching for Chip Multiprocessors”, Computer Architecture, ISCA '06. 33rd
International Symposium on, 2006, pp.264-276
• [6] S. Cho, L. Jin, "Managing Distributed, Shared L2 Caches through OS-Level Page Allocation“, Microarchitecture, 2006.
MICRO-39. 39th Annual IEEE/ACM International Symposium on, Dec. 2006, pp.455-468
• [7] H. Lee, S. Cho, B.R. Childers, "PERFECTORY: A Fault-Tolerant Directory Memory Architecture“, Computers, IEEE
Transactions on , vol.59, no.5, May 2010, p.638-650
• [8] D. Wentzlaff, P. Griffin, H. Hoffmann, L. Bao, B. Edwards, C. Ramey, M. Mattina, C.C. Miao, J.F. Brown, A. Agarwal,
"On-Chip Interconnection Architecture of the Tile Processor“, Micro, IEEE , vol.27, no.5, Sept.-Oct. 2007, pp.15-31
• [9] Linux Devices, “4-way chip gains Linux IDE, dev cards, design wins” [online], Linux Devices, Apr. 2008 [cited Oct. 21
2010] , available from World Wide Web: < http://thing1.linuxdevices.com/news/NS4811855366.html >
24