ChinaNetCloud training for Linux Memory Basics for SysAdmins.
This is an introduction to general Linux memory for troubleshooting, monitoring, and basic understanding.
Talk given on state of NUMA with Java databases such as Cassandra and how it can improved / ameliorated, and compared with traditional storage engines.
How to Meet Your P99 Goal While Overcommitting Another WorkloadScyllaDB
Meeting a tight P99 latency goal is hard, it's harder when running
multiple workloads with a mix of real time sensitive and analytical workloads.
In this presentation, I will cover the Scylla schedulers and controllers and demonstrate how they guarantee a good level of resource isolation.
In this webinar, we'll discuss the different ways to back up and restore your single servers, replica sets, and sharded clusters in case of a disaster scenario. We'll review various approaches, including taking filesystem snapshots, using mongodump and mongorestore, or leveraging MongoDB Management Service to backup and restore.
Talk given on state of NUMA with Java databases such as Cassandra and how it can improved / ameliorated, and compared with traditional storage engines.
How to Meet Your P99 Goal While Overcommitting Another WorkloadScyllaDB
Meeting a tight P99 latency goal is hard, it's harder when running
multiple workloads with a mix of real time sensitive and analytical workloads.
In this presentation, I will cover the Scylla schedulers and controllers and demonstrate how they guarantee a good level of resource isolation.
In this webinar, we'll discuss the different ways to back up and restore your single servers, replica sets, and sharded clusters in case of a disaster scenario. We'll review various approaches, including taking filesystem snapshots, using mongodump and mongorestore, or leveraging MongoDB Management Service to backup and restore.
A backup and recovery strategy is necessary to protect your mission critical data against the risk of catastrophic failure or human error. In this session, we'll discuss the different strategies to backing up and restoring your MongoDB clusters in case of a disaster scenario. We'll review the benefits and drawbacks of various approaches, including taking filesystem snapshots, using mongodump, or using MongoDB Management Service.
Redis : Database, cache, pub/sub and more at Jelly button gamesRedis Labs
Nir Shney-Dor of Jelly button games talks about how he uses Redis across many different use cases - as a persistent db, cache, for pub/sub, leaderboards etc etc..Fun walkthrough of all the uses one can put Redis to.
DB Latency Using DRAM + PMem in App Direct & Memory ModesScyllaDB
How does the latency of DDR4 DRAM compare to Intel Optane Persistent Memory when used in both App Direct and Memory Modes for In-Memory database access?
This talk is about the latency benchmarks that I performed by adding gettimeofday() calls around critical DB kernel operations.
This talk covers the technology, cache hit ratios, lots of histograms and lessons learned.
Redis as a Main Database, Scaling and HADave Nielsen
Iskren Chernev, an Independent developer, uses a lot of Redis. In this talk, Iskren will look at a particular Redis use-case -- using it as the main database (not cache). Iskren will show how to achieve reasonable guarantees about data integrity, speed, high-availability in an event of failure and infinite horizontal scalability. This particular approach has proven successful in managing clusters of up to 2400 nodes, and storing data north of 7TB before replication. We'll cover ways to separate your data appropriately into many nodes, performing different types of migrations (from another database, from one cluster to another, scaling migrations and migrating out of Redis), moving nodes without downtime, some configuration tips and monitoring.
The talk will elaborate on how to detect and Heal your MySQL topology with MySQL Orchestrator .This talk was delivered on Mydbops database Meetup on 27-04-2019 by Anil Yadav, Lead Database Engineer with OLA and Krishna Ramanathan Database Administrator III with OLA.
Despite all the buzz about it, building a horizontally scalable application for cloud deployment isn't all that different from building one for a physical deployment, except in its ability to change size on-the-fly. Bigger applications have been using commodity hardware and fault-tolerant design to achieve high availability and scalability for a while, but provisioning capacity remains troublesome there. The real addition the cloud brings architecturally is the ability to add new resources instantly, and even change your provisioning profile algorithmically.
Hardware once reserved to HPC systems is entering the datacenter. Cyprien will describe an effort to help developers leverage its new capabilities. Its integration to H2O, along with tools like Caffe, is accelerating and making the platform more powerful. #h2ony
There are two key choices when scaling a NoSQL data store:
choosing between a hash or a range based sharding and choosing the right sharding key. Any choice is a trade-off between scalability of read, append, and update workloads.
In this talk I will present the standard scaling techniques,
some non-universal sharding tricks, less obvious reasons for
hotspots, as well as techniques to avoid them.
Everyone is talking about shared memory versus message passing concurrency and lock-free algorithms in Java and other programming languages. But we forget that most applications use a database and all concurrent sessions end up being serialized there. Should we replace relational database with NoSQL instead? In this talk I will show what a programmer can do to improve concurrency in the database. I will use Oracle database as an example but most ideas apply to other relational databases as well.
A backup and recovery strategy is necessary to protect your mission critical data against the risk of catastrophic failure or human error. In this session, we'll discuss the different strategies to backing up and restoring your MongoDB clusters in case of a disaster scenario. We'll review the benefits and drawbacks of various approaches, including taking filesystem snapshots, using mongodump, or using MongoDB Management Service.
Redis : Database, cache, pub/sub and more at Jelly button gamesRedis Labs
Nir Shney-Dor of Jelly button games talks about how he uses Redis across many different use cases - as a persistent db, cache, for pub/sub, leaderboards etc etc..Fun walkthrough of all the uses one can put Redis to.
DB Latency Using DRAM + PMem in App Direct & Memory ModesScyllaDB
How does the latency of DDR4 DRAM compare to Intel Optane Persistent Memory when used in both App Direct and Memory Modes for In-Memory database access?
This talk is about the latency benchmarks that I performed by adding gettimeofday() calls around critical DB kernel operations.
This talk covers the technology, cache hit ratios, lots of histograms and lessons learned.
Redis as a Main Database, Scaling and HADave Nielsen
Iskren Chernev, an Independent developer, uses a lot of Redis. In this talk, Iskren will look at a particular Redis use-case -- using it as the main database (not cache). Iskren will show how to achieve reasonable guarantees about data integrity, speed, high-availability in an event of failure and infinite horizontal scalability. This particular approach has proven successful in managing clusters of up to 2400 nodes, and storing data north of 7TB before replication. We'll cover ways to separate your data appropriately into many nodes, performing different types of migrations (from another database, from one cluster to another, scaling migrations and migrating out of Redis), moving nodes without downtime, some configuration tips and monitoring.
The talk will elaborate on how to detect and Heal your MySQL topology with MySQL Orchestrator .This talk was delivered on Mydbops database Meetup on 27-04-2019 by Anil Yadav, Lead Database Engineer with OLA and Krishna Ramanathan Database Administrator III with OLA.
Despite all the buzz about it, building a horizontally scalable application for cloud deployment isn't all that different from building one for a physical deployment, except in its ability to change size on-the-fly. Bigger applications have been using commodity hardware and fault-tolerant design to achieve high availability and scalability for a while, but provisioning capacity remains troublesome there. The real addition the cloud brings architecturally is the ability to add new resources instantly, and even change your provisioning profile algorithmically.
Hardware once reserved to HPC systems is entering the datacenter. Cyprien will describe an effort to help developers leverage its new capabilities. Its integration to H2O, along with tools like Caffe, is accelerating and making the platform more powerful. #h2ony
There are two key choices when scaling a NoSQL data store:
choosing between a hash or a range based sharding and choosing the right sharding key. Any choice is a trade-off between scalability of read, append, and update workloads.
In this talk I will present the standard scaling techniques,
some non-universal sharding tricks, less obvious reasons for
hotspots, as well as techniques to avoid them.
Everyone is talking about shared memory versus message passing concurrency and lock-free algorithms in Java and other programming languages. But we forget that most applications use a database and all concurrent sessions end up being serialized there. Should we replace relational database with NoSQL instead? In this talk I will show what a programmer can do to improve concurrency in the database. I will use Oracle database as an example but most ideas apply to other relational databases as well.
Linux Memory Management
1.Memory Structure of Linux OS.
2.How Program is loaded into the memory.
3.Address Translation.
4.Feature for Multithreading and Multiprocessing.
Linux memory consumption - Why memory utilities show a little amount of free RAM? How does Linux kernel utilizes free RAM? What is the real amount of free RAM in the system?
MySQL backups overview. Characteristics of every backup type, including dumps, Xtrabackup and snapshots. Planning proper backup strategies. Why and how to test backups.
OpenNebulaConf 2013 - How Can OpenNebula Fit Your Needs: A European Project F...OpenNebula Project
BonFIRE is an european project which aims at providing a ”multi-site cloud facility for applications, services and systems research and experimentation”. Grouping different research cloud providers behind a common set of tools, APIs and services, it enables users to run their experiment against a heterogeneous set of infrastructure, hypervisors, networks, etc …
BonFIRE, and thus the (OpenNebula) testbeds, provide a relatively small set of images used to boot VMs. However, the experimental nature of BonFIRE projects results in a big ”turnover” of running VMs. Lot of VMs are used for a time period between a few hours and a few days, and an experiment startup can trigger deployment of many VMs at same time on a small set of OpenNebula workers, which does not correspond to usual Cloud workflow.
Default OpenNebula is not optimized for such usecase (small amount of worker nodes, high VMs turnover). However, thanks to its ability to be easily modified at each level of a Cloud deployment workflow, OpenNebula has been tuned to make it fit better with BonFIRE deployment process. This presentation will explain how to change OpenNebula TM and VMM to improve the parrallel deployment of many VMs in a short amount of time, reducing time needed to deploy an experiment to its lowest without lot of expensive hardware.
Bio:
I’m a french system engineer, working at Inria french research laboratory for 2 years, and involved in free software development and support (French Ubuntu community, House automation software, etc …). Inside Myriads team at Inria, I work on a few European research projects including BonFIRE (http://www.bonfire-project.eu), as well as on free Grid5000 project.
How Can OpenNebula Fit Your Needs: A European Project FeedbackNETWAYS
BonFIRE is an european project which aims at providing a ”multi-site cloud facility for applications, services and systems research and experimentation”. Grouping different research cloud providers behind a common set of tools, APIs and services, it enables users to run their experiment against a heterogeneous set of infrastructure, hypervisors, networks, etc …
BonFIRE, and thus the (OpenNebula) testbeds, provide a relatively small set of images used to boot VMs. However, the experimental nature of BonFIRE projects results in a big ”turnover” of running VMs. Lot of VMs are used for a time period between a few hours and a few days, and an experiment startup can trigger deployment of many VMs at same time on a small set of OpenNebula workers, which does not correspond to usual Cloud workflow.
Default OpenNebula is not optimized for such usecase (small amount of worker nodes, high VMs turnover). However, thanks to its ability to be easily modified at each level of a Cloud deployment workflow, OpenNebula has been tuned to make it fit better with BonFIRE deployment process. This presentation will explain how to change OpenNebula TM and VMM to improve the parrallel deployment of many VMs in a short amount of time, reducing time needed to deploy an experiment to its lowest without lot of expensive hardware.
Slides of a course that is given to teach embedded linux to engineers. The full course is 2-days; this is the first time a 'light' version was given lasting a single day.
Focus is on
. What is Linux
. How do I compile
. How do I flash
The Dark Side Of Go -- Go runtime related problems in TiDB in productionPingCAP
Ed Huang, CTO of PingCAP, talked at Go System Conference about dealing with the typical and profound issues related to Go’s runtime as your systems become more complex. Taking TiDB as an example, he demonstrated how these problems can be reproduced, located, and analyzed in production.
Scaling up and accelerating Drupal 8 with NoSQLOSInet
Drupal 8 can scale well and serve pages fast to many users, especially by offloading parts of the work load from the main SQL database to NoSQL solutions.
This presentation describes the strategies and technologies usable to achieve such gains, including specific configuration, contributed modules and custom coding strategies.
How can OpenNebula fit your needs - OpenNebulaConf 2013 Maxence Dunnewind
In the scope of a European Project (BonFIRE - www.bonfire-project.eu ), I had to tune openNebula to fit our requirement that are unusual in a private cloud environment (small hardware, small number of base images, but lot of vms created).
These slides explain how, thanks to how OpenNebula enables administrators to tune it, I updated the transfer manager scripts to improve our deployment speed by almost 8.
Nagios Conference 2014 - Andy Brist - Nagios XI Failover and HA SolutionsNagios
Andy Brist's presentation on High Availability and Failover Solutions for Nagios XI. The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
Software Design for Persistent Memory SystemsC4Media
Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2KhnfRX.
Howard Chu talks about both naive approaches to leveraging NVRAM, and reasons to avoid those approaches, as well as optimal, proven methods for building systems around persistent memory. Filmed at qconlondon.com.
Howard Chu’s work has spanned a wide range of computing topics, including most of the GNU utilities, networking protocols and tools, kernel and filesystem drivers, and focused on maximizing the useful work from a system. He founded Symas Corp. with 5 other partners and serves as its CTO. His current focus is database oriented, covering LDAP, LMDB, and other non-relational database technologies.
Redis Developers Day 2014 - Redis Labs TalksRedis Labs
These are the slides that the Redis Labs team had used to accompany the session that we gave during the first ever Redis Developers Day on October 2nd, 2014, London. It includes some of the ideas we've come up with to tackle operational challenges in the hyper-dense, multi-tenants Redis deployments that our service - Redis Cloud - consists of.
Slides presented at Percona Live Europe Open Source Database Conference 2019, Amsterdam, 2019-10-01.
Imagine a world where all Wikipedia articles disappear due to a human error or software bug. Sounds unreal? According to some estimations, it would take an excess of hundreds of million person-hours to be written again. To prevent that scenario from ever happening, our SRE team at Wikimedia recently refactored the relational database recovery system.
In this session, we will discuss how we backup 550TB of MariaDB data without impacting the 15 billion page views per month we get. We will cover what were our initial plans to replace the old infrastructure, how we achieved recovering 2TB databases in less than 30 minutes while maintaining per-table granularity, as well as the different types of backups we implemented. Lastly, we will talk about lessons learned, what went well, how our original plans changed and future work.
Similar to Linux Memory Basics for SysAdmins - ChinaNetCloud Training (20)
Introduction of OpsStack--Integrated Operation Platform developed by ChinaNetCloud
Operations Evolution Underway
Customer Challenges
OpsStack Killer Features
ChinaNetCloud Online Lecture：Something About TsharkChinaNetCloud
On Jul 26, Cecil Han, Senior engineer of ChinaNetCloud, shared insights on the topic of “Something about Tshark” with audience of ChinaNetCloud Webinar.
What is Tshark?
Why we need Tshark even we have tcpdump?
Common parameters
ChinaNetCloud Online Lecture： Fight Against External Attacks From Different L...ChinaNetCloud
Internet makes the world brand-new, but it also put the system safety in danger— security problems such as DDOS attacks, data theft, and BotNet always bothering IT operations teams. How can we defend ourselves from these types of attack? By implementing four layers of security protection: network, system, code, and operation maintenance.
On July 5, Wang Han, senior architect of ChinaNetCloud shared our view points about “How to resist external attacks” with dozens of audience through webinar.
云中漫步 颠覆创新_创业邦春季创新峰会主题演讲 Cloud Innovation in ChinaChinaNetCloud
President of ChinaNetCloud, Yinan Gu's keynote speech at 2016 DEMO CHINA in Beijing, May 11.
Cloud Innovation in China, Speech at DEMO China in May, 2016 in Bejing by Yinan Gu, ChinaNetCloud's President (in Mandarin)
ChinaNetCloud talk on Operations-as-a-Service at AWS China Summit 2015 in Shanghai. Covers OaaS cases, technology, and AWS.
Given December 17th, 2015, in Shanghai, in Chinese by Yinan Gu.
Running Internet Systems in China - The Details You Need to Succeed in Chines...ChinaNetCloud
There are a lot of misconceptions about how the Internet actually works in China.
China’s Internet has it’s own set of unique problems such as regulatory, legal, infrastructure, connectivity, etc.
In this presentation we will examine the modern Chinese Internet, it’s structure, challenges, and how to succeed in the world’s most populous country.
Internet Cloud Operations - ChinaNetcloud & AWS Event BeijingChinaNetCloud
Internet Operations in the cloud, presented by ChinaNetCloud at joint AWS event in Beijing June, 2015.
Covers background, how clouds are different, key issues, and how OaaS (Operations as a Service) helps.
Presentation on Internet Security by @SteveMushero of @ChinaNetCloud, given in Shanghai in May, 2015.
Covers threats, risks, and solutions.
Bi-lingual English & Chinese.
Cloud Operations Challenges - Talk by ChinaNetCloud at Joint Cisco eventChinaNetCloud
Talk given by ChinaNetCloud CEO Steve Mushero at joint event with Cisco in Beijing on April 2, 2015.
Covers what's wrong with Internet Operations today and how to fix it.
ChinaNetCloud - Public Clouds in China OverviewChinaNetCloud
Overview of the Public Clouds in China, including Aliyun, AWS, Tencent plus UCloud, QingCloud, Azure, and more.
Given by Steve Mushero at BarCamp Shanghai 2014 and other events.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Linux Memory Basics for SysAdmins - ChinaNetCloud Training
1. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetcloud Training Program
ChinaNetCloud Training
Linux Memory Basics
By ChinaNetCloud
Pioneers in OaaS – Operations-as-a-Service
October, 2013
www.ChinaNetCloud.com
2. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 2
Introduction
●
Linux memory is complex and interesting
●
Class summarizes:
●
Types of memory
●
How it's used
●
How to troubleshoot memory issues
3. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 3
Overview, Purpose & Use
●
Linux memory is one of the most important
areas for engineers to understand
●
Used by everything
●
Often not well-understood
●
Often the cause of problems
4. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 4
Introduction
●
Memory is simple, in theory
●
Complex in real use
●
Plenty of strange things, too
●
Important to understand
●
Important to monitor
5. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 5
Memory Types I
●
Physical RAM – Fixed amount, real RAM
●
Virtual Memory – Virtual, can swap
●
Shared – Between processes
●
Oracle & Postgres use this
●
Slab – Kernel memory
●
/proc/slabinfo & slabtop utility
●
Includes big RAM users
– tcpmem & inode cache
6. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 6
Memory Types II
●
Page Cache – File system cache
●
Dirty
●
Changed file system data waiting to write to disks
●
Important for high write systems
7. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 7
/proc/meminfo
Very useful but complex
●
These are items not found by 'free' or 'top'
●
MemTotal: Total usable ram (RAM minus kernel binary code)
●
MemFree: Total free memory, same as free's 'free' output
●
SwapCached: Memory swapped out, then back in, but still
also in the swapfile
8. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 8
/proc/meminfo
●
Active – Recently used, not reclaimed unless necessary
●
Inactive – Less recently used, likely to be reclaimed
●
Dirty – File / Page Cache waiting to be written to disk
●
Writeback – Memory which is now being written to the disk
●
Mapped – Memory Mapped files, such as libraries
●
Includes Mongo, Varnish, and many others
●
Slab – Kernel memory, usually 256-512MB
9. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 9
Memory Page Size
●
4KB page size
●
Important as many stats are in pages
– Be careful of units ! KB vs. Pages
●
sysctl items like tcpmem are mixed
●
Do not confuse with disk sector size of 0.5KB
10. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 10
Free -m
●
Total – Physical in machine
●
Used – by Apps plus Cache, Buffers and maybe Shared
●
Free – Totally unused RAM, not important, usually very small
●
Shared – by different processes like Oracle / Postgres
●
Buffers – Raw block cache to/from disks, not impotrant
●
Cached (Page Cache) – Used by disk files cached in RAM
●
Swap – Total, Used, Free – Used should be small
●
-/+ buffers/cache – Important numbers, inside box is key
●
PAY ATTENTION to the BOX number – it's all that matters
total used free shared buffers cached
Mem: 3961 3901 60 0 121 1232
-/+ buffers/cache: 2546 1414
Swap: 4683 756 3927
11. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 11
Top, ps Memory Output
●
VIRT – Virtual size
●
Includes mapped libraries, often very large
●
Not too useful – IGNORE THIS
●
RSS – Resident Set Size
●
Most important !
●
Real RAM used by applications, not including swap
●
SHR – Shared, such as Oracle / Postgres
●
Be careful to see processes not threads
●
Threads will share RSS, see htop in thread mode
12. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 12
Swap
●
Simple, but not simple
●
Managed by kswapd
●
Actual swap used calc
●
SwapTotal – SwapCached – SwapFree
●
Goal is zero swap on servers
●
Never let a system actively swap
●
But some systems will have small swap
●
Common to see 100-200MB, more on big system
●
NUMA defaults can cause swapping
13. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 13
Problem with Swap
●
Swapping FREEZES the swapping process
●
So, a 32MB byte swap of MySQL RAM
●
Freezes ALL of MySQL for many seconds
●
Very bad for applications
●
Bad for any single process system
●
MySQL, Nginx, HAProxy, etc. (not Apache)
●
Also uses valuable IO, slowing DBs, etc.
●
Goal is to NEVER swap real applications
14. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 14
Swappiness
●
Tells kernel which is more important
●
File system Cache vs. App Memory
●
Default is 60 – Means cache more important
●
Stupid for servers – will swap even with free RAM
●
Set to 1, always (used to be 0, now use 1)
●
Won't swap until all RAM used by applications
●
Note some swap anyway (see next slide)
●
Set by sysctl
●
Check in /proc/sys/vm/swappiness
15. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 15
Swap still happens
●
Linux likes to swap a little
●
Even with free RAM and swappiness = 0
●
Some kernel data wants to swap
●
Can pre-swap SwapCached data
●
Still a mystery, but it's okay
●
Very large systems (64GB+ might swap 1-2GB)
●
Turning off can cause kswapd to go crazy
●
Watch vmstat swapin/out, make sure very small
16. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 16
NUMA - Can cause swap
●
NUMA – Non-Uniform Memory Access
●
Each CPU has its own RAM
●
Slower to use 'other' CPU's RAM
●
Standard now on all Intel servers
●
Causes RAM inbalance on big RAM processes
●
Like MySQL, MongoDB, Java
●
Install & use numa-utils package, numactl app
●
Set all big processes to full interleave:
●
numactrl –interleave all mysqld . . .
17. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 17
Dirty Memory & Ratios
●
Causes strange problems & performance issues
●
See /proc/meminfo
●
Dirty is the changed FileSystem cache data
●
Must be written to disk
●
Can cause big problems if too high
●
Two ratios control
●
/proc/sys/vm/dirty_background_ratio / _bytes
●
/proc/sys/vm/dirty_ratio / _bytes
●
Freezes all writing processes when dirty > dirty_ratio
●
Calculation not clear, but keep dirty data < 20% of RAM
18. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 18
OOM Killer
●
Kills process when no more RAM or Swap
●
Chooses process to die via a score
●
Biggest RAM user (MySQL) often dies first
●
See /proc/<pid>/oom_score
●
Always a message in dmesg - /var/log/kernel
●
Can adjust score to control (advanced)
●
Best practice to monitor swap and log
19. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
Copyright 2013 ChinaNetCloud 19
Summary
●
Memory important to understand
●
Especially basics & definitions
●
Understand swap & swappiness
●
Watch NUMA on big systems
●
Memory is cheap – buy more !
21. ChinaNetCloudRunning the World's Internet Servers 管理全球服务器
ChinaNetCloud
Sales@ChinaNetCloud.com
www.ChinaNetCloud.com
Beijing Office:
Lee World Business Building #305
57 Happiness Village Road,
Chaoyang District
Beijing, 100027 China
Silicon Valley Office:
California Avenue
Palo Alto, 94123 USA
Shanghai Headquarters:
X2 Space 1-601, 1238 Xietu Lu
Shanghai, 200032 China
T: +86-21-6422-1946 F: +86-21-
6422-4911
