Fluentd meetup dive into fluent plugin (outdated)N Masahiro
Fluentd meetup in Japan. I talked about "Dive into Fluent plugin".
Some contents are outdated. See this slide: http://www.slideshare.net/repeatedly/dive-into-fluentd-plugin-v012
Fluentd meetup dive into fluent plugin (outdated)N Masahiro
Fluentd meetup in Japan. I talked about "Dive into Fluent plugin".
Some contents are outdated. See this slide: http://www.slideshare.net/repeatedly/dive-into-fluentd-plugin-v012
Snaps are a new packaging format that allows unmodified binaries to run across a wide variety of distributions. Snapd is the software that manages snaps on a running system. Learn about the basics of snaps, snapd and what is needed to port snapd to OpenSUSE.
Facebook is a company that operates at massive scale. In this talk we’ll talk about how we use Python at Facebook.
Be it building back-end services, fast prototyping, automation, scaling operations, or simply gluing together various pieces of our infrastructure, Python is at the heart of it and allows our engineers to quickly deliver working solutions.
We’ll talk about our review process, unit testing, deployment workflow and various open-source framework we use.
Keynote on Fluentd Meetup Summer
Related Slide
- Fluentd ServerEngine Integration & Windows Support http://www.slideshare.net/RittaNarita/fluentd-meetup-2016-serverengine-integration-windows-support
- Fluentd v0.14 Plugin API Details http://www.slideshare.net/tagomoris/fluentd-v014-plugin-api-details
“p4alu” is a P4 program who would parse UDP packet with payload in "p4alu header format" and apply calculation.
This program is tested using BMv2 simple_switch P4 target.
Snaps are a new packaging format that allows unmodified binaries to run across a wide variety of distributions. Snapd is the software that manages snaps on a running system. Learn about the basics of snaps, snapd and what is needed to port snapd to OpenSUSE.
Facebook is a company that operates at massive scale. In this talk we’ll talk about how we use Python at Facebook.
Be it building back-end services, fast prototyping, automation, scaling operations, or simply gluing together various pieces of our infrastructure, Python is at the heart of it and allows our engineers to quickly deliver working solutions.
We’ll talk about our review process, unit testing, deployment workflow and various open-source framework we use.
Keynote on Fluentd Meetup Summer
Related Slide
- Fluentd ServerEngine Integration & Windows Support http://www.slideshare.net/RittaNarita/fluentd-meetup-2016-serverengine-integration-windows-support
- Fluentd v0.14 Plugin API Details http://www.slideshare.net/tagomoris/fluentd-v014-plugin-api-details
“p4alu” is a P4 program who would parse UDP packet with payload in "p4alu header format" and apply calculation.
This program is tested using BMv2 simple_switch P4 target.
실전 애자일 게임 개발 (Agile Game Agile Game Development From The Trenches)Kay Kim
Noel Llopis가 Montreal International Game Summit 2006에서 발표한 내용을 번역.
Agile 중에서 XP에 대해서 주로 다룸.
자세한 것은 http://betterways.tistory.com/51 참조.
Source: http://www.gamesfromwithin.com/articles/0611/000112.html
Everyone knows installing PHP on IBM i is easy. Just download the save file and run the RSTLICPGM command. But not everyone knows what that RSTLICPGM does. What is it putting on the machine? Where is it putting it? How does it all work together? In this session, we'll take a look at the IFS directories added by the Zend Server installer, to see what's in them. We'll see where PHP and PHP extensions, Apache and FastCGI configurations, log files, and more are kept. We'll also talk about how these new file objects interact with existing prerequisite software on the IBM i to present dynamic pages to the web. This session is for PHP developers and administrators interested in learning how PHP fits in with the unique operating environment of IBM i.
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
There’s no such thing as fast enough. You can always make your website faster. This talk will show you how. The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get tired and leave.In this talk we’ll start with the basics and get progressively insane. We’ll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they’ve changed over the years. We’ll also look at some great tools to help you.
Presentation from OpenStack Summit Tokyo
Also check the video from
https://www.openstack.org/summit/tokyo-2015/videos/presentation/ntts-journey-with-openstack
How to Use Telegraf and Its Plugin EcosystemInfluxData
Telegraf is the open source server agent which is used to collect metrics from your stacks, sensors and systems. It is InfluxDB’s native data collector that supports over 250+ inputs and outputs. Learn how to send data from a variety of systems, apps, databases and services in the appropriate format to InfluxDB. Discover tips and tricks on how to write your own plugins.
Join this webinar as Jessica Ingrassellino and Samantha Wang dive into:
Types of Telegraf plugins (i.e. input, output, aggregator and processor)
Specific plugins including Execd input plugins and the Starlark processor plugin
How to create your own Telegraf plugin
Software Define your Current Storage with OpensourceAntonio Romeo
While Software Defined Storage is becoming one of the major trend topics in the Data Center, what do you do with your current “legacy” arrays?
Learn how with ViPR Controller, or its Open Source counterpart, CoprHD, you can automate and make your datacenter “software defined” with your current infrastructure.
Content from my Brighttalk webinar available here: https://www.brighttalk.com/webcast/10739/169959
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get fed up and leave.
In this talk we'll start with the basics and get progressively insane. We'll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
About
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
• Remote control: Parallel or serial interface.
• Compatible with MAFI CCR system.
• Compatible with IDM8000 CCR.
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
• Easy in configuration using DIP switches.
Technical Specifications
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
Key Features
Indigenized remote control interface card suitable for MAFI system CCR equipment. Compatible for IDM8000 CCR. Backplane mounted serial and TCP/Ethernet communication module for CCR remote access. IDM 8000 CCR remote control on serial and TCP protocol.
• Remote control: Parallel or serial interface
• Compatible with MAFI CCR system
• Copatiable with IDM8000 CCR
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
Application
• Remote control: Parallel or serial interface.
• Compatible with MAFI CCR system.
• Compatible with IDM8000 CCR.
• Compatible with Backplane mount serial communication.
• Compatible with commercial and Defence aviation CCR system.
• Remote control system for accessing CCR and allied system over serial or TCP.
• Indigenized local Support/presence in India.
• Easy in configuration using DIP switches.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Saudi Arabia stands as a titan in the global energy landscape, renowned for its abundant oil and gas resources. It's the largest exporter of petroleum and holds some of the world's most significant reserves. Let's delve into the top 10 oil and gas projects shaping Saudi Arabia's energy future in 2024.
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxR&R Consult
CFD analysis is incredibly effective at solving mysteries and improving the performance of complex systems!
Here's a great example: At a large natural gas-fired power plant, where they use waste heat to generate steam and energy, they were puzzled that their boiler wasn't producing as much steam as expected.
R&R and Tetra Engineering Group Inc. were asked to solve the issue with reduced steam production.
An inspection had shown that a significant amount of hot flue gas was bypassing the boiler tubes, where the heat was supposed to be transferred.
R&R Consult conducted a CFD analysis, which revealed that 6.3% of the flue gas was bypassing the boiler tubes without transferring heat. The analysis also showed that the flue gas was instead being directed along the sides of the boiler and between the modules that were supposed to capture the heat. This was the cause of the reduced performance.
Based on our results, Tetra Engineering installed covering plates to reduce the bypass flow. This improved the boiler's performance and increased electricity production.
It is always satisfying when we can help solve complex challenges like this. Do your systems also need a check-up or optimization? Give us a call!
Work done in cooperation with James Malloy and David Moelling from Tetra Engineering.
More examples of our work https://www.r-r-consult.dk/en/cases-en/
Governing Equations for Fundamental Aerodynamics_Anderson2010.pdf
Fluentd Hacking Guide at RubyKaigi 2014
1. Fluentd Hacking Guide
(Fluentd ソースコード完全解説)
September 20th, 2014
!
Naotoshi Seo @sonots
DeNA Co., Ltd.
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
2. 2
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Who am I?
・Naotoshi Seo @sonots
・DeNA Co, Ltd.
・Infrastructure Engineer
・Fluentd Commiter
3. ⁃ An application (or a framework) to process log streaming
⁃ Powerful plugin architecture (+250 plugins)
⁃ written in Ruby
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
4. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Disclaimer
4
1. I do not explain what is Fluentd, how to use Fluentd
2. I assume audiences have ever used Fluentd, have ever
created Fluentd plugins, have interests inside Fluentd
!
3. Source Codes are from v0.10 branch as of September 2014
5. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Topics
5
1. The bootstrap sequence of Fluentd, and how Fluentd loads
plugins
2. How an input plugin passes data to output plugins
3. How BufferedOutput plugin works
4. How Fluentd parses the config file
5. The event-driven programming using cool.io, and effects of
GVL
6. 6
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Let's Dive into
Source Codes
8. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Topics
8
1. The bootstrap sequence of Fluentd, and how Fluentd loads
plugins
2. How an input plugin passes data to output plugins
3. How BufferedOutput plugin works
9. The Bootstrap Sequence
9
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Fluent::Supervisor#run_configure
1. require
2. new
3. configure(conf )
Fluent::Supervisor#run_engine
4. start
5. shutdown (if signal received)
Input plugin creates threads on #start
Output plugin does nothing on #start (typically)
10. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Topics
10
1. The bootstrap sequence of Fluentd, and how Fluentd loads
plugins
2. How an input plugin passes data to output plugins
3. How BufferedOutput plugin works
11. Dataflow from Input to Output
11
Input Engine Output
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
emit(tag, es)
emit(tag, es)
If an input thread receives data, call Engine.emit
12. CAUTION: Output Plugin
12
Input Engine Output
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
emit(tag, es)
emit(tag, es)
BLOCK!!!
Can not receive new input during blocking
ex) HTTP POST
13. 13
Use BufferedOutput
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
to avoid blocking
14. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Topics
14
1. The bootstrap sequence of Fluentd, and how Fluentd loads
plugins
2. How an input plugin passes data to output plugins
3. How BufferedOutput plugin works
15. BufferedOutput Plugin
Input Engine BufferedOutput BasicBuffer
15
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
emit(tag,es)
emit(tag,es)
emit(tag,data)
enqueue
immediately
return!!
Can receive new input. No blocking.
16. BufferedOutput Plugin
16
BufferedOutput BasicBuffer
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
try_flush
(push)
OutputThread
pop
write(chunk)
do some EXPENSIVE things
Run heavy processing in OTHER threads
17. CAUTION: BufferedOutput
17
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Stuck if enqueued over its capability
!
HOW TO IMPROVE
1. Increase num_threads
2. Enlarge buffer_chunk_limit
3. Set smaller queued_chunk_flush_interval, try_flush_interval
(Secret parameters)
19. 19
Improving actual
processing throughputs
is most important
fluent-plugin-elasticsearch gets stuck? Then, tune Elasticsearch!
Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
20. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
Conclusion
20
1. Output plugin blocks
2. BufferedOutput does not block, but stuck if enqueued
over its capability
• Tune with option parameters such as num_threads
3. In either case, improving actual processing capability
itself is most important!!
4. I am happy if this talk helps your operation, your
plugin development, and contributions