The document discusses using Go language to build a distributed computing architecture with multiple machines. It covers using RabbitMQ and NSQ for queues, building a simple queue mechanism for single machine versions, and rewriting the architecture to address concurrency limits. It also discusses setting up a server-agent system with RPC communication and context cancellation to enable job cancellation.
1. what is the different unbuffered and buffered channel?
2. how to implement a job queue in golang?
3. how to stop the worker in a container?
4. Shutdown with Sigterm Handling
5. Canceling Workers without Context
6. Graceful shutdown with worker
7. How to auto-scaling build agent?
8. How to cancel the current Job?
Hiveminder - Everything but the Secret SauceJesse Vincent
Ten tools and techniques to help you:
Find bugs faster バグの検出をもっと素早く
Build web apps ウェブアプリの構築
Ship software ソフトのリリース
Get input from users ユーザからの入力を受けつける
Own the Inbox 受信箱を用意する
今日の話
How to make a large C++-code base manageablecorehard_by
My talk will cover how to work with a large C++ code base professionally. How to write code for debuggability, how to work effectively even due the long C++ compilation times, how and why to utilize the STL algorithms, how and why to keep interfaces clean. In addition, general convenience methods like making wrappers to make the code less error prone (for example ranged integers, listeners, concurrent values). Also a little bit about common architecture patterns to avoid (virtual classes), and patterns to encourage (pure functions), and how std::function/lambda functions can be used to make virtual classes copyable.
GDG Devfest 2019 - Build go kit microservices at kubernetes with easeKAI CHU CHUNG
Gokit is microservice tookit and use Service/Endpoint/Transport to strict separation of concerns design. This talk to use go-kit develop microservice application integrate with consul, zipkin, prometheus, etc service and deploy on Kubernetes.
Недавно работы комитета по стандартизации WG21 были завершены, и документ-черновик C++17 был отправлен на рассмотрение в Международную организацию по стандартизации (ISO). С этого момента технически можно считать, что стандарт C++17 у нас есть. Если вы ещё ознакомились с принятыми изменениями, то сейчас для этого самое время. В докладе будет сделан обзор нововведений. Рассмотрено текущее состояние дел у популярных компиляторов с поддержкой С++17
1. what is the different unbuffered and buffered channel?
2. how to implement a job queue in golang?
3. how to stop the worker in a container?
4. Shutdown with Sigterm Handling
5. Canceling Workers without Context
6. Graceful shutdown with worker
7. How to auto-scaling build agent?
8. How to cancel the current Job?
Hiveminder - Everything but the Secret SauceJesse Vincent
Ten tools and techniques to help you:
Find bugs faster バグの検出をもっと素早く
Build web apps ウェブアプリの構築
Ship software ソフトのリリース
Get input from users ユーザからの入力を受けつける
Own the Inbox 受信箱を用意する
今日の話
How to make a large C++-code base manageablecorehard_by
My talk will cover how to work with a large C++ code base professionally. How to write code for debuggability, how to work effectively even due the long C++ compilation times, how and why to utilize the STL algorithms, how and why to keep interfaces clean. In addition, general convenience methods like making wrappers to make the code less error prone (for example ranged integers, listeners, concurrent values). Also a little bit about common architecture patterns to avoid (virtual classes), and patterns to encourage (pure functions), and how std::function/lambda functions can be used to make virtual classes copyable.
GDG Devfest 2019 - Build go kit microservices at kubernetes with easeKAI CHU CHUNG
Gokit is microservice tookit and use Service/Endpoint/Transport to strict separation of concerns design. This talk to use go-kit develop microservice application integrate with consul, zipkin, prometheus, etc service and deploy on Kubernetes.
Недавно работы комитета по стандартизации WG21 были завершены, и документ-черновик C++17 был отправлен на рассмотрение в Международную организацию по стандартизации (ISO). С этого момента технически можно считать, что стандарт C++17 у нас есть. Если вы ещё ознакомились с принятыми изменениями, то сейчас для этого самое время. В докладе будет сделан обзор нововведений. Рассмотрено текущее состояние дел у популярных компиляторов с поддержкой С++17
Из презентации вы узнаете:
про большинство утилит из арсенала Go, предназначенных для оптимизации производительности;
— как и когда их (утилиты) использовать, а также мы посмотрим как они устроены внутри;
— про применимость linux утилиты perf для оптимизации программ на Go.
Кроме того, устроим небольшой crash course, в рамках которого поэтапно соптимизируем несколько небольших программ на Go с использованием вышеперечисленных утилит.
1. Why we moving API from REST to Graphql?
2. What is Graphql?
3. Graphql in Golang (Why we choose Golang)
4. How to testing Graphql in Golang
5. Deploy Graphql application
A journey through the wonderful world of Node.js C++ addons. This talk was given at the September 8, 2015 NodeMN meetup.
Code: https://github.com/cb1kenobi/nodemn
HHVM: Efficient and Scalable PHP/Hack Execution / Guilherme Ottoni (Facebook)Ontico
The HipHop Virtual Machine (HHVM) is the PHP and Hack execution engine developed at Facebook. HHVM is the fastest existing PHP/Hack engine, and it powers not only Facebook's servers, but also Wikipedia, Baidu, and many other sites across the web. In this talk, we'll give an overview of the design of HHVM, with particular focus on its JIT compiler. I'll share some of our experiences building this high performance system, as well as some of the key challenges that we've faced to improve PHP performance.
Small Node.js proxy to turn a paginated JSON REST API into a CSV streaming download. Examples of code and patterns.
Presented at the London Node User Group meetup, April 2014
Taking Jenkins Pipeline to the Extremeyinonavraham
Slide deck from Jenkins User Conference Tel Aviv 2018.
Talking about suggested (best?) practices, tips and tricks, using Jenkins pipeline scripts with shared libraries, managing shared libraries, using docker compose, and more.
node.js and native code extensions by examplePhilipp Fehre
Over the last years node.js has evolved to be a great language to build web applications. The reason for this is not only that it is based on JavaScript which already is established around "the web" but also that it provides excellent facilities for extensions, not only via JavaScript but also integration of native C libraries. Couchbase makes a lot of use of this fact making the Couchbase node.js SDK (Couchnode) a wrapper around the C library providing a node.js like API, but leveraging the power of a native C library underneat. So how is this done? How does such a package look like? Let me show you how integration of C in node.js works and how to "read" a package like Couchnode.
KubeCon EU 2016: Getting the Jobs Done With KubernetesKubeAcademy
When you hear words such as Kubernetes or OpenShift you immediately start thinking
about long running processes you can easily scale at will. However, Kubernetes includes a lesser known feature which allows you to run pretty much anything from simple tasks up to highly-complicated ones.
During this presentation, the author of the Job resource in Kubernetes will guide you through several techniques for performing anything ranging from simple Pi calculations to rendering a movie. No matter if you're a data scientist running large scale calculations across several data centers or a hobby programmer running simple day-to-day tasks, this presentation is to teach you how to efficiently use Kubernetes Jobs on their own or as the building blocks of something
bigger.
This presentation will feature a number of live demos to help illustrate the various ways that you can put Jobs to work. Don’t miss out on learning about one of the coolest features of Kubernetes!
Sched Link: http://sched.co/6BUw
Streams are a fundamental programming primitive for representing the flow of data through your system. It's time we brought this powerful tool to the web. What if we could stream data from a HTTP request, through a web worker that transforms it, and then into a <video> tag? Over the last year, I've been working on the WHATWG streams specification, which builds upon the lessons learned in Node.js, to provide a suitable abstraction for needs of the extensible web.
I'll discuss briefly why streams are important, what they enable, and the role we envision them playing in the future of the web platform. Mostly, though, I want to help you understand streams, at a deep level. In the course of writing this specification, I've learned a lot about streams, and I want to share that knowledge with you. At the core, they are a very simple and beautiful abstraction. I think we've done a good job capturing that abstraction, and producing an API the web can be proud of. I'd love to tell you all about it.
Source plugins are new in GHC 8.6 and provide a convenient mechanism for a user to inspect and modify the internal source code representation of a Haskell program. They are an extension to the existing plugins mechanism which adds passes which run after parsing, renaming and typechecking. This allows users to extend the compiler without having to modify GHC itself.
gRPC in Golang presentation
In this talk, I introduced gRPC, Protocol buffer, and how to use them with golang.
Source code used in the presentation: http://github.com/AlmogBaku/grpc-in-go
Из презентации вы узнаете:
про большинство утилит из арсенала Go, предназначенных для оптимизации производительности;
— как и когда их (утилиты) использовать, а также мы посмотрим как они устроены внутри;
— про применимость linux утилиты perf для оптимизации программ на Go.
Кроме того, устроим небольшой crash course, в рамках которого поэтапно соптимизируем несколько небольших программ на Go с использованием вышеперечисленных утилит.
1. Why we moving API from REST to Graphql?
2. What is Graphql?
3. Graphql in Golang (Why we choose Golang)
4. How to testing Graphql in Golang
5. Deploy Graphql application
A journey through the wonderful world of Node.js C++ addons. This talk was given at the September 8, 2015 NodeMN meetup.
Code: https://github.com/cb1kenobi/nodemn
HHVM: Efficient and Scalable PHP/Hack Execution / Guilherme Ottoni (Facebook)Ontico
The HipHop Virtual Machine (HHVM) is the PHP and Hack execution engine developed at Facebook. HHVM is the fastest existing PHP/Hack engine, and it powers not only Facebook's servers, but also Wikipedia, Baidu, and many other sites across the web. In this talk, we'll give an overview of the design of HHVM, with particular focus on its JIT compiler. I'll share some of our experiences building this high performance system, as well as some of the key challenges that we've faced to improve PHP performance.
Small Node.js proxy to turn a paginated JSON REST API into a CSV streaming download. Examples of code and patterns.
Presented at the London Node User Group meetup, April 2014
Taking Jenkins Pipeline to the Extremeyinonavraham
Slide deck from Jenkins User Conference Tel Aviv 2018.
Talking about suggested (best?) practices, tips and tricks, using Jenkins pipeline scripts with shared libraries, managing shared libraries, using docker compose, and more.
node.js and native code extensions by examplePhilipp Fehre
Over the last years node.js has evolved to be a great language to build web applications. The reason for this is not only that it is based on JavaScript which already is established around "the web" but also that it provides excellent facilities for extensions, not only via JavaScript but also integration of native C libraries. Couchbase makes a lot of use of this fact making the Couchbase node.js SDK (Couchnode) a wrapper around the C library providing a node.js like API, but leveraging the power of a native C library underneat. So how is this done? How does such a package look like? Let me show you how integration of C in node.js works and how to "read" a package like Couchnode.
KubeCon EU 2016: Getting the Jobs Done With KubernetesKubeAcademy
When you hear words such as Kubernetes or OpenShift you immediately start thinking
about long running processes you can easily scale at will. However, Kubernetes includes a lesser known feature which allows you to run pretty much anything from simple tasks up to highly-complicated ones.
During this presentation, the author of the Job resource in Kubernetes will guide you through several techniques for performing anything ranging from simple Pi calculations to rendering a movie. No matter if you're a data scientist running large scale calculations across several data centers or a hobby programmer running simple day-to-day tasks, this presentation is to teach you how to efficiently use Kubernetes Jobs on their own or as the building blocks of something
bigger.
This presentation will feature a number of live demos to help illustrate the various ways that you can put Jobs to work. Don’t miss out on learning about one of the coolest features of Kubernetes!
Sched Link: http://sched.co/6BUw
Streams are a fundamental programming primitive for representing the flow of data through your system. It's time we brought this powerful tool to the web. What if we could stream data from a HTTP request, through a web worker that transforms it, and then into a <video> tag? Over the last year, I've been working on the WHATWG streams specification, which builds upon the lessons learned in Node.js, to provide a suitable abstraction for needs of the extensible web.
I'll discuss briefly why streams are important, what they enable, and the role we envision them playing in the future of the web platform. Mostly, though, I want to help you understand streams, at a deep level. In the course of writing this specification, I've learned a lot about streams, and I want to share that knowledge with you. At the core, they are a very simple and beautiful abstraction. I think we've done a good job capturing that abstraction, and producing an API the web can be proud of. I'd love to tell you all about it.
Source plugins are new in GHC 8.6 and provide a convenient mechanism for a user to inspect and modify the internal source code representation of a Haskell program. They are an extension to the existing plugins mechanism which adds passes which run after parsing, renaming and typechecking. This allows users to extend the compiler without having to modify GHC itself.
gRPC in Golang presentation
In this talk, I introduced gRPC, Protocol buffer, and how to use them with golang.
Source code used in the presentation: http://github.com/AlmogBaku/grpc-in-go
In this webinar, we talked about hard-to-test patterns in C++ and show how to refactor them. The difficulty, in this context, does not lie in the code's inherent complexity.
The focus will be on patterns technically difficult to unit test because they may:
* Require irrelevant software to be tested too
* E.g.: 3rd party libraries, classes other than the one under test
* Delay the test execution
* E.g.: sleeps inside code under test
* Require intricate structures to be copied or written from scratch
* E.g.: fakes containing a lot of logic
* Require test details to be included in the production code
* E.g.: #ifdef UNIT_TESTS
* Make changes and/or are dependent on the runtime environment
* E.g.: Creating or reading from files
JVM Mechanics: When Does the JVM JIT & Deoptimize?Doug Hawkins
HotSpot promises to do the "right" thing for us by identifying our hot code and compiling "just-in-time", but how does HotSpot make those decisions?
This presentation aims to detail how HotSpot makes those decisions and how it corrects its mistakes through a series of demos that you run yourself.
An exploration into RxJava on Android for the experienced, yet uninitiated software engineer. This presentation explores Declarative vs Imperative programming paradigms and expands the discussion into Functional Reactive Programming. It explains the benefits of the observer contract, high-order functions, and schedulers available in RxJava. It also explains the purpose of the Android integration libraries: RxAndroid, RxLifecycle, and RxBindings.
Azul Virtual Machine Engineer Douglas Hawkins describes how decisions made by the JVM affect how your code is compiled and run. Learn how this affects application performance and what steps you can take to optimize how the JVM acts on your code.
go-git is a 100% Go libray used to interact with git repositories. Even if it already supports most of the functionality it still lags a bit in performance when compared with the git CLI or some other libraries. I'll explain some of the problems that we face when dealing with git repos and some examples of performance improvements done to the library.
This talk I will show you the golang project layout and some best practice like as the following:
1. RESTful api and GraphQL
2. Model testing (Postgres, SQLite, MySQL)
3. Software Quality
4. Data Metrics
5. Go Testing
* What is different GitHub Flow and Git Flow?
* What is GitHub Actions?
* How to write the simple workflow?
* What's problem in GitHub Actions UI?
* What's problem with Secrets in GitHub Actions?
* How to write your first GitHub Actions and upload to the marketplace?
* What's a problem with environment variables in GitHub Actions?
Drone is a Continuous Delivery platform built on Docker, written in Go
* Why I don’t choose Jenkins or GitLab CI?
* What is Drone CI?
* Drone Infrastructure
* How to install Drone in five minutes?
* Integrate your project
* Create your Drone plugin
* Try drone cli without drone server
Go 語言入門介紹,底下是大綱:
* Go 語言誕生
* Go 語言優勢
* Go 語言選擇
* 誰在用 Go 語言
最後會提到 goroutine 及 channel 這兩大特性。如果對 Go 語言有興趣,可以參考線上課程
https://www.udemy.com/golang-fight/?couponCode=GOLANG-INTRO
The following is my Agenda
* Why I don’t choose Jenkins or GitLab CI?
* What is Drone CI?
* Drone Infrastructure
* How to install Drone in five minutes?
* Integrate your project
* Create your Drone plugin
* Try drone CLI without drone server
more detail information you can find that in Udemy: https://www.udemy.com/devops-oneday/?couponCode=DRONE-DEVOPS
Go 語言 (又稱Golang) 是 Google 推出新一代的強大語言,今年 3 月 Google 公佈了去年底統計的問卷結果,發現 63% 用 Go 來寫網站,38% 用來開發系統程式,35% 用來做 DevOps,本次議題將會帶您瞭解為什麼 Go 語言適合打造微服務架構,Go 語言大給微服務什麼樣的特性以及 Go 語言適合用來開發什麼樣的系統?
Gitea is a painless self-hosted Git service. It is similar to GitHub, Bitbucket or Gitlab. The initial development have been done on Gogs but we have forked it and named it Gitea. If you want to read more about the reasons why we have done that please read this blog post.
https://docs.gitea.io/en-us/
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
2. About me
• Software Engineer in Mediatek
• Member of Drone CI/CD Platform
• Member of Gitea Platform
• Member of Gin Golang Framework
• Maintain Some GitHub Actions Plugins.
• Teacher of Udemy Platform: Golang + Drone
57. Watch the Cancel event (Agent)
go func() {
done, _ := r.Manager.Watch(ctx, id)
if done {
cancel()
}
}()
58. Handle cancel event on Server
subscribers: make(map[chan struct{}]int64),
cancelled: make(map[int64]time.Time),
59. User cancel running job
c.Lock()
c.cancelled[id] = time.Now().Add(time.Minute * 5)
for subscriber, build := range c.subscribers {
if id == build {
close(subscriber)
}
}
c.Unlock()
60. Agent subscribe the cancel event
for {
select {
case <-ctx.Done():
return false, ctx.Err()
case <-time.After(time.Minute):
c.Lock()
_, ok := c.cancelled[id]
c.Unlock()
if ok {
return true, nil
}
case <-subscriber:
return true, nil
}
}