Writing New Relic Plugins: NSQ
•Download as PPTX, PDF•
1 like•1,068 views
Lucas Fontes is a chief architect at Uken Games, which is a Rails shop that builds HTML5 and Unity3d games. They have lots of servers and data. He discussed the various tools they use for operations, monitoring, server monitoring/alerting, log crunching, visualization, and business intelligence. He then explained how New Relic can provide application monitoring out of the box or they can roll their own plugin using the New Relic Rubygem to handle configuration, API communication, and data aggregation. He provided an example NSQ plugin they created that is on GitHub.
Report
Share
Report
Share
Recommended
Sheep it
This document summarizes Lucas Fontes' approach to deploying applications with Docker and Consul at Uken Studios, which has around 150 servers running 70 application servers and 40 databases. It outlines challenges with their previous deployment process and infrastructure, and describes how their new system addresses these using Docker for building and running applications, Consul for service discovery and configuration, and Consul Template to reload applications when configurations change. The workflow involves building Docker images, running containers securely, and releasing new versions by pushing images to a registry and notifying Consul to trigger reloads.
Building Docker Containers @ Scale
This document discusses building Docker containers for a Rails application in a continuous delivery pipeline. It notes that as the company and number of developers grows, build times increase which can lead to bottlenecks. Various strategies are presented for optimizing the build process such as leveraging Docker layers and caching to speed up rebuild times. A new tool called Pipa is proposed that aims to provide fast, secure, sandboxed builds by leveraging Docker and build caches in a standardized way to help scale testing and deployments.
Chaos Engineering for Docker
Slides from my ContainerCamp UK 2017 session.
These slides present a practical chaos engineering approach for resilience testing of Docker based software systems.
DockerCon EU 2015: The Glue is the Hard Part: Making a Production-Ready PaaS
PaaSTA is Yelp's open-source Docker-based PaaS that glues together common PaaS components like scheduling with Mesos/Marathon, service delivery with Docker, discovery with SmartStack, and monitoring with Sensu. The document discusses how PaaSTA provides a production-ready platform by using stable components, reducing single points of failure, enabling graceful degradation, making upgrades painless, and incorporating self-healing and alerting capabilities. Lessons learned include the importance of interfaces, maintaining the app-infra boundary, using the right abstractions, and making iterative improvements.
"On-premises" FaaS on Kubernetes
In this webinar, Alex Casalboni will overview the main FaaS concepts and best practices (Function as a Service), explore the open-source FaaS options and discuss pros and cons of deploying and managing your own serverless platform on Kubernetes.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
TIAD 2016 : Migrating 100% of your production services to containers
The document summarizes BlaBlaCar's journey to migrating 100% of their production services to containers. It describes how they started with bare metal servers and evolved to using configuration management tools like Chef. They then standardized on CoreOS and rkt as their container platform. Key tools they developed include dgr for building container images and ggn for managing services running in containers. They also implemented service discovery using custom tools like Nerve and Synapse. The document shares many lessons learned from their large scale production container deployment.
Building applications with Serverless Framework and AWS Lambda - JavaZone 2019
Slides from intro workshop at JavaZone 2019 on building applications using Serverless Framework and AWS Lambda.
Recommended
Sheep it
This document summarizes Lucas Fontes' approach to deploying applications with Docker and Consul at Uken Studios, which has around 150 servers running 70 application servers and 40 databases. It outlines challenges with their previous deployment process and infrastructure, and describes how their new system addresses these using Docker for building and running applications, Consul for service discovery and configuration, and Consul Template to reload applications when configurations change. The workflow involves building Docker images, running containers securely, and releasing new versions by pushing images to a registry and notifying Consul to trigger reloads.
Building Docker Containers @ Scale
This document discusses building Docker containers for a Rails application in a continuous delivery pipeline. It notes that as the company and number of developers grows, build times increase which can lead to bottlenecks. Various strategies are presented for optimizing the build process such as leveraging Docker layers and caching to speed up rebuild times. A new tool called Pipa is proposed that aims to provide fast, secure, sandboxed builds by leveraging Docker and build caches in a standardized way to help scale testing and deployments.
Chaos Engineering for Docker
Slides from my ContainerCamp UK 2017 session.
These slides present a practical chaos engineering approach for resilience testing of Docker based software systems.
DockerCon EU 2015: The Glue is the Hard Part: Making a Production-Ready PaaS
PaaSTA is Yelp's open-source Docker-based PaaS that glues together common PaaS components like scheduling with Mesos/Marathon, service delivery with Docker, discovery with SmartStack, and monitoring with Sensu. The document discusses how PaaSTA provides a production-ready platform by using stable components, reducing single points of failure, enabling graceful degradation, making upgrades painless, and incorporating self-healing and alerting capabilities. Lessons learned include the importance of interfaces, maintaining the app-infra boundary, using the right abstractions, and making iterative improvements.
"On-premises" FaaS on Kubernetes
In this webinar, Alex Casalboni will overview the main FaaS concepts and best practices (Function as a Service), explore the open-source FaaS options and discuss pros and cons of deploying and managing your own serverless platform on Kubernetes.
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
One of the most underrated features of Kubernetes is namespaces. In the market, instead of using this feature, people are still stuck with having different clusters for their environments. This talk will try to break this approach, and will introduce how we end up using ephemeral namespaces within our CI/CD pipeline. It will cover the architecture of our system for running the user acceptance tests on isolated ephemeral namespaces with every bits and pieces running within pods. While doing this, we will set up our CI/CD pipeline on top of TravisCI, GoCD, and Selenium that is controlled by Nightwatch.js.
Sched Link: http://sched.co/6Bcb
TIAD 2016 : Migrating 100% of your production services to containers
The document summarizes BlaBlaCar's journey to migrating 100% of their production services to containers. It describes how they started with bare metal servers and evolved to using configuration management tools like Chef. They then standardized on CoreOS and rkt as their container platform. Key tools they developed include dgr for building container images and ggn for managing services running in containers. They also implemented service discovery using custom tools like Nerve and Synapse. The document shares many lessons learned from their large scale production container deployment.
Building applications with Serverless Framework and AWS Lambda - JavaZone 2019
Slides from intro workshop at JavaZone 2019 on building applications using Serverless Framework and AWS Lambda.
Scaling Jenkins with Kubernetes by Ami Mahloof
This document discusses scaling Jenkins with Kubernetes. Previously, Jenkins was run on single EC2 instances with issues like underutilization and port collisions. The new approach runs Jenkins on Kubernetes, allowing on-demand PODs for each build with complete isolation. A modified Kubernetes plugin was created to support multiple containers and persistent volumes per POD. Builds are queued and run on Kubernetes nodes, improving scalability. Demostrating this approach saves 70-90% over running Jenkins on always-on EC2 instances by using spot instances for Kubernetes nodes.
Kernel load-balancing for Docker containers using IPVS
IPVS is a Linux kernel module that provides high performance virtual server capabilities including load balancing, failover and dynamic configuration. GORB is an open source REST API and daemon that controls IPVS to provide load balancing for Docker containers. It supports TCP and HTTP health checks and integrates with Docker links. Using IPVS, GORB and Docker, high performance load balancing can be achieved without the need for expensive proprietary hardware load balancers.
Extend and build on Kubernetes
API basics, kinds, resource, apigroups, etcd storage, watches, discovery, ThirdPartyResources, custom apiservers, API aggregation
Enabling Microservices @Orbitz - DockerCon 2015
Slides from Orbitz's use case for microservices on docker & mesos at Velocity Santa Clara 2015 conference.
Details: http://sched.co/39S6
KubeCon EU 2016: Using Traffic Control to Test Apps in Kubernetes
Testing applications is important, as shown by the rise of continuous integration and automated testing. In this talk, I will focus on one area of testing that is difficult to automate: poor network connectivity. Developers usually work within reliable networking conditions so they might not notice issues that arise in other networking conditions. I will give examples of software that would benefit from test scenarios with varying connectivity. I will explain how traffic control on Linux can help to simulate various network connectivity. Finally, I will run a demo showing how an application running in Kubernetes behaves when changing network parameters.
Sched Link: http://sched.co/6Bb3
Beyond static configuration
Beyond static configuration management discusses how containerization and distributed configuration management are disrupting traditional system engineering. Key developments include specialized container-centric operating systems like CoreOS, orchestration tools like Docker, Mesos, and Kubernetes, as well as configuration stores like etcd, Consul, and Zookeeper that enable dynamic configuration of distributed systems. The talk argues this represents an exciting transition period for development and operations.
London Hug 19/5 - Terraform in Production
- Terraform allows infrastructure teams to more efficiently and agilely provision resources at scale across multiple production datacenters and regions.
- Key benefits include auto-scaling, self-service provisioning of services like Elasticsearch and Cassandra, and reducing new datacenter provisioning from over 12 months to just 2 months.
- Debugging and managing complex Terraform configurations, especially across modules, can currently be challenging due to limitations in Terraform's data handling and interpolation features.
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
Trilha Containers How we figured out we had a SRE team at - Traefik: Dynamic container load balancer for containers
Kubernetes on AWS
This document provides an overview of a workshop on running Kubernetes on AWS. It outlines the prerequisites including installing Git, AWS CLI, kubectl, and cloning a GitHub repository. The workshop will cover basic Kubernetes concepts like pods, labels, replication controllers, deployments and services. It will demonstrate how to build a Kubernetes cluster on AWS using CloudFormation for infrastructure as code. Hands-on portions will include deploying containers, creating services, and observing the cluster architecture and networking. Additional topics are cluster add-ons like Kubernetes Dashboard and DNS, deploying applications, and cleaning up resources.
Kubernetes @ Nanit by Chen Fisher
This document summarizes Chen Fisher's presentation on Kubernetes at nanit. The key points are:
1) Nanit uses Kubernetes for service orchestration, taking advantage of its built-in service discovery, high availability features, and port management capabilities.
2) Kubernetes is preferred over Amazon ECS due to ECS lacking service discovery and only supporting basic health checks.
3) Nanit runs two Kubernetes clusters for staging and production, with over 100 pods across more than 20 instances in production.
4) Nanit uses Kubernetes for deploying microservices from development to production, with Jenkins doing automated testing and deployment via Slack notifications.
Kubernetes API - deep dive into the kube-apiserver
The document discusses the Kubernetes API server and its RESTful HTTP API. It describes the API endpoints for accessing different Kubernetes resources, how API groups and versions are organized, how API requests are routed and processed, how Kubernetes objects are converted between different versions, and how storage and code generation are used.
London HUG 12/4
This document provides an overview of Terraform, an open-source infrastructure as code tool. It discusses Terraform's goals of providing a unified view of infrastructure, composing multiple tiers of infrastructure from IaaS to PaaS to SaaS, and safely changing infrastructure over time with one workflow. Key features highlighted include being open source, using infrastructure as code, resource providers that interface with cloud APIs, and the plan and apply workflow. The document also covers topics like collaboration and version history in Terraform Enterprise, file examples, the plan and apply commands, resource providers, and new features in recent Terraform versions like destroy provisioners, remote backends, state locking, and state environments.
Intro to kubernetes
A basic introduction to Kubernetes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
Infrastructure as code with Terraform
The document discusses using Terraform to implement infrastructure as code. It describes how Terraform allows building multiple environments like development, test, staging and production in an automated and repeatable way. It also provides code examples to demonstrate how to build a VPC, security group and EC2 instance using Terraform modules to reuse infrastructure components and simplify configuration.
Meetup - Principles of the kube api and how to extend it
The document discusses principles of the Kubernetes API and how to extend it. It describes how the Kubernetes API is organized into resources, kinds, and API groups. It also covers techniques for extending the Kubernetes API like ThirdPartyResources and API aggregation. API aggregation allows for powerful custom APIs to be added without modifying Kubernetes itself. The document provides examples of querying and modifying API resources to demonstrate how the Kubernetes API works.
DockerCon EU 2015: Trading Bitcoin with Docker
Presented by Sebastien Goasguen, VP, Apache CloudStack and Mathieu Buffenoir, co-founder, SBEX
Bity is an internet money gateway built by Swiss Bitcoin Exchange ( SBEX ). To trade bitcoin the entire infrastructure of Bity is running in Docker containers. All the components of the infrastructure are using Docker, from the frontend applications and load balancer, the Django based backend, replicated Postgres database, Bitcoin daemon and remittance engine. All software goes through a CI pipeline that starts with Docker images being built on private repositories in Docker hub. Developers take also advantage of a docker-compose definition that allows them to run the entire infrastructure on a single laptop. Finally the production deployments happen thanks to the Ansible Docker module on a CloudStack based public cloud. Everything has been automated to ease re-deployment and operations. This presentation will go through every component and how Docker has enabled us to go production in 4 months.
ILM - Pipeline in the cloud
An overview of our experiments at Industrial Light and Magic to create a fully cloud based pipeline, based on Mesos, Docker and automated with Ansible.
Composing services with Kubernetes
1. Kubernetes services can be used to abstract external dependencies like databases and SaaS services, decoupling consumers from providers.
2. For databases, a service allows using a single-instance container database in test environments instead of a production cluster, without consumers needing different configurations.
3. For SaaS services, an ExternalName service type maps the service to the real URL in production but to a virtualization pod in tests, providing fake responses without consumers knowing.
KubeCon EU 2016: Templatized Application Configuration on OpenShift and Kuber...
Kubernetes gives developers a platform on which to run images and many configuration objects to control those images, but constructing a cohesive application made up of images and configuration objects is currently a challenge. Reconstructing or sharing that configuration can also be a challenge. This talk will cover the Template feature implemented in OpenShift to simplify the process of defining and repeatably deploying coordinated objects, discuss what is coming to Kubernetes with respect to this capability, and touch on several other existing projects that enable templatizing application definitions.
Sched Link: http://sched.co/6BVH
KubeCon EU 2016: Transforming the Government
This talk is documents the UK Home Office's cloud-native journey, changing how we did devops forever!
At the UK Home Office, we run Kubernetes in production. This talk is about how we got there, where we came from, where we are right now and where do we want to go from here. We will also cover what things worked out and which things didn't.
From on-boarding projects into Kubernetes to continous delivery, this talk will give you a good understanding of what lies ahead if you decided to take the road to schedule containers in production.
Sched Link: http://sched.co/68xS
Splunk Dynamic lookup
This document discusses static and dynamic lookups in Splunk. Static lookups retrieve fields from a CSV file, while dynamic lookups retrieve fields from external sources like websites, databases, or caches. It provides examples of writing scripts for dynamic lookups from databases and caches to enrich event data. Dynamic lookups first check a persistent cache, and if not present, query the external source and cache the results. This allows combining external data sources with caching for improved performance.
Heterogenous Persistence
This document provides an overview of heterogeneous persistence and different database management systems (DBMS). It discusses why a single DBMS is often not sufficient and describes different types of DBMS including relational databases, key-value stores, and columnar databases. For each type, it outlines good and bad use cases, examples, considerations, and pros and cons. The document aims to help readers understand the different flavors of DBMS and how to choose the right ones for their specific data and access needs.
More Related Content
What's hot
Scaling Jenkins with Kubernetes by Ami Mahloof
This document discusses scaling Jenkins with Kubernetes. Previously, Jenkins was run on single EC2 instances with issues like underutilization and port collisions. The new approach runs Jenkins on Kubernetes, allowing on-demand PODs for each build with complete isolation. A modified Kubernetes plugin was created to support multiple containers and persistent volumes per POD. Builds are queued and run on Kubernetes nodes, improving scalability. Demostrating this approach saves 70-90% over running Jenkins on always-on EC2 instances by using spot instances for Kubernetes nodes.
Kernel load-balancing for Docker containers using IPVS
IPVS is a Linux kernel module that provides high performance virtual server capabilities including load balancing, failover and dynamic configuration. GORB is an open source REST API and daemon that controls IPVS to provide load balancing for Docker containers. It supports TCP and HTTP health checks and integrates with Docker links. Using IPVS, GORB and Docker, high performance load balancing can be achieved without the need for expensive proprietary hardware load balancers.
Extend and build on Kubernetes
API basics, kinds, resource, apigroups, etcd storage, watches, discovery, ThirdPartyResources, custom apiservers, API aggregation
Enabling Microservices @Orbitz - DockerCon 2015
Slides from Orbitz's use case for microservices on docker & mesos at Velocity Santa Clara 2015 conference.
Details: http://sched.co/39S6
KubeCon EU 2016: Using Traffic Control to Test Apps in Kubernetes
Testing applications is important, as shown by the rise of continuous integration and automated testing. In this talk, I will focus on one area of testing that is difficult to automate: poor network connectivity. Developers usually work within reliable networking conditions so they might not notice issues that arise in other networking conditions. I will give examples of software that would benefit from test scenarios with varying connectivity. I will explain how traffic control on Linux can help to simulate various network connectivity. Finally, I will run a demo showing how an application running in Kubernetes behaves when changing network parameters.
Sched Link: http://sched.co/6Bb3
Beyond static configuration
Beyond static configuration management discusses how containerization and distributed configuration management are disrupting traditional system engineering. Key developments include specialized container-centric operating systems like CoreOS, orchestration tools like Docker, Mesos, and Kubernetes, as well as configuration stores like etcd, Consul, and Zookeeper that enable dynamic configuration of distributed systems. The talk argues this represents an exciting transition period for development and operations.
London Hug 19/5 - Terraform in Production
- Terraform allows infrastructure teams to more efficiently and agilely provision resources at scale across multiple production datacenters and regions.
- Key benefits include auto-scaling, self-service provisioning of services like Elasticsearch and Cassandra, and reducing new datacenter provisioning from over 12 months to just 2 months.
- Debugging and managing complex Terraform configurations, especially across modules, can currently be challenging due to limitations in Terraform's data handling and interpolation features.
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
Trilha Containers How we figured out we had a SRE team at - Traefik: Dynamic container load balancer for containers
Kubernetes on AWS
This document provides an overview of a workshop on running Kubernetes on AWS. It outlines the prerequisites including installing Git, AWS CLI, kubectl, and cloning a GitHub repository. The workshop will cover basic Kubernetes concepts like pods, labels, replication controllers, deployments and services. It will demonstrate how to build a Kubernetes cluster on AWS using CloudFormation for infrastructure as code. Hands-on portions will include deploying containers, creating services, and observing the cluster architecture and networking. Additional topics are cluster add-ons like Kubernetes Dashboard and DNS, deploying applications, and cleaning up resources.
Kubernetes @ Nanit by Chen Fisher
This document summarizes Chen Fisher's presentation on Kubernetes at nanit. The key points are:
1) Nanit uses Kubernetes for service orchestration, taking advantage of its built-in service discovery, high availability features, and port management capabilities.
2) Kubernetes is preferred over Amazon ECS due to ECS lacking service discovery and only supporting basic health checks.
3) Nanit runs two Kubernetes clusters for staging and production, with over 100 pods across more than 20 instances in production.
4) Nanit uses Kubernetes for deploying microservices from development to production, with Jenkins doing automated testing and deployment via Slack notifications.
Kubernetes API - deep dive into the kube-apiserver
The document discusses the Kubernetes API server and its RESTful HTTP API. It describes the API endpoints for accessing different Kubernetes resources, how API groups and versions are organized, how API requests are routed and processed, how Kubernetes objects are converted between different versions, and how storage and code generation are used.
London HUG 12/4
This document provides an overview of Terraform, an open-source infrastructure as code tool. It discusses Terraform's goals of providing a unified view of infrastructure, composing multiple tiers of infrastructure from IaaS to PaaS to SaaS, and safely changing infrastructure over time with one workflow. Key features highlighted include being open source, using infrastructure as code, resource providers that interface with cloud APIs, and the plan and apply workflow. The document also covers topics like collaboration and version history in Terraform Enterprise, file examples, the plan and apply commands, resource providers, and new features in recent Terraform versions like destroy provisioners, remote backends, state locking, and state environments.
Intro to kubernetes
A basic introduction to Kubernetes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
Infrastructure as code with Terraform
The document discusses using Terraform to implement infrastructure as code. It describes how Terraform allows building multiple environments like development, test, staging and production in an automated and repeatable way. It also provides code examples to demonstrate how to build a VPC, security group and EC2 instance using Terraform modules to reuse infrastructure components and simplify configuration.
Meetup - Principles of the kube api and how to extend it
The document discusses principles of the Kubernetes API and how to extend it. It describes how the Kubernetes API is organized into resources, kinds, and API groups. It also covers techniques for extending the Kubernetes API like ThirdPartyResources and API aggregation. API aggregation allows for powerful custom APIs to be added without modifying Kubernetes itself. The document provides examples of querying and modifying API resources to demonstrate how the Kubernetes API works.
DockerCon EU 2015: Trading Bitcoin with Docker
Presented by Sebastien Goasguen, VP, Apache CloudStack and Mathieu Buffenoir, co-founder, SBEX
Bity is an internet money gateway built by Swiss Bitcoin Exchange ( SBEX ). To trade bitcoin the entire infrastructure of Bity is running in Docker containers. All the components of the infrastructure are using Docker, from the frontend applications and load balancer, the Django based backend, replicated Postgres database, Bitcoin daemon and remittance engine. All software goes through a CI pipeline that starts with Docker images being built on private repositories in Docker hub. Developers take also advantage of a docker-compose definition that allows them to run the entire infrastructure on a single laptop. Finally the production deployments happen thanks to the Ansible Docker module on a CloudStack based public cloud. Everything has been automated to ease re-deployment and operations. This presentation will go through every component and how Docker has enabled us to go production in 4 months.
ILM - Pipeline in the cloud
An overview of our experiments at Industrial Light and Magic to create a fully cloud based pipeline, based on Mesos, Docker and automated with Ansible.
Composing services with Kubernetes
1. Kubernetes services can be used to abstract external dependencies like databases and SaaS services, decoupling consumers from providers.
2. For databases, a service allows using a single-instance container database in test environments instead of a production cluster, without consumers needing different configurations.
3. For SaaS services, an ExternalName service type maps the service to the real URL in production but to a virtualization pod in tests, providing fake responses without consumers knowing.
KubeCon EU 2016: Templatized Application Configuration on OpenShift and Kuber...
Kubernetes gives developers a platform on which to run images and many configuration objects to control those images, but constructing a cohesive application made up of images and configuration objects is currently a challenge. Reconstructing or sharing that configuration can also be a challenge. This talk will cover the Template feature implemented in OpenShift to simplify the process of defining and repeatably deploying coordinated objects, discuss what is coming to Kubernetes with respect to this capability, and touch on several other existing projects that enable templatizing application definitions.
Sched Link: http://sched.co/6BVH
KubeCon EU 2016: Transforming the Government
This talk is documents the UK Home Office's cloud-native journey, changing how we did devops forever!
At the UK Home Office, we run Kubernetes in production. This talk is about how we got there, where we came from, where we are right now and where do we want to go from here. We will also cover what things worked out and which things didn't.
From on-boarding projects into Kubernetes to continous delivery, this talk will give you a good understanding of what lies ahead if you decided to take the road to schedule containers in production.
Sched Link: http://sched.co/68xS
What's hot (20)
Kernel load-balancing for Docker containers using IPVS
Kernel load-balancing for Docker containers using IPVS
KubeCon EU 2016: Using Traffic Control to Test Apps in Kubernetes
KubeCon EU 2016: Using Traffic Control to Test Apps in Kubernetes
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
TDC2017 | São Paulo - Trilha Containers How we figured out we had a SRE team ...
Kubernetes API - deep dive into the kube-apiserver
Kubernetes API - deep dive into the kube-apiserver
Meetup - Principles of the kube api and how to extend it
Meetup - Principles of the kube api and how to extend it
KubeCon EU 2016: Templatized Application Configuration on OpenShift and Kuber...
KubeCon EU 2016: Templatized Application Configuration on OpenShift and Kuber...
Viewers also liked
Splunk Dynamic lookup
This document discusses static and dynamic lookups in Splunk. Static lookups retrieve fields from a CSV file, while dynamic lookups retrieve fields from external sources like websites, databases, or caches. It provides examples of writing scripts for dynamic lookups from databases and caches to enrich event data. Dynamic lookups first check a persistent cache, and if not present, query the external source and cache the results. This allows combining external data sources with caching for improved performance.
Heterogenous Persistence
This document provides an overview of heterogeneous persistence and different database management systems (DBMS). It discusses why a single DBMS is often not sufficient and describes different types of DBMS including relational databases, key-value stores, and columnar databases. For each type, it outlines good and bad use cases, examples, considerations, and pros and cons. The document aims to help readers understand the different flavors of DBMS and how to choose the right ones for their specific data and access needs.
Accelerated Leadership
The document outlines a pilot leadership program called the Accelerated Leadership Class (ALC) at Peterson Air Force Base. The 7-session program will provide interactive leadership training to 12 junior airmen using experiential learning activities and models. Sessions will focus on developing leadership skills, emotional intelligence, giving and receiving feedback, and completing a group leadership project to benefit the base. The tentative schedule explores different timing options over 6 days, 3 sessions per week for 2 weeks, or twice monthly for 3 months.
Opensource approach to design and deployment of Microservices based VNF
Microservice is gaining increased adoption in the Telco NFV world. It is key to understand the design and deployment methodologies involved in developing Microservice based VNF. This talk provides an opensource practitioner approach to building and deploying a Microservice based VNF and includes the following: - Design patterns, workflow models - Design models for VNF placement, capacity management, scale-in/out and resiliency - Deployment considerations that includes handing of scale and fault tolerant VNF using well known Opensource tools.
About the presenter: Prem Sankar works for Ericsson Opensource Ecosystem team and part of the Opendaylight and OPNFV team in Ericsson. Prem evangelizes SDN and Cloud and has given many sessions and conducted workshops around SDN and ODL. Prem is PTL of ODL COE project and currently driving the Kuberenetes and ODL Integration in Opendaylight community. Prem is a frequent speaker at opensource summits and has presented in Opendaylight, OPNFV and Open networking summits.
Introduction to Volansys Technologies
Volansys is Next Generation Digital Transformation, Product Realization, and Data Sense company, which acts as a ‘One stop solution enabler’ to realize your ideas into life. We deliver consistent customer experience, business efficiency, and actionable insights by leveraging integrated set of leading technologies like Big Data Analytics, Mobility, Cloud Computing, Unified Communication, Sensorization, Hyperscaling, Business Intelligence, Perceptual Computing etc.
Microservices
This document discusses microservices and the process of setting up a new microservice. It covers topics such as defining the service scope, getting approvals, source control and packaging, running environments, logging and monitoring, and preparing the service for production use. The key aspects of setting up a new microservice include buy-in from management, external design reviews, source control and deployment automation, provisioning compute and storage resources, and integrating the service with monitoring and on-call systems.
Docker for PHP Developers - Madison PHP 2017
Docker is quickly becoming an invaluable development and deployment tool for many organizations. Come and spend the day learning about what Docker is, how to use it, how to integrate it into your workflow, and build an environment that works for you and the rest of your team. This hands-on tutorial will give you the kick-start needed to start using Docker effectively.
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
Prometheus is a next-generation monitoring system with a time series database at it's core. Once you have a time series database, what do you do with it though? This talk will look at getting data in, and more importantly how to use the data you collect productively.
Contact us at prometheus@robustperception.io
Tubular Labs - Using Elastic to Search Over 2.5B Videos
This document discusses optimizing Elasticsearch to search over 2.5 billion videos. It describes 4 steps to improve the user experience and 4 patterns to simplify architecture and reduce costs. The data size is 2.5 billion documents totaling 4 terabytes, with 200 million daily updates. Hardware includes 52 nodes with 832 cores and 16 terabytes of SSD storage. Recommendations include using bulk indexing, increasing refresh intervals, monitoring rejected bulk operations, understanding hardware utilization, benchmarking different numbers of shards, splitting queries from aggregations, reducing data cardinality, and implementing denormalization and partial updates to improve performance and reduce costs.
Persistence in the cloud with bosh
BOSH is an open source tool for release engineering, deployment, lifecycle management, and monitoring of distributed systems. It handles packaging, infrastructure provisioning across multiple clouds, application deployment with features like canary releases and rolling updates, configuration management of services rather than individual servers, and monitoring of processes, nodes, and the overall service. The presentation demonstrated deploying an ELK stack on AWS using BOSH and showed how it can scale the deployment vertically, horizontally, add more disk space, replace nodes, and add monitoring.
Secure Yourself, Practice what we preach - BSides Austin 2015
This document discusses various methods for securing personal devices and systems. It provides tips for using password managers, multi-factor authentication, browser security settings, encrypted backups, and restricting administrative privileges to help protect against malware and identity theft. The document also addresses security best practices for social media, wireless networks, and monitoring children's cell phone usage.
Turnkey Riak KV Cluster
Presented at the Chicago Riak Meetup, 20 Oct 2015. Slides walk through a demo of automating the building of a full stack Riak KV cluster from scratch, and then populating it with 250,000,000 time series data points.
Selma_CV1
This CV summarizes the qualifications and experience of Selma Ibrahim Mohammedelkhier. She has over 14 years of experience in monitoring and evaluation, most recently working as an M&E specialist for the Budget Capacity Strengthening Project in Sudan. She holds an MBA in Project Management and professional diplomas in translation and veterinary sciences. Her skills include planning, implementation, coordination, communication, M&E, and computer applications. She is seeking a management, M&E, or coordination position where she can apply her experience and skills.
Failing at Scale - PNWPHP 2016
The document discusses challenges with distributed applications including maintaining consistent configurations, updating code everywhere, and locating errors. It recommends setting up a centralized syslog server to collect logs from all servers and applications. This allows errors to be tracked even as servers and containers change. It also suggests automating build, deployment and configuration processes to reduce human errors and ensure consistency across environments. Tools recommended for this include Ansible, Puppet, Vagrant and tracking code changes.
Catálogo Elk Sport 2016 2017
Catálogo de material deportivo y equipamiento para clubes, centros educativos, centros deportivos, gimnasios y particulares.
B2B Digital Transformation - Case Study
The Complete Transformation from Offline to Online Magento Implementation of a B2B Platform – a Case Study. This case study shows how we managed to reach 100.000.000 EUR of online revenue, within only 3 quarters last year.
AWS re:Invent 2014 | (ARC202) Real-World Real-Time Analytics
Working with big volumes of data is a complicated task, but it's even harder if you have to do everything in real time and try to figure it all out yourself. This session will use practical examples to discuss architectural best practices and lessons learned when solving real-time social media analytics, sentiment analysis, and data visualization decision-making problems with AWS. Learn how you can leverage AWS services like Amazon RDS, AWS CloudFormation, Auto Scaling, Amazon S3, Amazon Glacier, and Amazon Elastic MapReduce to perform highly performant, reliable, real-time big data analytics while saving time, effort, and money. Gain insight from two years of real-time analytics successes and failures so you don't have to go down this path on your own.
Using a Canary Microservice to Validate the Software Delivery Pipeline
This document discusses using a canary microservice to validate the software delivery pipeline at DigitalGlobe. It describes how DigitalGlobe uses a canary microservice that is deployed through the entire pipeline to automatically test that the pipeline is functioning properly. The document also provides statistics on the availability and success of DigitalGlobe's continuous delivery pipeline and canary deployments that help them monitor and improve the pipeline over time.
Acts 6:1-7 ~ Organic Growth of the Early Church (pt. 1)
This document discusses several topics related to the growth and development of the early Christian church as described in Acts, including:
- The early church providing an exemplary model to follow due to the absence of evil and swift exclusion of it.
- Apostolic authority being rooted in the twelve apostles who were personally connected to Jesus.
- The church growing numerically, socially, and geographically through various developments.
- Debate around the identity and meaning of the terms "Hebrews" and "Hellenists" in the early church.
Viewers also liked (20)
Opensource approach to design and deployment of Microservices based VNF
Opensource approach to design and deployment of Microservices based VNF
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
Your data is in Prometheus, now what? (CurrencyFair Engineering Meetup, 2016)
Tubular Labs - Using Elastic to Search Over 2.5B Videos
Tubular Labs - Using Elastic to Search Over 2.5B Videos
Secure Yourself, Practice what we preach - BSides Austin 2015
Secure Yourself, Practice what we preach - BSides Austin 2015
AWS re:Invent 2014 | (ARC202) Real-World Real-Time Analytics
AWS re:Invent 2014 | (ARC202) Real-World Real-Time Analytics
Using a Canary Microservice to Validate the Software Delivery Pipeline
Using a Canary Microservice to Validate the Software Delivery Pipeline
Acts 6:1-7 ~ Organic Growth of the Early Church (pt. 1)
Acts 6:1-7 ~ Organic Growth of the Early Church (pt. 1)
Similar to Writing New Relic Plugins: NSQ
Instrumenting and Scaling Databases with Envoy
Every request to a database at Lyft is proxied by Envoy, providing complete visibility into the L3/L4 aspects of database interactions. This allows engineers to easily visualize changes to a database's load profile and pinpoint the root cause if necessary. Lyft has also open-sourced codecs for MongoDB, DynamoDB, and Redis. Protocol codecs in combination with custom filters yield benefits ranging from operation-level observability to horizontal scalability via sharding. Using Envoy for this purpose means that enhancements are implemented once and usable across a polyglot stack. The talk demonstrates Envoy's utility beyond traditional RPC service interactions in the network.
Log everything! @DC13
Deck36 is a small team of engineers who specialize in designing, implementing, and operating complex web systems. They discuss their approach to logging everything through a data pipeline that ingests data from producers, transports it via RabbitMQ, stores it in Hadoop HDFS and Amazon S3, runs analytics with Hadoop MapReduce and Amazon EMR, and performs real-time stream processing with Twitter Storm. They also live demo their JavaScript data collector client and a PHP/Storm example that processes click stream data.
StrongLoop Overview
OpenSource API Server based on Node.js API framework built on supported Node.js platform with Tooling and DevOps. Use cases are Omni-channel API Server, Mobile Backend as a Service (mBaaS) or Next Generation Enterprise Service Bus. Key functionality include built in enterprise connectors, ORM, Offline Sync, Mobile and JS SDKs, Isomorphic JavaScript and Graphical API creation tool.
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Michael Sun presented on CBS Interactive's use of Hadoop for web analytics processing. Some key points:
- CBS Interactive processes over 1 billion web logs daily from hundreds of websites on a Hadoop cluster with over 1PB of storage.
- They developed an ETL framework called Lumberjack in Python for extracting, transforming, and loading data from web logs into Hadoop and databases.
- Lumberjack uses streaming, filters, and schemas to parse, clean, lookup dimensions, and sessionize web logs before loading into a data warehouse for reporting and analytics.
- Migrating to Hadoop provided significant benefits including reduced processing time, fault tolerance, scalability, and cost effectiveness compared to their
Server Monitoring (Scaling while bootstrapped)
This document discusses server monitoring strategies for scaling applications. It recommends:
1) Using a load balancer and multiple scalable web and database servers to handle increased traffic loads from hundreds of thousands of users on multi-platform apps.
2) Monitoring key server metrics like CPU usage, bandwidth usage, and application errors to improve stability and catch issues.
3) Tracking logs, client errors, and infrastructure bottlenecks to debug problems and prevent outages like those that led some companies to be "acquired" due to technical failures.
Pivotal Real Time Data Stream Analytics
This document discusses using Pivotal's Big Data Suite to build a real-time analytics solution for processing taxi trip data streams. It presents an architecture that uses Spring XD for data ingestion, Spark Streaming for in-memory analytics on 10-second windows, Gemfire for fast data retrieval, and Pivotal HD for long-term storage. The solution demonstrates filtering inconsistent data, finding top traffic areas, and available taxis in real-time. The document highlights how the Big Data Suite provides a complete toolset for data-driven enterprises through its optimized Hadoop distribution, in-memory processing, stream processing, and low-latency data stores.
UnConference for Georgia Southern Computer Science March 31, 2015
I presented to the Georgia Southern Computer Science ACM group. Rather than one topic for 90 minutes, I decided to do an UnConference. I presented them a list of 8-9 topics, let them vote on what to talk about, then repeated.
Each presentation was ~8 minutes, (Except Career) and was by no means an attempt to explain the full concept or technology. Only to wake up their interest.
Enabling Microservices Frameworks to Solve Business Problems
Opening keynote at Mesoscon 2015 with announcements on creating an ecosystem for developing solutions to business problems leveraging Mesos, Mantl.io, Mesosphere Infinity, ZoomData, and Project Calico to create Fog nodes for IoE use cases.
Javantura v3 - Real-time BigData ingestion and querying of aggregated data – ...
Javantura v3 - Real-time BigData ingestion and querying of aggregated data – ...HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association
This document discusses Infobip's journey towards enabling real-time querying of aggregated data. Initially, Infobip had a monolithic architecture with a single database that became a bottleneck. They introduced multiple databases and microservices but querying spanned databases and results had to be joined. A data warehouse (GREEN) provided reporting but was not real-time. To enable real-time queries, Infobip implemented a lambda architecture using Kafka as the real-time data pipeline and Druid for real-time querying and aggregations, achieving sub-second responses and less than 2 seconds of data delay. This allows real-time insights from ingested messaging data while GREEN remains the batch/serving layer.Treasure Data and OSS
- Treasure Data is a cloud data service that provides data acquisition, storage, and analysis capabilities.
- It collects data from various sources using Fluentd and Embulk and stores it in its own columnar database called Plazma DB.
- It offers various computing frameworks like Hive, Pig, and Presto for analytics and visualization with tools like Tableau.
- Presto is an interactive SQL query engine that can query data in HDFS, Hive, Cassandra and other data stores.
I Love APIs 2015: Building Predictive Apps with Lamda and MicroServices
I Love APIs 2015
Machine learning, big data, and API technologies have drastically reduced the complexity of building predictive apps. But all these advances also mean that these apps require a new approach to system architecture. This talk discusses the lamda architecture and microservices, and best practices on decomposing your app into batch, near-realtime, an real-time services. Learn how Apigee uses both new architectures to implement predictive apps using Hadoop, Node.js, Cassandra, and ElasticSearch.
Seattle spark-meetup-032317
The document discusses the design and implementation of Spark Streaming connectors for real-time data sources like Azure Event Hubs. It covers key aspects like connecting Event Hubs to Spark Streaming, designing the connector to minimize resource usage, ensuring fault tolerance through checkpointing and recovery, and managing message offsets and processing rates in a distributed manner. The connector design addresses challenges like long-running receivers, extra resource requirements, and data loss during failures. Lessons from the initial receiver-based approach informed the design of a more efficient solution.
Data Analytics Service Company and Its Ruby Usage
This document summarizes Satoshi Tagomori's presentation on Treasure Data, a data analytics service company. It discusses Treasure Data's use of Ruby for various components of its platform including its logging (Fluentd), ETL (Embulk), scheduling (PerfectSched), and storage (PlazmaDB) technologies. The document also provides an overview of Treasure Data's architecture including how it collects, stores, processes, and visualizes customer data using open source tools integrated with services like Hadoop and Presto.
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
Thousands of services work in concert to deliver millions of hours of video streams to Netflix customers every day. These applications vary in size, function, and technology, but they all make use of the Netflix network to communicate. Understanding the interactions between these services is a daunting challenge both because of the sheer volume of traffic and the dynamic nature of deployments. In this talk, we’ll first discuss why Netflix chose Amazon Kinesis Streams over other streaming data solutions like Kafka to address these challenges at scale. We’ll then dive deep into how Netflix uses Amazon Kinesis Streams to enrich network traffic logs and identify usage patterns in real time. Lastly, we will cover how Netflix uses this system to build comprehensive dependency maps, increase network efficiency, and improve failure resiliency. From this talk, you’ll take away techniques and processes that you can apply to your large-scale networks and derive real-time, actionable insights.
Data science for infrastructure dev week 2022
The document discusses using data science and automation for infrastructure monitoring. It introduces Pixie, a tool that allows users to collect raw data, transform it into signals, and then take actions based on those signals. Two examples are provided: 1) detecting SQL injections from application logs and sending Slack alerts, and 2) automatically scaling a deployment based on HTTP request throughput metrics. Pixie uses an embedded domain-specific language called PxL to define logical data workflows and queries.
Choisir entre une API RPC, SOAP, REST, GraphQL?
Et si le problème était ai...
Choisir entre une API RPC, SOAP, REST, GraphQL?
Et si le problème était ai...François-Guillaume Ribreau
Posons-nous et profitons de ce talk pour prendre un peu de hauteur sur l’état de l’industrie tech autour de la création d’API de persistence (CRUD).
D’où venons-nous, ou allons-nous ? Pourquoi le choix entre RPC, SOAP, REST et GraphQL n’est peut-être qu’un sujet de surface qui cache un problème bien plus profond…
Youtube: https://www.youtube.com/watch?v=IskE3m3VjRYSpring Boot & Spring Cloud Apps on Pivotal Application Service - Daniel Lavoie
SpringOne Tour 2018 by Pivotal
Spring Boot & Spring Cloud Apps on Pivotal Application Service - Daniel Lavoie
Enterprise Data Lakes
How to create an enterprise data lake for enterprise-wide information storage and sharing? The data lake concept, architecture principles, support for data science and some use case review.
Building Super Fast Cloud-Native Data Platforms - Yaron Haviv, KubeCon 2017 EU
Deep dive into iguazio high-performance data platform architecture, using Kubernetes and Cloud-Native for elasticity and CI/CD, along with with extreme performance tricks
YouTube link: https://youtu.be/ujuWt6mvIig
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Learning Objectives:
- Understand how to build a serverless big data solution quickly and easily
- Learn how to discover and prepare all your data for analytics
- Learn how to query and visualize analytics on all your data to create actionable insights
Similar to Writing New Relic Plugins: NSQ (20)
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
Hadoop World 2011: Building Web Analytics Processing on Hadoop at CBS Interac...
UnConference for Georgia Southern Computer Science March 31, 2015
UnConference for Georgia Southern Computer Science March 31, 2015
Enabling Microservices Frameworks to Solve Business Problems
Enabling Microservices Frameworks to Solve Business Problems
Javantura v3 - Real-time BigData ingestion and querying of aggregated data – ...
Javantura v3 - Real-time BigData ingestion and querying of aggregated data – ...
I Love APIs 2015: Building Predictive Apps with Lamda and MicroServices
I Love APIs 2015: Building Predictive Apps with Lamda and MicroServices
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis
Choisir entre une API RPC, SOAP, REST, GraphQL?
Et si le problème était ai...
Choisir entre une API RPC, SOAP, REST, GraphQL?
Et si le problème était ai...
Spring Boot & Spring Cloud Apps on Pivotal Application Service - Daniel Lavoie
Spring Boot & Spring Cloud Apps on Pivotal Application Service - Daniel Lavoie
Building Super Fast Cloud-Native Data Platforms - Yaron Haviv, KubeCon 2017 EU
Building Super Fast Cloud-Native Data Platforms - Yaron Haviv, KubeCon 2017 EU
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Serverless Analytics with Amazon Redshift Spectrum, AWS Glue, and Amazon Quic...
Recently uploaded
Climate Impact of Software Testing at Nordic Testing Days
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...Edge AI and Vision Alliance
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.UiPath Test Automation using UiPath Test Suite series, part 6
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Best 20 SEO Techniques To Improve Website Visibility In SERP
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Artificial Intelligence for XMLDevelopment
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Recently uploaded (20)
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Writing New Relic Plugins: NSQ
- 2. Data in / out New Relic User Group - Toronto
- 3. $ whoami Lucas Fontes Chief Architect Uken Games @lxfontes
- 4. Rails shop HTML5 and Unity3d Lots of servers Lots of data
- 6. Operations Time series Graphite / InfluxDB / Cacti Server side Error Reporting Elasticsearch + Kibana Client side Error Reporting custom ( javascript / native ) Server Monitoring / Alerting Nagios / Sensu / collectd Columnar stores Redshift / CitusDB Log crunching Spark / Hadoop ecosystem Visualization R with Shiny / pydata Business Intelligence
- 9. Coverage so far Applications Servers Mobile (Native) Browser (Javascript)
- 10. Data in
- 12. Roll your own Rubygem: newrelic_plugin Handles Configuration, API communication, aggregation Setup one or more metrics (setup_metrics) Collect data at fixed interval (pool_cycle)
- 13. Use Case: NSQ
- 17. Data out