Communication tool & Environment for Remote WorkerShotaro Sakamaki
Shotaro Sakamaki is a front-end engineer at PixelGrid.Inc, a company that develops JavaScript applications. He discusses the communication tools and development environment used by PixelGrid's remote workers. Key tools mentioned include Slack for chat, esa.io for documentation sharing, GitHub for source control, and ZenHub as a GitHub extension. Costs for these paid services range from $3.99 to $6.67 per user per month. While costs may seem high, the speaker argues they replace expenses from maintaining multiple free tools and reduce invisible maintenance costs.
The document discusses how a startup has adopted new technologies and scaled rapidly over 10 months. It overhauled its web platform and launched a new product using React Native by focusing on engineering culture, adopting modern tools, and understanding requirements from a business perspective. Continuous delivery of new code, processing over 120 million numbers daily, and rewriting the platform with their own UI library helped the startup scale quickly while improving quality.
This document provides an introduction to Node.js, including:
- Node.js allows JavaScript to be run on the server-side using the V8 JavaScript engine.
- It uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
- Node.js supports module-driven development through NPM, which has over 250,000 packages available.
Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine. It allows JavaScript to run on the server-side and is used for building scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, as well as packaged modules to help build everything from web servers to utilities. Popular Node.js frameworks include Express and Restler for building web servers and APIs.
Presentation of a session about how we use AWS Neptune graph database and the road that we took from 0 to production as it was given at GeekTime Code conference by Ohad Israeli Tech Engineering Architect @ Natural Intelligence
Communication tool & Environment for Remote WorkerShotaro Sakamaki
Shotaro Sakamaki is a front-end engineer at PixelGrid.Inc, a company that develops JavaScript applications. He discusses the communication tools and development environment used by PixelGrid's remote workers. Key tools mentioned include Slack for chat, esa.io for documentation sharing, GitHub for source control, and ZenHub as a GitHub extension. Costs for these paid services range from $3.99 to $6.67 per user per month. While costs may seem high, the speaker argues they replace expenses from maintaining multiple free tools and reduce invisible maintenance costs.
The document discusses how a startup has adopted new technologies and scaled rapidly over 10 months. It overhauled its web platform and launched a new product using React Native by focusing on engineering culture, adopting modern tools, and understanding requirements from a business perspective. Continuous delivery of new code, processing over 120 million numbers daily, and rewriting the platform with their own UI library helped the startup scale quickly while improving quality.
This document provides an introduction to Node.js, including:
- Node.js allows JavaScript to be run on the server-side using the V8 JavaScript engine.
- It uses an event-driven, non-blocking I/O model that makes it lightweight and efficient.
- Node.js supports module-driven development through NPM, which has over 250,000 packages available.
Node.js is a JavaScript runtime built on Chrome's V8 JavaScript engine. It allows JavaScript to run on the server-side and is used for building scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, as well as packaged modules to help build everything from web servers to utilities. Popular Node.js frameworks include Express and Restler for building web servers and APIs.
Presentation of a session about how we use AWS Neptune graph database and the road that we took from 0 to production as it was given at GeekTime Code conference by Ohad Israeli Tech Engineering Architect @ Natural Intelligence
This document provides instructions for using Ansible to manage Google Compute Engine (GCE) instances. It outlines the steps to set up GCE instances, generate service account keys, configure Ansible inventory files to connect to GCE, and run playbooks to manage hosts on GCE. Additional resources are also referenced for more details on integrating Ansible with GCE, managing SSH keys, service accounts, and the demo GitHub repository.
This document provides an introduction to Node.js for beginners. It defines Node.js as a server-side JavaScript platform that uses non-blocking I/O and an event-driven architecture. It discusses key Node.js concepts like callbacks, event loops, streams, and asynchronous programming. It also highlights how Node.js is well suited for building real-time applications and APIs due to its asynchronous and modular nature.
This document discusses the benefits of implementing infrastructure, pipelines, images, servers, orchestration, and chat operations as code. Key benefits include being reproducible, saving time, avoiding human error, and not becoming outdated. It outlines tools like Terraform, Jenkins, Packer, Kubernetes, Docker, configuration management, and Hubot. Code samples and demos are provided to define infrastructure with Terraform, transform jobs into pipelines with Jenkins, build images with Packer, define containers in Kubernetes, build servers with Docker, and interact with services through Hubot chat. A link is provided to access additional sample code.
This document summarizes a presentation about migrating an existing ad data loading (ADLA) batch system to Amazon Batch Service. It outlines problems with the old single-server system like slowdowns from multiple batches running at once. The solution section then explains how Amazon Batch Service provides optimized compute resources and workload distribution. It details how the batches were refactored as Docker images and job definitions to run on Amazon Batch's compute environments and job queues. The last section previews a demo of the new system and lists pros like scalability and cons like increased management overhead.
Run C++ as serverless with GCP Cloud FunctionsRuncy Oommen
Runcy Oommen discusses using Google Cloud Functions with C++. Cloud Functions allows code to be written in Node.js or Python and executed in corresponding runtimes. It can access GCP services and be triggered by events from HTTP, Cloud Storage, Cloud Pub/Sub, and Firebase. The document walks through building a C++ addon for Cloud Functions using V8, binding.gyp, and Node.js to invoke the native code. It describes uploading the files, testing the function, and depicting the overall flow.
[HashiTalk Korea] OCP with Super Tengen Toppahyeongchae lee
The document discusses using Orchestrator, Consul, and ProxySQL for database orchestration. It provides an overview of how WiX Engineering uses Orchestrator to orchestrate ProxySQL with Open Containers Platform (OCP). It also discusses Github's use of Orchestrator for MySQL replication topology management and incident analysis. HashiCorp's Consul is presented as an alternative for service discovery and integration with tools like Orchestrator and ProxySQL. The document demonstrates how ProxySQL can be used with Consul and monitored using PMM. It concludes with a summary of using this "parasite architecture" approach with additional tools like Envoy, Gloo, Vault, and dnsmasq.
What is rest. Why is it part of the Rails way?. Railsconf 2014. Javier Ramirezjavier ramirez
When David Heinemeier Hansson started talking about REST in 2006, little could we suspect it'd become such a central part of Rails (and of web development in general). Back then a web service meant something you coded using XML and SOAP. Those were dark times. REST changed it all.
In this talk, I'll explain REST for beginners, I'll talk about why it is such an important architecture, and I'll show all the nice things Rails offers you for building a RESTful application or a RESTful API.
API Analytics with Redis and Bigquery. NoSQLmatters Cologne '14 edition. Javi...javier ramirez
At teowaki we have a system for API usage analytics, with Redis as a fast intermediate store and bigquery as a big data backend. As a result, we can launch aggregated queries on our traffic/usage data in just a few seconds and we can try and find for usage patterns that wouldn’t be obvious otherwise.In this session I will talk about how we entered the Big Data world, which alternatives we evaluated, and how we are using Redis and Bigquery to solve our problem.
At teowaki we have a system for API usage analytics, with Redis as a fast intermediate store and bigquery as a big data backend. As a result, we can launch aggregated queries on our traffic/usage data in just a few seconds and we can try and find for usage patterns that wouldn’t be obvious otherwise.
In this session I will talk about how we entered the Big Data world, which alternatives we evaluated, and how we are using Redis and Bigquery to solve our problem.
usable rest apis, by Javier Ramirez from teowaki (Apidays Mediterranea)javier ramirez
This summary provides the key points from the document in 3 sentences:
The document discusses various aspects of designing usable and effective REST APIs, highlighting the importance of usability principles and emphasizing that API usability is just as important as web usability. It also provides examples of different HTTP status codes and considerations for API design such as supporting different formats, enabling caching, implementing hypermedia and links, handling pagination and versioning, and ensuring the API is self-documenting. The overall message is that API designers should thoughtfully consider usability and the user experience when building RESTful interfaces.
Basics of the Highly Available Distributed Databases - teowaki - javier ramir...javier ramirez
Basic concepts of distributed systems, such as consensus, gossip and infection protocols, vector clocks, sharding storage, so you can create highly available distributed systems
Rapidly Building and Deploying Scalable Web ArchitecturesKeith Fitzgerald
The document provides an overview of how Keith Fitzgerald and his team of 4 developers were able to rapidly build and deploy the scalable web product TicketLeap using various cloud technologies. It discusses how they used Git for source control and bug tracking, Continuous Integration for testing, Trello for backlog management, Heroku for hosting, PostgreSQL/MongoDB for persistence, S3 and CloudFront for static assets, and common frameworks like Rails, Django, and Node. It emphasizes that architectures can scale rather than specific technologies, and that launching a product that can scale from the start allows a team to iterate quickly.
Peter Hoddie's keynote for IEEE at CES 2016. He explores upcoming trends for developers in the IoT space, scriptable IoT leading us to the right standards, and JavaScript for the IoT.
This document summarizes the Yatoto social crowdfunding platform. It provides details on its business model, which incorporates elements from lottery, Facebook, eBay, charities and Kickstarter. It discusses user and financial metrics from 2013-2016. It also describes the technical architecture including the Ruby on Rails backend, use of Docker for deployment, monitoring with Zabbix and error tracking with Errbit. Development practices like pair programming, code reviews and CI/CD are outlined.
What happens when you start transitioning from a monolithic PHP app to Go services running on AWS Lambda? Good things! I'd like to share the problems encountered, decisions made and lessons learned along the way.
Migrating to an Agile Architecture, Will Demaine, Engineer, Fat LlamaUXDXConf
Will Demaine, Engineer, Fat Llama Setup decisions: Planning your Agile architecture (Cloud migration path, platform choice, microservices/conainer architecture) ... Before you know everything about your product, how are you supposed to set it up
Fast, concurrent ruby web applications with EventMachine and EM::SynchronyKyle Drake
This document summarizes Kyle Drake's presentation on using EventMachine (EM) and EM::Synchrony to build fast, concurrent Ruby web applications without blocking I/O or callback hell. Key points include:
- EM implements the reactor pattern to handle blocking I/O without threads by using callbacks and kernel threads.
- EM::Synchrony avoids callback nesting by wrapping callbacks in fibers, allowing synchronous-looking code.
- Sinatra can be made concurrent by running each request in its own EM::Synchrony fiber with little code change.
- Ruby has strong options for building high-performance concurrent apps while maintaining productivity advantages over Node.js.
This document provides instructions for using Ansible to manage Google Compute Engine (GCE) instances. It outlines the steps to set up GCE instances, generate service account keys, configure Ansible inventory files to connect to GCE, and run playbooks to manage hosts on GCE. Additional resources are also referenced for more details on integrating Ansible with GCE, managing SSH keys, service accounts, and the demo GitHub repository.
This document provides an introduction to Node.js for beginners. It defines Node.js as a server-side JavaScript platform that uses non-blocking I/O and an event-driven architecture. It discusses key Node.js concepts like callbacks, event loops, streams, and asynchronous programming. It also highlights how Node.js is well suited for building real-time applications and APIs due to its asynchronous and modular nature.
This document discusses the benefits of implementing infrastructure, pipelines, images, servers, orchestration, and chat operations as code. Key benefits include being reproducible, saving time, avoiding human error, and not becoming outdated. It outlines tools like Terraform, Jenkins, Packer, Kubernetes, Docker, configuration management, and Hubot. Code samples and demos are provided to define infrastructure with Terraform, transform jobs into pipelines with Jenkins, build images with Packer, define containers in Kubernetes, build servers with Docker, and interact with services through Hubot chat. A link is provided to access additional sample code.
This document summarizes a presentation about migrating an existing ad data loading (ADLA) batch system to Amazon Batch Service. It outlines problems with the old single-server system like slowdowns from multiple batches running at once. The solution section then explains how Amazon Batch Service provides optimized compute resources and workload distribution. It details how the batches were refactored as Docker images and job definitions to run on Amazon Batch's compute environments and job queues. The last section previews a demo of the new system and lists pros like scalability and cons like increased management overhead.
Run C++ as serverless with GCP Cloud FunctionsRuncy Oommen
Runcy Oommen discusses using Google Cloud Functions with C++. Cloud Functions allows code to be written in Node.js or Python and executed in corresponding runtimes. It can access GCP services and be triggered by events from HTTP, Cloud Storage, Cloud Pub/Sub, and Firebase. The document walks through building a C++ addon for Cloud Functions using V8, binding.gyp, and Node.js to invoke the native code. It describes uploading the files, testing the function, and depicting the overall flow.
[HashiTalk Korea] OCP with Super Tengen Toppahyeongchae lee
The document discusses using Orchestrator, Consul, and ProxySQL for database orchestration. It provides an overview of how WiX Engineering uses Orchestrator to orchestrate ProxySQL with Open Containers Platform (OCP). It also discusses Github's use of Orchestrator for MySQL replication topology management and incident analysis. HashiCorp's Consul is presented as an alternative for service discovery and integration with tools like Orchestrator and ProxySQL. The document demonstrates how ProxySQL can be used with Consul and monitored using PMM. It concludes with a summary of using this "parasite architecture" approach with additional tools like Envoy, Gloo, Vault, and dnsmasq.
What is rest. Why is it part of the Rails way?. Railsconf 2014. Javier Ramirezjavier ramirez
When David Heinemeier Hansson started talking about REST in 2006, little could we suspect it'd become such a central part of Rails (and of web development in general). Back then a web service meant something you coded using XML and SOAP. Those were dark times. REST changed it all.
In this talk, I'll explain REST for beginners, I'll talk about why it is such an important architecture, and I'll show all the nice things Rails offers you for building a RESTful application or a RESTful API.
API Analytics with Redis and Bigquery. NoSQLmatters Cologne '14 edition. Javi...javier ramirez
At teowaki we have a system for API usage analytics, with Redis as a fast intermediate store and bigquery as a big data backend. As a result, we can launch aggregated queries on our traffic/usage data in just a few seconds and we can try and find for usage patterns that wouldn’t be obvious otherwise.In this session I will talk about how we entered the Big Data world, which alternatives we evaluated, and how we are using Redis and Bigquery to solve our problem.
At teowaki we have a system for API usage analytics, with Redis as a fast intermediate store and bigquery as a big data backend. As a result, we can launch aggregated queries on our traffic/usage data in just a few seconds and we can try and find for usage patterns that wouldn’t be obvious otherwise.
In this session I will talk about how we entered the Big Data world, which alternatives we evaluated, and how we are using Redis and Bigquery to solve our problem.
usable rest apis, by Javier Ramirez from teowaki (Apidays Mediterranea)javier ramirez
This summary provides the key points from the document in 3 sentences:
The document discusses various aspects of designing usable and effective REST APIs, highlighting the importance of usability principles and emphasizing that API usability is just as important as web usability. It also provides examples of different HTTP status codes and considerations for API design such as supporting different formats, enabling caching, implementing hypermedia and links, handling pagination and versioning, and ensuring the API is self-documenting. The overall message is that API designers should thoughtfully consider usability and the user experience when building RESTful interfaces.
Basics of the Highly Available Distributed Databases - teowaki - javier ramir...javier ramirez
Basic concepts of distributed systems, such as consensus, gossip and infection protocols, vector clocks, sharding storage, so you can create highly available distributed systems
Rapidly Building and Deploying Scalable Web ArchitecturesKeith Fitzgerald
The document provides an overview of how Keith Fitzgerald and his team of 4 developers were able to rapidly build and deploy the scalable web product TicketLeap using various cloud technologies. It discusses how they used Git for source control and bug tracking, Continuous Integration for testing, Trello for backlog management, Heroku for hosting, PostgreSQL/MongoDB for persistence, S3 and CloudFront for static assets, and common frameworks like Rails, Django, and Node. It emphasizes that architectures can scale rather than specific technologies, and that launching a product that can scale from the start allows a team to iterate quickly.
Peter Hoddie's keynote for IEEE at CES 2016. He explores upcoming trends for developers in the IoT space, scriptable IoT leading us to the right standards, and JavaScript for the IoT.
This document summarizes the Yatoto social crowdfunding platform. It provides details on its business model, which incorporates elements from lottery, Facebook, eBay, charities and Kickstarter. It discusses user and financial metrics from 2013-2016. It also describes the technical architecture including the Ruby on Rails backend, use of Docker for deployment, monitoring with Zabbix and error tracking with Errbit. Development practices like pair programming, code reviews and CI/CD are outlined.
What happens when you start transitioning from a monolithic PHP app to Go services running on AWS Lambda? Good things! I'd like to share the problems encountered, decisions made and lessons learned along the way.
Migrating to an Agile Architecture, Will Demaine, Engineer, Fat LlamaUXDXConf
Will Demaine, Engineer, Fat Llama Setup decisions: Planning your Agile architecture (Cloud migration path, platform choice, microservices/conainer architecture) ... Before you know everything about your product, how are you supposed to set it up
Fast, concurrent ruby web applications with EventMachine and EM::SynchronyKyle Drake
This document summarizes Kyle Drake's presentation on using EventMachine (EM) and EM::Synchrony to build fast, concurrent Ruby web applications without blocking I/O or callback hell. Key points include:
- EM implements the reactor pattern to handle blocking I/O without threads by using callbacks and kernel threads.
- EM::Synchrony avoids callback nesting by wrapping callbacks in fibers, allowing synchronous-looking code.
- Sinatra can be made concurrent by running each request in its own EM::Synchrony fiber with little code change.
- Ruby has strong options for building high-performance concurrent apps while maintaining productivity advantages over Node.js.
The document discusses how AT&T Interactive, formerly known as Yellowpages.com, uses Ruby and Ruby on Rails extensively. It overhauled Yellowpages.com using Rails and now has one of the largest Rails sites. It discusses how Rails is used for the website tier and a Merb-based services tier. It also describes using Ruby for other projects including mobile apps and internal tools. AT&T Interactive contributes to many open source Ruby projects and conducts research on new technologies using Ruby.
Standards are driving the growth of the Internet of Things by enabling new business models, rapid development, and global markets. Key standardization activities include work by the IETF on networking protocols like IPv6, CoAP, and security, as well as work by OMA, IPSO Alliance, and OneM2M on lightweight M2M, web objects, and a common M2M system standard. Building a Web of Things involves using standards like CoAP, 6LoWPAN, and IPSO web objects to connect M2M devices to backend systems over various network types.
This document provides an overview of Node.js including:
- What Node.js is and its event-driven, non-blocking architecture
- How to install Node.js and build applications
- How to use modules like Express.js and Socket.io
- Examples of deploying Node.js applications to Microsoft Azure
- A demonstration of building a collaborative drum machine app with Node.js, WebSockets, and the Web Audio API
Michael Tucker, GoDaddy_How GoDaddy used Spree to build their Online Store pr...Spark Solutions
Spree is an open source e-commerce platform built on Ruby on Rails. It has a large community and is highly customizable and extensible. This document discusses Spree's architecture and performance, including how it scales to support tens of thousands of live stores with high traffic. It also identifies some areas for improvement, such as maintaining order consistency and clearer upgrade paths.
Peter Hoddie, Kinoma VP, gave a talk at the IoT-themed API-Craft meet-up at the Tradeshift HQ in San Francisco. He discusses connectivity, the challenges and demands of IoT, and how Kinoma is building a set of APIs for the IoT.
Make your app idea a reality with Ruby On RailsNataly Tkachuk
This document provides an overview of Ruby on Rails including what it is, how to get started, learning resources, and why it may be suitable for building an app idea. Ruby on Rails is an open-source web application framework that is simple to learn, promotes programmer happiness through conventions, and has a large ecosystem of plugins and a supportive community. The document outlines options for learning Ruby on Rails such as online courses, books, screencasts, and community resources and emphasizes that it offers development simplicity and a lifestyle that can help bring ideas to life.
Move out from AppEngine, and Python PaaS alternativestzang ms
This document discusses moving a podcast hosting application called MyAudioCast off of Google App Engine (GAE) and onto other Python platforms as a result of high costs and limitations. Some key points:
- MyAudioCast was running on GAE for over a year but costs were rising to $120/month due to high storage, bandwidth, and processing usage.
- Performance on GAE was poor with high error rates for operations like inserting logs and updating counters.
- Development was slowed by GAE limitations like long deployment times and inability to easily use common Python packages.
- The author chose to migrate MyAudioCast to the Linode VPS and Heroku PaaS for better pricing,
This document provides an introduction to web applications and their components. It discusses what a web application is, how it is made up of front-end and back-end parts, and how Ruby on Rails is a framework that helps develop web applications. It then demonstrates Ruby programming concepts through interactive examples in IRB, covering basic operations, variables, methods, classes and objects. The document aims to give a high-level overview of web applications and introduce Ruby programming.
This document discusses building robots using Raspberry Pi. It recommends using a Raspberry Pi B+ as the inexpensive yet powerful hardware, along with a motor driver, sensors, and chassis. The total estimated cost is around $80. It then outlines the steps to assemble the hardware, wire everything, and program the robot using JavaScript and the Johnny-Five library. Examples are provided of initializing Johnny-Five, making the robot move, adding sensors, and improving the programming. References for further information and South African online retailers are also included.
【Primer】WebApp release in 3 hours! Programming from scratch Rails courseDIVE INTO CODE Corp.
The document outlines a seminar on programming using Ruby on Rails. It introduces the founder and CEO of Dive into Code Corp, Hiroyoshi Noro, and describes the seminar's goals of teaching basic programming knowledge, practicing development, and improving skills. The seminar covers the development process, programming languages like Ruby and frameworks like Rails, and tools for coding, version control, and deployment.
Building CLR/H Registration Site with ASP.NET MVC4 and EF4CodeFirstJun-ichi Sakamoto
This document discusses building a registration site using ASP.NET MVC4, Entity Framework Code First, and other technologies. It demonstrates creating a site that allows attendees to register by entering their name and email, with the list only shared among staff. Technologies used include MVC4, Razor syntax, Entity Framework Code First, SQL Server 2012 LocalDB, and libraries from NuGet. Authentication is added using HTTP Basic Authentication to allow access from any client like Excel. An API is also provided. Storing data in Google Spreadsheets is demonstrated as an alternative database.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
Hubo un tiempo en el que casi cualquier componente de software requería pagar una licencia. Afortunadamente, hoy en día gracias al software libre y de código abierto, se puede desarrollar prácticamente cualquier aplicación usando componentes gratuitos.
Pero, si el software es gratis, ¿Quién lo desarrolla? ¿Trabaja la comunidad de software libre de forma altruista? ¿Se puede desarrollar software libre de forma profesional? De hecho, hay quien dice que el código abierto tal y como lo conocimos ya no existe, y que lo que hay hoy en día es otra cosa.
En esta charla hablaré de cómo se puede monetizar el código libre, y de algunos posibles conflictos que puedes encontrarte en el camino.
Además, te contaré cómo hacemos desde QuestDB para desarrollar una base de datos de código abierto y mantener un equipo estable viviendo de ello. Comentaré también algunas situaciones problemáticas a las que proyectos muy destacados se han enfrentado, o que se enfrentan a día de hoy.
QuestDB: The building blocks of a fast open-source time-series databasejavier ramirez
(talk delivered at OSA CON 23)
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed.
We will learn how it deals with data ingestion, and which SQL extensions it implements for working with time-series efficiently.
We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or data deduplication.
Como creamos QuestDB Cloud, un SaaS basado en Kubernetes alrededor de QuestDB...javier ramirez
QuestDB es una base de datos open source de alto rendimiento. Mucha gente nos comentaba que les gustaría usarla como servicio, sin tener que gestionar las máquinas. Así que nos pusimos manos a la obra para desarrollar una solución que nos permitiese lanzar instancias de QuestDB con provisionado, monitorización, seguridad o actualizaciones totalmente gestionadas.
Unos cuantos clusters de Kubernetes más tarde, conseguimos lanzar nuestra oferta de QuestDB Cloud. Esta charla es la historia de cómo llegamos ahí. Hablaré de herramientas como Calico, Karpenter, CoreDNS, Telegraf, Prometheus, Loki o Grafana, pero también de retos como autenticación, facturación, multi-nube, o de a qué tienes que decir que no para poder sobrevivir en la nube.
Ingesting Over Four Million Rows Per Second With QuestDB Timeseries Database ...javier ramirez
How would you build a database to support sustained ingestion of several hundreds of thousands rows per second while running near real-time queries on top?
In this session I will go over some of the technical decisions and trade-offs we applied when building QuestDB, an open source time-series database developed mainly in JAVA, and how we can achieve over four million row writes per second on a single instance without blocking or slowing down the reads. There will be code and demos, of course.
We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
Deduplicating and analysing time-series data with Apache Beam and QuestDBjavier ramirez
Time series data pipelines tend to prioritise speed and freshness over completeness and integrity. In such scenarios, it is very common to ingest duplicate data, which may be fine for many analytical use cases, but is very inconvenient for others.
There are many open source databases built specifically for the speed and query semantics of time series, and most of them lack automatic deduplication of events in near real-time. One such database is QuestDB, which requires a manual batch process to deduplicate ingested data.
In this talk, we will see how we can successfully use Apache Beam to deduplicate streaming time series, which can then be analysed by a time series database.
Relational databases were created a long time ago for a simpler world. Even if they are still awesome tools for generic workloads, there are some things they cannot do well.
In this session I will speak about purpose-built databases that you can use for specific business scenarios. We will see the type of queries you can run on a Graph database, a Document Database, and a Time-Series database. We will then see how a relational database could also be used for the same use cases, just in a much more complex way.
Your Timestamps Deserve Better than a Generic Databasejavier ramirez
This document discusses the challenges of working with timestamped data in databases and introduces QuestDB as a time-series database designed to address these challenges. It highlights QuestDB's high performance for ingesting and querying large volumes of timestamped data. It also demonstrates several time-series focused query patterns in QuestDB like time range queries, sampling, filling missing data, retrieving the latest value, and approximate joins between tables. Finally, it outlines some areas QuestDB is exploring to further improve performance.
Cómo se diseña una base de datos que pueda ingerir más de cuatro millones de ...javier ramirez
En esta sesión voy a contar las decisiones técnicas que tomamos al desarrollar QuestDB, una base de datos Open Source para series temporales compatible con Postgres, y cómo conseguimos escribir más de cuatro millones de filas por segundo sin bloquear o enlentecer las consultas.
Hablaré de cosas como (zero) Garbage Collection, vectorización de instrucciones usando SIMD, reescribir en lugar de reutilizar para arañar microsegundos, aprovecharse de los avances en procesadores, discos duros y sistemas operativos, como por ejemplo el soporte de io_uring, o del balance entre experiencia de usuario y rendimiento cuando se plantean nuevas funcionalidades.
Processing and analysing streaming data with Python. Pycon Italy 2022javier ramirez
Data used to be a batch thing, but more and more we get unbounded streams of data, fast or slow, that we need to process and analyse in near real time.
In this talk I’ll show you how you can use Apache Flink and QuestDB to build reliable streaming data pipelines that can grow as much as you need.
QuestDB: ingesting a million time series per second on a single instance. Big...javier ramirez
In this session I will show you the technical decisions we made when building QuestDB, the open source, Postgres compatible, time-series database, and how we can achieve a million row writes per second without blocking or slowing down the reads.
Servicios e infraestructura de AWS y la próxima región en Aragónjavier ramirez
AWS está montando una región de infraestructura en Aragón. Vale, pero ¿Qué significa eso? ¿Es tan diferente de un centro de datos convencional o de otros proveedores de nube? (Spoiler: Sí). En esta sesión te cuento por qué. Hay video en https://catedrasamcadt.unizar.es/noticias/el-momento-tecnologico-actual-contado-por-trabajadores-de-amazon-web-services/
¿Qué es eso del desarrollo sin servidores? ¿Qué lenguajes puedo utilizar? ¿Cómo hago cosas como autenticación, o guardar en base de datos, o enviar notificaciones? ¿Esto escala? A todas estas preguntas, y a alguna más, intentaré dar respuesta en esta sesión, donde haré una pequeña demo de montar una app muy sencilla y desplegarla en la nube sin preocuparnos de gestionar infraestructura. Charla realizada por primera vez para AlcarriaConf 2021
AWS launched publicly on March 2006 with just one service, starting the age of the public cloud. You might think after 15 years everything in cloud has already been invented, but that's simply not the case.
In this session I want to show you how AWS is reinventing the cloud in areas like computing, machine learning, databases and analytics, or cloud infrastructure.
Analitica de datos en tiempo real con Apache Flink y Apache BEAMjavier ramirez
This document summarizes a presentation about real-time data analytics with Apache Flink and Apache BEAM. It discusses possible real-time and batch processing systems using AWS services, challenges of streaming systems including state management, and demos of analyzing user clickstreams and taxi trips with Apache Flink, Kafka, and Elasticsearch. It also covers advantages of Apache BEAM including a unified batch and streaming API that can run on different frameworks like Flink, benefits of native support for Java, Python, and Go, and how it allows mixing languages in pipelines.
In this webinar we explain which are some of the problems of streaming analytics, and why they are different to batch/big data analytics. Then we go into introducing some basic streaming concepts, like event queues, event processors, event vs processing time, and delivery guarantees. We end this first part of the series presenting a few of the most common open source components for streaming (Kafka, Spark, Flink, Cassandra, or ElasticSearch) and we mention the different options you have to run them on AWS.
Getting started with streaming analytics: Setting up a pipelinejavier ramirez
In this session I will show you how to create a simple streaming analytics pipeline, first using open source tools and developing locally, then moving to a VM, then moving to fully managed AWS services. The session will serve as an introduction to some details of Apache Kafka, Apache Flink, ElasticSearch, Amazon Managed Streaming for Kafka, Kinesis Data Analytics, and Amazon ElasticSearch. It will be an almost slideless presentation, as I will spent most of the time at the command line and the IDE.
Getting started with streaming analytics: Deep Divejavier ramirez
Now that we know how to create simple streaming analytics pipelines, it is time to learn something more interesting. In this session I will show you how to add Complex Event Processing to your Apache Flink (or Kinesis Data Analytics) application using JAVA. For those of you that prefer SQL, I will show you how to run streaming analytics using only SQL.
Getting started with streaming analytics: streaming basics (1 of 3)javier ramirez
In this webinar we explain which are some of the problems of streaming analytics, and why they are different to batch/big data analytics. Then we go into introducing some basic streaming concepts, like event queues, event processors, event vs processing time, and delivery guarantees. We end this first part of the series presenting a few of the most common open source components for streaming (Kafka, Spark, Flink, Cassandra, or ElasticSearch) and we mention the different options you have to run them on AWS.
Adani Group's Active Interest In Increasing Its Presence in the Cement Manufa...Adani case
Time and again, the business group has taken up new business ventures, each of which has allowed it to expand its horizons further and reach new heights. Even amidst the Adani CBI Investigation, the firm has always focused on improving its cement business.
AI Transformation Playbook: Thinking AI-First for Your BusinessArijit Dutta
I dive into how businesses can stay competitive by integrating AI into their core processes. From identifying the right approach to building collaborative teams and recognizing common pitfalls, this guide has got you covered. AI transformation is a journey, and this playbook is here to help you navigate it successfully.
SATTA MATKA DPBOSS KALYAN MATKA RESULTS KALYAN CHART KALYAN MATKA MATKA RESULT KALYAN MATKA TIPS SATTA MATKA MATKA COM MATKA PANA JODI TODAY BATTA SATKA MATKA PATTI JODI NUMBER MATKA RESULTS MATKA CHART MATKA JODI SATTA COM INDIA SATTA MATKA MATKA TIPS MATKA WAPKA ALL MATKA RESULT LIVE ONLINE MATKA RESULT KALYAN MATKA RESULT DPBOSS MATKA 143 MAIN MATKA KALYAN MATKA RESULTS KALYAN CHART
Enhancing Adoption of AI in Agri-food: IntroductionCor Verdouw
Introduction to the Panel on: Pathways and Challenges: AI-Driven Technology in Agri-Food, AI4Food, University of Guelph
“Enhancing Adoption of AI in Agri-food: a Path Forward”, 18 June 2024
➒➌➎➏➑➐➋➑➐➐ Satta Matka Dpboss Matka Guessing Indian Matka Satta Matta Matka KALYAN MATKA | MATKA RESULT | KALYAN MATKA TIPS | SATTA MATKA | MATKA.COM | MATKA PANA JODI TODAY | BATTA SATKA | MATKA PATTI JODI NUMBER | MATKA RESULTS | MATKA CHART | MATKA JODI | SATTA COM | FULL RATE GAME | MATKA GAME | MATKA WAPKA | ALL MATKA RESULT LIVE ONLINE | MATKA RESULT | KALYAN MATKA RESULT | DPBOSS MATKA 143 | MAIN MATKA
➒➌➎➏➑➐➋➑➐➐ Satta Matka Dpboss Matka Guessing Indian Matka
KALYAN MATKA | MATKA RESULT | KALYAN MATKA TIPS | SATTA MATKA | MATKA.COM | MATKA PANA JODI TODAY | BATTA SATKA | MATKA PATTI JODI NUMBER | MATKA RESULTS | MATKA CHART | MATKA JODI | SATTA COM | FULL RATE GAME | MATKA GAME | MATKA WAPKA | ALL MATKA RESULT LIVE ONLINE | MATKA RESULT | KALYAN MATKA RESULT | DPBOSS MATKA 143 | MAIN MATKA
Tired of chasing down expiring contracts and drowning in paperwork? Mastering contract management can significantly enhance your business efficiency and productivity. This guide unveils expert secrets to streamline your contract management process. Learn how to save time, minimize risk, and achieve effortless contract management.
SATTA MATKA DPBOSS KALYAN MATKA RESULTS KALYAN CHART KALYAN MATKA MATKA RESULT KALYAN MATKA TIPS SATTA MATKA MATKA COM MATKA PANA JODI TODAY BATTA SATKA MATKA PATTI JODI NUMBER MATKA RESULTS MATKA CHART MATKA JODI SATTA COM INDIA SATTA MATKA MATKA TIPS MATKA WAPKA ALL MATKA RESULT LIVE ONLINE MATKA RESULT KALYAN MATKA RESULT DPBOSS MATKA 143 MAIN MATKA KALYAN MATKA RESULTS KALYAN CHART
SATTA MATKA DPBOSS KALYAN MATKA RESULTS KALYAN CHART KALYAN MATKA MATKA RESULT KALYAN MATKA TIPS SATTA MATKA MATKA COM MATKA PANA JODI TODAY BATTA SATKA MATKA PATTI JODI NUMBER MATKA RESULTS MATKA CHART MATKA JODI SATTA COM INDIA SATTA MATKA MATKA TIPS MATKA WAPKA ALL MATKA RESULT LIVE ONLINE MATKA RESULT KALYAN MATKA RESULT DPBOSS MATKA 143 MAIN MATKA KALYAN MATKA RESULTS KALYAN CHART
japanese language course in delhi near meheyfairies7
Next is the Nihon Language Academy in East Delhi, renowned for its comprehensive curriculum and interactive teaching methods. They boast a faculty of experienced educators with a blend of both Indian and Japanese nationals. The academy provides extensive support for JLPT exam preparation along with personalized tutoring sessions if needed. Nihon Language Academy also arranges exchange programs with partner institutes in Japan, which provides students an opportunity to experience Japanese culture and language first-hand.
Adani Group Requests For Additional Land For Its Dharavi Redevelopment Projec...Adani case
It will bring about growth and development not only in Maharashtra but also in our country as a whole, which will experience prosperity. The project will also give the Adani Group an opportunity to rise above the controversies that have been ongoing since the Adani CBI Investigation.
➒➌➎➏➑➐➋➑➐➐ Satta Matka Dpboss Matka Guessing Indian Matka Satta Matta Matka KALYAN MATKA | MATKA RESULT | KALYAN MATKA TIPS | SATTA MATKA | MATKA.COM | MATKA PANA JODI TODAY | BATTA SATKA | MATKA PATTI JODI NUMBER | MATKA RESULTS | MATKA CHART | MATKA JODI | SATTA COM | FULL RATE GAME | MATKA GAME | MATKA WAPKA | ALL MATKA RESULT LIVE ONLINE | MATKA RESULT | KALYAN MATKA RESULT | DPBOSS MATKA 143
SATTA MATKA DPBOSS KALYAN MATKA RESULTS KALYAN CHART KALYAN MATKA MATKA RESULT KALYAN MATKA TIPS SATTA MATKA MATKA COM MATKA PANA JODI TODAY BATTA SATKA MATKA PATTI JODI NUMBER MATKA RESULTS MATKA CHART MATKA JODI SATTA COM INDIA SATTA MATKA MATKA TIPS MATKA WAPKA ALL MATKA RESULT LIVE ONLINE MATKA RESULT KALYAN MATKA RESULT DPBOSS MATKA 143 MAIN MATKA KALYAN MATKA RESULTS KALYAN CHART
28. Hypermedia RESTful API (Ruby on Rails)
Kind-of-responsive web application (AngularJS)
Background jobs queue (Redis)
Background workers (Ruby on Rails)
Postgresql master/slave, for text search too
Redis master/slave (with LUA scripting)
Memcached
29. Devops
Ansible for provisioning
Monit and Munin for server health check
Google Bigquery for bigdata processing
S3 and Glacier for backups (every minute for
postgresql, every hour for Redis and stats)
30. If you love your developers, please tell them about
https://teowaki.com
Cheers!
techcity talks
javier ramirez
j@teowaki.com